Release 3.5.0

Closes Gallopsled/pwntools#921

.github/ISSUE_TEMPLATE.md

@@ -6,4 +6,4 @@ When reporting an issue, be sure that you are running the latest released versio
 
 Please verify that your issue occurs on 64-bit Ubuntu 14.04.  You can use the Dockerfile in `extra/docker/Dockerfile` to save time.
 
-If possible, provide a proof-of-concept which demonstrates the problem.  Include any binaries or scripts necessary to reproduce the issue.
+If possible, provide a proof-of-concept which demonstrates the problem.  Include any binaries or scripts necessary to reproduce the issue, and please include the full debug output via setting the environment variable `PWNLIB_DEBUG=1`.

.travis.yml

@@ -13,6 +13,7 @@ addons:
     - mksh
     - zsh
     - pandoc
+    - gdb
 cache:
     - pip
     - directories:
@@ -22,11 +23,15 @@ cache:
 python:
   - "2.7"
 before_install:
+  - echo $-
   - source travis/install.sh
   - source travis/ssh_setup.sh
+  - echo $-
 install:
+  - pip install --upgrade pip
+  - pip install --upgrade appdirs # https://github.com/ActiveState/appdirs/issues/89#issuecomment-282326570
   - pip install --upgrade --editable .
-  - pip install --requirement docs/requirements.txt
+  - pip install --upgrade --requirement docs/requirements.txt
 before_script:
   - PWNLIB_NOTERM=1 python -c 'from pwn import *; print pwnlib.term.term_mode'
   - PWNLIB_NOTERM=1 python -c 'from pwn import *; print pwnlib.term.term_mode'

CHANGELOG.md

@@ -9,9 +9,18 @@ The table below shows which release corresponds to each branch, and what date th
 
 | Version          | Branch   | Release Date           |
 | ---------------- | -------- | ---------------------- |
-| [3.4.0](#340)    | `dev`    | Feb 4, 2017 (planned)
-| [3.3.0](#330)    | `beta`   | Dec 24, 2016 (planned)
-| [3.2.0](#320)    | `stable` | Nov 12, 2016
+| [3.7.0](#350)    | `dev`    | Jun 17, 2017 (planned)
+| [3.6.0](#350)    | `beta`   | May 6, 2017 (planned)
+| [3.5.0](#350)    | `stable` | Mar 26, 2017
+| [3.4.1](#341)    |          | Feb 17, 2017
+| [3.4.0](#340)    |          | Feb 13, 2017
+| [3.3.4](#334)    |          | Jan 12, 2016
+| [3.3.3](#333)    |          | Jan 10, 2016
+| [3.3.2](#332)    |          | Jan 10, 2016
+| [3.3.1](#331)    |          | Jan 10, 2016
+| [3.3.0](#330)    |          | Dec 24, 2016
+| [3.2.1](#321)    |          | Dec 24, 2016
+| [3.2.0](#320)    |          | Nov 12, 2016
 | [3.1.1](#311)    |          | Oct 23, 2016
 | [3.1.0](#310)    |          | Oct 2, 2016
 | [3.0.4](#304)    |          | Sept 19, 2016
@@ -21,19 +30,226 @@ The table below shows which release corresponds to each branch, and what date th
 | [3.0.0](#300)    |          | Aug 20, 2016
 | [2.2.0](#220)    |          | Jan 5, 2015
 
+## 3.7.0
+
+To be on Jun 17, 2017.
+
+## 3.6.0
+
+To be released on May 6, 2017.
+
+- [#895][895] Added a Dockerfile to simplify testing setup and allow testing on OSX
+- [#897][897] Fixed some incorrect AArch64 syscals
+- [#893][893] Added the `pwnlib.config` module
+    + Configuration options can now be set in `~/.pwn.conf`
+    + This replaces the old, **undocumented** mechanism for changing logging colors.  Only @br0ns and @ebeip90 were likely using this.
+    + More information is available in the documentation [here](http://docs.pwntools.com/en/dev/config.html).
+- [#899][899] Pwntools now uses Unicorn Engine to emulate PLT instructions to ensure correct mapping of PIE / RELRO binaries.
+- [#904][904] Enhancements to the accuracy of the `pwn checksec` command.
+- [#905][905] Added a `pwn debug` command-line utility which automates the process of `gdb.attach(process(...))` to spawn GDB
+    + More information is available in the documentation [here](http://docs.pwntools.com/en/dev/commandline.html#pwn-debug)
+- [#919][919] Added a `pwn template` command-line utility to simplify the process of bootstrapping a new exploit.
+    + More information is available in the documentation [here](http://docs.pwntools.com/en/dev/commandline.html#pwn-template).
+
+[895]: https://github.com/Gallopsled/pwntools/pull/895
+[897]: https://github.com/Gallopsled/pwntools/pull/897
+[893]: https://github.com/Gallopsled/pwntools/pull/893
+[899]: https://github.com/Gallopsled/pwntools/pull/899
+[904]: https://github.com/Gallopsled/pwntools/pull/904
+[905]: https://github.com/Gallopsled/pwntools/pull/905
+[919]: https://github.com/Gallopsled/pwntools/pull/919
+
+
+## 3.5.0
+
+- [b584ca3][b584ca3] Fixed an issue running `setup.py` on ARM
+- [#822][822] Enabled relative leaks with `MemLeak`
+    + This should be useful for e.g. heap-relative leaks
+- [#832][832] Changed all internal imports to use absolute imports (no functional changes)
+- [a12d0b6][a12d0b6] Move `STDOUT`, `PIPE`, `PTY` constants to globals
+    + `process(..., stdin=process.PTY)` --> `process(..., stdin=PTY)`
+- [#828][828] Use `PR_SET_PTRACER` for all `process()` and `ssh.process()` instances
+    + This simplifies debugging on systems with YAMA ptrace enabled
+- Various documentation enhancements
+    + In particular, the [gdb][gdb], [elf][elf], and [ssh][ssh] docs are much better
+- [#833][833] Performance enhancements for `adb` module
+- [d0267f3][d0267f3] `packing.fit()` now treats large offsets as cyclic patterns (e.g. `0x61616161` behaves the same as `"aaaa"`)
+- [#835][835] Added `ssh.checksec`
+    + Reports the kernel version and other relevant information on connection
+- [#857][857] Slightly shortened `execve` shellcode
+- [300f8e0][300f8e0] Slightly speed up processing of large ELF files
+- [#861][861] Adds support for extracting `IKCONFIG` configs from Linux kernel images, and extends `checksec` to report on any insecure configurations discovered
+- [#871][871] Moves all of the basic syscall templates to `shellcraft/common` and exposes them via symlinks.  Closed [#685][685]
+    + Should not have any visible effects from any documented APIs
+    + `shellcraft.arch.os.syscall_function()` still works the same
+    + We now have the ability to differentiate between the `connect` syscall, and a TCP `connect` helper
+- [#887][887] `sh_string` now returns a quoted empty string `''` rather than just an empty string
+- [#839][839] Exposes a huge amount of functionality via corefiles which was not previously availble.  See the [docs][corefile_docs] for examples.
+    + `process().corefile` will automatically instantiate a Corefile for the process
+    + QEMU-emulated processes are supported
+    + Native processes are supported, including extraction of coredumps from `apport` crash logs
+    + Native processes can be dumped *while running*, in a manner similar to `GDB`'s `gcore` script
+- [#875][857] Added [documentation][aarch64] (and tests) for AArch64 shellcode
+- [#882][882] The `ROP` class now respects `context.bytes` instead of using the hard-coded value of `4` (fixed [#879][879])
+- [#869][869] Added several fields to the `process` class (`uid`, `gid`, `suid`, `sgid`) which are recorded at execution time, based on the file permissions
+- [#868][868] Changed the way that `ssh.process()` works internally, and it now returns a more specialized class, `ssh_process`.
+    + Added `ssh_process.corefile` for fetching remote corefiles
+    + Added `ssh_process.ELF` for getting an ELF of the remote executable
+    + The `uid`, `gid`, and `suid`, and `sgid` which are recorded at execution time, based on the file permissions
+- [#865][865] Fixes `ELF.read` to support contiguous memory reads across non-contiguous file-backed segments
+- [#862][862] Adds a `symlink=` argument to `ssh.set_working_directory`, which will automatically symlink all of the files in the "old" working directory into the "new" working directory
+
+[ssh]: http://docs.pwntools.com/en/dev/tubes/ssh.html
+[gdb]: http://docs.pwntools.com/en/dev/gdb.html
+[elf]: http://docs.pwntools.com/en/dev/elf.html
+[corefile_docs]: http://docs.pwntools.com/en/dev/elf/corefile.html
+[aarch64]: http://docs.pwntools.com/en/dev/shellcraft/aarch64.html
+
+[685]: https://github.com/Gallopsled/pwntools/pull/685
+[822]: https://github.com/Gallopsled/pwntools/pull/822
+[828]: https://github.com/Gallopsled/pwntools/pull/828
+[832]: https://github.com/Gallopsled/pwntools/pull/832
+[833]: https://github.com/Gallopsled/pwntools/pull/833
+[835]: https://github.com/Gallopsled/pwntools/pull/835
+[839]: https://github.com/Gallopsled/pwntools/pull/839
+[857]: https://github.com/Gallopsled/pwntools/pull/857
+[861]: https://github.com/Gallopsled/pwntools/pull/861
+[862]: https://github.com/Gallopsled/pwntools/pull/862
+[865]: https://github.com/Gallopsled/pwntools/pull/865
+[868]: https://github.com/Gallopsled/pwntools/pull/868
+[869]: https://github.com/Gallopsled/pwntools/pull/869
+[871]: https://github.com/Gallopsled/pwntools/pull/871
+[875]: https://github.com/Gallopsled/pwntools/pull/857
+[879]: https://github.com/Gallopsled/pwntools/issues/879
+[882]: https://github.com/Gallopsled/pwntools/pull/882
+[887]: https://github.com/Gallopsled/pwntools/pull/887
+
+
+[b584ca3]: https://github.com/Gallopsled/pwntools/commit/b584ca3
+[a12d0b6]: https://github.com/Gallopsled/pwntools/commit/a12d0b6
+[d0267f3]: https://github.com/Gallopsled/pwntools/commit/d0267f3
+[300f8e0]: https://github.com/Gallopsled/pwntools/commit/300f8e0
+
+## 3.4.1
+
+- [#894][894] Fix a bug when using `gdb.debug()` over ssh.
+- [e021f57][e021f57] Fix a bug ([#891][891]) in `rop` when needing to insert padding to fix alignment
+
+[e021f57]: https://github.com/Gallopsled/pwntools/commit/e021f57
+[894]: https://github.com/Gallopsled/pwntools/pull/894
+[891]: https://github.com/Gallopsled/pwntools/issues/891
+
 ## 3.4.0
 
-To be released on Feb 4, 2017.
+- [#800][800] Add `shell=` option to `ssh.process()`
+- [#806][806] Add `context.buffer_size` for fine-tuning `tube` performance
+    + Also adds `buffer_fill_size=` argument for all tubes
+- [b83a6c7][b83a6c7] Fix undocumented `process.leak` function
+- [546061e][546061e] Modify `coredump_filter` of all spawned processes, so that core dumps are more complete
+- [#809][809] Add several functions to `adb` (`unlink`, `mkdir`, `makedirs`, `isdir`, `exists`)
+- [#817][817] Make disconnection detection more robust
+
+[800]: https://github.com/Gallopsled/pwntools/pull/800
+[806]: https://github.com/Gallopsled/pwntools/pull/806
+[809]: https://github.com/Gallopsled/pwntools/pull/809
+[817]: https://github.com/Gallopsled/pwntools/pull/817
+[5d9792f]: https://github.com/Gallopsled/pwntools/commit/5d9792f
+[b83a6c7]: https://github.com/Gallopsled/pwntools/commit/b83a6c7
+[546061e]: https://github.com/Gallopsled/pwntools/commit/546061e
+
+## 3.3.4
+
+- [#850][850] and [#846][846] fix issues with `hexdump` and the `phd` command-line utility, when using pipes (e.g. `echo foo | phd`)
+- [#852][852] Fixes register ordering in `regsort`
+- [#853][853] Fixes the registers restored in `shellcraft.amd64.popad`
+
+[846]: https://github.com/gallopsled/pwntools/pull/846
+[850]: https://github.com/gallopsled/pwntools/pull/850
+[852]: https://github.com/gallopsled/pwntools/pull/852
+[853]: https://github.com/gallopsled/pwntools/pull/853
+
+## 3.3.3
+
+- [#843][843] fixed a bug in `amd64.mov`.
+
+[843]: https://github.com/gallopsled/pwntools/pull/843
+
+## 3.3.2
+
+- [#840][840] fixed a regression introduced by [#837][837].
+
+[840]: https://github.com/gallopsled/pwntools/pull/840
+
+## 3.3.1
+
+- [#833][833] Fixed a performance-impacting bug in the adb module.
+- [#837][837] Fixed a bug([#836][836]) causing `hexdump(cyclic=True)` to throw an exception.
+
+[833]: https://github.com/Gallopsled/pwntools/pull/833
+[837]: https://github.com/Gallopsled/pwntools/pull/837
+[836]: https://github.com/Gallopsled/pwntools/issues/836
 
 ## 3.3.0
 
-To be released on Dec 24, 2016.
+- [b198ec8][b198ec8] Added `tube.stream()` function, which is like `tube.interact()` without a prompt or keyboard input.
+    + Effectively, this is similar to `cat file` and just prints data as fast as it is received.
+- [aec3fa6][aec3fa6] Disable update checks against GitHub
+    + These checks frequently broke due to GitHub query limits
+- [#757][757] Fixed `adb.wait_for_device()` re-use of the same connection
+- [f9133b1][f9133b1] Add a `STDERR` magic argument to make logging go to `stderr` instead of `stdout`
+    + Usage is e.g. `python foo.py STDERR` or `PWNLIB_STDERR=1 python foo.py`
+    + Also adds `context.log_console` to log to any file or terminal
+- [67e11a9][67e11a9] Add faster error checking to `cyclic()` when provided very large values
+- [5fda658][5fda658] Expose BitPolynom in `globals()`
+- [#765][765] Added `-d` option for hex-escaped output for `shellcraft` command-line tool
+- [#772][772] Fixed bash completion regressions
+- [30c34b7][30c34b7] Fix `ROP.call()` with `Function` objects from `ELF.functions`
+- [fa402ce][fa402ce] Add `adb.uptime` and `adb.boot_time`
+- [82312ba][82312ba] Add `cyclic_metasploit` and `cyclic_metasploit_find`
+
+[757]: https://github.com/Gallopsled/pwntools/pull/757
+[765]: https://github.com/Gallopsled/pwntools/pull/765
+[772]: https://github.com/Gallopsled/pwntools/pull/772
+[b198ec8]: https://github.com/Gallopsled/pwntools/commit/b198ec8
+[aec3fa6]: https://github.com/Gallopsled/pwntools/commit/aec3fa6
+[f9133b1]: https://github.com/Gallopsled/pwntools/commit/f9133b1
+[67e11a9]: https://github.com/Gallopsled/pwntools/commit/67e11a9
+[5fda658]: https://github.com/Gallopsled/pwntools/commit/5fda658
+[30c34b7]: https://github.com/Gallopsled/pwntools/commit/30c34b7
+[fa402ce]: https://github.com/Gallopsled/pwntools/commit/fa402ce
+[82312ba]: https://github.com/Gallopsled/pwntools/commit/82312ba
+
+## 3.2.1
+
+Multiple bug fixes.
+
+- [#783][783] Fix `adb.uninstall` typo
+- [#787][787] Added error handling for `ssh.process` argument `preexec_fn`
+- [#793][793] Fixed progress message in `remote()` when connections failed
+- [#802][802] Fixed partition listing in `adb.partitions`, which accidentally shelled out to the `adb` binary
+- [#804][804] Fix error message for 32-bit distributions
+- [#805][805] Fix exception in `Core.segments` when a segment has no name
+- [#811][811] Fixes and performance improvements for `adb.wait_for_device()`
+- [#813][813] Fixed a release script
+- [#814][814] Fixed exceptions thrown if the `$HOME` directory is not writable
+- [#815][815] Properly handle `None` in `MemLeak`
+
+[783]: https://github.com/Gallopsled/pwntools/pull/783
+[787]: https://github.com/Gallopsled/pwntools/pull/787
+[793]: https://github.com/Gallopsled/pwntools/pull/793
+[802]: https://github.com/Gallopsled/pwntools/pull/802
+[804]: https://github.com/Gallopsled/pwntools/pull/804
+[805]: https://github.com/Gallopsled/pwntools/pull/805
+[811]: https://github.com/Gallopsled/pwntools/pull/811
+[813]: https://github.com/Gallopsled/pwntools/pull/813
+[814]: https://github.com/Gallopsled/pwntools/pull/814
+[815]: https://github.com/Gallopsled/pwntools/pull/815
 
 ## 3.2.0
 
 - [#695][695] Fixed a performance regression in `phd`.
-- [452605e][452605e] Fixed [#629][#629] related to correct removal of temporary files.
-- [ea94ee4][ea94ee4] Disallows semi-colons in for the `run_in_terminal` function, since it did not work properly in all cases. 
+- [452605e][452605e] Fixed [#629][629] related to correct removal of temporary files.
+- [ea94ee4][ea94ee4] Disallows semi-colons in for the `run_in_terminal` function, since it did not work properly in all cases.
 - [6376d07][6376d07] Added the mips shellcode `pushstr_array`.
 - [#700][700] Added missing MIPS shellcode documentation to readthedocs, and enabled unit tests
 - [#701][701] Command line tools refactored to have a common `pwn` entry point.
@@ -60,11 +276,12 @@ To be released on Dec 24, 2016.
 - [1e414af][1e414af] Added `connect()` alias for `remote()`
     + For example, `io=connect('google.com', 80)`
     + This also works with `tcp(...)` and `udp(...)` aliases
-- [869ec42][869ec42] Added `ssh.read()` and `ssh.write()` aiases
+- [869ec42][869ec42] Added `ssh.read()` and `ssh.write()` aliases
 - [2af55c9][2af55c9] `AdbDevice` objects exposed via e.g. `adb.devices()` now offer scoped access to all `adb` module properties
     + It is now possible to e.g. `map(lambda d: d.process(['id']).recvall(), adb.devices())`
 
 
+[629]: https://github.com/Gallopsled/pwntools/issues/629
 [695]: https://github.com/Gallopsled/pwntools/pull/695
 [700]: https://github.com/Gallopsled/pwntools/pull/700
 [701]: https://github.com/Gallopsled/pwntools/pull/701
@@ -146,7 +363,7 @@ Android support via a new `adb` module, `context.device`, `context.adb_host`, an
 - Added `context.log_file` setting for sending logs to a file.  This can be set with the `LOG_FILE` magic command-line option.
 - Added `context.noptrace` setting for disabling actions which require `ptrace` support.  This is useful for turning all `gdb.debug` and `gdb.attach` options into no-ops, and can be set via the `NOPTRACE` magic command-line option.
 - Added `context.proxy` which hooks all connections and sends them to a SOCKS4/SOCKS5.  This can be set via the `PROXY` magic command-line option.
-- Added `context.randomize` to control randommization of settings like XOR keys and register ordering (default off).
+- Added `context.randomize` to control randomization of settings like XOR keys and register ordering (default off).
 - Added `context.terminal` for setting how to launch commands in a new terminal.
 
 #### DynELF and MemLeak Module
@@ -159,7 +376,7 @@ Android support via a new `adb` module, `context.device`, `context.adb_host`, an
 #### Encoders Module
 
 - Added a `pwnlib.encoders` module for assembled-shellcode encoders/decoders
-- Includes position-indepentent basic XOR encoders
+- Includes position-independent basic XOR encoders
 - Includes position-independent delta encoders
 - Includes non-position-independent alphanumeric encoders for Intel
 - Includes position-independent alphanumeric encoders for ARM/Thumb

MANIFEST.in

@@ -2,6 +2,6 @@ graft examples
 graft extra
 include *.md *.txt *.sh *.yml MANIFEST.in
 recursive-include docs   *.rst *.png Makefile *.py *.txt
-recursive-include pwnlib *.py *.asm *.rst *.md *.txt *.sh
+recursive-include pwnlib *.py *.asm *.rst *.md *.txt *.sh __doc__
 recursive-include pwn    *.py *.asm *.rst *.md *.txt *.sh
 recursive-exclude *.pyc

README.md

@@ -2,7 +2,7 @@
 ![pwntools logo](https://github.com/Gallopsled/pwntools/blob/stable/docs/source/logo.png?raw=true)
 
 [![Docs](https://readthedocs.org/projects/pwntools/badge/?version=stable)](https://docs.pwntools.com/)
-[![PyPI](https://img.shields.io/badge/pypi-v3.2.0-green.svg?style=flat)](https://pypi.python.org/pypi/pwntools/)
+[![PyPI](https://img.shields.io/badge/pypi-v3.5.0-green.svg?style=flat)](https://pypi.python.org/pypi/pwntools/)
 [![Travis](https://travis-ci.org/Gallopsled/pwntools.svg)](https://travis-ci.org/Gallopsled/pwntools)
 [![Coveralls](https://img.shields.io/coveralls/Gallopsled/pwntools/dev.svg)](https://coveralls.io/github/Gallopsled/pwntools?branch=dev)
 [![Twitter](https://img.shields.io/badge/twitter-pwntools-4099FF.svg?style=flat)](https://twitter.com/pwntools)
@@ -38,7 +38,8 @@ Most of the functionality of pwntools is self-contained and Python-only.  You sh
 
 ```sh
 apt-get update
-apt-get install python2.7 python-pip python-dev git libssl-dev
+apt-get install python2.7 python-pip python-dev git libssl-dev libffi-dev build-essential
+pip install --upgrade pip
 pip install --upgrade pwntools
 ```
 

TESTING.md

@@ -5,7 +5,7 @@ Pwntools makes extensive use of unit tests and integration tests to ensure every
 ## Test Suite
 
 
-To run the test suite, it is beste to use Ubuntu 12.04 or 14.04, and run the following commands.  **Be aware** that this will add a user to the machine, and create a public key for SSH login!
+To run the test suite, it is best to use Ubuntu 12.04 or 14.04, and run the following commands.  **Be aware** that this will add a user to the machine, and create a public key for SSH login!
 
 ```sh
 bash travis/install.sh

docs/source/args.rst

@@ -0,0 +1,9 @@
+.. testsetup:: *
+
+   from pwn import *
+
+:mod:`pwnlib.args` --- Magic Command-Line Arguments
+=====================================================
+
+.. automodule:: pwnlib.args
+   :members:

docs/source/asm.rst

@@ -13,3 +13,13 @@
 
 .. automodule:: pwnlib.asm
    :members:
+
+Internal Functions
+=========================================
+
+These are only included so that their tests are run.
+
+You should never need these.
+
+.. autofunction:: pwnlib.asm.dpkg_search_for_binutils
+.. autofunction:: pwnlib.asm.print_binutils_instructions

docs/source/conf.py

@@ -44,6 +44,10 @@ extensions = [
     'sphinxcontrib.napoleon'
 ]
 
+# Napoleon settings
+napoleon_use_ivar = True
+napoleon_use_rtype = False
+
 doctest_global_setup = '''
 import sys, os
 os.environ['PWNLIB_NOTERM'] = '1'
@@ -52,17 +56,29 @@ import pwnlib
 pwnlib.context.context.reset_local()
 pwnlib.context.ContextType.defaults['log_level'] = 'ERROR'
 pwnlib.context.ContextType.defaults['randomize'] = False
+pwnlib.util.fiddling.default_style = {}
 pwnlib.term.text.when = 'never'
 pwnlib.log.install_default_handler()
 pwnlib.log.rootlogger.setLevel(1)
+
+# Sphinx modifies sys.stdout, and context.log_terminal has
+# a reference to the original instance.  We need to update
+# it for logging to be captured.
+class stdout(object):
+    def __getattr__(self, name):
+        return getattr(sys.stdout, name)
+    def __setattr__(self, name, value):
+        return setattr(sys.stdout, name, value)
+pwnlib.context.ContextType.defaults['log_console'] = stdout()
 '''
 
-autodoc_member_order = 'alphabetical'
+autoclass_content = 'both'
+autodoc_member_order = 'groupwise' # 'alphabetical'
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
 
-doctest_test_doctest_blocks = True
+doctest_test_doctest_blocks = 'true'
 
 # The suffix of source filenames.
 source_suffix = '.rst'
@@ -221,7 +237,7 @@ latex_documents = [
 ]
 
 intersphinx_mapping = {'python': ('https://docs.python.org/2.7', None),
-                       'paramiko': ('https://paramiko-docs.readthedocs.org/en/1.15/', None)}
+                       'paramiko': ('https://paramiko-docs.readthedocs.org/en/2.1/', None)}
 
 # The name of an image file (relative to this directory) to place at the top of
 # the title page.
@@ -337,3 +353,9 @@ if build_dash:
         html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
 
     # otherwise, readthedocs.org uses their theme by default, so no need to specify it
+
+
+# -- Customization to Sphinx autodoc generation --------------------------------------------
+import sphinx.ext.autodoc
+
+sphinx.ext.autodoc

docs/source/context.rst

@@ -3,10 +3,32 @@
    from pwn import *
    import logging
    log = pwnlib.log.getLogger('pwnlib.context')
+   context.clear()
 
 :mod:`pwnlib.context` --- Setting runtime variables
 =====================================================
 
+Many settings in ``pwntools`` are controlled via the global variable :data:`.context, such as the selected target operating system, architecture, and bit-width.
+
+In general, exploits will start with something like:
+
+.. code-block:: python
+
+    from pwn import *
+    context.arch = 'amd64'
+
+Which sets up everything in the exploit for exploiting a 64-bit Intel binary.
+
+The recommended method is to use ``context.binary``  to automagically set all of the appropriate values.
+
+.. code-block:: python
+
+    from pwn import *
+    context.binary = './challenge-binary'
+
+Module Members
+=====================================================
+
 .. autodata:: pwnlib.context.context
 
 .. autoclass:: pwnlib.context.ContextType

docs/source/elf.rst

@@ -3,8 +3,16 @@
    from pwnlib.elf import *
    from pwnlib.util.misc import which
 
-:mod:`pwnlib.elf` --- Working with ELF binaries
-===============================================
+:mod:`pwnlib.elf` --- ELF Executables and Libraries
+===================================================
 
 .. automodule:: pwnlib.elf
-   :members:
+
+
+ELF Modules
+-------------------
+
+.. toctree::
+    elf/elf
+    elf/config
+    elf/corefile

docs/source/elf/config.rst

@@ -0,0 +1,10 @@
+.. testsetup:: *
+
+   from pwn import *
+   from pwnlib.elf.config import parse_kconfig
+
+:mod:`pwnlib.elf.config` --- Kernel Config Parsing
+===========================================================
+
+.. automodule:: pwnlib.elf.config
+  :members:

docs/source/elf/corefile.rst

@@ -0,0 +1,15 @@
+.. testsetup:: *
+
+   from pwn import *
+
+:mod:`pwnlib.elf.corefile` --- Core Files
+===========================================================
+
+.. automodule:: pwnlib.elf.corefile
+
+  .. autoclass:: pwnlib.elf.corefile.Corefile
+     :members:
+     :show-inheritance:
+
+  .. autoclass:: pwnlib.elf.corefile.Mapping
+     :members:

docs/source/elf/elf.rst

@@ -0,0 +1,27 @@
+.. testsetup:: *
+
+   from pwn import *
+
+:mod:`pwnlib.elf.elf` --- ELF Files
+===========================================================
+
+.. automodule:: pwnlib.elf.elf
+
+  .. autoclass:: pwnlib.elf.elf.ELF
+     :members:
+     :show-inheritance:
+     :inherited-members:
+     :exclude-members: address_offsets,
+                       get_data,
+                       get_dwarf_info,
+                       get_section,
+                       get_segment,
+                       has_dwarf_info,
+                       iter_sections,
+                       iter_segments
+
+  .. autoclass:: pwnlib.elf.elf.Function
+     :members:
+
+  .. autoclass:: pwnlib.elf.elf.dotdict
+     :members:

docs/source/gdb.rst

@@ -1,3 +1,7 @@
+.. testsetup:: *
+
+    from pwn import *
+
 :mod:`pwnlib.gdb` --- Working with GDB
 ======================================
 

docs/source/globals.rst

@@ -13,11 +13,11 @@ Which imports a bazillion things into the global namespace to make your life eas
 
 This is a quick list of most of the objects and routines imported, in rough order of importance and frequency of use.
 
-- ``context``
+- :obj:`.context`
     - :data:`pwnlib.context.context`
     - Responsible for most of the pwntools convenience settings
     - Set `context.log_level = 'debug'` when troubleshooting your exploit
-    - Scope-aware, so you can disable logging for a subsection of code via ``pwnlib.context.ContextType.local``
+    - Scope-aware, so you can disable logging for a subsection of code via :meth:`.ContextType.local`
 - ``remote``, ``listen``, ``ssh``, ``process``
     - :mod:`pwnlib.tubes`
     - Super convenient wrappers around all of the common functionality for CTF challenges
@@ -57,11 +57,11 @@ This is a quick list of most of the objects and routines imported, in rough orde
 - ``gdb.debug`` and ``gdb.attach``
     - :mod:`pwnlib.gdb`
     - Launch a binary under GDB and pop up a new terminal to interact with it.  Automates setting breakpoints and makes iteration on exploits MUCH faster.
-    - Alternately, attach to a running process given a PID, ``pwnlib.tubes`` object, or even just a socket that's connected to it
+    - Alternately, attach to a running process given a PID, :mod:`pwnlib.tubes` object, or even just a socket that's connected to it
 - ``args``
-    - Dictionary contining all-caps command-line arguments for quick access
+    - Dictionary containing all-caps command-line arguments for quick access
     - Run via ``python foo.py REMOTE=1`` and ``args['REMOTE'] == '1'``.
-    - Can also control logging verbosity and terminal fancyness
+    - Can also control logging verbosity and terminal fanciness
         - `NOTERM`
         - `SILENT`
         - `DEBUG`
@@ -78,7 +78,7 @@ This is a quick list of most of the objects and routines imported, in rough orde
     - It's the new ``getch``
 - ``safeeval``
     - :mod:`pwnlib.util.safeeval`
-    - Functions for safely evalutaing python code without nasty side-effects.
+    - Functions for safely evaluating python code without nasty side-effects.
 
 These are all pretty self explanatory, but are useful to have in the global namespace.
 

docs/source/index.rst

@@ -43,6 +43,7 @@ Each of the ``pwntools`` modules is documented here.
    :glob:
 
    adb
+   args
    asm
    atexception
    atexit
@@ -57,6 +58,7 @@ Each of the ``pwntools`` modules is documented here.
    gdb
    log
    memleak
+   protocols
    replacements
    rop
    rop/*

docs/source/install.rst

@@ -24,7 +24,9 @@ pwntools is available as a ``pip`` package.
 
 .. code-block:: bash
 
-    $ apt-get install python2.7 python2.7-dev python-pip
+    $ apt-get update
+    $ apt-get install python2.7 python-pip python-dev git libssl-dev libffi-dev build-essential
+    $ pip install --upgrade pip
     $ pip install --upgrade pwntools
 
 Development

docs/source/intro.rst

@@ -212,8 +212,8 @@ Stop hard-coding things!  Look them up at runtime with :mod:`pwnlib.elf`.
 You can even patch and save the files.
 
     >>> e = ELF('/bin/cat')
-    >>> e.read(e.address+1, 3)
-    'ELF'
+    >>> e.read(e.address, 4)
+    '\x7fELF'
     >>> e.asm(e.address, 'ret')
     >>> e.save('/tmp/quiet-cat')
     >>> disasm(file('/tmp/quiet-cat','rb').read(1))

docs/source/protocols.rst

@@ -0,0 +1,15 @@
+:mod:`pwnlib.protocols` --- Wire Protocols
+=============================================
+
+.. automodule:: pwnlib.protocols
+
+
+Supported Protocols
+-------------------
+
+.. toctree::
+    :maxdepth: 3
+    :glob:
+
+    protocols/*
+

docs/source/protocols/adb.rst

@@ -0,0 +1,10 @@
+.. testsetup:: *
+
+   from pwn import *
+   from pwnlib.protocols.adb import AdbClient
+
+:mod:`pwnlib.protocols.adb` --- ADB Protocol Implementation
+===========================================================
+
+.. automodule:: pwnlib.protocols.adb
+   :members:

docs/source/shellcraft/aarch64.rst

@@ -0,0 +1,19 @@
+.. testsetup:: *
+
+   from pwn import *
+   context.clear(arch='aarch64')
+
+:mod:`pwnlib.shellcraft.aarch64` --- Shellcode for AArch64
+===========================================================
+
+:mod:`pwnlib.shellcraft.aarch64`
+--------------------------------
+
+.. automodule:: pwnlib.shellcraft.aarch64
+   :members:
+
+:mod:`pwnlib.shellcraft.aarch64.linux`
+--------------------------------------
+
+.. automodule:: pwnlib.shellcraft.aarch64.linux
+   :members:

docs/source/tubes.rst

@@ -27,4 +27,4 @@ Types of Tubes
   .. autoclass:: pwnlib.tubes.tube.tube()
      :members:
      :exclude-members: recv_raw, send_raw, settimeout_raw,
-                       can_recv_raw
+                       can_recv_raw, shutdown_raw, connected_raw,

docs/source/util/fiddling.rst

@@ -1,6 +1,7 @@
 .. testsetup:: *
 
-   from pwnlib.util.fiddling import *
+    from pwn import *
+    from pwnlib.util.fiddling import *
 
 
 :mod:`pwnlib.util.fiddling` --- Utilities bit fiddling

examples/sigreturn_corefile_aarch64.py

@@ -0,0 +1,31 @@
+from pwn import *
+context.arch='aarch64'
+frame = SigreturnFrame()
+
+registers = ['x%i' % i for i in range(0, 31)]
+registers += ['sp', 'pc']
+
+
+for index, register in enumerate(registers):
+    setattr(frame, register, index)
+
+assembly = '\n'.join([
+    shellcraft.read(constants.STDIN_FILENO, 'sp', 1024),
+    shellcraft.syscall(constants.SYS_rt_sigreturn)
+])
+
+binary = ELF.from_assembly(assembly)
+
+io = binary.process()
+io.flat(frame)
+io.wait()
+assert io.poll() == -11
+
+corefile = io.corefile
+
+for index, register in enumerate(registers):
+    value = getattr(corefile, register)
+    if index != value:
+        log.error("%s != %i (%i)" % (register, index, value))
+    else:
+        log.success("%s == %i" % (register, value))

examples/sigreturn_corefile_amd64.py

@@ -0,0 +1,32 @@
+from pwn import *
+context.arch='amd64'
+frame = SigreturnFrame()
+
+registers = ['rax', 'rbx', 'rcx', 'rdx',
+             'rdi', 'rsi', 'rbp', 'rsp',
+             'rip',
+             'r8', 'r9', 'r10', 'r11', 'r12', 'r13', 'r14', 'r15']
+
+for index, register in enumerate(registers):
+    setattr(frame, register, index)
+
+assembly = '\n'.join([
+    shellcraft.read(constants.STDIN_FILENO, 'rsp', 1024),
+    shellcraft.sigreturn()
+])
+
+binary = ELF.from_assembly(assembly)
+
+io = binary.process()
+io.flat(frame)
+io.wait()
+assert io.poll() == -11
+
+corefile = io.corefile
+
+for index, register in enumerate(registers):
+    value = getattr(corefile, register)
+    if index != value:
+        log.error("%s != %i (%i)" % (register, index, value))
+    else:
+        log.success("%s == %i" % (register, value))

examples/sigreturn_corefile_arm.py

@@ -0,0 +1,32 @@
+from pwn import *
+
+context.arch='arm'
+frame = SigreturnFrame()
+
+registers = ['r0', 'r1', 'r2', 'r3', 'r4', 'r5', 'r6', 'r7',
+             'r8', 'r9', 'r10', 'fp', 'ip', 'sp', 'pc', 'lr']
+
+
+for index, register in enumerate(registers):
+    setattr(frame, register, index)
+
+assembly = '\n'.join([
+    shellcraft.read(constants.STDIN_FILENO, 'sp', 1024),
+    shellcraft.sigreturn()
+])
+
+binary = ELF.from_assembly(assembly)
+
+io = binary.process()
+io.flat(frame)
+io.wait()
+assert io.poll() == -11
+
+corefile = io.corefile
+
+for index, register in enumerate(registers):
+    value = getattr(corefile, register)
+    if index != value:
+        log.error("%s != %i (%i)" % (register, index, value))
+    else:
+        log.success("%s == %i" % (register, value))

examples/sigreturn_corefile_i386.py

@@ -0,0 +1,31 @@
+from pwn import *
+context.kernel='amd64'
+frame = SigreturnFrame()
+
+registers = ['eax', 'ebx', 'ecx', 'edx',
+             'edi', 'esi', 'ebp', 'esp',
+             'eip']
+
+for index, register in enumerate(registers):
+    setattr(frame, register, index)
+
+assembly = '\n'.join([
+    shellcraft.read(constants.STDIN_FILENO, 'esp', 1024),
+    shellcraft.sigreturn()
+])
+
+binary = ELF.from_assembly(assembly)
+
+io = binary.process()
+io.flat(frame)
+io.wait()
+assert io.poll() == -11
+
+corefile = io.corefile
+
+for index, register in enumerate(registers):
+    value = getattr(corefile, register)
+    if index != value:
+        log.error("%s != %i (%i)" % (register, index, value))
+    else:
+        log.success("%s == %i" % (register, value))

extra/docker/Dockerfile

@@ -1,58 +0,0 @@
-############################################################
-# Dockerfile to build Pwntools container
-# Based on Ubuntu
-############################################################
-
-FROM ubuntu:trusty
-MAINTAINER Maintainer Gallopsled et al.
-
-RUN apt-get update && apt-get install -y software-properties-common
-RUN apt-add-repository --yes ppa:pwntools/binutils
-RUN apt-get update
-RUN apt-get install -y \
-	binutils \
-	binutils-aarch64-linux-gnu \
-	binutils-arm-linux-gnueabihf \
-	curl \
-	gdb \
-	gdbserver \
-	gdb-multiarch \
-	git \
-	libc6-i386 \
-	libc6-arm64-cross \
-	libc6-armhf-cross \
-	libc6-dev-i386 \
-	libffi-dev \
-	libglib2.0-dev \
-	libncurses5-dev \
-	libncursesw5-dev \
-	ltrace \
-	python-dev \
-	python-pip \
-	python3-pip \
-	python2.7 \
-	qemu \
-	qemu-user-static \
-	strace \
-	tmux \
-	wget
-RUN pip install --upgrade ipython
-RUN useradd -m zerocool
-RUN usermod -U zerocool
-
-RUN mkdir -p /etc/qemu-binfmt
-RUN ln -s /usr/aarch64-linux-gnu /etc/qemu-binfmt/aarch64
-RUN ln -s /usr/arm-linux-gnueabihf /etc/qemu-binfmt/arm
-
-RUN pip install --upgrade pip pycparser psutil python-ptrace
-RUN pip3 install --upgrade pip pycparser psutil python-ptrace
-
-RUN pip install --upgrade git+https://github.com/Gallopsled/pwntools.git
-
-RUN locale-gen en_US.UTF-8
-ENV LANG en_US.UTF-8
-ENV LANGUAGE en_US:en
-ENV LC_ALL en_US.UTF-8
-
-RUN chown -R zerocool.zerocool /home/zerocool
-RUN chsh -s /bin/bash zerocool

extra/docker/Makefile

@@ -0,0 +1,13 @@
+subdirs=xenial stable $(wildcard */)
+makefiles=$(wildcard */Makefile)
+tags=stable beta dev
+
+all: $(makefiles) $(subdirs)
+
+$(subdirs):
+	docker build -t pwntools/pwntools:$(shell basename $(@)) "$@"
+
+$(makefiles):
+	make -C $(shell dirname $@)
+
+.PHONY: all $(subdirs) $(makefiles)

extra/docker/beta/Dockerfile

@@ -0,0 +1,6 @@
+FROM pwntools/pwntools:stable
+
+USER root
+RUN pip install --upgrade git+https://github.com/Gallopsled/pwntools@beta
+RUN PWNLIB_NOTERM=1 pwn update
+USER pwntools

extra/docker/dev/Dockerfile

@@ -0,0 +1,6 @@
+FROM pwntools/pwntools:stable
+
+USER root
+RUN pip install --upgrade git+https://github.com/Gallopsled/pwntools@dev
+RUN PWNLIB_NOTERM=1 pwn update
+USER pwntools

extra/docker/kali/Dockerfile

@@ -0,0 +1,26 @@
+############################################################
+# Dockerfile to build Pwntools container
+# Based on Ubuntu
+############################################################
+
+FROM kalilinux/kali-linux-docker
+MAINTAINER Maintainer Gallopsled et al.
+
+RUN apt-get update
+RUN apt-get install -y build-essential
+RUN apt-get install -y git
+RUN apt-get install -y libssl-dev
+RUN apt-get install -y libffi-dev
+RUN apt-get install -y python2.7
+RUN apt-get install -y python-pip
+RUN apt-get install -y python-dev
+RUN pip install --upgrade pip
+RUN python -m pip install --upgrade pwntools
+RUN PWNLIB_NOTERM=1 pwn update
+
+RUN apt-get install -y sudo
+RUN useradd -m pwntools
+RUN passwd --delete --unlock pwntools
+RUN echo "pwntools ALL=(ALL:ALL) NOPASSWD: ALL" > /etc/sudoers.d/pwntools
+USER pwntools
+WORKDIR /home/pwntools

extra/docker/kali/Makefile

@@ -0,0 +1,8 @@
+all: Dockerfile
+
+Dockerfile: ../stable/Dockerfile FORCE
+	sed "s|ubuntu:xenial|kalilinux/kali-linux-docker|" < "$<" > "$@"
+
+FORCE:
+
+.PHONY: all

extra/docker/precise/Dockerfile

@@ -0,0 +1,26 @@
+############################################################
+# Dockerfile to build Pwntools container
+# Based on Ubuntu
+############################################################
+
+FROM ubuntu:precise
+MAINTAINER Maintainer Gallopsled et al.
+
+RUN apt-get update
+RUN apt-get install -y build-essential
+RUN apt-get install -y git
+RUN apt-get install -y libssl-dev
+RUN apt-get install -y libffi-dev
+RUN apt-get install -y python2.7
+RUN apt-get install -y python-pip
+RUN apt-get install -y python-dev
+RUN pip install --upgrade pip
+RUN python -m pip install --upgrade pwntools
+RUN PWNLIB_NOTERM=1 pwn update
+
+RUN apt-get install -y sudo
+RUN useradd -m pwntools
+RUN passwd --delete --unlock pwntools
+RUN echo "pwntools ALL=(ALL:ALL) NOPASSWD: ALL" > /etc/sudoers.d/pwntools
+USER pwntools
+WORKDIR /home/pwntools

extra/docker/precise/Makefile

@@ -0,0 +1,8 @@
+all: Dockerfile
+
+Dockerfile: ../xenial/Dockerfile FORCE
+	sed "s|ubuntu:xenial|ubuntu:precise|" < "$<" > "$@"
+
+FORCE:
+
+.PHONY: all

extra/docker/stable/Dockerfile

@@ -0,0 +1,26 @@
+############################################################
+# Dockerfile to build Pwntools container
+# Based on Ubuntu
+############################################################
+
+FROM ubuntu:xenial
+MAINTAINER Maintainer Gallopsled et al.
+
+RUN apt-get update
+RUN apt-get install -y build-essential
+RUN apt-get install -y git
+RUN apt-get install -y libssl-dev
+RUN apt-get install -y libffi-dev
+RUN apt-get install -y python2.7
+RUN apt-get install -y python-pip
+RUN apt-get install -y python-dev
+RUN pip install --upgrade pip
+RUN python -m pip install --upgrade pwntools
+RUN PWNLIB_NOTERM=1 pwn update
+
+RUN apt-get install -y sudo
+RUN useradd -m pwntools
+RUN passwd --delete --unlock pwntools
+RUN echo "pwntools ALL=(ALL:ALL) NOPASSWD: ALL" > /etc/sudoers.d/pwntools
+USER pwntools
+WORKDIR /home/pwntools

extra/docker/stable/Makefile

@@ -0,0 +1,8 @@
+all: Dockerfile
+
+Dockerfile: ../xenial/Dockerfile FORCE
+	cp "$<" "$@"
+
+FORCE:
+
+.PHONY: all

extra/docker/trusty/Dockerfile

@@ -0,0 +1,26 @@
+############################################################
+# Dockerfile to build Pwntools container
+# Based on Ubuntu
+############################################################
+
+FROM ubuntu:trusty
+MAINTAINER Maintainer Gallopsled et al.
+
+RUN apt-get update
+RUN apt-get install -y build-essential
+RUN apt-get install -y git
+RUN apt-get install -y libssl-dev
+RUN apt-get install -y libffi-dev
+RUN apt-get install -y python2.7
+RUN apt-get install -y python-pip
+RUN apt-get install -y python-dev
+RUN pip install --upgrade pip
+RUN python -m pip install --upgrade pwntools
+RUN PWNLIB_NOTERM=1 pwn update
+
+RUN apt-get install -y sudo
+RUN useradd -m pwntools
+RUN passwd --delete --unlock pwntools
+RUN echo "pwntools ALL=(ALL:ALL) NOPASSWD: ALL" > /etc/sudoers.d/pwntools
+USER pwntools
+WORKDIR /home/pwntools

extra/docker/trusty/Makefile

@@ -0,0 +1,8 @@
+all: Dockerfile
+
+Dockerfile: ../xenial/Dockerfile FORCE
+	sed "s|ubuntu:xenial|ubuntu:trusty|" < "$<" > "$@"
+
+FORCE:
+
+.PHONY: all

extra/docker/xenial/Dockerfile

@@ -0,0 +1,26 @@
+############################################################
+# Dockerfile to build Pwntools container
+# Based on Ubuntu
+############################################################
+
+FROM ubuntu:xenial
+MAINTAINER Maintainer Gallopsled et al.
+
+RUN apt-get update
+RUN apt-get install -y build-essential
+RUN apt-get install -y git
+RUN apt-get install -y libssl-dev
+RUN apt-get install -y libffi-dev
+RUN apt-get install -y python2.7
+RUN apt-get install -y python-pip
+RUN apt-get install -y python-dev
+RUN pip install --upgrade pip
+RUN python -m pip install --upgrade pwntools
+RUN PWNLIB_NOTERM=1 pwn update
+
+RUN apt-get install -y sudo
+RUN useradd -m pwntools
+RUN passwd --delete --unlock pwntools
+RUN echo "pwntools ALL=(ALL:ALL) NOPASSWD: ALL" > /etc/sudoers.d/pwntools
+USER pwntools
+WORKDIR /home/pwntools

pwn/__init__.py

@@ -1,5 +1,7 @@
 # Promote useful stuff to toplevel
-from .toplevel import *
+from __future__ import absolute_import
+
+from pwn.toplevel import *
 
 pwnlib.args.initialize()
 pwnlib.log.install_default_handler()
@@ -13,4 +15,5 @@ if not platform.architecture()[0].startswith('64'):
     See Gallopsled/pwntools#518 for more information."""
     log.warn_once('Pwntools does not support 32-bit Python.  Use a 64-bit release.')
 
-pwnlib.update.check_automatically()
+with context.local(log_console=sys.stderr):
+    pwnlib.update.check_automatically()

pwn/toplevel.py

@@ -24,15 +24,15 @@ from pwnlib.context import Thread
 from pwnlib.context import context
 from pwnlib.dynelf import DynELF
 from pwnlib.encoders import *
-from pwnlib.elf import Core
-from pwnlib.elf import ELF
-from pwnlib.elf import load
+from pwnlib.elf.corefile import Core, Corefile, Coredump
+from pwnlib.elf.elf import ELF, load
 from pwnlib.encoders import *
 from pwnlib.exception import PwnlibException
+from pwnlib.gdb import attach, debug, debug_assembly, debug_shellcode
 from pwnlib.flag import *
 from pwnlib.fmtstr import FmtStr, fmtstr_payload
 from pwnlib.log import getLogger
-from pwnlib.memleak import MemLeak
+from pwnlib.memleak import MemLeak, RelativeMemLeak
 from pwnlib.regsort import *
 from pwnlib.replacements import *
 from pwnlib.rop import ROP
@@ -40,7 +40,7 @@ from pwnlib.rop.srop import SigreturnFrame
 from pwnlib.runner import *
 from pwnlib.timeout import Timeout
 from pwnlib.tubes.listen import listen
-from pwnlib.tubes.process import process
+from pwnlib.tubes.process import process, PTY, PIPE, STDOUT
 from pwnlib.tubes.remote import remote, tcp, udp, connect
 from pwnlib.tubes.serialtube import serialtube
 from pwnlib.tubes.ssh import ssh
@@ -51,6 +51,7 @@ from pwnlib.util import iters
 from pwnlib.util import net
 from pwnlib.util import proc
 from pwnlib.util import safeeval
+from pwnlib.util.crc import BitPolynom
 from pwnlib.util.cyclic import *
 from pwnlib.util.fiddling import *
 from pwnlib.util.getdents import *

pwnlib/__init__.py

@@ -1,6 +1,8 @@
+from __future__ import absolute_import
+
 import importlib
 
-from .version import __version__
+from pwnlib.version import __version__
 
 version = __version__
 

pwnlib/abi.py

@@ -1,6 +1,8 @@
 # -*- coding: utf-8 -*-
-from .context import LocalContext
-from .context import context
+from __future__ import absolute_import
+
+from pwnlib.context import LocalContext
+from pwnlib.context import context
 
 
 class ABI(object):
@@ -24,7 +26,8 @@ class ABI(object):
     #: Indicates that this ABI returns to the next address on the slot
     returns            = True
 
-    def __init__(self, regs, align, minimum):
+    def __init__(self, stack, regs, align, minimum):
+        self.stack              = stack
         self.register_arguments = regs
         self.arg_alignment      = align
         self.stack_minimum      = minimum
@@ -32,35 +35,52 @@ class ABI(object):
     @staticmethod
     @LocalContext
     def default():
+        if context.os == 'android':
+            context.os = 'linux'
+
         return {
         (32, 'i386', 'linux'):  linux_i386,
+        (64, 'aarch64', 'linux'): linux_aarch64,
         (64, 'amd64', 'linux'): linux_amd64,
         (32, 'arm', 'linux'):   linux_arm,
         (32, 'thumb', 'linux'):   linux_arm,
         (32, 'mips', 'linux'):   linux_mips,
         (32, 'i386', 'windows'):  windows_i386,
         (64, 'amd64', 'windows'): windows_amd64,
+        (32, 'powerpc', 'linux'): linux_ppc,
+        (64, 'powerpc', 'linux'): linux_ppc64,
         }[(context.bits, context.arch, context.os)]
 
     @staticmethod
     @LocalContext
     def syscall():
+        if context.os == 'android':
+            context.os = 'linux'
+
         return {
         (32, 'i386', 'linux'):  linux_i386_syscall,
         (64, 'amd64', 'linux'): linux_amd64_syscall,
+        (64, 'aarch64', 'linux'): linux_aarch64_syscall,
         (32, 'arm', 'linux'):   linux_arm_syscall,
         (32, 'thumb', 'linux'):   linux_arm_syscall,
         (32, 'mips', 'linux'):   linux_mips_syscall,
+        (64, 'aarch64', 'linux'):   linux_aarch64_syscall,
+        (32, 'powerpc', 'linux'): linux_ppc_syscall,
+        (64, 'powerpc', 'linux'): linux_ppc64_syscall,
         }[(context.bits, context.arch, context.os)]
 
     @staticmethod
     @LocalContext
     def sigreturn():
+        if context.os == 'android':
+            context.os = 'linux'
+
         return {
         (32, 'i386', 'linux'):  linux_i386_sigreturn,
         (64, 'amd64', 'linux'): linux_amd64_sigreturn,
         (32, 'arm', 'linux'):   linux_arm_sigreturn,
         (32, 'thumb', 'linux'):   linux_arm_sigreturn,
+        (64, 'aarch64', 'linux'):   linux_aarch64_sigreturn,
         }[(context.bits, context.arch, context.os)]
 
 class SyscallABI(ABI):
@@ -68,9 +88,9 @@ class SyscallABI(ABI):
     The syscall ABI treats the syscall number as the zeroth argument,
     which must be loaded into the specified register.
     """
-    def __init__(self, register_arguments, *a, **kw):
-        super(SyscallABI, self).__init__(register_arguments, *a, **kw)
-        self.syscall_register = register_arguments[0]
+    def __init__(self, *a, **kw):
+        super(SyscallABI, self).__init__(*a, **kw)
+        self.syscall_register = self.register_arguments[0]
 
 class SigreturnABI(SyscallABI):
     """
@@ -81,26 +101,31 @@ class SigreturnABI(SyscallABI):
     returns = False
 
 
-linux_i386   = ABI([], 4, 0)
-linux_amd64  = ABI(['rdi','rsi','rdx','rcx','r8','r9'], 8, 0)
-linux_arm    = ABI(['r0', 'r1', 'r2', 'r3'], 8, 0)
-linux_aarch64 = ABI(['x0', 'x1', 'x2', 'x3'], 16, 0)
-linux_mips  = ABI(['$a0','$a1','$a2','$a3'], 4, 0)
+linux_i386   = ABI('esp', [], 4, 0)
+linux_amd64  = ABI('rsp', ['rdi','rsi','rdx','rcx','r8','r9'], 8, 0)
+linux_arm    = ABI('sp', ['r0', 'r1', 'r2', 'r3'], 8, 0)
+linux_aarch64 = ABI('sp', ['x0', 'x1', 'x2', 'x3'], 16, 0)
+linux_mips  = ABI('$sp', ['$a0','$a1','$a2','$a3'], 4, 0)
+linux_ppc = ABI('sp', ['r3', 'r4', 'r5', 'r6', 'r7', 'r8', 'r9', 'r10'], 4, 0)
+linux_ppc64 = ABI('sp', ['r3', 'r4', 'r5', 'r6', 'r7', 'r8', 'r9', 'r10'], 8, 0)
 
-linux_i386_syscall = SyscallABI(['eax', 'ebx', 'ecx', 'edx', 'esi', 'edi', 'ebp'], 4, 0)
-linux_amd64_syscall = SyscallABI(['rax','rdi', 'rsi', 'rdx', 'r10', 'r8', 'r9'],   8, 0)
-linux_arm_syscall   = SyscallABI(['r7', 'r0', 'r1', 'r2', 'r3', 'r4', 'r5', 'r6'], 4, 0)
-linux_aarch64_syscall   = SyscallABI(['x8', 'x0', 'x1', 'x2', 'x3', 'x4', 'x5', 'x6'], 16, 0)
-linux_mips_syscall  = ABI(['$v0', '$a0','$a1','$a2','$a3'], 4, 0)
+linux_i386_syscall = SyscallABI('esp', ['eax', 'ebx', 'ecx', 'edx', 'esi', 'edi', 'ebp'], 4, 0)
+linux_amd64_syscall = SyscallABI('rsp', ['rax', 'rdi', 'rsi', 'rdx', 'r10', 'r8', 'r9'],   8, 0)
+linux_arm_syscall   = SyscallABI('sp', ['r7', 'r0', 'r1', 'r2', 'r3', 'r4', 'r5', 'r6'], 4, 0)
+linux_aarch64_syscall   = SyscallABI('sp', ['x8', 'x0', 'x1', 'x2', 'x3', 'x4', 'x5', 'x6'], 16, 0)
+linux_mips_syscall  = SyscallABI('$sp', ['$v0','$a0','$a1','$a2','$a3'], 4, 0)
+linux_ppc_syscall = ABI('sp', ['r0', 'r3', 'r4', 'r5', 'r6', 'r7', 'r8', 'r9'], 4, 0)
+linux_ppc64_syscall = ABI('sp', ['r0', 'r3', 'r4', 'r5', 'r6', 'r7', 'r8', 'r9'], 8, 0)
 
-linux_i386_sigreturn = SigreturnABI(['eax'], 4, 0)
-linux_amd64_sigreturn = SigreturnABI(['rax'], 4, 0)
-linux_arm_sigreturn = SigreturnABI(['r7'], 4, 0)
+linux_i386_sigreturn = SigreturnABI('esp', ['eax'], 4, 0)
+linux_amd64_sigreturn = SigreturnABI('rsp', ['rax'], 4, 0)
+linux_arm_sigreturn = SigreturnABI('sp', ['r7'], 4, 0)
+linux_aarch64_sigreturn = SigreturnABI('sp', ['x8'], 16, 0)
 
-windows_i386  = ABI([], 4, 0)
-windows_amd64 = ABI(['rcx','rdx','r8','r9'], 32, 32)
+windows_i386  = ABI('esp', [], 4, 0)
+windows_amd64 = ABI('rsp', ['rcx','rdx','r8','r9'], 32, 32)
 
 # Fake ABIs used by SROP
-linux_i386_srop = ABI(['eax'], 4, 0)
-linux_amd64_srop = ABI(['rax'], 4, 0)
-linux_arm_srop = ABI(['r7'], 4, 0)
+linux_i386_srop = ABI('esp', ['eax'], 4, 0)
+linux_amd64_srop = ABI('rsp', ['rax'], 4, 0)
+linux_arm_srop = ABI('sp', ['r7'], 4, 0)

pwnlib/adb/__init__.py

@@ -1,2 +1,4 @@
-from .adb import *
-from .protocol import Client
+from __future__ import absolute_import
+
+from pwnlib.adb.adb import *
+from pwnlib.adb.protocol import Client

pwnlib/adb/adb.py

@@ -1,6 +1,7 @@
 """Provides utilities for interacting with Android devices via the Android Debug Bridge.
 
-# Using Android Devices with Pwntools
+Using Android Devices with Pwntools
+-----------------------------------
 
 Pwntools tries to be as easy as possible to use with Android devices.
 
@@ -10,14 +11,42 @@ If you have multiple devices, you have a handful of options to select one, or it
 over the devices.
 
 First and most important is the ``context.device`` property, which declares the "currently"
-seelcted device in any scope.  It can be set manually to a serial number, or to a ``Device``
+selected device in any scope.  It can be set manually to a serial number, or to a ``Device``
 instance.
 
 .. code-block:: python
 
-    >>> context.device = ''
+    # Take the first available device
+    context.device = adb.wait_for_device()
+
+    # Set a device by serial number
+    context.device = 'ZX1G22LH8S'
+
+    # Set a device by its product name
+    for device in adb.devices():
+        if device.product == 'shamu':
+            break
+    else:
+        error("Could not find any shamus!")
+
+Once a device is selected, you can operate on it with any of the functions in
+the :mod:`pwnlib.adb` module.
+
+.. code-block:: python
+
+    # Get a process listing
+    print adb.process(['ps']).recvall()
+
+    # Fetch properties
+    print adb.properties.ro.build.fingerprint
+
+    # Read and write files
+    print adb.read('/proc/version')
+    adb.write('/data/local/tmp/foo', 'my data')
 
 """
+from __future__ import absolute_import
+
 import functools
 import glob
 import logging
@@ -31,14 +60,14 @@ import time
 
 import dateutil.parser
 
-from .. import atexit
-from .. import tubes
-from ..context import LocalContext
-from ..context import context
-from ..device import Device
-from ..log import getLogger
-from ..util import misc
-from .protocol import Client
+from pwnlib import atexit
+from pwnlib import tubes
+from pwnlib.context import LocalContext
+from pwnlib.context import context
+from pwnlib.device import Device
+from pwnlib.log import getLogger
+from pwnlib.protocols.adb import AdbClient
+from pwnlib.util import misc
 
 log = getLogger(__name__)
 
@@ -59,10 +88,10 @@ def adb(argv, *a, **kw):
 
     return tubes.process.process(context.adb + argv, *a, **kw).recvall()
 
-@context.quiet
+@context.quietfunc
 def devices(serial=None):
     """Returns a list of ``Device`` objects corresponding to the connected devices."""
-    with Client() as c:
+    with AdbClient() as c:
         lines = c.devices(long=True)
     result = []
 
@@ -117,7 +146,7 @@ def root():
     log.info("Enabling root on %s" % context.device)
 
     with context.quiet:
-        with Client() as c:
+        with AdbClient() as c:
             reply = c.root()
 
     if 'already running as root' in reply:
@@ -146,7 +175,7 @@ def reboot(wait=True):
     """
     log.info('Rebooting device %s' % context.device)
 
-    with Client() as c:
+    with AdbClient() as c:
         c.reboot()
 
     if wait:
@@ -159,11 +188,49 @@ def reboot_bootloader():
     """
     log.info('Rebooting %s to bootloader' % context.device)
 
-    with Client() as c:
+    with AdbClient() as c:
         c.reboot_bootloader()
 
+@with_device
+def uptime():
+    """uptime() -> float
+
+    Returns:
+        Uptime of the device, in seconds
+    """
+    up, idle = map(float, read('/proc/uptime').split())
+    return up
+
+@with_device
+def boot_time():
+    """boot_time() -> int
+
+    Returns:
+        Boot time of the device, in Unix time, rounded to the
+        nearest second.
+    """
+    for line in read('/proc/stat').splitlines():
+        name, value = line.split(None, 1)
+        if name == 'btime':
+            return int(value)
+
 class AdbDevice(Device):
-    """Encapsulates information about a connected device."""
+    """Encapsulates information about a connected device.
+
+    Example:
+
+        >>> device = adb.wait_for_device()
+        >>> device.arch
+        'arm'
+        >>> device.bits
+        32
+        >>> device.os
+        'android'
+        >>> device.product
+        'sdk_phone_armv7'
+        >>> device.serial
+        'emulator-5554'
+    """
     def __init__(self, serial, type, port=None, product='unknown', model='unknown', device='unknown', features=None, **kw):
         self.serial  = serial
         self.type    = type
@@ -176,13 +243,45 @@ class AdbDevice(Device):
         if product == 'unknown':
             return
 
-        with context.local(device=serial):
+        # Deferred fields
+        self._initialized = False
+        self._arch = None
+        self._bits = None
+        self._endian = None
+        self._avd = None
+
+    @property
+    def arch(self):
+        self.__do_deferred_initialization()
+        return self._arch
+
+    @property
+    def avd(self):
+        self.__do_deferred_initialization()
+        return self._avd
+
+    @property
+    def bits(self):
+        self.__do_deferred_initialization()
+        return self._bits
+
+    @property
+    def endian(self):
+        self.__do_deferred_initialization()
+        return self._endian
+
+
+    def __do_deferred_initialization(self):
+        if self._initialized:
+            return
+
+        with context.local(device=self.serial):
             abi = str(properties.ro.product.cpu.abi)
             context.clear()
             context.arch = str(abi)
-            self.arch = context.arch
-            self.bits = context.bits
-            self.endian = context.endian
+            self._arch = context.arch
+            self._bits = context.bits
+            self._endian = context.endian
 
         if self.port == 'emulator':
             emulator, port = self.serial.split('-')
@@ -195,7 +294,8 @@ class AdbDevice(Device):
                     self.avd = r.recvline().strip()
             except:
                 pass
-            # r = remote('localhost')
+
+        self._initialized = True
 
     def __str__(self):
         return self.serial
@@ -279,7 +379,7 @@ def wait_for_device(kick=False):
         >>> device = adb.wait_for_device()
     """
     with log.waitfor("Waiting for device to come online") as w:
-        with Client() as c:
+        with AdbClient() as c:
             if kick:
                 try:
                     c.reconnect()
@@ -290,13 +390,15 @@ def wait_for_device(kick=False):
             if context.device:
                 serial = str(context.device)
 
-        with Client() as c:
+        with AdbClient() as c:
             c.wait_for_device(serial)
 
         for device in devices():
             if context.device == device:
                 return device
-            break
+
+            if not serial:
+                break
         else:
             log.error("Could not find any devices")
 
@@ -316,7 +418,7 @@ def disable_verity():
     with log.waitfor("Disabling dm-verity on %s" % context.device) as w:
         root()
 
-        with Client() as c:
+        with AdbClient() as c:
             reply = c.disable_verity()
 
         if 'Verity already disabled' in reply:
@@ -335,7 +437,7 @@ def remount():
         disable_verity()
         root()
 
-        with Client() as c:
+        with AdbClient() as c:
             reply = c.remount()
 
         if 'remount succeeded' not in reply:
@@ -346,7 +448,7 @@ def unroot():
     """Restarts adbd as AID_SHELL."""
     log.info("Unrooting %s" % context.device)
     with context.quiet:
-        with Client() as c:
+        with AdbClient() as c:
             reply  = c.unroot()
 
     if '0006closed' == reply:
@@ -409,10 +511,14 @@ def push(local_path, remote_path):
         local_path(str): Path to the local file to push.
         remote_path(str): Path or directory to store the file on the device.
 
+    Returns:
+        Remote path of the file.
+
     Example:
 
         >>> write('./filename', 'contents')
-        >>> _=adb.push('./filename', '/data/local/tmp')
+        >>> adb.push('./filename', '/data/local/tmp')
+        '/data/local/tmp/filename'
         >>> adb.read('/data/local/tmp/filename')
         'contents'
         >>> adb.push('./filename', '/does/not/exist')
@@ -427,7 +533,7 @@ def push(local_path, remote_path):
         msg += ' (%s)' % context.device
 
     with log.waitfor(msg) as w:
-        with Client() as c:
+        with AdbClient() as c:
 
             # We need to discover whether remote_path is a directory or not.
             # If we cannot stat the full path, assume it's a path-plus-filename,
@@ -447,10 +553,13 @@ def push(local_path, remote_path):
             if stat.S_ISDIR(mode):
                 remote_path = os.path.join(remote_path, remote_filename)
 
-            return c.write(remote_path,
-                           misc.read(local_path),
-                           callback=_create_adb_push_pull_callback(w))
-@context.quiet
+            c.write(remote_path,
+                    misc.read(local_path),
+                    callback=_create_adb_push_pull_callback(w))
+
+    return remote_path
+
+@context.quietfunc
 @with_device
 def read(path, target=None, callback=None):
     """Download a file from the device, and extract its contents.
@@ -460,7 +569,7 @@ def read(path, target=None, callback=None):
         target(str): Optional, location to store the file.
             Uses a temporary file by default.
         callback(callable): See the documentation for
-            ``adb.protocol.Client.read``.
+            ``adb.protocol.AdbClient.read``.
 
     Examples:
 
@@ -471,7 +580,7 @@ def read(path, target=None, callback=None):
         ...
         PwnlibException: Could not stat '/does/not/exist'
     """
-    with Client() as c:
+    with AdbClient() as c:
         stat = c.stat(path)
         if not stat:
             log.error('Could not stat %r' % path)
@@ -482,7 +591,7 @@ def read(path, target=None, callback=None):
 
     return data
 
-@context.quiet
+@context.quietfunc
 @with_device
 def write(path, data=''):
     """Create a file on the device with the provided contents.
@@ -500,14 +609,158 @@ def write(path, data=''):
         misc.write(temp.name, data)
         push(temp.name, path)
 
+@context.quietfunc
+@with_device
+def mkdir(path):
+    """Create a directory on the target device.
+
+    Note:
+        Silently succeeds if the directory already exists.
+
+    Arguments:
+        path(str): Directory to create.
+
+    Examples:
+
+        >>> adb.mkdir('/')
+
+        >>> path = '/data/local/tmp/mkdir_test'
+        >>> adb.exists(path)
+        False
+        >>> adb.mkdir(path)
+        >>> adb.exists(path)
+        True
+
+        >>> adb.mkdir('/init')
+        Traceback (most recent call last):
+        ...
+        PwnlibException: mkdir failed for /init, File exists
+    """
+    if not path.startswith('/'):
+        log.error("Must provide an absolute path: %r" % path)
+
+    with AdbClient() as c:
+        st = c.stat(path)
+
+        # Don't re-create existing directories
+        if st and stat.S_ISDIR(st['mode']):
+            return
+
+        result = process(['mkdir', path]).recvall()
+
+        # Any output at all is an error
+        if result:
+            log.error(result)
+
+@context.quietfunc
+@with_device
+def makedirs(path):
+    """Create a directory and all parent directories on the target device.
+
+    Note:
+        Silently succeeds if the directory already exists.
+
+    Examples:
+
+        >>> adb.makedirs('/data/local/tmp/this/is/a/directory/heirarchy')
+        >>> adb.listdir('/data/local/tmp/this/is/a/directory')
+        ['heirarchy']
+    """
+    if path != '/':
+        makedirs(os.path.dirname(path))
+
+    mkdir(path)
+
+@context.quietfunc
+@with_device
+def exists(path):
+    """Return :const:`True` if ``path`` exists on the target device.
+
+    Examples:
+
+        >>> adb.exists('/')
+        True
+        >>> adb.exists('/init')
+        True
+        >>> adb.exists('/does/not/exist')
+        False
+    """
+    with AdbClient() as c:
+        return bool(c.stat(path))
+
+@context.quietfunc
+@with_device
+def isdir(path):
+    """Return :const:`True` if ``path`` is a on the target device.
+
+    Examples:
+
+        >>> adb.isdir('/')
+        True
+        >>> adb.isdir('/init')
+        False
+        >>> adb.isdir('/does/not/exist')
+        False
+    """
+    with AdbClient() as c:
+        st = c.stat(path)
+        return bool(st and stat.S_ISDIR(st['mode']))
+
+@context.quietfunc
+@with_device
+def unlink(path, recursive=False):
+    """Unlinks a file or directory on the target device.
+
+    Examples:
+
+        >>> adb.unlink("/does/not/exist")
+        Traceback (most recent call last):
+        ...
+        PwnlibException: Could not unlink '/does/not/exist': Does not exist
+
+        >>> filename = '/data/local/tmp/unlink-test'
+        >>> adb.write(filename, 'hello')
+        >>> adb.exists(filename)
+        True
+        >>> adb.unlink(filename)
+        >>> adb.exists(filename)
+        False
+
+        >>> adb.mkdir(filename)
+        >>> adb.write(filename + '/contents', 'hello')
+        >>> adb.unlink(filename)
+        Traceback (most recent call last):
+        ...
+        PwnlibException: Cannot delete non-empty directory '/data/local/tmp/unlink-test' without recursive=True
+
+        >>> adb.unlink(filename, recursive=True)
+        >>> adb.exists(filename)
+        False
+    """
+    with AdbClient() as c:
+        st = c.stat(path)
+        if not st:
+            log.error("Could not unlink %r: Does not exist" % path)
+
+        # If the directory is not empty, do not delete it
+        if isdir(path) and c.list(path) and not recursive:
+            log.error("Cannot delete non-empty directory %r without recursive=True" % path)
+
+        flags = '-rf' if recursive else '-r'
+
+        output = c.execute(['rm', flags, path]).recvall()
+
+        if output:
+            log.error(output)
+
 @with_device
 def process(argv, *a, **kw):
     """Execute a process on the device.
 
-    See ``pwnlib.tubes.process.process`` documentation for more info.
+    See :class:`pwnlib.tubes.process.process` documentation for more info.
 
     Returns:
-        A ``process`` tube.
+        A :class:`pwnlib.tubes.process.process` tube.
 
     Examples:
 
@@ -524,7 +777,7 @@ def process(argv, *a, **kw):
         if argv != [argv[0]]: message += ' argv=%r ' % argv
 
     with log.progress(message) as p:
-        return Client().execute(argv)
+        return AdbClient().execute(argv)
 
 @with_device
 def interactive(**kw):
@@ -537,27 +790,59 @@ def shell(**kw):
     return process(['sh', '-i'], **kw)
 
 @with_device
-def which(name):
-    """Retrieves the full path to a binary in ``PATH`` on the device
+def which(name, all = False, *a, **kw):
+    """Retrieves the full path to a binary in ``$PATH`` on the device
 
-    >>> adb.which('sh')
-    '/system/bin/sh'
+    Arguments:
+        name(str): Binary name
+        all(bool): Whether to return all paths, or just the first
+        *a: Additional arguments for :func:`.adb.process`
+        **kw: Additional arguments for :func:`.adb.process`
+
+    Returns:
+        Either a path, or list of paths
+
+    Example:
+
+        >>> adb.which('sh')
+        '/system/bin/sh'
+        >>> adb.which('sh', all=True)
+        ['/system/bin/sh']
+
+        >>> adb.which('foobar') is None
+        True
+        >>> adb.which('foobar', all=True)
+        []
     """
     # Unfortunately, there is no native 'which' on many phones.
     which_cmd = '''
-IFS=:
-BINARY=%s
-P=($PATH)
-for path in "${P[@]}"; do \
-    if [ -e "$path/$BINARY" ]; then \
-        echo "$path/$BINARY";
-        break
-    fi
+echo $PATH | while read -d: directory; do
+    [ -x "$directory/{name}" ] || continue;
+    echo -n "$directory/{name}\\x00";
 done
-''' % name
+'''.format(name=name)
 
     which_cmd = which_cmd.strip()
-    return process(['sh','-c',which_cmd]).recvall().strip()
+    data = process(['sh','-c', which_cmd], *a, **kw).recvall()
+    result = []
+
+    for path in data.split('\x00'):
+        # Skip empty entries
+        if not path:
+            continue
+
+        # Return the first entry if all=False
+        if not all:
+            return path
+
+        # Accumulate all entries if all=True
+        result.append(path)
+
+    if all:
+        return result
+
+    return None
+
 
 @with_device
 def whoami():
@@ -570,7 +855,7 @@ def forward(port):
     start_forwarding = adb(['forward', tcp_port, tcp_port])
     atexit.register(lambda: adb(['forward', '--remove', tcp_port]))
 
-@context.quiet
+@context.quietfunc
 @with_device
 def logcat(stream=False):
     """Reads the system log file.
@@ -578,12 +863,12 @@ def logcat(stream=False):
     By default, causes logcat to exit after reading the file.
 
     Arguments:
-        stream(bool): If ``True``, the contents are streamed rather than
-            read in a one-shot manner.  Default is ``False``.
+        stream(bool): If :const:`True`, the contents are streamed rather than
+            read in a one-shot manner.  Default is :const:`False`.
 
     Returns:
-        If ``stream`` is ``False``, returns a string containing the log data.
-        Otherwise, it returns a ``tube`` connected to the log output.
+        If ``stream`` is :const:`False`, returns a string containing the log data.
+        Otherwise, it returns a :class:`pwnlib.tubes.tube.tube` connected to the log output.
     """
 
     if stream:
@@ -659,18 +944,17 @@ def listdir(directory='/'):
         Otherwise, less files may be returned due to restrictive SELinux
         policies on adbd.
     """
-    return list(sorted(Client().list(directory)))
+    return list(sorted(AdbClient().list(directory)))
 
+@with_device
 def fastboot(args, *a, **kw):
     """Executes a fastboot command.
 
     Returns:
         The command output.
     """
-    serial = context.device
-    if not serial:
-        log.error("Unknown device")
-    return tubes.process.process(['fastboot', '-s', serial] + list(args), **kw).recvall()
+    argv = ['fastboot', '-s', str(context.device)] + list(args)
+    return tubes.process.process(argv, *a, **kw).recvall()
 
 @with_device
 def fingerprint():
@@ -695,7 +979,7 @@ def unlock_bootloader():
     Note:
         This requires physical interaction with the device.
     """
-    Client().reboot_bootloader()
+    AdbClient().reboot_bootloader()
     fastboot(['oem', 'unlock'])
     fastboot(['continue'])
 
@@ -707,7 +991,7 @@ class Kernel(object):
         return self.symbols['_text']
 
     @property
-    @context.quiet
+    @context.quietfunc
     def symbols(self):
         """Returns a dictionary of kernel symbols"""
         result = {}
@@ -719,7 +1003,7 @@ class Kernel(object):
         return result
 
     @property
-    @context.quiet
+    @context.quietfunc
     def kallsyms(self):
         """Returns the raw output of kallsyms"""
         if not self._kallsyms:
@@ -730,20 +1014,20 @@ class Kernel(object):
         return self._kallsyms
 
     @property
-    @context.quiet
+    @context.quietfunc
     def version(self):
         """Returns the kernel version of the device."""
         root()
         return read('/proc/version').strip()
 
     @property
-    @context.quiet
+    @context.quietfunc
     def cmdline(self):
         root()
         return read('/proc/cmdline').strip()
 
     @property
-    @context.quiet
+    @context.quietfunc
     def lastmsg(self):
         root()
         if 'last_kmsg' in listdir('/proc'):
@@ -850,7 +1134,7 @@ _android_mk_template = '''
 LOCAL_PATH := $(call my-dir)
 
 include $(CLEAR_VARS)
-LOCAL_MODULE := poc
+LOCAL_MODULE := %(local_module)s
 LOCAL_SRC_FILES := %(local_src_files)s
 
 include $(BUILD_EXECUTABLE)
@@ -880,6 +1164,8 @@ def _generate_ndk_project(file_list, abi='arm-v7a', platform_version=21):
     # Create the directories
 
     # Populate Android.mk
+    local_module = os.path.basename(file_list[0])
+    local_module, _ = os.path.splitext(local_module)
     local_src_files = ' '.join(list(map(os.path.basename, file_list)))
     Android_mk = os.path.join(jni_directory, 'Android.mk')
     with open(Android_mk, 'w+') as f:
@@ -914,7 +1200,7 @@ def compile(source):
         abi = 'armeabi-v7a'
         sdk = '21'
 
-        # If we have an atatched device, use its settings.
+        # If we have an attached device, use its settings.
         if context.device:
             abi = str(properties.ro.product.cpu.abi)
             sdk = str(properties.ro.build.version.sdk)
@@ -951,18 +1237,57 @@ class Partition(object):
         with log.waitfor('Fetching %r partition (%s)' % (self.name, self.path)):
             return read(self.path)
 
+@with_device
+def walk(top, topdown=True):
+    join = os.path.join
+    isdir = lambda x: stat.S_ISDIR(x['mode'])
+    client = AdbClient()
+    names = client.list(top)
+
+    dirs, nondirs = [], []
+    for name, metadata in names.items():
+        if isdir(metadata):
+            dirs.append(name)
+        else:
+            nondirs.append(name)
+
+    if topdown:
+        yield top, dirs, nondirs
+    for name in dirs:
+        new_path = join(top, name)
+        for x in walk(new_path, topdown):
+            yield x
+    if not topdown:
+        yield top, dirs, nondirs
+
+@with_device
+def find(top, name):
+    for root, dirs, files in walk(top):
+        if name in files or name in dirs:
+            yield os.path.join(root, name)
+
+@with_device
+def readlink(path):
+    path = process(['readlink', path]).recvall()
+
+    # Readlink will emit a single newline
+    # We can't use the '-n' flag since old versions don't support it
+    if path.endswith('\n'):
+        path = path[:-1]
+
+    return path
+
 class Partitions(object):
     @property
-    @context.quiet
+    @context.quietfunc
     def by_name_dir(self):
-        cmd = ['shell','find /dev/block/platform -type d -name by-name']
-        return adb(cmd).strip()
+        return next(find('/dev/block/platform','by-name'))
 
-    @context.quiet
+    @context.quietfunc
     def __dir__(self):
         return list(self)
 
-    @context.quiet
+    @context.quietfunc
     @with_device
     def __iter__(self):
         root()
@@ -971,7 +1296,8 @@ class Partitions(object):
         for name in listdir(self.by_name_dir):
             yield name
 
-    @context.quiet
+    @context.quietfunc
+    @with_device
     def __getattr__(self, attr):
         for name in self:
             if name == attr:
@@ -982,7 +1308,7 @@ class Partitions(object):
         path = os.path.join(self.by_name_dir, name)
 
         # Find the actual path of the device
-        devpath = process(['readlink', '-n', path]).recvall()
+        devpath = readlink(path)
         devname = os.path.basename(devpath)
 
         # Get the size of the partition
@@ -1036,13 +1362,12 @@ def uninstall(package, *arguments):
         package(str): Name of the package to uninstall (e.g. ``'com.foo.MyPackage'``)
         arguments: Supplementary arguments to ``'pm install'``, e.g. ``'-k'``.
     """
-    with log.progress("Installing package {}".format(package)):
+    with log.progress("Uninstalling package {}".format(package)):
         with context.quiet:
-            return process(['pm','uninstall',apk] + list(arguments)).recvall()
+            return process(['pm','uninstall',package] + list(arguments)).recvall()
 
-@context.quiet
+@context.quietfunc
 def packages():
     """Returns a list of packages installed on the system"""
     packages = process(['pm', 'list', 'packages']).recvall()
     return [line.split('package:', 1)[-1] for line in packages.splitlines()]
-

pwnlib/adb/protocol.py

@@ -1,495 +1,5 @@
 """
-Implementation of the Android Debug Bridge (ADB) protocol, as far as Binjitsu needs it.
-
-Documentation is available here:
-https://android.googlesource.com/platform/system/core/+/master/adb/protocol.txt
+This file exists only for backward compatibility
 """
-import functools
-import time
-
-from ..context import context
-from ..log import Logger
-from ..log import getLogger
-from ..tubes.listen import listen
-from ..tubes.process import process
-from ..tubes.remote import remote
-from ..util.lists import group
-from ..util.misc import size
-from ..util.packing import p32
-from ..util.proc import pidof
-from ..util.sh_string import sh_string
-
-log = getLogger(__name__)
-
-def pack(val):
-    return '%04x' % val
-
-def unpack(val):
-    return int(val, 16)
-
-OKAY = "OKAY"
-FAIL = "FAIL"
-
-class Message(object):
-    """An ADB hex-length-prefixed message"""
-    def __init__(self, string):
-        self.string = string
-    def __str__(self):
-        return ('%04x' % len(self.string)) + self.string
-    def __flat__(self):
-        return str(self)
-
-class Connection(remote):
-    """Connection to the ADB server"""
-    def __init__(self, host, port, level=None, *a, **kw):
-
-        # Try to make sure ADB is running if it's on the default host and port.
-        if host == context.defaults['adb_host'] \
-        and port == context.defaults['adb_port']:
-            process(context.adb + ['start-server'], level='error').wait_for_close()
-
-        with context.quiet:
-            super(Connection, self).__init__(host, port, level=level, *a, **kw)
-
-        self._executable = None
-        self._argv       = None
-        self._pid        = None
-        self._cwd        = None
-        self._env        = None
-
-    def close(self):
-        with context.quiet:
-            super(Connection, self).close()
-
-    def adb_send(self, message):
-        self.send(str(Message(message)))
-        return self.recvn(4)
-
-    def adb_unpack(self):
-        return unpack(self.recvn(4))
-
-    def flat(self, *a, **kw):
-        kw.setdefault('word_size', 32)
-        return super(Connection, self).flat(*a, **kw)
-
-class Process(Connection):
-    """Duck-typed ``tubes.remote`` object to add properties of a ``tubes.process``"""
-
-class Client(Logger):
-    """ADB Client"""
-    def __init__(self, level=None):
-        super(Client, self).__init__()
-
-        if level is not None:
-            self.setLevel(level)
-
-        self.host = context.adb_host
-        self.port = context.adb_port
-        self._c   = None
-
-    @property
-    def c(self):
-        """Client's connection to the ADB server"""
-        if not self._c:
-            self._c = Connection(self.host, self.port, level=self.level)
-        return self._c
-
-    def _autoclose(fn):
-        """Decorator which automatically closes the connection to the ADB server
-        after calling the decorated function."""
-        @functools.wraps(fn)
-        def wrapper(self, *a, **kw):
-            rv = fn(self, *a, **kw)
-            if self._c:
-                self._c.close()
-                self._c = None
-            return rv
-        return wrapper
-
-    def _with_transport(fn):
-        """Decorator which automatically selects a device transport before calling
-        the decorated function, and closes the connection afterward."""
-        @functools.wraps(fn)
-        def wrapper(self, *a, **kw):
-            self.transport()
-            rv = fn(self, *a, **kw)
-            if self._c:
-                self._c.close()
-                self._c = None
-            return rv
-        return wrapper
-
-    def send(self, *a, **kw):
-        """Sends data to the ADB server"""
-        return self.c.adb_send(*a, **kw)
-
-    def unpack(self, *a, **kw):
-        """Receives a hex-ascii packed integer from the ADB server"""
-        return self.c.adb_unpack(*a, **kw)
-
-    def recvl(self):
-        """Receives a length-prefixed data buffer from the ADB server"""
-        length = self.c.adb_unpack()
-        return self.c.recvn(length)
-
-    @_autoclose
-    def kill(self):
-        """Kills the remote ADB server"""
-        try:
-            self.send('host:kill')
-        except EOFError:
-            pass
-
-    def version(self):
-        """
-        Returns:
-            Tuple containing the ``(major, minor)`` version from the ADB server
-
-        Example:
-
-            >>> adb.protocol.Client().version() # doctest: +SKIP
-            (4, 36)
-        """
-        response = self.send('host:version')
-        if response == OKAY:
-            return (self.c.adb_unpack(), self.c.adb_unpack())
-        self.error("Could not fetch version")
-
-    @_autoclose
-    def devices(self, long=False):
-        """
-        Arguments:
-            long(bool): If ``True``, fetch the long-format listing.
-        Returns:
-            String representation of all available devices.
-        """
-        msg = 'host:devices'
-        if long:
-            msg += '-l'
-        response = self.send(msg)
-        if response == 'OKAY':
-            return self.recvl()
-        self.error("Could not enumerate devices")
-
-    @_autoclose
-    def track_devices(self):
-        """
-        Returns:
-            Generator which returns a short-format listing of available
-            devices each time a device state changes.
-        """
-        self.send('host:track-devices')
-        while True:
-            yield self.recvl()
-
-    def transport(self, serial=None):
-        """Sets the Transport on the rmeote device.
-
-        Examples:
-
-            >>> adb.protocol.Client().transport()
-        """
-        msg = 'host:transport:%s' % (serial or context.device)
-        if self.send(msg) == FAIL:
-            self.error("Could not set transport to %r" % serial)
-
-    @_autoclose
-    @_with_transport
-    def execute(self, argv):
-        r"""Executes a program on the device.
-
-        Returns:
-            A ``tube`` which is connected to the process.
-
-        Examples:
-
-            >>> adb.protocol.Client().execute(['echo','hello']).recvall()
-            'hello\n'
-        """
-        self.transport(context.device)
-        if isinstance(argv, str):
-            argv = [argv]
-        argv = list(map(sh_string, argv))
-        cmd = 'exec:%s' % (' '.join(argv))
-        if OKAY == self.send(cmd):
-            rv = self._c
-            self._c = None
-            return rv
-
-    def _basic_wrapper(string):
-        @_autoclose
-        @_with_transport
-        def wrapper(self):
-            self.send(string)
-            return self.c.recvall()
-        return wrapper
-
-    @_autoclose
-    @_with_transport
-    def remount(self):
-        self.send('remount:')
-        return self.c.recvall()
-
-    @_autoclose
-    @_with_transport
-    def root(self):
-        self.send('root:')
-        return self.c.recvall()
-
-    @_autoclose
-    @_with_transport
-    def unroot(self):
-        self.send('unroot:')
-        return self.c.recvall()
-
-    @_autoclose
-    @_with_transport
-    def disable_verity(self):
-        self.send('disable-verity:')
-        return self.c.recvall()
-
-    @_autoclose
-    @_with_transport
-    def enable_verity(self):
-        self.send('enable-verity:')
-        return self.c.recvall()
-
-    @_autoclose
-    @_with_transport
-    def reconnect(self):
-        self.send('reconnect:')
-        return self.c.recvall()
-
-    @_autoclose
-    @_with_transport
-    def reboot(self):
-        self.send('reboot:')
-        return self.c.recvall()
-
-    @_autoclose
-    @_with_transport
-    def reboot_bootloader(self):
-        self.send('reboot:bootloader')
-        return self.c.recvall()
-
-    @_autoclose
-    def wait_for_device(self, serial=''):
-        response = self.send('host-serial:%s:wait-for-any-device' % serial)
-
-        # The first OKAY is that the command was understood
-        if response != 'OKAY':
-            self.error("An error occurred while waiting for device with serial %r" % serial)
-
-        # The second OKAY is that the device is available
-        response = self.c.recvn(4)
-        if response != 'OKAY':
-            self.error("An error occurred while waiting for device with serial %r" % serial)
-
-    def _sync(fn):
-        """Decorator which enters 'sync:' mode to the selected transport,
-        then invokes the decorated funciton."""
-        @functools.wraps(fn)
-        def wrapper(self, *a, **kw):
-            rv = None
-            if FAIL != self.send('sync:'):
-                rv = fn(self, *a, **kw)
-            return rv
-        return wrapper
-
-    @_with_transport
-    @_sync
-    def list(self, path):
-        """Execute the ``LIST`` command of the ``SYNC`` API.
-
-        Arguments:
-            path(str): Path of the directory to list.
-
-        Return:
-            A dictionary, where the keys are relative filenames,
-            and the values are a dictionary containing the same
-            values as ``stat()`` supplies.
-
-        Note:
-            In recent releases of Android (e.g. 7.0), the domain that
-            adbd executes from does not have access to everything that
-            the shell user does.
-
-            Because of this, while the shell user can get listings of
-            e.g. the root directory ('/'), adbd cannot.
-
-            The SYNC APIs are executed within the adbd context, not the
-            shell user context.
-
-            This issue is not a problem if the phone is rooted via
-            'adb root', since adbd then runs in the ``su`` domain.
-
-        Examples:
-
-            >>> pprint(adb.Client().list('/data/user'))
-            {'0': {'mode': 41471, 'size': 11, 'time': ...}}
-            >>> adb.Client().list('/does/not/exist')
-            {}
-        """
-        self.c.flat('LIST', len(path), path)
-        files = {}
-        while True:
-            response = self.c.recvn(4)
-
-            if response == 'DONE':
-                break
-
-            if response != 'DENT':
-                self.error('Unexpected response: %r' % response)
-
-            mode = self.c.u32()
-            size = self.c.u32()
-            time = self.c.u32()
-            name = self.c.recvn(self.c.u32())
-
-            # Ignore the current directory and parent
-            if name in ('', '.', '..'):
-                continue
-
-            files[name] = {'mode': mode,
-                           'size': size,
-                           'time': time}
-
-        return files
-
-    @_with_transport
-    @_sync
-    def stat(self, path):
-        """Execute the ``STAT`` command of the ``SYNC`` API.
-
-        Arguments:
-            path(str): Path to the file to stat.
-
-        Return:
-            On success, a dictionary mapping the values returned.
-            If the file cannot be ``stat()``ed, None is returned.
-
-        Example:
-
-            >>> expected = {'mode': 16749, 'size': 0, 'time': 0}
-            >>> adb.protocol.Client().stat('/proc')           == expected
-            True
-            >>> adb.protocol.Client().stat('/does/not/exist') == None
-            True
-        """
-        self.c.flat('STAT', len(path), path)
-        if self.c.recvn(4) != 'STAT':
-            self.error("An error occured while attempting to STAT a file.")
-
-        mode = self.c.u32()
-        size = self.c.u32()
-        time = self.c.u32()
-
-        if (mode,size,time) == (0,0,0):
-            return None
-
-        return {'mode': mode, 'size': size, 'time': time}
-
-    @_with_transport
-    @_sync
-    def write(self, path, data, mode=0o755, timestamp=None, callback=None):
-        path += ',' + str(mode)
-        self.c.flat('SEND', len(path), path)
-
-        sent = 0
-
-        # Data needs to be broken up into chunks!
-        for chunk in group(0x10000, data):
-            self.c.flat('DATA', len(chunk), chunk)
-            if callback:
-                callback(path, data[:sent], len(data), chunk, len(chunk))
-            sent += len(chunk)
-
-        # Send completion notification and timestamp
-        if timestamp is None:
-            timestamp = int(time.time())
-        self.c.flat('DONE', timestamp)
-
-        result = self.c.recvn(4)
-        if result != 'OKAY':
-            log.error("Sync write failed: %r (expected OKAY)" % result)
-
-        return
-
-    @_with_transport
-    @_sync
-    def read(self, path, filesize=0, callback=lambda *a: True):
-        """Execute the ``READ`` command of the ``SYNC`` API.
-
-        Arguments:
-            path(str): Path to the file to read
-            filesize(int): Size of the file, in bytes.  Optional.
-            callback(callable): Callback function invoked as data
-                becomes available.  Arguments provided are:
-
-                - File path
-                - All data
-                - Expected size of all data
-                - Current chunk
-                - Expected size of chunk
-
-        Return:
-            The data received as a string.
-        """
-        self.c.send('RECV' + p32(len(path)) + path)
-
-        # Accumulate all data here
-        all_data = ''
-
-        while True:
-            magic = self.c.recvn(4)
-
-            # adbd says there is no more data to send
-            if magic == 'DONE':
-                break
-
-            if magic == 'FAIL':
-                self.error('Could not read file %r: Got FAIL.' % path)
-
-            # did we expect to be done?
-            if magic != 'DATA':
-                self.error('Error after file read: %r (expected DATA)' % magic)
-
-            # receive all of the data in the chunk
-            chunk_size = self.c.u32()
-            chunk_data  = ''
-            while len(chunk_data) != chunk_size:
-                chunk_data += self.c.recv(chunk_size - len(chunk_data))
-
-                if callback:
-                    callback(path,
-                             all_data,
-                             filesize,
-                             chunk_data,
-                             chunk_size)
-
-            # add the chunk onto what we have
-            all_data += chunk_data
-
-        zero = self.c.u32()
-        if zero != 0:
-            self.error('Error after file read: %r (expected ZERO)' % zero)
-
-        return all_data
-
-    @_with_transport
-    def forward(self, device, host_proto, host_port, device_proto, device_port):
-        self.send('host:forward:%s:%s;%s:%s' % (host_proto, host_port, device_proto, device_port))
-        self.c.recvall()
-
-    def __enter__(self, *a, **kw):
-        return self
-
-    @_autoclose
-    def __exit__(self, *a, **kw): pass
-
-def proxy(port=9999):
-    """Starts an ADB proxy on the specified port, for debugging purposes."""
-    l = listen(port)
-    l.wait_for_connection()
-    r = remote(context.adb_host, context.adb_port, level='debug')
-    l <> r
+from pwnlib.protocols.adb import AdbClient
+Client = AdbClient

pwnlib/args.py

@@ -1,18 +1,70 @@
+
 #!/usr/bin/env python2
 """
+Pwntools exposes several magic command-line arguments and environment
+variables when operating in `from pwn import *` mode.
+
+The arguments extracted from the command-line and removed from ``sys.argv``.
+
+Arguments can be set by appending them to the command-line, or setting
+them in the environment prefixed by ``PWNLIB_``.
+
+The easiest example is to enable more verbose debugging.  Just set ``DEBUG``.
+
+.. code-block:: bash
+
+    $ PWNLIB_DEBUG=1 python exploit.py
+    $ python exploit.py DEBUG
+
+These arguments are automatically extracted, regardless of their name, and
+exposed via :mod:`pwnlib.args.args`, which is exposed as the global variable
+:data:`args`.  Arguments which ``pwntools`` reserves internally are not exposed
+this way.
+
+.. code-block:: bash
+
+    $ python -c 'from pwn import *; print args' A=1 B=Hello HOST=1.2.3.4 DEBUG
+    defaultdict(<type 'str'>, {'A': '1', 'HOST': '1.2.3.4', 'B': 'Hello'})
+
+This is very useful for conditional code, for example determining whether to
+run an exploit locally or to connect to a remote server.  Arguments which are
+not specified evaluate to an empty string.
+
+.. code-block:: python
+
+    if args['REMOTE']:
+        io = remote('exploitme.com', 4141)
+    else:
+        io = process('./pwnable')
+
+The full list of supported "magic arguments" and their effects are listed
+below.
 """
+from __future__ import absolute_import
+
 import collections
 import logging
 import os
 import string
 import sys
 
-from . import term
-from .context import context
+from pwnlib import term
+from pwnlib.context import context
 
 term_mode  = True
 args       = collections.defaultdict(str)
 env_prefix = 'PWNLIB_'
+free_form  = True
+
+# Check to see if we were invoked as one of the 'pwn xxx' scripts.
+# If so, we don't want to remove e.g. "SYS_" from the end of the command
+# line, as this breaks things like constgrep.
+import pwnlib.commandline
+basename = os.path.basename(sys.argv[0])
+
+if basename == 'pwn' or basename in pwnlib.commandline.__all__:
+    free_form = False
+
 
 def isident(s):
     """
@@ -42,46 +94,67 @@ def asbool(s):
     else:
         raise ValueError('must be integer or boolean: %r' % s)
 
-def set_log_level(x):
+def LOG_LEVEL(x):
+    """Sets the logging verbosity used via ``context.log_level``,
+    e.g. ``LOG_LEVEL=debug``.
+    """
     with context.local(log_level=x):
         context.defaults['log_level']=context.log_level
 
-def set_log_file(x):
+def LOG_FILE(x):
+    """Sets a log file to be used via ``context.log_file``, e.g.
+    ``LOG_FILE=./log.txt``"""
     context.log_file=x
 
-def set_log_level_error(x):
-    set_log_level('error')
+def SILENT(x):
+    """Sets the logging verbosity to ``error`` which silences most
+    output."""
+    LOG_LEVEL('error')
 
-def set_log_level_debug(x):
-    set_log_level('debug')
+def DEBUG(x):
+    """Sets the logging verbosity to ``debug`` which displays much
+    more information, including logging each byte sent by tubes."""
+    LOG_LEVEL('debug')
 
-def set_noterm(v):
+def NOTERM(v):
+    """Disables pretty terminal settings and animations."""
     if asbool(v):
         global term_mode
         term_mode = False
 
-def set_timeout(v):
+def TIMEOUT(v):
+    """Sets a timeout for tube operations (in seconds) via
+    ``context.timeout``, e.g. ``TIMEOUT=30``"""
     context.defaults['timeout'] = int(v)
 
-def set_randomize(v):
+def RANDOMIZE(v):
+    """Enables randomization of various pieces via ``context.randomize``"""
     context.defaults['randomize'] = asbool(v)
 
-def set_aslr(v):
+def NOASLR(v):
+    """Disables ASLR via ``context.aslr``"""
     context.defaults['aslr'] = not asbool(v)
 
-def set_noptrace(v):
+def NOPTRACE(v):
+    """Disables facilities which require ``ptrace`` such as ``gdb.attach()``
+    statements, via ``context.noptrace``."""
     context.defaults['noptrace'] = asbool(v)
 
+def STDERR(v):
+    """Sends logging to ``stderr`` by default, instead of ``stdout``"""
+    context.log_console = sys.stderr
+
 hooks = {
-    'LOG_LEVEL': set_log_level,
-    'LOG_FILE': set_log_file,
-    'DEBUG': set_log_level_debug,
-    'NOTERM': set_noterm,
-    'SILENT': set_log_level_error,
-    'RANDOMIZE': set_randomize,
-    'TIMEOUT': set_timeout,
-    'NOASLR': set_aslr,
-    'NOPTRACE': set_noptrace,
+    'LOG_LEVEL': LOG_LEVEL,
+    'LOG_FILE': LOG_FILE,
+    'DEBUG': DEBUG,
+    'NOTERM': NOTERM,
+    'SILENT': SILENT,
+    'RANDOMIZE': RANDOMIZE,
+    'TIMEOUT': TIMEOUT,
+    'NOASLR': NOASLR,
+    'NOPTRACE': NOPTRACE,
+    'STDERR': STDERR,
 }
 
 def initialize():
@@ -113,7 +186,7 @@ def initialize():
             sys.argv.remove(orig)
             hooks[arg](value)
 
-        elif isident(arg):
+        elif free_form and isident(arg):
             sys.argv.remove(orig)
             args[arg] = value
 

pwnlib/asm.py

@@ -39,6 +39,8 @@ Disassembly
     '   0:   b8 0b 00 00 00          mov    eax,0xb'
 
 """
+from __future__ import absolute_import
+
 import errno
 import os
 import platform
@@ -53,11 +55,11 @@ from glob import glob
 from os import environ
 from os import path
 
-from . import atexit
-from . import shellcraft
-from .context import LocalContext
-from .context import context
-from .log import getLogger
+from pwnlib import atexit
+from pwnlib import shellcraft
+from pwnlib.context import LocalContext
+from pwnlib.context import context
+from pwnlib.log import getLogger
 
 log = getLogger(__name__)
 
@@ -71,6 +73,11 @@ def dpkg_search_for_binutils(arch, util):
 
     Returns:
         A list of candidate package names.
+
+    ::
+
+        >>> pwnlib.asm.dpkg_search_for_binutils('aarch64', 'as')
+        ['binutils-aarch64-linux-gnu']
     """
 
     # Example output:
@@ -97,6 +104,16 @@ def dpkg_search_for_binutils(arch, util):
 def print_binutils_instructions(util, context):
     """On failure to find a binutils utility, inform the user of a way
     they can get it easily.
+
+    Doctest:
+
+        >>> context.clear(arch = 'amd64')
+        >>> pwnlib.asm.print_binutils_instructions('as', context)
+        Traceback (most recent call last):
+        ...
+        PwnlibException: Could not find 'as' installed for ContextType(arch = 'amd64', bits = 64, endian = 'little')
+        Try installing binutils for this architecture:
+        $ sudo apt-get install binutils
     """
     # This links to our instructions on how to manually install binutils
     # for several architectures.
@@ -173,10 +190,12 @@ def which_binutils(util):
     for arch in arches:
         for gutil in utils:
             # e.g. objdump
-            if arch is None: pattern = gutil
+            if arch is None:
+                pattern = gutil
 
             # e.g. aarch64-linux-gnu-objdump
-            else:       pattern = '%s*linux*-%s' % (arch,gutil)
+            else:
+                pattern = '%s*linux*-%s' % (arch,gutil)
 
             for dir in environ['PATH'].split(':'):
                 res = sorted(glob(path.join(dir, pattern)))
@@ -380,16 +399,14 @@ def cpp(shellcode):
 
     Examples:
 
-        .. doctest::
-
-            >>> cpp("mov al, SYS_setresuid", arch = "i386", os = "linux")
-            'mov al, 164\n'
-            >>> cpp("weee SYS_setresuid", arch = "arm", os = "linux")
-            'weee (0+164)\n'
-            >>> cpp("SYS_setresuid", arch = "thumb", os = "linux")
-            '(0+164)\n'
-            >>> cpp("SYS_setresuid", os = "freebsd")
-            '311\n'
+        >>> cpp("mov al, SYS_setresuid", arch = "i386", os = "linux")
+        'mov al, 164\n'
+        >>> cpp("weee SYS_setresuid", arch = "arm", os = "linux")
+        'weee (0+164)\n'
+        >>> cpp("SYS_setresuid", arch = "thumb", os = "linux")
+        '(0+164)\n'
+        >>> cpp("SYS_setresuid", os = "freebsd")
+        '311\n'
     """
     arch = context.arch
     os   = context.os
@@ -406,17 +423,39 @@ def cpp(shellcode):
     return _run(cmd, code).strip('\n').rstrip() + '\n'
 
 @LocalContext
-def make_elf_from_assembly(assembly, vma = None, extract=False, shared = False):
-    r"""
-    Builds an ELF file with the specified assembly as its
-    executable code.
+def make_elf_from_assembly(assembly,
+                           vma=None,
+                           extract=False,
+                           shared=False,
+                           strip=False,
+                           **kwargs):
+    r"""make_elf_from_assembly(assembly, vma=None, extract=None, shared=False, strip=False, **kwargs) -> str
+
+    Builds an ELF file with the specified assembly as its executable code.
+
+    This differs from :func:`.make_elf` in that all ELF symbols are preserved,
+    such as labels and local variables.  Use :func:`.make_elf` if size matters.
+    Additionally, the default value for ``extract`` in :func:`.make_elf` is
+    different.
+
+    Note:
+        This is effectively a wrapper around :func:`.asm`. with setting
+        ``extract=False``, ``vma=0x10000000``, and marking the resulting
+        file as executable (``chmod +x``).
+
+    Note:
+        ELF files created with `arch=thumb` will prepend an ARM stub
+        which switches to Thumb mode.
 
     Arguments:
-
-        assembly(str): Assembly
+        assembly(str): Assembly code to build into an ELF
         vma(int): Load address of the binary
-        extract(bool): Whether to return the data extracted from the file created,
-                       or the path to it.
+            (Default: ``0x10000000``, or ``0`` if ``shared=True``)
+        extract(bool): Extract the full ELF data from the file.
+            (Default: ``False``)
+        shared(bool): Create a shared library
+            (Default: ``False``)
+        kwargs(dict): Arguments to pass to :func:`.asm`.
 
     Returns:
 
@@ -425,6 +464,9 @@ def make_elf_from_assembly(assembly, vma = None, extract=False, shared = False):
 
     Example:
 
+        This example shows how to create a shared library, and load it via
+        ``LD_PRELOAD``.
+
         >>> context.clear()
         >>> context.arch = 'amd64'
         >>> sc = 'push rbp; mov rbp, rsp;'
@@ -433,6 +475,18 @@ def make_elf_from_assembly(assembly, vma = None, extract=False, shared = False):
         >>> solib = make_elf_from_assembly(sc, shared=1)
         >>> subprocess.check_output(['echo', 'World'], env={'LD_PRELOAD': solib})
         'Hello\nWorld\n'
+
+        The same thing can be done with :func:`.make_elf`, though the sizes
+        are different.  They both
+
+        >>> file_a = make_elf(asm('nop'), extract=True)
+        >>> file_b = make_elf_from_assembly('nop', extract=True)
+        >>> file_a[:4] == file_b[:4]
+        True
+        >>> len(file_a) < 0x200
+        True
+        >>> len(file_b) > 0x1000
+        True
     """
     if shared and vma:
         log.error("Cannot specify a VMA for a shared library.")
@@ -449,38 +503,44 @@ def make_elf_from_assembly(assembly, vma = None, extract=False, shared = False):
         if not assembly.startswith(to_thumb):
             assembly = to_thumb + assembly
 
-    path = asm(assembly, vma = vma, extract = extract, shared = shared)
+    result = asm(assembly, vma = vma, shared = shared, extract = False, **kwargs)
 
     if not extract:
-        os.chmod(path, 0755)
+        os.chmod(result, 0755)
+    else:
+        with open(result, 'rb') as io:
+            result = io.read()
 
-    return path
+    return result
 
 @LocalContext
-def make_elf(data, vma = None, strip=True, extract=True, shared=False):
-    r"""
-    Builds an ELF file with the specified binary data as its
-    executable code.
+def make_elf(data,
+             vma=None,
+             strip=True,
+             extract=True,
+             shared=False):
+    r"""make_elf(data, vma=None, strip=True, extract=True, shared=False, **kwargs) -> str
+
+    Builds an ELF file with the specified binary data as its executable code.
 
     Arguments:
         data(str): Assembled code
         vma(int):  Load address for the ELF file
+        strip(bool): Strip the resulting ELF file. Only matters if ``extract=False``.
+            (Default: ``True``)
+        extract(bool): Extract the assembly from the ELF file.
+            If ``False``, the path of the ELF file is returned.
+            (Default: ``True``)
+        shared(bool): Create a Dynamic Shared Object (DSO, i.e. a ``.so``)
+            which can be loaded via ``dlopen`` or ``LD_PRELOAD``.
 
     Examples:
-
         This example creates an i386 ELF that just does
         execve('/bin/sh',...).
 
-        >>> context.clear()
-        >>> context.arch = 'i386'
-        >>> context.bits = 32
-        >>> filename = tempfile.mktemp()
+        >>> context.clear(arch='i386')
         >>> bin_sh = '6a68682f2f2f73682f62696e89e331c96a0b5899cd80'.decode('hex')
-        >>> data = make_elf(bin_sh)
-        >>> with open(filename,'wb+') as f:
-        ...     f.write(data)
-        ...     f.flush()
-        >>> os.chmod(filename,0777)
+        >>> filename = make_elf(bin_sh, extract=False)
         >>> p = process(filename)
         >>> p.sendline('echo Hello; exit')
         >>> p.recvline()
@@ -548,38 +608,37 @@ def make_elf(data, vma = None, strip=True, extract=True, shared=False):
 
 @LocalContext
 def asm(shellcode, vma = 0, extract = True, shared = False):
-    r"""asm(code, vma = 0, extract = True, ...) -> str
+    r"""asm(code, vma = 0, extract = True, shared = False, ...) -> str
 
     Runs :func:`cpp` over a given shellcode and then assembles it into bytes.
 
     To see which architectures or operating systems are supported,
     look in :mod:`pwnlib.contex`.
 
-    To support all these architecture, we bundle the GNU assembler
-    and objcopy with pwntools.
+    Assembling shellcode requires that the GNU assembler is installed
+    for the target architecture.
+    See :doc:`Installing Binutils </install/binutils>` for more information.
 
     Arguments:
-      shellcode(str): Assembler code to assemble.
-      vma(int):       Virtual memory address of the beginning of assembly
-      extract(bool):  Extract the raw assembly bytes from the assembled
-                      file.  If ``False``, returns the path to an ELF file
-                      with the assembly embedded.
-
-    Kwargs:
-        Any arguments/properties that can be set on ``context``
+        shellcode(str): Assembler code to assemble.
+        vma(int):       Virtual memory address of the beginning of assembly
+        extract(bool):  Extract the raw assembly bytes from the assembled
+                        file.  If :const:`False`, returns the path to an ELF file
+                        with the assembly embedded.
+        shared(bool):   Create a shared object.
+        kwargs(dict):   Any attributes on :data:`.context` can be set, e.g.set
+                        ``arch='arm'``.
 
     Examples:
 
-        .. doctest::
-
-            >>> asm("mov eax, SYS_select", arch = 'i386', os = 'freebsd')
-            '\xb8]\x00\x00\x00'
-            >>> asm("mov eax, SYS_select", arch = 'amd64', os = 'linux')
-            '\xb8\x17\x00\x00\x00'
-            >>> asm("mov rax, SYS_select", arch = 'amd64', os = 'linux')
-            'H\xc7\xc0\x17\x00\x00\x00'
-            >>> asm("mov r0, #SYS_select", arch = 'arm', os = 'linux', bits=32)
-            'R\x00\xa0\xe3'
+        >>> asm("mov eax, SYS_select", arch = 'i386', os = 'freebsd')
+        '\xb8]\x00\x00\x00'
+        >>> asm("mov eax, SYS_select", arch = 'amd64', os = 'linux')
+        '\xb8\x17\x00\x00\x00'
+        >>> asm("mov rax, SYS_select", arch = 'amd64', os = 'linux')
+        'H\xc7\xc0\x17\x00\x00\x00'
+        >>> asm("mov r0, #SYS_select", arch = 'arm', os = 'linux', bits=32)
+        'R\x00\xa0\xe3'
     """
     result = ''
 
@@ -611,11 +670,20 @@ def asm(shellcode, vma = 0, extract = True, shared = False):
             ldflags = ['-z', 'execstack', '-o', step3, step2]
             if vma:
                 ldflags += ['--section-start=.shellcode=%#x' % vma,
-                            '--entry=%#x' % vma,
-                            '-z', 'max-page-size=4096',
-                            '-z', 'common-page-size=4096']
+                            '--entry=%#x' % vma]
             elif shared:
                 ldflags += ['-shared', '-init=_start']
+
+            # In order to ensure that we generate ELF files with 4k pages,
+            # and not e.g. 65KB pages (AArch64), force the page size.
+            # This is a result of GNU Gold being silly.
+            #
+            # Introduced in commit dd58f409 without supporting evidence,
+            # this shouldn't do anything except keep consistent page granularity
+            # across architectures.
+            ldflags += ['-z', 'max-page-size=4096',
+                        '-z', 'common-page-size=4096']
+
             _run(linker + ldflags)
 
         elif file(step2,'rb').read(4) == '\x7fELF':
@@ -667,24 +735,21 @@ def disasm(data, vma = 0, byte = True, offset = True, instructions = True):
 
     Examples:
 
-        .. doctest::
-
-          >>> print disasm('b85d000000'.decode('hex'), arch = 'i386')
-             0:   b8 5d 00 00 00          mov    eax,0x5d
-          >>> print disasm('b85d000000'.decode('hex'), arch = 'i386', byte = 0)
-             0:   mov    eax,0x5d
-          >>> print disasm('b85d000000'.decode('hex'), arch = 'i386', byte = 0, offset = 0)
-          mov    eax,0x5d
-          >>> print disasm('b817000000'.decode('hex'), arch = 'amd64')
-             0:   b8 17 00 00 00          mov    eax,0x17
-          >>> print disasm('48c7c017000000'.decode('hex'), arch = 'amd64')
-             0:   48 c7 c0 17 00 00 00    mov    rax,0x17
-          >>> print disasm('04001fe552009000'.decode('hex'), arch = 'arm')
-             0:   e51f0004        ldr     r0, [pc, #-4]   ; 0x4
-             4:   00900052        addseq  r0, r0, r2, asr r0
-          >>> print disasm('4ff00500'.decode('hex'), arch = 'thumb', bits=32)
-             0:   f04f 0005       mov.w   r0, #5
-          >>>
+        >>> print disasm('b85d000000'.decode('hex'), arch = 'i386')
+           0:   b8 5d 00 00 00          mov    eax,0x5d
+        >>> print disasm('b85d000000'.decode('hex'), arch = 'i386', byte = 0)
+           0:   mov    eax,0x5d
+        >>> print disasm('b85d000000'.decode('hex'), arch = 'i386', byte = 0, offset = 0)
+        mov    eax,0x5d
+        >>> print disasm('b817000000'.decode('hex'), arch = 'amd64')
+           0:   b8 17 00 00 00          mov    eax,0x17
+        >>> print disasm('48c7c017000000'.decode('hex'), arch = 'amd64')
+           0:   48 c7 c0 17 00 00 00    mov    rax,0x17
+        >>> print disasm('04001fe552009000'.decode('hex'), arch = 'arm')
+           0:   e51f0004        ldr     r0, [pc, #-4]   ; 0x4
+           4:   00900052        addseq  r0, r0, r2, asr r0
+        >>> print disasm('4ff00500'.decode('hex'), arch = 'thumb', bits=32)
+           0:   f04f 0005       mov.w   r0, #5
     """
     result = ''
 

pwnlib/atexception.py

@@ -2,12 +2,13 @@
 Analogous to atexit, this module allows the programmer to register functions to
 be run if an unhandled exception occurs.
 """
+from __future__ import absolute_import
 
 import sys
 import threading
 import traceback
 
-from .context import context
+from pwnlib.context import context
 
 __all__ = ['register', 'unregister']
 

pwnlib/atexit.py

@@ -7,12 +7,13 @@ this replacement module also defines :func:`unregister`.
 This module also fixes a the issue that exceptions raised by an exit handler is
 printed twice when the standard :mod:`atexit` is used.
 """
+from __future__ import absolute_import
 
 import sys
 import threading
 import traceback
 
-from .context import context
+from pwnlib.context import context
 
 __all__ = ['register', 'unregister']
 

pwnlib/commandline/__init__.py

@@ -1 +1,20 @@
 #!/usr/bin/env python2
+__all__ = [
+    'asm',
+    'checksec',
+    'common',
+    'constgrep',
+    'cyclic',
+    'disasm',
+    'elfdiff',
+    'elfpatch',
+    'errno',
+    'hex',
+    'main',
+    'phd',
+    'pwnstrip',
+    'scramble',
+    'shellcraft',
+    'unhex',
+    'update'
+]

pwnlib/commandline/asm.py

@@ -1,10 +1,11 @@
 #!/usr/bin/env python2
+from __future__ import absolute_import
+
 import argparse
 import sys
 
 from pwn import *
-
-from . import common
+from pwnlib.commandline import common
 
 parser = common.parser_commands.add_parser(
     'asm',

pwnlib/commandline/checksec.py

@@ -1,10 +1,11 @@
 #!/usr/bin/env python2
+from __future__ import absolute_import
+
 import argparse
 import sys
 
 from pwn import *
-
-from . import common
+from pwnlib.commandline import common
 
 parser = common.parser_commands.add_parser(
     'checksec',

pwnlib/commandline/common.py

@@ -5,8 +5,6 @@ import sys
 import pwnlib
 from pwnlib.context import context
 
-pwnlib.log.console.stream = sys.stderr
-
 choices = map(str, [16,32,64])
 choices += list(context.oses)
 choices += list(context.architectures)

pwnlib/commandline/constgrep.py

@@ -1,12 +1,15 @@
 #!/usr/bin/env python2
+from __future__ import absolute_import
 
 import argparse
 import os
 import re
 
-from pwn import *
+import pwnlib
+pwnlib.args.free_form = False
 
-from . import common
+from pwn import *
+from pwnlib.commandline import common
 
 p = common.parser_commands.add_parser(
     'constgrep',
@@ -75,6 +78,7 @@ def main(args):
         else:
             matcher = re.compile(args.regex)
 
+
         # Evaluate the given constant
         if args.constant:
             try:

pwnlib/commandline/cyclic.py

@@ -1,11 +1,15 @@
 #!/usr/bin/env python2
+from __future__ import absolute_import
+
 import argparse
 import string
 import sys
 
-from pwn import *
+import pwnlib
+pwnlib.args.free_form = False
 
-from . import common
+from pwn import *
+from pwnlib.commandline import common
 
 parser = common.parser_commands.add_parser(
     'cyclic',

pwnlib/commandline/disasm.py

@@ -1,11 +1,15 @@
 #!/usr/bin/env python2
+from __future__ import absolute_import
+
 import argparse
 import string
 import sys
 
-from pwn import *
+import pwnlib
+pwnlib.args.free_form = False
 
-from . import common
+from pwn import *
+from pwnlib.commandline import common
 
 parser = common.parser_commands.add_parser(
     'disasm',

pwnlib/commandline/elfdiff.py

@@ -1,13 +1,17 @@
 #!/usr/bin/env python2
+from __future__ import absolute_import
+
 import shutil
 from argparse import ArgumentParser
 from subprocess import CalledProcessError
 from subprocess import check_output
 from tempfile import NamedTemporaryFile
 
-from pwn import *
+import pwnlib
+pwnlib.args.free_form = False
 
-from . import common
+from pwn import *
+from pwnlib.commandline import common
 
 
 def dump(objdump, path):

pwnlib/commandline/elfpatch.py

@@ -1,10 +1,14 @@
 #!/usr/bin/env python2
+from __future__ import absolute_import
+
 import argparse
 import sys
 
-from pwn import *
+import pwnlib
+pwnlib.args.free_form = False
 
-from . import common
+from pwn import *
+from pwnlib.commandline import common
 
 p = common.parser_commands.add_parser(
     'elfpatch',

pwnlib/commandline/errno.py

@@ -1,7 +1,9 @@
+from __future__ import absolute_import
+
 import argparse
 import os
 
-from . import common
+from pwnlib.commandline import common
 
 parser = common.parser_commands.add_parser(
     'errno',

pwnlib/commandline/hex.py

@@ -1,8 +1,10 @@
 #!/usr/bin/env python2
+from __future__ import absolute_import
+
 import argparse
 import sys
 
-from . import common
+from pwnlib.commandline import common
 
 parser = common.parser_commands.add_parser(
     'hex',

pwnlib/commandline/main.py

@@ -1,20 +1,25 @@
-from . import asm
-from . import checksec
-from . import common
-from . import constgrep
-from . import cyclic
-from . import disasm
-from . import elfdiff
-from . import elfpatch
-from . import errno
-from . import hex
-from . import phd
-from . import pwnstrip
-from . import scramble
-from . import shellcraft
-from . import unhex
-from . import update
-from .common import parser
+from __future__ import absolute_import
+
+import sys
+
+from pwnlib.commandline import asm
+from pwnlib.commandline import checksec
+from pwnlib.commandline import common
+from pwnlib.commandline import constgrep
+from pwnlib.commandline import cyclic
+from pwnlib.commandline import disasm
+from pwnlib.commandline import elfdiff
+from pwnlib.commandline import elfpatch
+from pwnlib.commandline import errno
+from pwnlib.commandline import hex
+from pwnlib.commandline import phd
+from pwnlib.commandline import pwnstrip
+from pwnlib.commandline import scramble
+from pwnlib.commandline import shellcraft
+from pwnlib.commandline import unhex
+from pwnlib.commandline import update
+from pwnlib.commandline.common import parser
+from pwnlib.context import context
 
 commands = {
     'asm': asm.main,
@@ -36,7 +41,8 @@ commands = {
 
 def main():
     args = parser.parse_args()
-    commands[args.command](args)
+    with context.local(log_console = sys.stderr):
+        commands[args.command](args)
 
 if __name__ == '__main__':
     main()

pwnlib/commandline/phd.py

@@ -1,11 +1,15 @@
 #!/usr/bin/env python2
+from __future__ import absolute_import
+
 import argparse
 import os
 import sys
 
-from pwn import *
+import pwnlib
+pwnlib.args.free_form = False
 
-from . import common
+from pwn import *
+from pwnlib.commandline import common
 
 parser = common.parser_commands.add_parser(
     'phd',

pwnlib/commandline/pwnstrip.py

@@ -1,8 +1,12 @@
+from __future__ import absolute_import
+
 import argparse
 
-from pwn import *
+import pwnlib
+pwnlib.args.free_form = False
 
-from . import common
+from pwn import *
+from pwnlib.commandline import common
 
 p = common.parser_commands.add_parser(
     'pwnstrip',

pwnlib/commandline/scramble.py

@@ -1,9 +1,13 @@
+from __future__ import absolute_import
+
 import argparse
 import sys
 
-from pwn import *
+import pwnlib
+pwnlib.args.free_form = False
 
-from . import common
+from pwn import *
+from pwnlib.commandline import common
 
 parser = common.parser_commands.add_parser(
     'scramble',

pwnlib/commandline/shellcraft.py

@@ -1,12 +1,16 @@
 #!/usr/bin/env python2
+from __future__ import absolute_import
+
 import argparse
 import os
 import sys
 import types
 
-from pwn import *
+import pwnlib
+pwnlib.args.free_form = False
 
-from . import common
+from pwn import *
+from pwnlib.commandline import common
 
 
 #  ____  _          _ _                 __ _
@@ -57,9 +61,10 @@ p.add_argument(
                'p',
                'i', 'hexii',
                'e', 'elf',
+               'd', 'escaped',
                'default'],
     default = 'default',
-    help = 'Output format (default: hex), choose from {r}aw, {s}tring, {c}-style array, {h}ex string, hex{i}i, {a}ssembly code, {p}reprocssed code',
+    help = 'Output format (default: hex), choose from {e}lf, {r}aw, {s}tring, {c}-style array, {h}ex string, hex{i}i, {a}ssembly code, {p}reprocssed code, escape{d} hex string',
 )
 
 p.add_argument(
@@ -167,7 +172,7 @@ def get_template(name):
 
 def is_not_a_syscall_template(name):
     template_src = shellcraft._get_source(name)
-    return 'man 2' not in read(template_src)
+    return '/syscalls' not in template_src
 
 def main(args):
     if args.list:
@@ -329,7 +334,8 @@ def main(args):
         code = pwnlib.util.fiddling.enhex(code) + '\n'
     elif args.format in ['i', 'hexii']:
         code = hexii(code) + '\n'
-
+    elif args.format in ['d', 'escaped']:
+        code = ''.join('\\x%02x' % ord(c) for c in code) + '\n'
     if not sys.stdin.isatty():
         args.out.write(sys.stdin.read())
 

pwnlib/commandline/unhex.py

@@ -1,9 +1,11 @@
 #!/usr/bin/env python2
+from __future__ import absolute_import
+
 import argparse
 import sys
 from string import whitespace
 
-from . import common
+from pwnlib.commandline import common
 
 parser = common.parser_commands.add_parser(
     'unhex',

pwnlib/commandline/update.py

@@ -1,9 +1,13 @@
 #!/usr/bin/env python2
+from __future__ import absolute_import
+
 import subprocess
 
-from pwn import *
+import pwnlib
+pwnlib.args.free_form = False
 
-from . import common
+from pwn import *
+from pwnlib.commandline import common
 
 p = common.parser_commands.add_parser(
     'update',
@@ -21,7 +25,7 @@ p.add_argument('--pre', action='store_true', help='''
 def main(a):
     result = pwnlib.update.perform_check(prerelease=a.pre)
     if a.install:
-        subprocess.check_call(result, shell=True)
+        subprocess.check_call(result, shell=False)
 
 if __name__ == '__main__':
     pwnlib.common.main(__file__)

pwnlib/constants/__init__.py

@@ -40,13 +40,15 @@ Example:
     4
 
 """
+from __future__ import absolute_import
+
 import importlib
 import sys
 from types import ModuleType
 
-from ..context import context
-from ..util import safeeval
-from .constant import Constant
+from pwnlib.constants.constant import Constant
+from pwnlib.context import context
+from pwnlib.util import safeeval
 
 
 class ConstantsModule(ModuleType):
@@ -137,6 +139,11 @@ class ConstantsModule(ModuleType):
         if not isinstance(string, str):
             return string
 
+        simple = getattr(self, string, None)
+
+        if simple is not None:
+            return simple
+
         key = context.os, context.arch
         if key not in self._env_store:
             self._env_store[key] = {key: getattr(self, key) for key in dir(self) if not key.endswith('__')}

pwnlib/context/__init__.py

@@ -4,12 +4,15 @@
 Implements context management so that nested/scoped contexts and threaded
 contexts work properly and as expected.
 """
+from __future__ import absolute_import
+
 import collections
 import functools
 import logging
 import os
 import platform
 import socket
+import stat
 import string
 import subprocess
 import sys
@@ -18,8 +21,8 @@ import time
 
 import socks
 
-from ..device import Device
-from ..timeout import Timeout
+from pwnlib.device import Device
+from pwnlib.timeout import Timeout
 
 _original_socket = socket.socket
 
@@ -260,7 +263,7 @@ class ContextType(object):
     r"""
     Class for specifying information about the target machine.
     Intended for use as a pseudo-singleton through the global
-    variable ``pwnlib.context.context``, available via
+    variable :data:`.context`, available via
     ``from pwn import *`` as ``context``.
 
     The context is usually specified at the top of the Python file for clarity. ::
@@ -329,12 +332,16 @@ class ContextType(object):
         'aslr': True,
         'binary': None,
         'bits': 32,
+        'buffer_size': 4096,
+        'delete_corefiles': False,
         'device': os.getenv('ANDROID_SERIAL', None) or None,
         'endian': 'little',
         'kernel': None,
         'log_level': logging.INFO,
         'log_file': _devnull(),
+        'log_console': sys.stdout,
         'randomize': False,
+        'rename_corefiles': True,
         'newline': '\n',
         'noptrace': False,
         'os': 'linux',
@@ -525,12 +532,13 @@ class ContextType(object):
     def quiet(self, function=None):
         """Disables all non-error logging within the enclosed scope,
         *unless* the debugging level is set to 'debug' or lower."""
-        if not function:
-            level = 'error'
-            if context.log_level <= logging.DEBUG:
-                level = None
-            return self.local(function, log_level=level)
+        level = 'error'
+        if context.log_level <= logging.DEBUG:
+            level = None
+        return self.local(function, log_level=level)
 
+    def quietfunc(self, function):
+        """Similar to :attr:`quiet`, but wraps a whole function."""
         @functools.wraps(function)
         def wrapper(*a, **kw):
             level = 'error'
@@ -560,6 +568,7 @@ class ContextType(object):
         Examples:
 
             >>> # Default value
+            >>> context.clear()
             >>> context.arch == 'i386'
             True
             >>> context.arch = 'arm'
@@ -686,7 +695,7 @@ class ContextType(object):
         """
         ASLR settings for new processes.
 
-        If ``False``, attempt to disable ASLR in all processes which are
+        If :const:`False`, attempt to disable ASLR in all processes which are
         created via ``personality`` (``setarch -R``) and ``setrlimit``
         (``ulimit -s unlimited``).
 
@@ -753,7 +762,7 @@ class ContextType(object):
 
         """
         # Cyclic imports... sorry Idolf.
-        from ..elf     import ELF
+        from pwnlib.elf     import ELF
 
         if not isinstance(binary, ELF):
             binary = ELF(binary)
@@ -922,6 +931,24 @@ class ContextType(object):
         value.flush()
         return value
 
+    @_validator
+    def log_console(self, stream):
+        """
+        Sets the default logging console target.
+
+        Examples:
+
+            >>> context.log_level = 'warn'
+            >>> log.warn("Hello")
+            [!] Hello
+            >>> context.log_console=open('/dev/null', 'w')
+            >>> log.warn("Hello")
+            >>> context.clear()
+        """
+        if isinstance(stream, str):
+            stream = open(stream, 'wt')
+        return stream
+
     @property
     def mask(self):
         return (1 << self.bits) - 1
@@ -964,7 +991,7 @@ class ContextType(object):
 
         Can be set to any non-string truthy value, or the specific string
         values ``'signed'`` or ``'unsigned'`` which are converted into
-        ``True`` and ``False`` correspondingly.
+        :const:`True` and :const:`False` correspondingly.
 
         Examples:
 
@@ -1107,8 +1134,13 @@ class ContextType(object):
 
     @property
     def adb(self):
-        """Returns an argument array for connecting to adb."""
-        command = ['adb']
+        """Returns an argument array for connecting to adb.
+
+        Unless ``$ADB_PATH`` is set, uses the default ``adb`` binary in ``$PATH``.
+        """
+        ADB_PATH = os.environ.get('ADB_PATH', 'adb')
+
+        command = [ADB_PATH]
 
         if self.adb_host != self.defaults['adb_host']:
             command += ['-H', self.adb_host]
@@ -1121,6 +1153,79 @@ class ContextType(object):
 
         return command
 
+    @_validator
+    def buffer_size(self, size):
+        """Internal buffer size to use for :class:`pwnlib.tubes.tube.tube` objects.
+
+        This is not the maximum size of the buffer, but this is the amount of data
+        which is passed to each raw ``read`` syscall (or equivalent).
+        """
+        return int(size)
+
+    @property
+    def cache_dir(self):
+        """Directory used for caching data.
+
+        Note:
+            May be either a path string, or :const:`None`.
+
+        Example:
+
+            >>> cache_dir = context.cache_dir
+            >>> cache_dir is not None
+            True
+            >>> os.chmod(cache_dir, 0o000)
+            >>> context.cache_dir is None
+            True
+            >>> os.chmod(cache_dir, 0o755)
+            >>> cache_dir == context.cache_dir
+            True
+        """
+        home = os.path.expanduser('~')
+
+        if not os.access(home, os.W_OK):
+            return None
+
+        cache = os.path.join(home, '.pwntools-cache')
+
+        if not os.path.exists(cache):
+            try:
+                os.mkdir(cache)
+            except OSError:
+                return None
+
+        # Some wargames e.g. pwnable.kr have created dummy directories
+        # which cannot be modified by the user account (owned by root).
+        if not os.access(cache, os.W_OK):
+            return None
+
+        return cache
+
+    @_validator
+    def delete_corefiles(self, v):
+        """Whether pwntools automatically deletes corefiles after exiting.
+        This only affects corefiles accessed via :attr:`.process.corefile`.
+
+        Default value is ``False``.
+        """
+        return bool(v)
+
+    @_validator
+    def rename_corefiles(self, v):
+        """Whether pwntools automatically renames corefiles.
+
+        This is useful for two things:
+
+        - Prevent corefiles from being overwritten, if ``kernel.core_pattern``
+          is something simple like ``"core"``.
+        - Ensure corefiles are generated, if ``kernel.core_pattern`` uses ``apport``,
+          which refuses to overwrite any existing files.
+
+        This only affects corefiles accessed via :attr:`.process.corefile`.
+
+        Default value is ``True``.
+        """
+        return bool(v)
 
     #*************************************************************************
     #                               ALIASES
@@ -1196,11 +1301,14 @@ class ContextType(object):
     Thread = Thread
 
 
-#: Global ``context`` object, used to store commonly-used pwntools settings.
+#: Global :class:`.ContextType` object, used to store commonly-used pwntools settings.
+#:
 #: In most cases, the context is used to infer default variables values.
-#: For example, :meth:`pwnlib.asm.asm` can take an ``os`` parameter as a
-#: keyword argument.  If it is not supplied, the ``os`` specified by
-#: ``context`` is used instead.
+#: For example, :func:`.asm` can take an ``arch`` parameter as a
+#: keyword argument.
+#:
+#: If it is not supplied, the ``arch`` specified by ``context`` is used instead.
+#:
 #: Consider it a shorthand to passing ``os=`` and ``arch=`` to every single
 #: function call.
 context = ContextType()

pwnlib/data/syscalls/Makefile

@@ -0,0 +1,10 @@
+ROOT=$(shell git rev-parse --show-toplevel)
+TEMPLATE_DIR="$ROOT/pwnlib/shellcraft/templates/common/linux/syscalls"
+
+all: generate.py functions.py
+	python generate.py "$(ROOT)/pwnlib/shellcraft/templates/common/linux/syscalls"
+
+functions.py:
+	wget https://raw.githubusercontent.com/zachriggle/functions/master/functions.py
+
+.phony: all

pwnlib/data/syscalls/generate.py

@@ -0,0 +1,283 @@
+#!/usr/bin/env python2
+import argparse
+import os
+
+from pwnlib import constants
+from pwnlib.context import context
+
+# github.com/zachriggle/functions
+from functions import functions, Function, Argument
+
+ARCHITECTURES = ['i386', 'amd64', 'arm', 'aarch64', 'mips']
+
+HEADER = '''
+<%
+import collections
+import pwnlib.abi
+import pwnlib.constants
+import pwnlib.shellcraft
+%>
+'''
+
+DOCSTRING = '''
+<%docstring>{name}({arguments_comma_separated}) -> str
+
+Invokes the syscall {name}.
+
+See 'man 2 {name}' for more information.
+
+Arguments:
+{arg_docs}
+Returns:
+    {return_type}
+</%docstring>
+'''
+
+ARGUMENTS = """
+<%page args="{arguments_default_values}"/>
+"""
+
+CALL = """
+<%
+    abi = pwnlib.abi.ABI.syscall()
+    stack = abi.stack
+    regs = abi.register_arguments[1:]
+    allregs = pwnlib.shellcraft.registers.current()
+
+    can_pushstr = {string_arguments!r}
+    can_pushstr_array = {array_arguments!r}
+
+    argument_names = {argument_names!r}
+    argument_values = [{arguments_comma_separated!s}]
+
+    # Load all of the arguments into their destination registers / stack slots.
+    register_arguments = dict()
+    stack_arguments = collections.OrderedDict()
+    string_arguments = dict()
+    dict_arguments = dict()
+    array_arguments = dict()
+    syscall_repr = []
+
+    for name, arg in zip(argument_names, argument_values):
+        if arg is not None:
+            syscall_repr.append('%s=%r' % (name, arg))
+
+        # If the argument itself (input) is a register...
+        if arg in allregs:
+            index = argument_names.index(name)
+            if index < len(regs):
+                target = regs[index]
+                register_arguments[target] = arg
+            elif arg is not None:
+                stack_arguments[index] = arg
+
+        # The argument is not a register.  It is a string value, and we
+        # are expecting a string value
+        elif name in can_pushstr and isinstance(arg, str):
+            string_arguments[name] = arg
+
+        # The argument is not a register.  It is a dictionary, and we are
+        # expecting K:V paris.
+        elif name in can_pushstr_array and isinstance(arg, dict):
+            array_arguments[name] = ['%s=%s' % (k,v) for (k,v) in arg.items()]
+
+        # The arguent is not a register.  It is a list, and we are expecting
+        # a list of arguments.
+        elif name in can_pushstr_array and isinstance(arg, (list, tuple)):
+            array_arguments[name] = arg
+
+        # The argument is not a register, string, dict, or list.
+        # It could be a constant string ('O_RDONLY') for an integer argument,
+        # an actual integer value, or a constant.
+        else:
+            index = argument_names.index(name)
+            if index < len(regs):
+                target = regs[index]
+                register_arguments[target] = arg
+            elif arg is not None:
+                stack_arguments[target] = arg
+
+    # Some syscalls have different names on various architectures.
+    # Determine which syscall number to use for the current architecture.
+    for syscall in {syscalls!r}:
+        if hasattr(pwnlib.constants, syscall):
+            break
+    else:
+        raise Exception("Could not locate any syscalls: %r" % syscalls)
+%>
+    /* {name}(${{', '.join(syscall_repr)}}) */
+%for name, arg in string_arguments.items():
+    ${{pwnlib.shellcraft.pushstr(arg, append_null=('\\x00' not in arg))}}
+    ${{pwnlib.shellcraft.mov(regs[argument_names.index(name)], abi.stack)}}
+%endfor
+%for name, arg in array_arguments.items():
+    ${{pwnlib.shellcraft.pushstr_array(regs[argument_names.index(name)], arg)}}
+%endfor
+%for name, arg in stack_arguments.items():
+    ${{pwnlib.shellcraft.push(arg)}}
+%endfor
+    ${{pwnlib.shellcraft.setregs(register_arguments)}}
+    ${{pwnlib.shellcraft.syscall(syscall)}}
+"""
+
+
+def can_be_constant(arg):
+    if arg.derefcnt == 0:
+        return True
+
+
+def can_be_string(arg):
+    if arg.type == 'char' and arg.derefcnt == 1:
+        return True
+    if arg.type == 'void' and arg.derefcnt == 1:
+        return True
+
+def can_be_array(arg):
+    if arg.type == 'char' and arg.derefcnt == 2:
+        return True
+    if arg.type == 'void' and arg.derefcnt == 2:
+        return True
+
+
+def fix_bad_arg_names(func, arg):
+    if arg.name == 'str':
+        return 'str_'
+    if arg.name == 'len':
+        return 'length'
+    if arg.name == 'repr':
+        return 'repr_'
+
+    if func.name == 'open' and arg.name == 'vararg':
+        return 'mode'
+
+    return arg.name
+
+
+def get_arg_default(arg):
+    return 0
+
+def fix_rt_syscall_name(name):
+    if name.startswith('rt_'):
+        return name[3:]
+    return name
+
+def fix_syscall_names(name):
+    # Do not use old_mmap
+    if name == 'SYS_mmap':
+        return ['SYS_mmap2', name]
+    # Some arches don't have vanilla sigreturn
+    if name.endswith('_sigreturn'):
+        return ['SYS_sigreturn', 'SYS_rt_sigreturn']
+    return [name]
+
+
+def main(target):
+    for arch in ARCHITECTURES:
+        with context.local(arch=arch):
+            generate_one(target)
+
+def generate_one(target):
+    SYSCALL_NAMES = [c for c in dir(constants) if c.startswith('SYS_')]
+
+    for syscall in SYSCALL_NAMES:
+        name = syscall[4:]
+
+        # Skip anything with uppercase
+        if name.lower() != name:
+            print 'Skipping %s' % name
+            continue
+
+        # Skip anything that starts with 'unused' or 'sys' after stripping
+        if name.startswith('unused'):
+            print 'Skipping %s' % name
+            continue
+
+        function = functions.get(name, None)
+
+        if name.startswith('rt_'):
+            name = name[3:]
+
+        # If we can't find a function, just stub it out with something
+        # that has a vararg argument.
+        if function is None:
+            print 'Stubbing out %s' % name
+            args = [Argument('int', 0, 'vararg')]
+            function = Function('long', 0, name, args)
+
+        # Some syscalls have different names on different architectures,
+        # or are superceded.  We try to do the "best" thing at runtime.
+        syscalls = fix_syscall_names(syscall)
+
+        # Set up the argument string for Mako
+        argument_names = []
+        argument_defaults = []
+
+        #
+
+        for arg in function.args:
+            argname = fix_bad_arg_names(function, arg)
+            default = get_arg_default(arg)
+
+            # Mako is unable to use *vararg and *kwarg, so we just stub in
+            # a whole bunch of additional arguments.
+            if argname == 'vararg':
+                for j in range(5):
+                    argname = 'vararg_%i' % j
+                    argument_names.append(argname)
+                    argument_defaults.append('%s=%s' % (argname, None))
+                break
+
+            argument_names.append(argname)
+            argument_defaults.append('%s=%s' % (argname, default))
+
+        arguments_default_values = ', '.join(argument_defaults)
+        arguments_comma_separated = ', '.join(argument_names)
+
+        string_arguments = []
+        array_arguments = []
+        arg_docs = []
+
+        for arg in function.args:
+
+            if can_be_array(arg):
+                array_arguments.append(arg.name)
+
+            if can_be_string(arg):
+                string_arguments.append(arg.name)
+
+            argname = arg.name
+            argtype = str(arg.type) + ('*' * arg.derefcnt)
+            arg_docs.append(
+                '    {argname}({argtype}): {argname}'.format(argname=argname,
+                                                             argtype=argtype))
+
+        return_type = str(function.type) + ('*' * function.derefcnt)
+        arg_docs = '\n'.join(arg_docs)
+
+        template_variables = {
+            'name': name,
+            'arg_docs': arg_docs,
+            'syscalls': syscalls,
+            'arguments_default_values': arguments_default_values,
+            'arguments_comma_separated': arguments_comma_separated,
+            'return_type': return_type,
+            'string_arguments': string_arguments,
+            'array_arguments': array_arguments,
+            'argument_names': argument_names,
+        }
+
+        lines = [
+            HEADER,
+            DOCSTRING.format(**template_variables),
+            ARGUMENTS.format(**template_variables),
+            CALL.format(**template_variables)
+        ]
+
+        with open(os.path.join(target, name + '.asm'), 'wt+') as f:
+            f.write('\n'.join(map(str.strip, lines)))
+
+if __name__ == '__main__':
+    p = argparse.ArgumentParser()
+    p.add_argument('target_directory')
+    args = p.parse_args()
+    main(args.target_directory)

pwnlib/dynelf.py

@@ -47,20 +47,22 @@ Example
 
 DynELF
 """
+from __future__ import absolute_import
+
 import ctypes
 
 from elftools.elf.enums import ENUM_D_TAG
 
-from . import elf
-from . import libcdb
-from .context import context
-from .elf import ELF
-from .elf import constants
-from .log import getLogger
-from .memleak import MemLeak
-from .util.fiddling import enhex
-from .util.packing import unpack
-from .util.web import wget
+from pwnlib import elf
+from pwnlib import libcdb
+from pwnlib.context import context
+from pwnlib.elf import ELF
+from pwnlib.elf import constants
+from pwnlib.log import getLogger
+from pwnlib.memleak import MemLeak
+from pwnlib.util.fiddling import enhex
+from pwnlib.util.packing import unpack
+from pwnlib.util.web import wget
 
 log    = getLogger(__name__)
 sizeof = ctypes.sizeof
@@ -512,7 +514,7 @@ class DynELF(object):
               If set to ``'libc'``, ``'libc.so'`` is assumed.
 
         Returns:
-            Address of the named symbol, or ``None``.
+            Address of the named symbol, or :const:`None`.
         """
         result = None
 

pwnlib/elf/__init__.py

@@ -1,6 +1,14 @@
-from .corefile import Core
-from .datatypes import *
-from .elf import ELF
-from .elf import load
+"""
+Most exploitable CTF challenges are provided in the Executable and Linkable
+Format (``ELF``).  Generally, it is very useful to be able to interact with
+these files to extract data such as function addresses, ROP gadgets, and
+writable page addresses.
+"""
+from __future__ import absolute_import
+
+from pwnlib.elf.corefile import Core
+from pwnlib.elf.datatypes import *
+from pwnlib.elf.elf import ELF
+from pwnlib.elf.elf import load
 
 __all__ = ['load', 'ELF', 'Core'] + sorted(filter(lambda x: not x.startswith('_'), datatypes.__dict__.keys()))

pwnlib/elf/config.py

@@ -0,0 +1,258 @@
+"""
+Kernel-specific ELF functionality
+"""
+
+
+class KernelConfig(object):
+    def __init__(self, name, title, requires=[], excludes=[], minver=0, maxver=99):
+
+        #: Name of the configuration option
+        self.name = name
+
+        #: Section to which the configuration point belongs
+        self.title = title
+
+        #: List of configuration items, one of which must be present,
+        #: for this checker to be used.
+        self.requires = set(requires)
+
+        #: List of configuration items, NONE of which must be present,
+        #: for this checker to be used.
+        self.excludes = set(excludes)
+
+        #: Kernel version that this check should be enforced on
+        self.minver = map(int, str(minver).split('.'))
+        self.maxver = map(int, str(maxver).split('.'))
+
+    def relevant(self, config):
+
+        # If any of the excluded options are ENABLED,
+        # this config is not relevant.
+        if self.excludes:
+            for value in self.excludes & set(config):
+                if config.get(value, False):
+                    return False
+
+        # If any of the required options are PRESENT but DISABLED,
+        # this config is not relevant.
+        if self.requires:
+            for value in self.requires & set(config):
+                if config.get(value, False):
+                    return True
+
+            # We did not find a required value
+            return False
+
+        # If we are not in the correct version range, bail
+        if 'version' in config:
+            version = config['version']
+
+            if not (self.minver <= version < self.maxver):
+                return False
+
+        return True
+
+    def check(self, value):
+        raise NotImplementedError()
+
+    def __call__(self, config):
+        """__call__(config) -> str
+
+        Check whether the configuration point is set correctly.
+
+        Arguments:
+            config(dict): Dictionary of all configuration points
+
+        Returns:
+            A tuple (result, message) where result is whether the
+            option is correctly configured, and message is an optional
+            message describing the error.
+        """
+        if not self.relevant(config):
+            return (True, '')
+        return self.check(config.get(self.name, None))
+
+class Enabled(KernelConfig):
+    def __init__(self, name, **kw):
+        super(Enabled, self).__init__(name, 'Not Enabled', **kw)
+    def check(self, config):
+        return (bool(config), '')
+
+class EnabledIfPresent(Enabled):
+    def check(self, config):
+        if config is None:
+            return (True, '')
+        return super(EnabledIfPresent, self).check(config)
+
+class Disabled(KernelConfig):
+    def __init__(self, name, **kw):
+        super(Disabled, self).__init__(name, 'Not Disabled', **kw)
+    def check(self, config):
+        return (not bool(config), '')
+
+class Minimum(KernelConfig):
+    def __init__(self, name, minimum, **kw):
+        super(Minimum, self).__init__(name, 'Misconfigured', **kw)
+        self.minimum = minimum
+    def check(self, value):
+        if value is None or value < self.minimum:
+            return (False, "Should be at least {}".format(self.minimum))
+        return (True, '')
+
+kernel_configuration = [
+# Enabled
+    Enabled('STRICT_DEVMEM', requires=['DEVMEM']),
+    Enabled('IO_STRICT_DEVMEM', requires=['DEVMEM']),
+    Enabled('SYN_COOKIES'),
+    Enabled('DEBUG_CREDENTIALS'),
+    Enabled('DEBUG_NOTIFIERS'),
+    Enabled('DEBUG_LIST'),
+    Enabled('SECCOMP'),
+    Enabled('SECCOMP_FILTER', minver=3.5, requires=['HAVE_ARCH_SECCOMP_FILTER']),
+    Enabled('SECURITY'),
+    # Enabled('SECURITY_YAMA'),
+    Enabled('HARDENED_USERCOPY', minver=4.8),
+    Enabled('SLAB_FREELIST_RANDOM', minver=4.7),
+    # Enabled('SLUB_DEBUG'),
+    # Enabled('PAGE_POISONING'),
+    # Enabled('PAGE_POISONING_NO_SANITY'),
+    # Enabled('PAGE_POISONING_ZERO'),
+    Enabled('CC_STACKPROTECTOR', excludes=['CC_STACKPROTECTOR_STRONG']),
+    Enabled('CC_STACKPROTECTOR_STRONG', minver=3.14),
+    Enabled('PANIC_ON_OOPS', minver=3.5),
+    Enabled('DEBUG_SET_MODULE_RONX', requires=['MODULES']),
+    Enabled('DEBUG_RODATA'),
+    Enabled('SECURITY_KPTR_RESTRICT'),
+    Enabled('SECURITY_PERF_EVENTS_RESTRICT'),
+    Enabled('SECURITY_DMESG_RESTRICT'),
+    Enabled('SECURITY_NETWORK'),
+    Enabled('SECURITY_SELINUX'),
+    EnabledIfPresent('MSM_KERNEL_PROTECT'),
+
+# Not really security relevant, but come on guys
+    Enabled('COREDUMP'),
+
+# Disabled
+    Disabled('ACPI_CUSTOM_METHOD'),
+    Disabled('COMPAT_BRK'),
+    Disabled('DEVKMEM'),
+    Disabled('DEVMEM'),
+    Disabled('COMPAT_VDSO'),
+    Disabled('KEXEC'),
+    Disabled('BINFMT_MISC'),
+    Disabled('LEGACY_PTYS'),
+    Disabled('MODULES'),
+    Disabled('KEXEC'),
+    Disabled('SECURITY_SELINUX_DISABLE'),
+    Disabled('MISC_FILESYSTEMS'),
+    Disabled('SCSI_TGT'),
+    Disabled('SCSI_CONSTANTS'),
+    Disabled('SCSI_LOGGING'),
+    Disabled('SCSI_SCAN_ASYNC'),
+    Disabled('CONFIG_MEDIA_RADIO_SUPPORT'),
+    Disabled('CONFIG_PFT'),
+    Disabled('CONFIG_SYSVIPC'),
+
+
+    # Permits reloading the kernel from disk
+    Disabled('HIBERNATION'),
+
+    # Prior to v4.1, assists heap memory attacks; best to keep interface disabled
+    Disabled('INET_DIAG', maxver=4.1),
+
+
+# x86-specific
+    Enabled('HIGHMEM64G', requires=['X86_32']),
+    Enabled('X86_PAE', requires=['X86_32']),
+    Disabled('LEGACY_VSYSCALL_NONE', requires=['X86_32', 'X86_64'], minver=4.4),
+    Disabled('CONFIG_IA32_EMULATION'),
+    Disabled('X86_X32'),
+    Disabled('MODIFY_LDT_SYSCALL'),
+    Minimum('DEFAULT_MMAP_MIN_ADDR', 65536, requires=['X86_32', 'X86_64']),
+    Enabled('RANDOMIZE_BASE', excludes=['ARM']),
+    Enabled('RANDOMIZE_MEMORY', requires=['X86_64']),
+
+# ARM specific
+    Enabled('VMSPLIT_3G', requires=['ARM']),
+    Disabled('STRICT_MEMORY_RWX', requires=['ARM', 'ARM64']),
+    Minimum('DEFAULT_MMAP_MIN_ADDR', 32768, requires=['ARM', 'ARM64']),
+    Minimum('ARCH_MMAP_RND_BITS', 16, requires=['ARM']),
+    Minimum('ARCH_MMAP_RND_BITS', 24, requires=['ARM64']),
+    Minimum('ARCH_MMAP_RND_COMPAT_BITS', 16, requires=['ARM64']),
+    Enabled('CPU_SW_DOMAIN_PAN', requires=['ARM'], minver=4.3),
+    Enabled('CONFIG_ARM64_PAN', requires=['ARM64'], minver=4.3),
+    Disabled('OABI_COMPAT'),
+
+# Only relevant of MODULES are enabled
+    Enabled('MODULE_SIG', requires=['MODULES']),
+    Enabled('MODULE_SIG_FORCE', requires=['MODULES']),
+    Enabled('MODULE_SIG_ALL', requires=['MODULES']),
+    Enabled('MODULE_SIG_SHA512', requires=['MODULES']),
+    Enabled('MODULE_SIG_KEY', requires=['MODULES']),
+
+]
+
+def parse_kconfig(data):
+    """Parses configuration data from a kernel .config.
+
+    Arguments:
+        data(str): Configuration contents.
+
+    Returns:
+        A :class:`dict`  mapping configuration options.
+        "Not set" is converted into ``None``, ``y`` and ``n`` are converted
+        into :class:`bool`.  Numbers are converted into :class:`int`.
+        All other values are as-is.
+        Each key has ``CONFIG_`` stripped from the beginning.
+
+    Examples:
+
+        >>> parse_kconfig('FOO=3')
+        {'FOO': 3}
+        >>> parse_kconfig('FOO=y')
+        {'FOO': True}
+        >>> parse_kconfig('FOO=n')
+        {'FOO': False}
+        >>> parse_kconfig('FOO=bar')
+        {'FOO': 'bar'}
+        >>> parse_kconfig('# FOO is not set')
+        {'FOO': None}
+    """
+    config = {}
+
+    NOT_SET=' is not set'
+
+    if not data:
+        return
+
+    for line in data.splitlines():
+
+        # Not set? Then set to None.
+        if NOT_SET in line:
+            line = line.split(NOT_SET, 1)[0]
+            name = line.strip('# ')
+            config[name] = None
+
+        # Set to a value? Extract it
+        if '=' in line:
+            k, v = line.split('=', 1)
+
+            # Boolean conversions
+            if v == 'y':
+                v = True
+            elif v == 'n':
+                v = False
+            else:
+
+                # Integer conversions
+                try:
+                    v = int(v, 0)
+                except ValueError:
+                    pass
+
+            config[k] = v
+
+
+    # Strip off all of the CONFIG_ prefixes
+    config = ({k.replace('CONFIG_', ''): v for k,v in config.items()})
+    return config

pwnlib/elf/datatypes.py

@@ -533,19 +533,102 @@ class user_regs_struct_amd64(ctypes.Structure):
 
 assert ctypes.sizeof(user_regs_struct_amd64) == 0xd8
 
+class user_regs_struct_arm(ctypes.Structure):
+    _fields_ = [('r%i' % i, ctypes.c_uint32) for i in range(18)]
+
+    @property
+    def cpsr(self):
+        return self.r16
+    @property
+    def pc(self):
+        return self.r15
+    @property
+    def lr(self):
+        return self.r14
+    @property
+    def sp(self):
+        return self.r13
+    @property
+    def ip(self):
+        return self.r12
+    @property
+    def fp(self):
+        return self.r11
+
+
+class user_regs_struct_aarch64(ctypes.Structure):
+    _fields_ = [('x%i' % i, ctypes.c_uint64) for i in range(31)] \
+             + [('sp', ctypes.c_uint64),
+                ('pc', ctypes.c_uint64),
+                ('pstate', ctypes.c_uint64)]
+
+    @property
+    def lr(self):
+        return self.x30
+
+    def __getattr__(self, name):
+        if name.startswith('r'):
+            name = 'x' + name[1:]
+            return getattr(name) & 0xffffffff
+
 class elf_prstatus_i386(ctypes.Structure):
     _fields_ = generate_prstatus_common(32, user_regs_struct_i386)
 
 assert ctypes.sizeof(elf_prstatus_i386) == 0x90
 
 class elf_prstatus_amd64(ctypes.Structure):
-    _fields_ = generate_prstatus_common(64, user_regs_struct_amd64)
+    _fields_ = generate_prstatus_common(64, user_regs_struct_amd64) \
+             + [('padding', ctypes.c_uint32)]
 
 assert ctypes.sizeof(elf_prstatus_amd64) == 0x150
 
+class elf_prstatus_arm(ctypes.Structure):
+    _fields_ = generate_prstatus_common(32, user_regs_struct_arm)
+
+class elf_prstatus_aarch64(ctypes.Structure):
+    _fields_ = generate_prstatus_common(64, user_regs_struct_aarch64)
+
 class Elf32_auxv_t(ctypes.Structure):
     _fields_ = [('a_type', ctypes.c_uint32),
                 ('a_val', ctypes.c_uint32),]
 class Elf64_auxv_t(ctypes.Structure):
     _fields_ = [('a_type', ctypes.c_uint64),
                 ('a_val', ctypes.c_uint64),]
+
+def generate_prspinfo(long):
+    return [
+        ('pr_state', byte),
+        ('pr_sname', char),
+        ('pr_zomb', byte),
+        ('pr_nice', byte),
+        ('pr_flag', long),
+        ('pr_uid', ctypes.c_ushort),
+        ('pr_gid', ctypes.c_ushort),
+        ('pr_pid', ctypes.c_int),
+        ('pr_ppid', ctypes.c_int),
+        ('pr_pgrp', ctypes.c_int),
+        ('pr_sid', ctypes.c_int),
+        ('pr_fname', char * 16),
+        ('pr_psargs', char * 80)
+    ]
+
+class elf_prspinfo_32(ctypes.Structure):
+    _fields_ = generate_prspinfo(Elf32_Addr)
+
+class elf_prspinfo_64(ctypes.Structure):
+    _fields_ = generate_prspinfo(Elf64_Addr)
+
+def generate_siginfo(int_t, long_t):
+    class siginfo_t(ctypes.Structure):
+        _fields_ = [('si_signo', int_t),
+                    ('si_errno', int_t),
+                    ('si_code', int_t),
+                    ('sigfault_addr', long_t),
+                    ('sigfault_trapno', int_t)]
+    return siginfo_t
+
+class elf_siginfo_32(generate_siginfo(ctypes.c_uint32, ctypes.c_uint32)):
+    pass
+
+class elf_siginfo_64(generate_siginfo(ctypes.c_uint32, ctypes.c_uint64)):
+    pass

pwnlib/encoders/__init__.py

@@ -2,14 +2,16 @@
 """
 Encode shellcode to avoid input filtering and impress your friends!
 """
-from . import amd64
-from . import arm
-from . import i386
-from . import mips
-from .encoder import Encoder
-from .encoder import alphanumeric
-from .encoder import encode
-from .encoder import line
-from .encoder import null
-from .encoder import printable
-from .encoder import scramble
+from __future__ import absolute_import
+
+from pwnlib.encoders import amd64
+from pwnlib.encoders import arm
+from pwnlib.encoders import i386
+from pwnlib.encoders import mips
+from pwnlib.encoders.encoder import Encoder
+from pwnlib.encoders.encoder import alphanumeric
+from pwnlib.encoders.encoder import encode
+from pwnlib.encoders.encoder import line
+from pwnlib.encoders.encoder import null
+from pwnlib.encoders.encoder import printable
+from pwnlib.encoders.encoder import scramble

pwnlib/encoders/amd64/__init__.py

@@ -1 +1,3 @@
-from . import delta
+from __future__ import absolute_import
+
+from pwnlib.encoders.amd64 import delta

pwnlib/encoders/amd64/delta.py

@@ -1,4 +1,6 @@
-from ..i386.delta import i386DeltaEncoder
+from __future__ import absolute_import
+
+from pwnlib.encoders.i386.delta import i386DeltaEncoder
 
 
 class amd64DeltaEncoder(i386DeltaEncoder):

pwnlib/encoders/arm/__init__.py

@@ -1,2 +1,4 @@
-from . import alphanumeric
-from . import xor
+from __future__ import absolute_import
+
+from pwnlib.encoders.arm import alphanumeric
+from pwnlib.encoders.arm import xor

pwnlib/encoders/arm/alphanumeric/__init__.py

@@ -1,12 +1,13 @@
+from __future__ import absolute_import
+
 import binascii
 import random
 import string
 import sys
 
-import builder
-
-from ....context import context
-from ...encoder import Encoder
+from pwnlib.context import context
+from pwnlib.encoders.arm.alphanumeric import builder
+from pwnlib.encoders.encoder import Encoder
 
 
 class ArmEncoder(Encoder):

pwnlib/encoders/arm/xor.py

@@ -1,10 +1,12 @@
-from ... import shellcraft
-from ...asm import asm
-from ...context import context
-from ...util.fiddling import xor_key
-from ...util.lists import group
-from ...util.packing import u8
-from ..encoder import Encoder
+from __future__ import absolute_import
+
+from pwnlib import shellcraft
+from pwnlib.asm import asm
+from pwnlib.context import context
+from pwnlib.encoders.encoder import Encoder
+from pwnlib.util.fiddling import xor_key
+from pwnlib.util.lists import group
+from pwnlib.util.packing import u8
 
 
 class ArmXorEncoder(Encoder):

pwnlib/encoders/encoder.py

@@ -1,13 +1,15 @@
 # -*- coding: utf-8 -*-
+from __future__ import absolute_import
+
 import collections
 import random
 import re
 import string
 
-from ..context import LocalContext
-from ..context import context
-from ..log import getLogger
-from ..util.fiddling import hexdump
+from pwnlib.context import LocalContext
+from pwnlib.context import context
+from pwnlib.log import getLogger
+from pwnlib.util.fiddling import hexdump
 
 log = getLogger(__name__)
 

pwnlib/encoders/i386/__init__.py

@@ -1,2 +1,4 @@
-from . import delta
-from . import xor
+from __future__ import absolute_import
+
+from pwnlib.encoders.i386 import delta
+from pwnlib.encoders.i386 import xor

pwnlib/encoders/i386/delta.py

@@ -1,12 +1,14 @@
+from __future__ import absolute_import
+
 import collections
 from random import choice
 from random import randint
 
-from ...asm import asm
-from ...asm import disasm
-from ...context import context
-from ...util.fiddling import hexdump
-from ..encoder import Encoder
+from pwnlib.asm import asm
+from pwnlib.asm import disasm
+from pwnlib.context import context
+from pwnlib.encoders.encoder import Encoder
+from pwnlib.util.fiddling import hexdump
 
 
 '''

pwnlib/encoders/i386/xor.py

@@ -10,13 +10,14 @@
 #; This program is distributed in the hope that it will be useful,
 #; but WITHOUT ANY WARRANTY; without even the implied warranty of
 #; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+from __future__ import absolute_import
 
-from ... import shellcraft
-from ...asm import asm
-from ...context import context
-from ...util.fiddling import xor_pair
-from ...util.lists import group
-from ..encoder import Encoder
+from pwnlib import shellcraft
+from pwnlib.asm import asm
+from pwnlib.context import context
+from pwnlib.encoders.encoder import Encoder
+from pwnlib.util.fiddling import xor_pair
+from pwnlib.util.lists import group
 
 
 # Note shellcode assumes it's based at ecx

pwnlib/encoders/mips/__init__.py

@@ -1 +1,3 @@
-from . import xor
+from __future__ import absolute_import
+
+from pwnlib.encoders.mips import xor

pwnlib/encoders/mips/xor.py

@@ -23,12 +23,13 @@
 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
+from __future__ import absolute_import
 
-from ... import shellcraft
-from ...asm import asm
-from ...context import context
-from ...util.fiddling import xor_key
-from ..encoder import Encoder
+from pwnlib import asm
+from pwnlib import shellcraft
+from pwnlib.context import context
+from pwnlib.encoders.encoder import Encoder
+from pwnlib.util.fiddling import xor_key
 
 decoders = {
     'little': ''.join([

pwnlib/flag/__init__.py

@@ -1,4 +1,6 @@
-from .flag import submit_flag
+from __future__ import absolute_import
+
+from pwnlib.flag.flag import submit_flag
 
 __all__ = [
     'submit_flag',

pwnlib/flag/flag.py

@@ -1,10 +1,12 @@
 """Describes a way to submit a key to a key server.
 """
+from __future__ import absolute_import
+
 import os
 
-from ..args import args
-from ..tubes.remote import remote
-from ..log import getLogger
+from pwnlib.args import args
+from pwnlib.log import getLogger
+from pwnlib.tubes.remote import remote
 
 env_server  = args.get('FLAG_HOST', 'flag-submission-server').strip()
 env_port    = args.get('FLAG_PORT', '31337').strip()

pwnlib/fmtstr.py

@@ -40,7 +40,7 @@ Examples:
     ...
     >>> autofmt = FmtStr(exec_fmt)
     >>> offset = autofmt.offset
-    >>> p = process(program, stderr=subprocess.PIPE)
+    >>> p = process(program, stderr=PIPE)
     >>> addr = unpack(p.recv(4))
     >>> payload = fmtstr_payload(offset, {addr: 0x1337babe})
     >>> p.sendline(payload)

pwnlib/gdb.py

@@ -1,33 +1,136 @@
+# -*- coding: utf-8 -*-
+"""
+During exploit development, it is frequently useful to debug the
+target binary under GDB.
+
+Pwntools makes this easy-to-do with a handful of helper routines, designed
+to make your exploit-debug-update cycles much faster.
+
+Useful Functions
+----------------
+
+- :func:`attach` - Attach to an existing process
+- :func:`debug` - Start a new process under a debugger, stopped at the first instruction
+- :func:`debug_shellcode` - Build a binary with the provided shellcode, and start it under a debugger
+
+Debugging Tips
+--------------
+
+The :func:`attach` and :func:`debug` functions will likely be your bread and
+butter for debugging.
+
+Both allow you to provide a script to pass to GDB when it is started, so that
+it can automatically set your breakpoints.
+
+Attaching to Processes
+~~~~~~~~~~~~~~~~~~~~~~
+
+To attach to an existing process, just use :func:`attach`.  It is surprisingly
+versatile, and can attach to a :class:`.process` for simple
+binaries, or will automatically find the correct process to attach to for a
+forking server, if given a :class:`.remote` object.
+
+Spawning New Processes
+~~~~~~~~~~~~~~~~~~~~~~
+
+Attaching to processes with :func:`attach` is useful, but the state the process
+is in may vary.  If you need to attach to a process very early, and debug it from
+the very first instruction (or even the start of ``main``), you instead should use
+:func:`debug`.
+
+When you use :func:`debug`, the return value is a :class:`.tube` object
+that you interact with exactly like normal.
+
+Tips and Troubleshooting
+------------------------
+
+``NOPTRACE`` magic argument
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It's quite cumbersom to comment and un-comment lines containing `attach`.
+
+You can cause these lines to be a no-op by running your script with the
+``NOPTRACE`` argument appended, or with ``PWNLIB_NOPTRACE=1`` in the environment.
+
+::
+
+    $ python exploit.py NOPTRACE
+    [+] Starting local process '/bin/bash': Done
+    [!] Skipping debug attach since context.noptrace==True
+    ...
+
+Kernel Yama ptrace_scope
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+The Linux kernel v3.4 introduced a security mechanism called ``ptrace_scope``,
+which is intended to prevent processes from debugging eachother unless there is
+a direct parent-child relationship.
+
+This causes some issues with the normal Pwntools workflow, since the process
+heirarchy looks like this:
+
+::
+
+    python ---> target
+           `--> gdb
+
+Note that ``python`` is the parent of ``target``, not ``gdb``.
+
+In order to avoid this being a problem, Pwntools uses the function
+``prctl(PR_SET_PTRACER, PR_SET_PTRACER_ANY)``.  This disables Yama
+for any processes launched by Pwntools via :class:`.process` or via
+:meth:`.ssh.process`.
+
+Older versions of Pwntools did not perform the ``prctl`` step, and
+required that the Yama security feature was disabled systemwide, which
+requires ``root`` access.
+
+Member Documentation
+===============================
+"""
+from __future__ import absolute_import
+
 import os
 import random
 import re
 import shlex
 import tempfile
+import time
 
-from . import adb
-from . import atexit
-from . import elf
-from . import tubes
-from .asm import _bfdname
-from .asm import make_elf
-from .asm import make_elf_from_assembly
-from .context import LocalContext
-from .context import context
-from .log import getLogger
-from .qemu import get_qemu_user
-from .util import misc
-from .util import proc
+from pwnlib import adb
+from pwnlib import atexit
+from pwnlib import elf
+from pwnlib import tubes
+from pwnlib.asm import _bfdname
+from pwnlib.asm import make_elf
+from pwnlib.asm import make_elf_from_assembly
+from pwnlib.context import LocalContext
+from pwnlib.context import context
+from pwnlib.log import getLogger
+from pwnlib.qemu import get_qemu_user
+from pwnlib.util import misc
+from pwnlib.util import proc
 
 log = getLogger(__name__)
 
 @LocalContext
-def debug_assembly(asm, execute=None, vma=None):
-    """
-    Creates an ELF file, and launches it with GDB.
+def debug_assembly(asm, gdbscript=None, vma=None):
+    """debug_assembly(asm, gdbscript=None, vma=None) -> tube
+
+    Creates an ELF file, and launches it under a debugger.
 
     This is identical to debug_shellcode, except that
     any defined symbols are available in GDB, and it
     saves you the explicit call to asm().
+
+    Arguments:
+        asm(str): Assembly code to debug
+        gdbscript(str): Script to run in GDB
+        vma(int): Base address to load the shellcode at
+        **kwargs: Override any :obj:`.context` values.
+
+    Returns:
+        :class:`.process`
     """
     tmp_elf = make_elf_from_assembly(asm, vma=vma, extract=False)
     os.chmod(tmp_elf, 0777)
@@ -39,19 +142,21 @@ def debug_assembly(asm, execute=None, vma=None):
         adb.push(tmp_elf, android_path)
         tmp_elf = android_path
 
-    return debug(tmp_elf, execute=execute, arch=context.arch)
+    return debug(tmp_elf, gdbscript=gdbscript, arch=context.arch)
 
 @LocalContext
-def debug_shellcode(data, execute=None, vma=None):
+def debug_shellcode(data, gdbscript=None, vma=None):
     """
-    Creates an ELF file, and launches it with GDB.
+    Creates an ELF file, and launches it under a debugger.
 
     Arguments:
         data(str): Assembled shellcode bytes
-        kwargs(dict): Arguments passed to context (e.g. arch='arm')
+        gdbscript(str): Script to run in GDB
+        vma(int): Base address to load the shellcode at
+        **kwargs: Override any :obj:`.context` values.
 
     Returns:
-        A ``process`` tube connected to the shellcode on stdin/stdout/stderr.
+        :class:`.process`
     """
     if isinstance(data, unicode):
         log.error("Shellcode is cannot be unicode.  Did you mean debug_assembly?")
@@ -65,7 +170,7 @@ def debug_shellcode(data, execute=None, vma=None):
         adb.push(tmp_elf, android_path)
         tmp_elf = android_path
 
-    return debug(tmp_elf, execute=execute, arch=context.arch)
+    return debug(tmp_elf, gdbscript=gdbscript, arch=context.arch)
 
 def _gdbserver_args(pid=None, path=None, args=None, which=None):
     """_gdbserver_args(pid=None, path=None) -> list
@@ -163,24 +268,90 @@ def _get_runner(ssh=None):
     else:                          return tubes.process.process
 
 @LocalContext
-def debug(args, execute=None, exe=None, ssh=None, env=None, **kwargs):
+def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
     """debug(args) -> tube
 
     Launch a GDB server with the specified command line,
     and launches GDB to attach to it.
 
     Arguments:
-        args: Same args as passed to pwnlib.tubes.process
-        ssh: Remote ssh session to use to launch the process.
-          Automatically sets up port forwarding so that gdb runs locally.
+        args(list): Arguments to the process, similar to :class:`.process`.
+        gdbscript(str): GDB script to run.
+        exe(str): Path to the executable on disk
+        env(dict): Environment to start the binary in
+        ssh(:class:`.ssh`): Remote ssh session to use to launch the process.
 
     Returns:
-        A tube connected to the target process
-    """
-    if context.noptrace:
-        log.warn_once("Skipping debugger since context.noptrace==True")
-        return tubes.process.process(args, executable=exe, env=env)
+        :class:`.process` or :class:`.ssh_channel`: A tube connected to the target process
 
+    Notes:
+
+        The debugger is attached automatically, and you can debug everything
+        from the very beginning.  This requires that both ``gdb`` and ``gdbserver``
+        are installed on your machine.
+
+        .. code-block:: python
+
+            # Create a new process, and stop it at 'main'
+            io = gdb.debug('bash', '''
+            break main
+            continue
+            ''')
+
+        When GDB opens via :func:`debug`, it will initially be stopped on the very first
+        instruction of the dynamic linker (``ld.so``) for dynamically-linked binaries.
+
+        Only the target binary and the linker will be loaded in memory, so you cannot
+        set breakpoints on shared library routines like ``malloc`` since ``libc.so``
+        has not even been loaded yet.
+
+        There are several ways to handle this:
+
+        1. Set a breakpoint on the executable's entry point (generally, ``_start``)
+            - This is only invoked after all of the required shared libraries
+              are loaded.
+            - You can generally get the address via the GDB command ``info file``.
+        2. Use pending breakpoints via ``set breakpoint pending on``
+            - This has the side-effect of setting breakpoints for **every** function
+              which matches the name.  For ``malloc``, this will generally set a
+              breakpoint in the executable's PLT, in the linker's internal ``malloc``,
+              and eventaully in ``libc``'s malloc.
+        3. Wait for libraries to be loaded with ``set stop-on-solib-event 1``
+            - There is no way to stop on any specific library being loaded, and sometimes
+              multiple libraries are loaded and only a single breakpoint is issued.
+            - Generally, you just add a few ``continue`` commands until things are set up
+              the way you want it to be.
+
+        .. code-block:: python
+
+            # Create a new process, and stop it at 'main'
+            io = gdb.debug('bash', '''
+            # Wait until we hit the main executable's entry point
+            break _start
+            continue
+
+            # Now set breakpoint on shared library routines
+            break malloc
+            break free
+            continue
+            ''')
+
+        You can use :func:`debug` to spawn new processes on remote machines as well,
+        by using the ``ssh=`` keyword to pass in your :class:`.ssh` instance.
+
+        .. code-block:: python
+
+            # Connect to the SSH server
+            shell = ssh('passcode', 'pwnable.kr', 2222, password='guest')
+
+            # Start a process on the server
+            bash = gdb.debug(['bash'],
+                            ssh=shell,
+                            gdbscript='''
+            break main
+            continue
+            ''')
+    """
     if isinstance(args, (int, tubes.process.process, tubes.ssh.ssh_channel)):
         log.error("Use gdb.attach() to debug a running process")
 
@@ -195,6 +366,10 @@ def debug(args, execute=None, exe=None, ssh=None, env=None, **kwargs):
     runner = _get_runner(ssh)
     which  = _get_which(ssh)
 
+    if context.noptrace:
+        log.warn_once("Skipping debugger since context.noptrace==True")
+        return runner(args, executable=exe, env=env)
+
     if ssh or context.native or (context.os == 'android'):
         args = _gdbserver_args(args=args, which=which)
     else:
@@ -226,7 +401,7 @@ def debug(args, execute=None, exe=None, ssh=None, env=None, **kwargs):
     if not ssh and context.os == 'android':
         host = context.adb_host
 
-    attach((host, port), exe=exe, execute=execute, need_ptrace_scope = False)
+    attach((host, port), exe=exe, gdbscript=gdbscript, need_ptrace_scope = False, ssh=ssh)
 
     # gdbserver outputs a message when a client connects
     garbage = gdbserver.recvline(timeout=1)
@@ -245,75 +420,146 @@ def get_gdb_arch():
         'thumb': 'arm'
     }.get(context.arch, context.arch)
 
-
-@LocalContext
-def attach(target, execute = None, exe = None, need_ptrace_scope = True):
-    """attach(target, execute = None, exe = None, arch = None) -> None
-
-    Start GDB in a new terminal and attach to `target`.
-    :func:`pwnlib.util.proc.pidof` is used to find the PID of `target` except
-    when `target` is a ``(host, port)``-pair.  In that case `target` is assumed
-    to be a GDB server.
-
-    If it is running locally and `exe` is not given we will try to find the path
-    of the target binary from parsing the command line of the program running
-    the GDB server (e.g. qemu or gdbserver).  Notice that if the PID is known
-    (when `target` is not a GDB server) `exe` will be read from
-    ``/proc/<pid>/exe``.
-
-    If `gdb-multiarch` is installed we use that or 'gdb' otherwise.
-
-    Arguments:
-      target: The target to attach to.
-      execute (str or file): GDB script to run after attaching.
-      exe (str): The path of the target binary.
-      arch (str): Architechture of the target binary.  If `exe` known GDB will
-      detect the architechture automatically (if it is supported).
+def binary():
+    """binary() -> str
 
     Returns:
-      :const:`None`
-"""
+        str: Path to the appropriate ``gdb`` binary to use.
+    """
+    gdb = misc.which('gdb')
+
+    if not context.native:
+        multiarch = misc.which('gdb-multiarch')
+
+        if multiarch:
+            return multiarch
+        log.warn_once('Cross-architecture debugging usually requires gdb-multiarch\n' \
+                      '$ apt-get install gdb-multiarch')
+
+    if not gdb:
+        log.error('GDB is not installed\n'
+                  '$ apt-get install gdb')
+
+    return gdb
+
+@LocalContext
+def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_args = None, ssh = None):
+    """attach(target, gdbscript = None, exe = None, arch = None, ssh = None) -> None
+
+    Start GDB in a new terminal and attach to `target`.
+
+    Arguments:
+        target: The target to attach to.
+        gdbscript(:obj:`str` or :obj:`file`): GDB script to run after attaching.
+        exe(str): The path of the target binary.
+        arch(str): Architechture of the target binary.  If `exe` known GDB will
+          detect the architechture automatically (if it is supported).
+        gdb_args(list): List of additional arguments to pass to GDB.
+
+    Returns:
+        PID of the GDB process (or the window which it is running in).
+
+    Notes:
+
+        The ``target`` argument is very robust, and can be any of the following:
+
+        :obj:`int`
+            PID of a process
+        :obj:`str`
+            Process name.  The youngest process is selected.
+        :obj:`tuple`
+            Host, port pair of a listening ``gdbserver``
+        :class:`.process`
+            Process to connect to
+        :class:`.sock`
+            Connected socket. The executable on the other end of the connection is attached to.
+            Can be any socket type, including :class:`.listen` or :class:`.remote`.
+        :class:`.ssh_channel`
+            Remote process spawned via :meth:`.ssh.process`.
+            This will use the GDB installed on the remote machine.
+            If a password is required to connect, the ``sshpass`` program must be installed.
+
+        .. code-block:: python
+
+            # Attach directly to pid 1234
+            gdb.attach(1234)
+
+        .. code-block:: python
+
+            # Attach to the youngest "bash" process
+            gdb.attach('bash')
+
+        .. code-block:: python
+
+            # Start a process
+            bash = process('bash')
+
+            # Attach the debugger
+            gdb.attach(bash, '''
+            set follow-fork-mode child
+            break execve
+            continue
+            ''')
+
+            # Interact with the process
+            bash.sendline('whoami')
+
+        .. code-block:: python
+
+            # Start a forking server
+            server = process(['socat', 'tcp-listen:1234,fork,reuseaddr', 'exec:/bin/sh'])
+
+            # Connect to the server
+            io = remote('localhost', 1234)
+
+            # Connect the debugger to the server-spawned process
+            gdb.attach(io, '''
+            break exit
+            continue
+            ''')
+
+            # Talk to the spawned 'sh'
+            io.sendline('exit')
+
+        .. code-block:: python
+
+            # Connect to the SSH server
+            shell = ssh('bandit0', 'bandit.labs.overthewire.org', password='bandit0')
+
+            # Start a process on the server
+            cat = shell.process(['cat'])
+
+            # Attach a debugger to it
+            gdb.attach(cat, '''
+            break exit
+            continue
+            ''')
+
+            # Cause `cat` to exit
+            cat.close()
+    """
     if context.noptrace:
         log.warn_once("Skipping debug attach since context.noptrace==True")
         return
 
-    # if execute is a file object, then read it; we probably need to run some
+    # if gdbscript is a file object, then read it; we probably need to run some
     # more gdb script anyway
-    if execute:
-        if isinstance(execute, file):
-            fd = execute
-            execute = fd.read()
-            fd.close()
+    if isinstance(gdbscript, file):
+        with gdbscript:
+            gdbscript = gdbscript.read()
 
     # enable gdb.attach(p, 'continue')
-    if execute and not execute.endswith('\n'):
-        execute += '\n'
+    if gdbscript and not gdbscript.endswith('\n'):
+        gdbscript += '\n'
 
-    # gdb script to run before `execute`
+    # gdb script to run before `gdbscript`
     pre = ''
     if not context.native:
-        if not misc.which('gdb-multiarch'):
-            log.warn_once('Cross-architecture debugging usually requires gdb-multiarch\n' \
-                '$ apt-get install gdb-multiarch')
         pre += 'set endian %s\n' % context.endian
         pre += 'set architecture %s\n' % get_gdb_arch()
 
         if context.os == 'android':
             pre += 'set gnutarget ' + _bfdname() + '\n'
-    else:
-        # If ptrace_scope is set and we're not root, we cannot attach to a
-        # running process.
-        # We assume that we do not need this to be set if we are debugging on
-        # a different architecture (e.g. under qemu-user).
-        try:
-            ptrace_scope = open('/proc/sys/kernel/yama/ptrace_scope').read().strip()
-            if need_ptrace_scope and ptrace_scope != '0' and os.geteuid() != 0:
-                msg =  'Disable ptrace_scope to attach to running processes.\n'
-                msg += 'More info: https://askubuntu.com/q/41629'
-                log.warning(msg)
-                return
-        except IOError:
-            pass
 
     # let's see if we can find a pid to attach to
     pid = None
@@ -340,11 +586,13 @@ def attach(target, execute = None, exe = None, need_ptrace_scope = True):
         shell = target.parent
 
         tmpfile = shell.mktemp()
-        execute = 'shell rm %s\n%s' % (tmpfile, execute)
-        shell.upload_data(execute or '', tmpfile)
+        gdbscript = 'shell rm %s\n%s' % (tmpfile, gdbscript)
+        shell.upload_data(gdbscript or '', tmpfile)
 
         cmd = ['ssh', '-C', '-t', '-p', str(shell.port), '-l', shell.user, shell.host]
         if shell.password:
+            if not misc.which('sshpass'):
+                log.error("sshpass must be installed to debug ssh processes")
             cmd = ['sshpass', '-p', shell.password] + cmd
         if shell.keyfile:
             cmd += ['-i', shell.keyfile]
@@ -377,6 +625,8 @@ def attach(target, execute = None, exe = None, need_ptrace_scope = True):
                     return os.path.join(proc.cwd(spid), exe)
 
         exe = exe or findexe()
+    elif isinstance(target, elf.corefile.Corefile):
+        pre += 'target core %s\n' % target.path
     else:
         log.error("don't know how to attach to target: %r" % target)
 
@@ -390,19 +640,20 @@ def attach(target, execute = None, exe = None, need_ptrace_scope = True):
     if not pid and not exe:
         log.error('could not find target process')
 
-    cmd = None
-    for p in ('gdb-multiarch', 'gdb'):
-        if misc.which(p):
-            cmd = p
-            break
-    else:
-        log.error('no gdb installed')
+    cmd = binary()
+
+    if gdb_args:
+        cmd += ' '
+        cmd += ' '.join(gdb_args)
 
     cmd += ' -q '
 
     if exe and context.native:
+        if ssh:
+            ssh.download_file(exe)
+            exe = os.path.basename(exe)
         if not os.path.isfile(exe):
-            log.error('no such file: %s' % exe)
+            log.error('No such file: %s' % exe)
         cmd += ' "%s"' % exe
 
     if pid and not context.os == 'android':
@@ -417,38 +668,40 @@ def attach(target, execute = None, exe = None, need_ptrace_scope = True):
         host    = context.adb_host
         pre    += 'target remote %s:%i' % (context.adb_host, port)
 
-    execute = pre + (execute or '')
+    gdbscript = pre + (gdbscript or '')
 
-    if execute:
+    if gdbscript:
         tmp = tempfile.NamedTemporaryFile(prefix = 'pwn', suffix = '.gdb',
                                           delete = False)
-        log.debug('Wrote gdb script to %r\n%s' % (tmp.name, execute))
-        execute = 'shell rm %s\n%s' % (tmp.name, execute)
+        log.debug('Wrote gdb script to %r\n%s' % (tmp.name, gdbscript))
+        gdbscript = 'shell rm %s\n%s' % (tmp.name, gdbscript)
 
-        tmp.write(execute)
+        tmp.write(gdbscript)
         tmp.close()
         cmd += ' -x "%s"' % (tmp.name)
 
     log.info('running in new terminal: %s' % cmd)
-    misc.run_in_new_terminal(cmd)
+
+    gdb_pid = misc.run_in_new_terminal(cmd)
+
     if pid and context.native:
         proc.wait_for_debugger(pid)
-    return pid
 
-def ssh_gdb(ssh, process, execute = None, arch = None, **kwargs):
-    if isinstance(process, (list, tuple)):
-        exe = process[0]
-        process = ["gdbserver", "127.0.0.1:0"] + process
-    else:
-        exe = process
-        process = "gdbserver 127.0.0.1:0 " + process
+    return gdb_pid
+
+def ssh_gdb(ssh, argv, gdbscript = None, arch = None, **kwargs):
+    if not isinstance(argv, (list, tuple)):
+        argv = [argv]
+
+    exe = argv[0]
+    argv = ["gdbserver", "127.0.0.1:0"] + argv
 
     # Download the executable
     local_exe = os.path.basename(exe)
-    ssh.download_file(exe, local_exe)
+    ssh.download_file(ssh.which(exe), local_exe)
 
     # Run the process
-    c = ssh.run(process, **kwargs)
+    c = ssh.process(argv, **kwargs)
 
     # Find the port for the gdb server
     c.recvuntil('port ')
@@ -460,7 +713,7 @@ def ssh_gdb(ssh, process, execute = None, arch = None, **kwargs):
     l = tubes.listen.listen(0)
     forwardport = l.lport
 
-    attach(('127.0.0.1', forwardport), execute, local_exe, arch)
+    attach(('127.0.0.1', forwardport), gdbscript, local_exe, arch, ssh=ssh)
     l.wait_for_connection() <> ssh.connect_remote('127.0.0.1', gdbport)
     return c
 
@@ -501,8 +754,8 @@ def find_module_addresses(binary, ssh=None, ulimit=False):
     Arguments:
         binary(str): Path to the binary on the remote server
         ssh(pwnlib.tubes.tube): SSH connection through which to load the libraries.
-            If left as ``None``, will use a ``pwnlib.tubes.process.process``.
-        ulimit(bool): Set to ``True`` to run "ulimit -s unlimited" before GDB.
+            If left as :const:`None`, will use a :class:`pwnlib.tubes.process.process`.
+        ulimit(bool): Set to :const:`True` to run "ulimit -s unlimited" before GDB.
 
     Returns:
         A list of pwnlib.elf.ELF objects, with correct base addresses.
@@ -589,3 +842,71 @@ def find_module_addresses(binary, ssh=None, ulimit=False):
         rv.append(lib)
 
     return rv
+
+def corefile(process):
+    r"""Drops a core file for the process.
+
+    Arguments:
+        process: Process to dump
+
+    Returns:
+        :class:`.Core`: The generated core file
+    """
+
+    if context.noptrace:
+        log.warn_once("Skipping corefile since context.noptrace==True")
+        return
+
+    corefile_path = './core.%s.%i' % (os.path.basename(process.executable),
+                                    process.pid)
+
+    # Due to https://sourceware.org/bugzilla/show_bug.cgi?id=16092
+    # will disregard coredump_filter, and will not dump private mappings.
+    if version() < (7,11):
+        log.warn_once('The installed GDB (%s) does not emit core-dumps which '
+                      'contain all of the data in the process.\n'
+                      'Upgrade to GDB >= 7.11 for better core-dumps.' % binary())
+
+    # This is effectively the same as what the 'gcore' binary does
+    gdb_args = ['-batch',
+                '-q',
+                '--nx',
+                '-ex', '"set pagination off"',
+                '-ex', '"set height 0"',
+                '-ex', '"set width 0"',
+                '-ex', '"set use-coredump-filter on"',
+                '-ex', '"generate-core-file %s"' % corefile_path,
+                '-ex', 'detach']
+
+    with context.local(terminal = ['sh', '-c']):
+        with context.quiet:
+            pid = attach(process, gdb_args=gdb_args)
+            os.waitpid(pid, 0)
+
+    return elf.corefile.Core(corefile_path)
+
+def version(program='gdb'):
+    """Gets the current GDB version.
+
+    Note:
+        Requires that GDB version meets the following format:
+
+        ``GNU gdb (GDB) 7.12``
+
+    Returns:
+        tuple: A tuple containing the version numbers
+
+    Example:
+
+        >>> (7,0) <= gdb.version() <= (8,0)
+        True
+    """
+    program = misc.which(program)
+    expr = r'([0-9]+\.?)+'
+
+    with tubes.process.process([program, '--version'], level='error') as gdb:
+        version = gdb.recvline()
+
+    versions = re.search(expr, version).group()
+
+    return tuple(map(int, versions.split('.')))

pwnlib/libcdb.py

@@ -1,21 +1,23 @@
 """
 Fetch a LIBC binary based on some heuristics.
 """
+from __future__ import absolute_import
+
 import codecs
 import json
 import os
 import tempfile
 import urlparse
 
-from .context import context
-from .elf import ELF
-from .log import getLogger
-from .util.fiddling import b64d
-from .util.fiddling import hexdump
-from .util.misc import read
-from .util.misc import write
-from .util.safeeval import const
-from .util.web import wget
+from pwnlib.context import context
+from pwnlib.elf import ELF
+from pwnlib.log import getLogger
+from pwnlib.util.fiddling import b64d
+from pwnlib.util.fiddling import hexdump
+from pwnlib.util.misc import read
+from pwnlib.util.misc import write
+from pwnlib.util.safeeval import const
+from pwnlib.util.web import wget
 
 log = getLogger(__name__)
 
@@ -33,7 +35,7 @@ def search_by_build_id(hex_encoded_id):
             Hex-encoded Build ID (e.g. 'ABCDEF...') of the library
 
     Returns:
-        Path to the downloaded library on disk, or ``None``.
+        Path to the downloaded library on disk, or :const:`None`.
     """
     cache = cache_dir + '-libc.so.' + hex_encoded_id
 

pwnlib/log.py

@@ -40,7 +40,7 @@ Pwnlib Developers
 -----------------
 A module-specific logger can be imported into the module via::
 
-    from .log import getLogger
+    from pwnlib.log import getLogger
     log = getLogger(__name__)
 
 This provides an easy way to filter logging programmatically
@@ -91,10 +91,7 @@ unless :data:`context.log_level` is also set to ``'DEBUG'``.  Other handlers
 will however see the extra log records generated by the ``'pwnlib.tubes.ssh'``
 logger.
 """
-
-__all__ = [
-    'getLogger', 'install_default_handler', 'rootlogger'
-]
+from __future__ import absolute_import
 
 import ConfigParser
 import logging
@@ -105,15 +102,21 @@ import sys
 import threading
 import time
 
-from . import term
-from .context import Thread
-from .context import context
-from .exception import PwnlibException
-from .term import spinners
-from .term import text
+from pwnlib import term
+from pwnlib.context import Thread
+from pwnlib.context import context
+from pwnlib.exception import PwnlibException
+from pwnlib.term import spinners
+from pwnlib.term import text
+
+__all__ = [
+    'getLogger', 'install_default_handler', 'rootlogger'
+]
+
+
 
 # list of prefixes to use for the different message types.  note that the `text`
-# module won't add any escape codes if `sys.stderr.isatty()` is `False`
+# module won't add any escape codes if `pwnlib.context.log_console.isatty()` is `False`
 _msgtype_prefixes = {
     'status'       : [text.magenta, 'x'],
     'success'      : [text.bold_green, '+'],
@@ -166,7 +169,6 @@ class Progress(object):
     using :meth:`Logger.progress`.
     """
     def __init__(self, logger, msg, status, level, args, kwargs):
-        global _progressid
         self._logger = logger
         self._msg = msg
         self._status = status
@@ -264,7 +266,7 @@ class Logger(object):
             # This is a minor hack to permit user-defined classes which inherit
             # from a tube (which do not actually reside in the pwnlib library)
             # to receive logging abilities that behave as they would expect from
-            # the resto f the library
+            # the rest of the library
             module = self.__module__
             if not module.startswith('pwnlib'):
                 module = 'pwnlib.' + module
@@ -490,10 +492,14 @@ class Handler(logging.StreamHandler):
     logger will not be emitted but rather an animated progress line will be
     created.
 
-    This handler outputs to ``sys.stderr``.
-
     An instance of this handler is added to the ``'pwnlib'`` logger.
     """
+    @property
+    def stream(self):
+        return context.log_console
+    @stream.setter
+    def stream(self, value):
+        pass
     def emit(self, record):
         """
         Emit a log record or create/update an animated progress logger
@@ -652,7 +658,7 @@ log_file.setFormatter(logging.Formatter(fmt, iso_8601))
 #     logger.addHandler(myCoolPitchingHandler)
 #
 rootlogger = getLogger('pwnlib')
-console   = Handler(sys.stdout)
+console   = Handler()
 formatter = Formatter()
 console.setFormatter(formatter)
 
@@ -663,8 +669,6 @@ def install_default_handler():
     the ``pwnlib`` root logger.  This function is automatically called from when
     importing :mod:`pwn`.
     '''
-    console.stream = sys.stdout
-
     logger         = logging.getLogger('pwnlib')
 
     if console not in logger.handlers:

pwnlib/memleak.py

@@ -1,11 +1,13 @@
+from __future__ import absolute_import
+
 import ctypes
 import functools
 import string
 
-from .context import context
-from .log import getLogger
-from .util.packing import pack
-from .util.packing import unpack
+from pwnlib.context import context
+from pwnlib.log import getLogger
+from pwnlib.util.packing import pack
+from pwnlib.util.packing import unpack
 
 log = getLogger(__name__)
 
@@ -31,46 +33,52 @@ class MemLeak(object):
 
     Example:
 
-        .. doctest:: leaker
+        >>> import pwnlib
+        >>> binsh = pwnlib.util.misc.read('/bin/sh')
+        >>> @pwnlib.memleak.MemLeak
+        ... def leaker(addr):
+        ...     print "leaking 0x%x" % addr
+        ...     return binsh[addr:addr+4]
+        >>> leaker.s(0)[:4]
+        leaking 0x0
+        leaking 0x4
+        '\\x7fELF'
+        >>> leaker[:4]
+        '\\x7fELF'
+        >>> hex(leaker.d(0))
+        '0x464c457f'
+        >>> hex(leaker.clearb(1))
+        '0x45'
+        >>> hex(leaker.d(0))
+        leaking 0x1
+        '0x464c457f'
+        >>> @pwnlib.memleak.MemLeak
+        ... def leaker_nonulls(addr):
+        ...     print "leaking 0x%x" % addr
+        ...     if addr & 0xff == 0:
+        ...         return None
+        ...     return binsh[addr:addr+4]
+        >>> leaker_nonulls.d(0) == None
+        leaking 0x0
+        True
+        >>> leaker_nonulls[0x100:0x104] == binsh[0x100:0x104]
+        leaking 0x100
+        leaking 0xff
+        leaking 0x103
+        True
 
-            >>> import pwnlib
-            >>> binsh = pwnlib.util.misc.read('/bin/sh')
-            >>> @pwnlib.memleak.MemLeak
-            ... def leaker(addr):
-            ...     print "leaking 0x%x" % addr
-            ...     return binsh[addr:addr+4]
-            >>> leaker.s(0)[:4]
-            leaking 0x0
-            leaking 0x4
-            '\\x7fELF'
-            >>> leaker[:4]
-            '\\x7fELF'
-            >>> hex(leaker.d(0))
-            '0x464c457f'
-            >>> hex(leaker.clearb(1))
-            '0x45'
-            >>> hex(leaker.d(0))
-            leaking 0x1
-            '0x464c457f'
-            >>> @pwnlib.memleak.MemLeak
-            ... def leaker_nonulls(addr):
-            ...     print "leaking 0x%x" % addr
-            ...     if addr & 0xff == 0:
-            ...         return None
-            ...     return binsh[addr:addr+4]
-            >>> leaker_nonulls.d(0) == None
-            leaking 0x0
-            True
-            >>> leaker_nonulls[0x100:0x104] == binsh[0x100:0x104]
-            leaking 0x100
-            leaking 0xff
-            leaking 0x103
-            True
+        >>> memory = {-4+i: c for i,c in enumerate('wxyzABCDE')}
+        >>> def relative_leak(index):
+        ...     return memory.get(index, None)
+        >>> leak = pwnlib.memleak.MemLeak(relative_leak, relative = True)
+        >>> leak[-1:2]
+        'zAB'
     """
-    def __init__(self, f, search_range = 20, reraise = True):
+    def __init__(self, f, search_range = 20, reraise = True, relative = False):
         self.leak = f
         self.search_range = search_range
         self.reraise = reraise
+        self.relative = relative
 
         # Map of address: byte for all bytes received
         self.cache = {}
@@ -129,6 +137,8 @@ class MemLeak(object):
         size   = obj.size
         offset = obj.offset
         data   = self.n(address + offset, size)
+        if not data:
+            return None
         return unpack(data, size*8)
 
     def field_compare(self, address, obj, expected):
@@ -162,9 +172,9 @@ class MemLeak(object):
         Leak ``n`` consecutive bytes starting at ``addr``.
 
         Returns:
-            A string of length ``n``, or ``None``.
+            A string of length ``n``, or :const:`None`.
         """
-        if addr < 0:
+        if not self.relative and addr < 0:
             return None
 
         addresses = [addr+i for i in xrange(n)]
@@ -604,3 +614,19 @@ class MemLeak(object):
             return result
 
         return MemLeak(string_wrapper)
+
+    # Aliases for convenience
+    u64 = q
+    u32 = d
+    u16 = w
+    u8 = b
+
+    p64 = setq
+    p32 = setd
+    p16 = setw
+    p8 = setb
+
+class RelativeMemLeak(MemLeak):
+    def __init__(self, *a, **kw):
+        kw.setdefault('relative', True)
+        super(RelativeMemLeak, self).__init__(*a, **kw)