Release 3.11.0

Update changelog for 3.11 and 3.12
Merge branch 'stable' into beta
2026-11-08 17:13:31 -10:00 · 2018-01-03 08:19:05 +01:00 · 2018-01-03 08:03:28 +01:00 · 2018-01-03 08:02:50 +01:00 · 2018-01-02 11:36:08 -06:00 · 2017-12-11 17:47:21 -06:00
29 changed files with 989 additions and 235 deletions
@@ -9,9 +9,11 @@ The table below shows which release corresponds to each branch, and what date th

 | Version          | Branch   | Release Date           |
 | ---------------- | -------- | ---------------------- |
-| [3.11.0](#3110)  | `dev`    | Dec 2, 2017 (planned)
-| [3.10.0](#3100)  | `beta`   | Oct 21, 2017 (planned)
-| [3.9.2](#392)    | `stable` | Oct 5, 2017
+| [3.13.0](#3130)  | `dev`    | Mar 31, 2018 (planned)
+| [3.12.0](#3120)  | `beta`   | Feb 17, 2018 (planned)
+| [3.11.0](#3110)  | `stable` | Jan 3, 2018
+| [3.10.0](#3100)  |          | Oct 25, 2017
+| [3.9.2](#392)    |          | Oct 5, 2017
 | [3.9.1](#391)    |          | Sep 28, 2017
 | [3.9.0](#390)    |          | Sep 11, 2017
 | [3.8.0](#380)    |          | Jul 29, 2017
@@ -39,13 +41,60 @@ The table below shows which release corresponds to each branch, and what date th
 | [3.0.0](#300)    |          | Aug 20, 2016
 | [2.2.0](#220)    |          | Jan 5, 2015

+## 3.12.0
+
+To be released on Feb 17, 2018.
+
+- [#1083][1083] Better error messages for `gdb` when `LD_PRELOAD` is incorrect
+- [#1085][1085] Add support for extracting Android `BOOTLDR!` images
+- [#1075][1075] Add support for detecting GNU Screen for `run_in_new_terminal`
+- [#1074][1074] Add support for running `pwntools-gdb` wrapper script instead of `gdb`
+- [#1067][1067] Add `pwnlib.tubes.server` module, which adds a reusable `server` listener
+- [#1063][1063] Add support for labels in `fit()`, allowing dynamic contents to be injected.  (This feature is really cool, check out the pull request!)
+
+[1083]: https://github.com/Gallopsled/pwntools/pull/1083
+[1085]: https://github.com/Gallopsled/pwntools/pull/1085
+[1075]: https://github.com/Gallopsled/pwntools/pull/1075
+[1074]: https://github.com/Gallopsled/pwntools/pull/1074
+[1067]: https://github.com/Gallopsled/pwntools/pull/1067
+[1063]: https://github.com/Gallopsled/pwntools/pull/1063
+
+
 ## 3.11.0

-To be released on Dec 2, 2017.
+- [#1044][1044] Enhancements to ROP
+    + Much better support for 64-bit Intel (amd64) ROP
+    + ROP gadget selection is optimized to favor multi-pops instead of multiple single-pop gadgets
+    + Added support for blacklisting byte values in ROP gadget addresses
+- [#1049][1049] Enhancements to `cyclic`
+    + `context` now has two additional attributes, `cyclic_alphabet` and `cyclic_length`, which correspond to the arguments `alphabet` and `n` to `cyclic()` and `cyclic_find()` and related routines.
+    + The motivation for this change is to allow setting the `alphabet` globally, so that any padding / patterns generated internally to pwntools can be controlled.  The specific motivation is blacklisting values in ROP padding.
+- [#1052][1052] Enhancements for detecting `QEMU_LD_PREFIX` used by QEMU user-mode emulation for sysroots
+- [#1035][1035] Minor documentation changes
+- [#1032][1032] Enhancements to `pwn template`
+- [#1031][1031] More accurate `Coredump.fault_addr` on amd64
+- [#1084][1084] Fix broken tests due to `ftp.debian.org` going down
+
+[1044]: https://github.com/Gallopsled/pwntools/pull/1044
+[1049]: https://github.com/Gallopsled/pwntools/pull/1049
+[1052]: https://github.com/Gallopsled/pwntools/pull/1052
+[1035]: https://github.com/Gallopsled/pwntools/pull/1035
+[1032]: https://github.com/Gallopsled/pwntools/pull/1032
+[1031]: https://github.com/Gallopsled/pwntools/pull/1031
+[1084]: https://github.com/Gallopsled/pwntools/pull/1084

 ## 3.10.0

-To be released on Oct 21, 2017.
+- [#1007][1007] Add support for setting a `gdbinit` file in the context
+- [#1055][1055] Fixes for `Corefile` stack parsing, speed up `ELF.string()`
+- [#1057][1057] Fix a variable name typo in `DynELF` logging which results in an exception being thrown
+- [#1058][1058] Fix an edge case in `ssh_process.exe`
+
+[1007]: https://github.com/Gallopsled/pwntools/pull/1007
+[1055]: https://github.com/Gallopsled/pwntools/pull/1055
+[1057]: https://github.com/Gallopsled/pwntools/pull/1057
+[1058]: https://github.com/Gallopsled/pwntools/pull/1058
+

 ## 3.9.2

@@ -2,7 +2,7 @@
 ![pwntools logo](https://github.com/Gallopsled/pwntools/blob/stable/docs/source/logo.png?raw=true)

 [![Docs](https://readthedocs.org/projects/pwntools/badge/?version=stable)](https://docs.pwntools.com/)
-[![PyPI](https://img.shields.io/badge/pypi-v3.9.2-green.svg?style=flat)](https://pypi.python.org/pypi/pwntools/)
+[![PyPI](https://img.shields.io/badge/pypi-v3.11.0-green.svg?style=flat)](https://pypi.python.org/pypi/pwntools/)
 [![Travis](https://travis-ci.org/Gallopsled/pwntools.svg)](https://travis-ci.org/Gallopsled/pwntools)
 [![Coveralls](https://img.shields.io/coveralls/Gallopsled/pwntools/dev.svg)](https://coveralls.io/github/Gallopsled/pwntools?branch=dev)
 [![Twitter](https://img.shields.io/badge/twitter-pwntools-4099FF.svg?style=flat)](https://twitter.com/pwntools)
@@ -8,7 +8,7 @@
 :mod:`pwnlib.context` --- Setting runtime variables
 =====================================================

-Many settings in ``pwntools`` are controlled via the global variable :data:`.context, such as the selected target operating system, architecture, and bit-width.
+Many settings in ``pwntools`` are controlled via the global variable :data:`.context`, such as the selected target operating system, architecture, and bit-width.

 In general, exploits will start with something like:

@@ -13,7 +13,7 @@ Which imports a bazillion things into the global namespace to make your life eas

 This is a quick list of most of the objects and routines imported, in rough order of importance and frequency of use.

- :obj:`.context`
+- :mod:`pwnlib.context`
    - :data:`pwnlib.context.context`
    - Responsible for most of the pwntools convenience settings
    - Set `context.log_level = 'debug'` when troubleshooting your exploit
@@ -61,6 +61,7 @@ Each of the ``pwntools`` modules is documented here.
   log
   memleak
   protocols
+   qemu
   replacements
   rop
   rop/*
@@ -27,7 +27,7 @@ This exposes a standard interface to talk to processes, sockets, serial ports,
 and all manner of things, along with some nifty helpers for common tasks.
 For example, remote connections via :mod:`pwnlib.tubes.remote`.

-    >>> conn = remote('ftp.debian.org',21)
+    >>> conn = remote('ftp.ubuntu.org',21)
    >>> conn.recvline() # doctest: +ELLIPSIS
    '220 ...'
    >>> conn.send('USER anonymous\r\n')
@@ -0,0 +1,10 @@
+.. testsetup:: *
+
+   from pwn import *
+
+
+:mod:`pwnlib.qemu` --- QEMU Utilities
+==========================================
+
+.. automodule:: pwnlib.qemu
+   :members:
@@ -1,5 +1,6 @@
 .. testsetup:: *

+   from pwn import *
   from pwnlib.util.cyclic import *


@@ -164,6 +164,12 @@ p.add_argument(
    help='List available shellcodes, optionally provide a filter'
 )

+p.add_argument(
+    '-s', '--shared',
+    action='store_true',
+    help='Generated ELF is a shared library'
+)
+
 def get_template(name):
    func = shellcraft
    for attr in name.split('.'):
@@ -296,11 +302,11 @@ def main(args):


        if not args.avoid:
-            code = read(make_elf_from_assembly(assembly, vma=vma))
+            code = read(make_elf_from_assembly(assembly, vma=vma, shared=args.shared))
        else:
            code = asm(assembly)
            code = encode(code, args.avoid)
-            code = make_elf(code, vma=vma)
+            code = make_elf(code, vma=vma, shared=args.shared)
            # code = read(make_elf(encode(asm(code), args.avoid)))
    else:
        code = encode(asm(assembly), args.avoid)
@@ -19,6 +19,7 @@ parser.add_argument('--port', help='Remote port / SSH port', type=int)
 parser.add_argument('--user', help='SSH Username')
 parser.add_argument('--pass', help='SSH Password', dest='password')
 parser.add_argument('--path', help='Remote path of file on SSH server')
+parser.add_argument('--quiet', help='Less verbose template comments', action='store_true')

 def main(args):
    cache = None
@@ -49,7 +50,8 @@ def main(args):
                             args.port,
                             args.user,
                             args.password,
-                             args.path)
+                             args.path,
+                             args.quiet)

    # Fix Mako formatting bs
    output = re.sub('\n\n\n', '\n\n', output)
@@ -150,7 +150,7 @@ class _Tls_DictStack(threading.local, _DictStack):

 def _validator(validator):
    """
-    Validator that tis tightly coupled to the implementation
+    Validator that is tightly coupled to the implementation
    of the classes here.

    This expects that the object has a ._tls property which
@@ -337,9 +337,12 @@ class ContextType(object):
        'binary': None,
        'bits': 32,
        'buffer_size': 4096,
+        'cyclic_alphabet': string.ascii_lowercase,
+        'cyclic_size': 4,
        'delete_corefiles': False,
        'device': os.getenv('ANDROID_SERIAL', None) or None,
        'endian': 'little',
+        'gdbinit': "",
        'kernel': None,
        'log_level': logging.INFO,
        'log_file': _devnull(),
@@ -1231,6 +1234,49 @@ class ContextType(object):
        """
        return bool(v)

+
+    @_validator
+    def gdbinit(self, value):
+        """Path to the gdbinit that is used when running GDB locally.
+
+        This is useful if you want pwntools-launched GDB to include some additional modules,
+        like PEDA but you do not want to have GDB include them by default.
+
+        The setting will only apply when GDB is launched locally since remote hosts may not have
+        the necessary requirements for the gdbinit.
+
+        If set to an empty string, GDB will use the default `~/.gdbinit`.
+
+        Default value is ``""``.
+        """
+        return str(value)
+
+    @_validator
+    def cyclic_alphabet(self, alphabet):
+        """Cyclic alphabet.
+
+        Default value is `string.ascii_lowercase`.
+        """
+
+        # Do not allow multiple occurrences
+        if len(set(alphabet)) != len(alphabet):
+            raise AttributeError("cyclic alphabet cannot contain duplicates")
+
+        return str(alphabet)
+
+    @_validator
+    def cyclic_size(self, size):
+        """Cyclic pattern size.
+
+        Default value is `4`.
+        """
+        size = int(size)
+
+        if size > self.bytes:
+            raise AttributeError("cyclic pattern size cannot be larger than word size")
+
+        return size
+
    #*************************************************************************
    #                               ALIASES
    #*************************************************************************
@@ -1,13 +1,19 @@
-<%page args="binary, host=None, port=None, user=None, password=None, remote_path=None"/>\
+<%page args="binary, host=None, port=None, user=None, password=None, remote_path=None, quiet=False"/>\
 <%
+import os
+import sys
+
 from pwnlib.context import context as ctx
 from pwnlib.elf.elf import ELF
+from pwnlib.util.sh_string import sh_string
 from elftools.common.exceptions import ELFError

-import os
+argv = list(sys.argv)
+argv[0] = os.path.basename(argv[0])
+
 try:
    if binary:
-        ctx.binary = ELF(binary, checksec=False)
+       ctx.binary = ELF(binary, checksec=False)
 except ELFError:
    pass

@@ -28,9 +34,15 @@ binary_repr = repr(binary)
 %>\
 #!/usr/bin/env python2
 # -*- coding: utf-8 -*-
+%if not quiet:
+# This exploit template was generated via:
+# $ ${' '.join(map(sh_string, argv))}
+%endif
 from pwn import *

+%if not quiet:
 # Set up pwntools for the correct architecture
+%endif
 %if ctx.binary:
 exe = context.binary = ELF(${binary_repr})
 <% binary_repr = 'exe.path' %>
@@ -40,6 +52,7 @@ exe = ${binary_repr}
 <% binary_repr = 'exe' %>
 %endif

+%if not quiet:
 # Many built-in settings can be controlled on the command-line and show up
 # in "args".  For example, to dump all data sent/received, and disable ASLR
 # for all created processes...
@@ -47,6 +60,7 @@ exe = ${binary_repr}
 %if host or port or user:
 # ./exploit.py GDB HOST=example.com PORT=4141
 %endif
+%endif
 %if host:
 host = args.HOST or ${repr(host)}
 %endif
@@ -61,10 +75,60 @@ password = args.PASSWORD or ${repr(password)}
 remote_path = ${repr(remote_path)}
 %endif

+%if ssh:
+# Connect to the remote SSH server
+shell = None
+if not args.LOCAL:
+    shell = ssh(user, host, port, password)
+    shell.set_working_directory(symlink=True)
+%endif
+
+%if host:
+def local(argv=[], *a, **kw):
+    '''Execute the target binary locally'''
+    if args.GDB:
+        return gdb.debug([${binary_repr}] + argv, gdbscript=gdbscript, *a, **kw)
+    else:
+        return process([${binary_repr}] + argv, *a, **kw)
+
+def remote(argv=[], *a, **kw):
+  %if ssh:
+    '''Execute the target binary on the remote host'''
+    if args.GDB:
+        return gdb.debug([remote_path] + argv, gdbscript=gdbscript, ssh=shell, *a, **kw)
+    else:
+        return shell.process([remote_path] + argv, *a, **kw)
+  %else:
+    '''Connect to the process on the remote host'''
+    io = connect(host, port)
+    if args.GDB:
+        gdb.attach(io, gdbscript=gdbscript)
+    return io
+  %endif
+%endif
+
+%if host:
+def start(argv=[], *a, **kw):
+    '''Start the exploit against the target.'''
+    if args.LOCAL:
+        return local(argv, *a, **kw)
+    else:
+        return remote(argv, *a, **kw)
+%else:
+def start(argv=[], *a, **kw):
+    '''Start the exploit against the target.'''
+    if args.GDB:
+        return gdb.debug([${binary_repr}] + argv, gdbscript=gdbscript, *a, **kw)
+    else:
+        return process([${binary_repr}] + argv, *a, **kw)
+%endif
+
 %if exe or remote_path:
+%if not quiet:
 # Specify your GDB script here for debugging
 # GDB will be launched if the exploit is run via e.g.
 # ./exploit.py GDB
+%endif
 gdbscript = '''
 %if ctx.binary:
  %if 'main' in ctx.binary.symbols:
@@ -77,56 +141,27 @@ continue
 '''.format(**locals())
 %endif

-%if ssh:
-# Connect to the remote SSH server
-shell = None
-if not args.LOCAL:
-    shell = ssh(user, host, port, password)
-    shell.set_working_directory(symlink=True)
-%endif
-
-%if host:
-# Execute the target binary locally
-def local(argv=[], *a, **kw):
-    if args.GDB:
-        return gdb.debug([exe.path] + argv, gdbscript=gdbscript, *a, **kw)
-    else:
-        return process([exe.path] + argv, *a, **kw)
-
-%if ssh:
-# Execute the target binary on the remote host
-%else:
-# Connect to the process on the remote host
-%endif
-def remote(argv=[], *a, **kw):
-  %if ssh:
-    if args.GDB:
-        return gdb.debug([remote_path] + argv, gdbscript=gdbscript, ssh=shell, *a, **kw)
-    else:
-        return shell.process([remote_path] + argv, *a, **kw)
-  %else:
-    io = connect(host, port)
-    if args.GDB:
-        gdb.attach(io, gdbscript=gdbscript)
-    return io
-  %endif
-%endif
-
-%if host:
-start = local if args.LOCAL else remote
-%else:
-def start(argv=[], *a, **kw):
-    if args.GDB:
-        return gdb.debug([${binary_repr}] + argv, gdbscript=gdbscript, *a, **kw)
-    else:
-        return process([${binary_repr}] + argv, *a, **kw)
-%endif

+%if not quiet:
 #===========================================================
 #                    EXPLOIT GOES HERE
 #===========================================================
+%else:
+# -- Exploit goes here --
+%endif
+%if ctx.binary and not quiet:
+# ${'%-10s%s-%s-%s' % ('Arch:',
+                       ctx.binary.arch,
+                       ctx.binary.bits,
+                       ctx.binary.endian)}
+%for line in ctx.binary.checksec(color=False).splitlines():
+# ${line}
+%endfor
+%endif
+
 io = start()

+%if not quiet:
 # shellcode = asm(shellcraft.sh())
 # payload = fit({
 #     32: 0xdeadbeef,
@@ -135,5 +170,6 @@ io = start()
 # io.send(payload)
 # flag = io.recv(...)
 # log.success(flag)
+%endif

 io.interactive()
@@ -447,7 +447,7 @@ class DynELF(object):
            if debug:
                w.status("r_debug.linkmap")
                linkmap = self.leak.field(debug, r_debug.r_map)
-                w.status("r_debug.linkmap %#x" % result)
+                w.status("r_debug.linkmap %#x" % linkmap)

        if not linkmap:
            w.failure("Could not find DT_PLTGOT or DT_DEBUG")
@@ -389,7 +389,7 @@ class Corefile(ELF):

        >>> core.vdso.data[:4]
        '\x7fELF'
-        >>> core.libc # docteset: +ELLIPSIS
+        >>> core.libc # doctest: +ELLIPSIS
        Mapping('/lib/x86_64-linux-gnu/libc-...', ...)

        The corefile also contains a :attr:`.Corefile.stack` property, which gives
@@ -421,7 +421,7 @@ class Corefile(ELF):

        >>> s = ssh('travis', 'example.pwnme')
        >>> _ = s.set_working_directory()
-        >>> elf = ELF.from_assembly('int3')
+        >>> elf = ELF.from_assembly(shellcraft.trap())
        >>> path = s.upload(elf.path)
        >>> _ =s.chmod('+x', path)
        >>> io = s.process(path)
@@ -429,6 +429,55 @@ class Corefile(ELF):
        -1
        >>> io.corefile.signal == signal.SIGTRAP # doctest: +SKIP
        True
+
+        Make sure fault_addr synthesis works for amd64 on ret.
+
+        >>> context.clear(arch='amd64')
+        >>> elf = ELF.from_assembly('push 1234; ret')
+        >>> io = elf.process()
+        >>> io.wait()
+        >>> io.corefile.fault_addr
+        1234
+
+    Tests:
+
+        These are extra tests not meant to serve as examples.
+
+        Corefile.getenv() works correctly, even if the environment variable's
+        value contains embedded '='. Corefile is able to find the stack, even
+        if the stack pointer doesn't point at the stack.
+
+        >>> elf = ELF.from_assembly(shellcraft.crash())
+        >>> io = elf.process(env={'FOO': 'BAR=BAZ'})
+        >>> io.wait()
+        >>> core = io.corefile
+        >>> core.getenv('FOO')
+        'BAR=BAZ'
+        >>> core.sp == 0
+        True
+        >>> core.sp in core.stack
+        False
+
+        Corefile gracefully handles the stack being filled with garbage, including
+        argc / argv / envp being overwritten.
+
+        >>> context.clear(arch='i386')
+        >>> assembly = '''
+        ... LOOP:
+        ...   mov dword ptr [esp], 0x41414141
+        ...   pop eax
+        ...   jmp LOOP
+        ... '''
+        >>> elf = ELF.from_assembly(assembly)
+        >>> io = elf.process()
+        >>> io.wait()
+        >>> core = io.corefile
+        >>> core.argc, core.argv, core.env
+        (0, [], {})
+        >>> core.stack.data.endswith('AAAA')
+        True
+        >>> core.fault_addr == core.sp
+        True
    """

    _fill_gaps = False
@@ -454,6 +503,9 @@ class Corefile(ELF):
        #: variable.
        #:
        #: Note: Use with the :meth:`.ELF.string` method to extract them.
+        #:
+        #: Note: If FOO=BAR is in the environment, self.env['FOO'] is the
+        #:       address of the string "BAR\x00".
        self.env = {}

        #: :class:`list`: List of addresses of arguments on the stack.
@@ -664,7 +716,22 @@ class Corefile(ELF):

    @property
    def signal(self):
-        """:class:`int`: Signal which caused the core to be dumped."""
+        """:class:`int`: Signal which caused the core to be dumped.
+
+        Example:
+
+            >>> elf = ELF.from_assembly(shellcraft.trap())
+            >>> io = elf.process()
+            >>> io.wait()
+            >>> io.corefile.signal == signal.SIGTRAP
+            True
+
+            >>> elf = ELF.from_assembly(shellcraft.crash())
+            >>> io = elf.process()
+            >>> io.wait()
+            >>> io.corefile.signal == signal.SIGSEGV
+            True
+        """
        if self.siginfo:
            return int(self.siginfo.si_signo)
        if self.prstatus:
@@ -675,10 +742,40 @@ class Corefile(ELF):
        """:class:`int`: Address which generated the fault, for the signals
            SIGILL, SIGFPE, SIGSEGV, SIGBUS.  This is only available in native
            core dumps created by the kernel.  If the information is unavailable,
-            this returns the address of the instruction pointer."""
-        if self.siginfo:
-            return int(self.siginfo.sigfault_addr)
+            this returns the address of the instruction pointer.

+
+        Example:
+
+            >>> elf = ELF.from_assembly('mov eax, 0xdeadbeef; jmp eax', arch='i386')
+            >>> io = elf.process()
+            >>> io.wait()
+            >>> io.corefile.fault_addr == io.corefile.eax == 0xdeadbeef
+            True
+        """
+        fault_addr = 0
+        if self.siginfo:
+            fault_addr = int(self.siginfo.sigfault_addr)
+
+            # The fault_addr on AMD64 is zero if the crash occurs
+            # after a "ret" instruction, if the "ret" would return
+            # to an invalid address.  We need to extract the address
+            # manually, as a convenience.
+            if fault_addr == 0 and self.arch == 'amd64' and self.pc and self.sp:
+                try:
+                    code = self.read(self.pc, 1)
+                    if code != '\xc3':
+                        return fault_addr
+                    address = self.unpack(self.sp)
+                    return address
+                except Exception:
+                    # Could not read $rsp or $rip
+                    pass
+
+            return fault_addr
+
+        # No embedded siginfo structure, so just return the
+        # current instruction pointer.
        return getattr(self, 'pc', 0)

    @property
@@ -848,11 +945,31 @@ class Corefile(ELF):
        # Now we can fill in the environment
        env_pointer_data = stack[start_of_envp:p_last_env_addr+self.bytes]
        for pointer in unpack_many(env_pointer_data):
-            end = stack.find('=', last_env_addr)

-            if pointer in stack and end in stack:
-                name = stack[pointer:end]
-                self.env[name] = pointer
+            # If the stack is corrupted, the pointer will be outside of
+            # the stack.
+            if pointer not in stack:
+                continue
+
+            try:
+                name_value = self.string(pointer)
+            except Exception:
+                continue
+
+            name, value = name_value.split('=', 1)
+
+            # "end" points at the byte after the null terminator
+            end = pointer + len(name_value) + 1
+
+            # Do not mark things as environment variables if they point
+            # outside of the stack itself, or we had to cross into a different
+            # mapping (after the stack) to read it.
+            # This may occur when the entire stack is filled with non-NUL bytes,
+            # and we NULL-terminate on a read failure in .string().
+            if end not in stack:
+                continue
+
+            self.env[name] = pointer + len(name) + len('=')

        # May as well grab the arguments off the stack as well.
        # argc comes immediately before argv[0] on the stack, but
@@ -904,15 +1021,32 @@ class Corefile(ELF):

        Returns:
            :class:`str`: The contents of the environment variable.
+
+        Example:
+
+            >>> elf = ELF.from_assembly(shellcraft.trap())
+            >>> io = elf.process(env={'GREETING': 'Hello!'})
+            >>> io.wait()
+            >>> io.corefile.getenv('GREETING')
+            'Hello!'
        """
        if name not in self.env:
            log.error("Environment variable %r not set" % name)

-        return self.string(self.env[name]).split('=',1)[-1]
+        return self.string(self.env[name])

    @property
    def registers(self):
-        """:class:`dict`: All available registers in the coredump."""
+        """:class:`dict`: All available registers in the coredump.
+
+        Example:
+
+            >>> elf = ELF.from_assembly('mov eax, 0xdeadbeef;' + shellcraft.trap(), arch='i386')
+            >>> io = elf.process()
+            >>> io.wait()
+            >>> io.corefile.registers['eax'] == 0xdeadbeef
+            True
+        """
        if not self.prstatus:
            return {}

@@ -1020,6 +1154,7 @@ class CorefileFinder(object):
            new_path = 'core.%i' % core_pid
            if core_pid > 0 and new_path != self.core_path:
                write(new_path, self.read(self.core_path))
+                self.unlink(self.core_path)
                self.core_path = new_path

        # Check the PID
@@ -60,6 +60,7 @@ from elftools.elf.segments import InterpSegment
 import intervaltree

 from pwnlib import adb
+from pwnlib import qemu
 from pwnlib.asm import *
 from pwnlib.context import LocalContext
 from pwnlib.context import context
@@ -68,7 +69,6 @@ from pwnlib.elf.config import parse_kconfig
 from pwnlib.elf.datatypes import constants
 from pwnlib.elf.plt import emulate_plt_instructions
 from pwnlib.log import getLogger
-from pwnlib.qemu import get_qemu_arch
 from pwnlib.term import text
 from pwnlib.tubes.process import process
 from pwnlib.util import misc
@@ -261,6 +261,11 @@ class ELF(ELFFile):
                self.arch = 'mips64'
                self.bits = 64

+        # Is this a native binary? Should we be checking QEMU?
+        with context.local(arch=self.arch):
+            #: Whether this ELF should be able to run natively
+            self.native = context.native
+
        self._address = 0
        if self.elftype != 'DYN':
            for seg in self.iter_segments_by_type('PT_LOAD'):
@@ -409,10 +414,10 @@ class ELF(ELFFile):
        import pwnlib.gdb
        return pwnlib.gdb.debug([self.path] + argv, *a, **kw)

-    def _describe(self):
+    def _describe(self, *a, **kw):
        log.info_once('\n'.join((repr(self.path),
                                '%-10s%s-%s-%s' % ('Arch:', self.arch, self.bits, self.endian),
-                                self.checksec())))
+                                self.checksec(*a, **kw))))

    def __repr__(self):
        return "ELF(%r)" % self.path
@@ -630,10 +635,11 @@ class ELF(ELFFile):
                if os.path.exists(lib):
                    continue

-                qemu_lib = '/etc/qemu-binfmt/%s/%s' % (get_qemu_arch(arch=self.arch), lib)
-
-                if os.path.exists(qemu_lib):
-                    libs[os.path.realpath(qemu_lib)] = libs.pop(lib)
+                if not self.native:
+                    ld_prefix = qemu.ld_prefix()
+                    qemu_lib = os.path.exists(os.path.join(ld_prefix, lib))
+                    if qemu_lib:
+                        libs[os.path.realpath(qemu_lib)] = libs.pop(lib)

            self.libs = libs

@@ -1499,9 +1505,9 @@ class ELF(ELFFile):
        if not dt_runpath:
            return None

-        return self.dynamic_string(dt_rpath.entry.d_ptr)
+        return self.dynamic_string(dt_runpath.entry.d_ptr)

-    def checksec(self, banner=True):
+    def checksec(self, banner=True, color=True):
        """checksec(banner=True)

        Prints out information in the binary, similar to ``checksec.sh``.
@@ -1509,9 +1515,9 @@ class ELF(ELFFile):
        Arguments:
            banner(bool): Whether to print the path to the ELF binary.
        """
-        red    = text.red
-        green  = text.green
-        yellow = text.yellow
+        red    = text.red if color else str
+        green  = text.green if color else str
+        yellow = text.yellow if color else str

        res = []

@@ -1686,16 +1692,33 @@ class ELF(ELFFile):
        return packing.unpack(self.read(address, context.bytes), *a, **kw)

    def string(self, address):
-        """Reads a null-terminated string from the specified ``address``"""
+        """string(address) -> str
+
+        Reads a null-terminated string from the specified ``address``
+
+        Returns:
+            A ``str`` with the string contents (NUL terminator is omitted),
+            or an empty string if no NUL terminator could be found.
+        """
        data = ''
        while True:
-            c = self.read(address, 1)
+            read_size = 0x1000
+            partial_page = address & 0xfff
+
+            if partial_page:
+                read_size -= partial_page
+
+            c = self.read(address, read_size)
+
            if not c:
                return ''
-            if c == '\x00':
-                return data
+
            data += c
-            address += 1
+
+            if '\x00' in c:
+                return data[:data.index('\x00')]
+
+            address += len(c)

    def flat(self, address, *a, **kw):
        """Writes a full array of values to the specified address.
@@ -14,11 +14,9 @@ class ArmEncoder(Encoder):
    arch = 'arm'

    blacklist  = {chr(c) for c in range(256) if chr(c) in (string.ascii_letters + string.digits)}
+    icache_flush = 1

    def __call__(self, input, avoid, pcreg=None):
-
-        icache_flush = 1
-
        # If randomization is disabled, ensure that the seed
        # is always the same for the builder.
        state = random.getstate()
@@ -29,9 +27,9 @@ class ArmEncoder(Encoder):
            b = builder.builder()

            enc_data = b.enc_data_builder(input)
-            dec_loop = b.DecoderLoopBuilder(icache_flush)
+            dec_loop = b.DecoderLoopBuilder(self.icache_flush)
            enc_dec_loop = b.encDecoderLoopBuilder(dec_loop)
-            dec = b.DecoderBuilder(dec_loop, icache_flush)
+            dec = b.DecoderBuilder(dec_loop, self.icache_flush)

            output,dec = b.buildInit(dec);

@@ -100,6 +100,7 @@ import time
 from pwnlib import adb
 from pwnlib import atexit
 from pwnlib import elf
+from pwnlib import qemu
 from pwnlib import tubes
 from pwnlib.asm import _bfdname
 from pwnlib.asm import make_elf
@@ -107,7 +108,6 @@ from pwnlib.asm import make_elf_from_assembly
 from pwnlib.context import LocalContext
 from pwnlib.context import context
 from pwnlib.log import getLogger
-from pwnlib.qemu import get_qemu_user
 from pwnlib.util import misc
 from pwnlib.util import proc

@@ -127,10 +127,19 @@ def debug_assembly(asm, gdbscript=None, vma=None):
        asm(str): Assembly code to debug
        gdbscript(str): Script to run in GDB
        vma(int): Base address to load the shellcode at
-        **kwargs: Override any :obj:`.context` values.
+        **kwargs: Override any :obj:`pwnlib.context.context` values.

    Returns:
        :class:`.process`
+
+    Example:
+
+        .. code-block:: python
+
+            assembly = shellcraft.echo("Hello world!\n")
+            io = gdb.debug_assembly(assembly)
+            io.recvline()
+            # 'Hello world!'
    """
    tmp_elf = make_elf_from_assembly(asm, vma=vma, extract=False)
    os.chmod(tmp_elf, 0777)
@@ -153,10 +162,20 @@ def debug_shellcode(data, gdbscript=None, vma=None):
        data(str): Assembled shellcode bytes
        gdbscript(str): Script to run in GDB
        vma(int): Base address to load the shellcode at
-        **kwargs: Override any :obj:`.context` values.
+        **kwargs: Override any :obj:`pwnlib.context.context` values.

    Returns:
        :class:`.process`
+
+    Example:
+
+        .. code-block:: python
+
+            assembly = shellcraft.echo("Hello world!\n")
+            shellcode = asm(assembly)
+            io = gdb.debug_shellcode(shellcode)
+            io.recvline()
+            # 'Hello world!'
    """
    if isinstance(data, unicode):
        log.error("Shellcode is cannot be unicode.  Did you mean debug_assembly?")
@@ -290,14 +309,6 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
        from the very beginning.  This requires that both ``gdb`` and ``gdbserver``
        are installed on your machine.

-        .. code-block:: python
-
-            # Create a new process, and stop it at 'main'
-            io = gdb.debug('bash', '''
-            break main
-            continue
-            ''')
-
        When GDB opens via :func:`debug`, it will initially be stopped on the very first
        instruction of the dynamic linker (``ld.so``) for dynamically-linked binaries.

@@ -322,6 +333,22 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
            - Generally, you just add a few ``continue`` commands until things are set up
              the way you want it to be.

+    Examples:
+
+        .. code-block:: python
+
+            # Create a new process, and stop it at 'main'
+            io = gdb.debug('bash', '''
+            break main
+            continue
+            ''')
+
+            # Send a command to Bash
+            io.sendline("echo hello")
+
+            # Interact with the process
+            io.interactive()
+
        .. code-block:: python

            # Create a new process, and stop it at 'main'
@@ -336,6 +363,12 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
            continue
            ''')

+            # Send a command to Bash
+            io.sendline("echo hello")
+
+            # Interact with the process
+            io.interactive()
+
        You can use :func:`debug` to spawn new processes on remote machines as well,
        by using the ``ssh=`` keyword to pass in your :class:`.ssh` instance.

@@ -345,12 +378,18 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
            shell = ssh('passcode', 'pwnable.kr', 2222, password='guest')

            # Start a process on the server
-            bash = gdb.debug(['bash'],
+            io = gdb.debug(['bash'],
                            ssh=shell,
                            gdbscript='''
            break main
            continue
            ''')
+
+            # Send a command to Bash
+            io.sendline("echo hello")
+
+            # Interact with the process
+            io.interactive()
    """
    if isinstance(args, (int, tubes.process.process, tubes.ssh.ssh_channel)):
        log.error("Use gdb.attach() to debug a running process")
@@ -365,6 +404,8 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):

    runner = _get_runner(ssh)
    which  = _get_which(ssh)
+    sysroot = None
+    gdbscript = gdbscript or ''

    if context.noptrace:
        log.warn_once("Skipping debugger since context.noptrace==True")
@@ -374,7 +415,11 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
        args = _gdbserver_args(args=args, which=which)
    else:
        qemu_port = random.randint(1024, 65535)
-        args = [get_qemu_user(), '-g', str(qemu_port)] + args
+        qemu_user = qemu.user_path()
+        sysroot = qemu.ld_prefix(env)
+        if not qemu_user:
+            log.error("Cannot debug %s binaries without appropriate QEMU binaries" % context.arch)
+        args = [qemu_user, '-g', str(qemu_port)] + args

    # Make sure gdbserver/qemu is installed
    if not which(args[0]):
@@ -383,6 +428,8 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
    exe = exe or which(orig_args[0])
    if not exe:
        log.error("%s does not exist" % orig_args[0])
+    else:
+        gdbscript = 'file %s\n%s' % (exe, gdbscript)

    # Start gdbserver/qemu
    # (Note: We override ASLR here for the gdbserver process itself.)
@@ -401,7 +448,7 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
    if not ssh and context.os == 'android':
        host = context.adb_host

-    attach((host, port), exe=exe, gdbscript=gdbscript, need_ptrace_scope = False, ssh=ssh)
+    attach((host, port), exe=exe, gdbscript=gdbscript, need_ptrace_scope = False, ssh=ssh, sysroot=sysroot)

    # gdbserver outputs a message when a client connects
    garbage = gdbserver.recvline(timeout=1)
@@ -425,6 +472,11 @@ def binary():

    Returns:
        str: Path to the appropriate ``gdb`` binary to use.
+
+    Example:
+
+        >>> gdb.binary() # doctest: +SKIP
+        '/usr/bin/gdb'
    """
    gdb = misc.which('gdb')

@@ -443,7 +495,7 @@ def binary():
    return gdb

@LocalContext
-def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_args = None, ssh = None):
+def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_args = None, ssh = None, sysroot = None):
    """attach(target, gdbscript = None, exe = None, arch = None, ssh = None) -> None

    Start GDB in a new terminal and attach to `target`.
@@ -455,6 +507,7 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
        arch(str): Architechture of the target binary.  If `exe` known GDB will
          detect the architechture automatically (if it is supported).
        gdb_args(list): List of additional arguments to pass to GDB.
+        sysroot(str): Foreign-architecture sysroot, used for QEMU-emulated binaries

    Returns:
        PID of the GDB process (or the window which it is running in).
@@ -479,6 +532,8 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
            This will use the GDB installed on the remote machine.
            If a password is required to connect, the ``sshpass`` program must be installed.

+    Examples:
+
        .. code-block:: python

            # Attach directly to pid 1234
@@ -557,6 +612,8 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
    if not context.native:
        pre += 'set endian %s\n' % context.endian
        pre += 'set architecture %s\n' % get_gdb_arch()
+        if sysroot:
+            pre += 'set sysroot %s\n' % sysroot

        if context.os == 'android':
            pre += 'set gnutarget ' + _bfdname() + '\n'
@@ -611,6 +668,7 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
        pid = pids[0]
    elif isinstance(target, tubes.process.process):
        pid = proc.pidof(target)[0]
+        exe = exe or target.executable
    elif isinstance(target, tuple) and len(target) == 2:
        host, port = target
        pre += 'target remote %s:%d\n' % (host, port)
@@ -640,12 +698,19 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
    if not pid and not exe:
        log.error('could not find target process')

+    if exe:
+        pre += 'file %s\n' % exe
+
    cmd = binary()

    if gdb_args:
        cmd += ' '
        cmd += ' '.join(gdb_args)

+    if context.gdbinit:
+        cmd += ' -nh '                     # ignore ~/.gdbinit
+        cmd += ' -x %s ' % context.gdbinit # load custom gdbinit
+
    cmd += ' -q '

    if exe and context.native:
@@ -1,5 +1,81 @@
+"""Run foreign-architecture binaries
+
+Overview
+--------
+
+So you want to exploit ARM binaries on your Intel PC?
+
+Pwntools has a good level of integration with QEMU user-mode emulation,
+in order to run, debug, and pwn foreign architecture binaries.
+
+In general, everything magic happens "behind the scenes", and pwntools
+attempts to make your life easier.
+
+When using :class:`.process.process`, pwntools will attempt to blindly
+execute the binary, in case your system is configured to use ``binfmt-misc``.
+
+If this fails, pwntools will attempt to manually launch the binary under
+qemu user-mode emulation.  Preference is given to statically-linked variants,
+i.e. ``qemu-arm-static`` will be selected before ``qemu-arm``.
+
+Debugging
+~~~~~~~~~
+
+When debugging binaries with :func:`.gdb.debug`, pwntools automatically adds
+the appropriate command-line flags to QEMU to start its GDB stub, and
+automatically informs GDB of the correct architecture and sysroot.
+
+Sysroot
+~~~~~~~
+You can override the default sysroot by setting the ``QEMU_LD_PREFIX``
+environment variable.  This affects where ``qemu`` will look for files when
+``open()`` is called, e.g. when the linker is attempting to resolve ``libc.so``.
+
+Required Setup
+--------------
+
+For Ubuntu 16.04 and newer, the setup is relatively straightforward for most
+architectures.
+
+First, install the QEMU emulator itself.  If your binary is statically-linked,
+thsi is sufficient.
+
+    $ sudo apt-get install qemu-user
+
+If your binary is dynamically linked, you need to install libraries like libc.
+Generally, this package is named ``libc6-$ARCH-cross``, e.g. ``libc-mips-cross``.
+ARM comes in both soft-float and hard-float variants, e.g. ``armhf``.
+
+    $ sudo apt-get install libc6-arm64-cross
+
+If your binary relies on additional libraries, you can generally find them
+easily with ``apt-cache search``.  For example, if it's a C++ binary it
+may require ``libstdc++``.
+
+    $ apt-cache search 'libstdc++' | grep arm64
+
+Any other libraries that you require you'll have to find some other way.
+
+Telling QEMU Where Libraries Are
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The libraries are now installed on your system at e.g. ``/usr/aarch64-linux-gnu``.
+
+QEMU does not know where they are, and expects them to be at e.g. ``/etc/qemu-binfmt/aarch64``.
+If you try to run your library now, you'll probably see an error about ``libc.so.6`` missing.
+
+Create the ``/etc/qemu-binfmt`` directory if it does not exist, and create a symlink to
+the appropriate path.
+
+    $ sudo mkdir /etc/qemu-binfmt
+    $ sudo ln -s /usr/aarch64-linux-gnu /etc/qemu-binfmt/aarch64
+
+Now QEMU should be able to run the libraries.
+"""
 from __future__ import absolute_import

+import os
+
 from pwnlib.context import LocalContext
 from pwnlib.context import context
 from pwnlib.log import getLogger
@@ -8,14 +84,14 @@ from pwnlib.util import misc
 log = getLogger(__name__)

@LocalContext
-def get_qemu_arch():
+def archname():
    """
    Returns the name which QEMU uses for the currently selected
    architecture.

-    >>> get_qemu_arch()
+    >>> pwnlib.qemu.archname()
    'i386'
-    >>> get_qemu_arch(arch='powerpc')
+    >>> pwnlib.qemu.archname(arch='powerpc')
    'ppc'
    """
    return {
@@ -31,17 +107,17 @@ def get_qemu_arch():
    }.get((context.arch, context.endian), context.arch)

@LocalContext
-def get_qemu_user():
+def user_path():
    """
    Returns the path to the QEMU-user binary for the currently
    selected architecture.

-    >>> get_qemu_user()
+    >>> pwnlib.qemu.user_path()
    'qemu-i386-static'
-    >>> get_qemu_user(arch='thumb')
+    >>> pwnlib.qemu.user_path(arch='thumb')
    'qemu-arm-static'
    """
-    arch   = get_qemu_arch()
+    arch   = archname()
    normal = 'qemu-' + arch
    static = normal + '-static'

@@ -52,3 +128,32 @@ def get_qemu_user():
        return normal

    log.warn_once("Neither %r nor %r are available" % (normal, static))
+
+@LocalContext
+def ld_prefix(path=None, env=None):
+    """Returns the linker prefix for the selected qemu-user binary
+
+    >>> pwnlib.qemu.ld_prefix(arch='arm')
+    '/etc/qemu-binfmt/arm'
+    """
+    if path is None:
+        path = user_path()
+
+    # Did we explicitly specify the path in an environment variable?
+    if env and 'QEMU_LD_PREFIX' in env:
+        return env['QEMU_LD_PREFIX']
+
+    if 'QEMU_LD_PREFIX' in os.environ:
+        return os.environ['QEMU_LD_PREFIX']
+
+    # Cyclic imports!
+    from pwnlib.tubes.process import process
+
+    with context.quiet:
+        with process([path, '--help'], env=env) as io:
+            line = io.recvline_regex('QEMU_LD_PREFIX *=')
+
+    name, libpath = line.split('=', 1)
+
+    return libpath.strip()
+
@@ -40,7 +40,7 @@ standard Linux ABIs.
    0x0004:       0x64636261
    0x0008:              0x2
    0x000c:       0xdeadbeef read(4, 5, 6)
-    0x0010:           'eaaa' <pad>
+    0x0010:           'eaaa' <return address>
    0x0014:              0x4 arg0
    0x0018:              0x5 arg1
    0x001c:              0x6 arg2
@@ -55,19 +55,18 @@ The stack is automatically adjusted for the next frame
    0x0004:       0x64636261
    0x0008:              0x2
    0x000c:       0xdeadbeef read(4, 5, 6)
-    0x0010:       0x10000000 <adjust: add esp, 0x10; ret>
+    0x0010:       0x10000000 <adjust @0x24> add esp, 0x10; ret
    0x0014:              0x4 arg0
    0x0018:              0x5 arg1
    0x001c:              0x6 arg2
    0x0020:           'iaaa' <pad>
    0x0024:       0xdecafbad write(7, 8, 9)
-    0x0028:       0x10000000 <adjust: add esp, 0x10; ret>
+    0x0028:       0x10000000 <adjust @0x3c> add esp, 0x10; ret
    0x002c:              0x7 arg0
    0x0030:              0x8 arg1
    0x0034:              0x9 arg2
    0x0038:           'oaaa' <pad>
    0x003c:       0xfeedface exit()
-    0x0040:           'qaaa' <pad>

 ROP Example
 -------------------
@@ -107,19 +106,18 @@ Finally, let's build our ROP stack
    >>> rop.exit()
    >>> print rop.dump()
    0x0000:       0x10000012 write(STDOUT_FILENO, 268435494, 8)
-    0x0004:       0x1000000e <adjust: add esp, 0x10; ret>
+    0x0004:       0x1000000e <adjust @0x18> add esp, 0x10; ret
    0x0008:              0x1 arg0
    0x000c:       0x10000026 flag
    0x0010:              0x8 arg2
    0x0014:           'faaa' <pad>
    0x0018:       0x1000002f exit()
-    0x001c:           'haaa' <pad>

 The raw data from the ROP stack is available via `str`.

    >>> raw_rop = str(rop)
    >>> print enhex(raw_rop)
-    120000100e000010010000002600001008000000666161612f00001068616161
+    120000100e000010010000002600001008000000666161612f000010

 Let's try it out!

@@ -128,6 +126,48 @@ Let's try it out!
    >>> print p.recvall(timeout=5)
    The flag

+ROP Example (amd64)
+-------------------
+
+For amd64 binaries, the registers are loaded off the stack.  Pwntools can do
+basic reasoning about simple "pop; pop; add; ret"-style gadgets, and satisfy
+requirements so that everything "just works".
+
+    >>> context.clear(arch='amd64')
+    >>> assembly = 'pop rdx; pop rdi; pop rsi; add rsp, 0x20; ret; target: ret'
+    >>> binary = ELF.from_assembly(assembly)
+    >>> rop = ROP(binary)
+    >>> rop.target(1,2,3)
+    >>> print rop.dump()
+    0x0000:       0x10000000 pop rdx; pop rdi; pop rsi; add rsp, 0x20; ret
+    0x0008:              0x3 [arg2] rdx = 3
+    0x0010:              0x1 [arg0] rdi = 1
+    0x0018:              0x2 [arg1] rsi = 2
+    0x0020:       'iaaajaaa' <pad 0x20>
+    0x0028:       'kaaalaaa' <pad 0x18>
+    0x0030:       'maaanaaa' <pad 0x10>
+    0x0038:       'oaaapaaa' <pad 0x8>
+    0x0040:       0x10000008 target
+    >>> rop.target(1)
+    >>> print rop.dump()
+    0x0000:       0x10000000 pop rdx; pop rdi; pop rsi; add rsp, 0x20; ret
+    0x0008:              0x3 [arg2] rdx = 3
+    0x0010:              0x1 [arg0] rdi = 1
+    0x0018:              0x2 [arg1] rsi = 2
+    0x0020:       'iaaajaaa' <pad 0x20>
+    0x0028:       'kaaalaaa' <pad 0x18>
+    0x0030:       'maaanaaa' <pad 0x10>
+    0x0038:       'oaaapaaa' <pad 0x8>
+    0x0040:       0x10000008 target
+    0x0048:       0x10000001 pop rdi; pop rsi; add rsp, 0x20; ret
+    0x0050:              0x1 [arg0] rdi = 1
+    0x0058:       'waaaxaaa' <pad rsi>
+    0x0060:       'yaaazaab' <pad 0x20>
+    0x0068:       'baabcaab' <pad 0x18>
+    0x0070:       'daabeaab' <pad 0x10>
+    0x0078:       'faabgaab' <pad 0x8>
+    0x0080:       0x10000008 target
+
 ROP + Sigreturn
 -----------------------

@@ -160,7 +200,7 @@ That's all there is to it.

    >>> print rop.dump()
    0x0000:       0x1000000e pop eax; ret
-    0x0004:             0x77
+    0x0004:             0x77 [arg0] eax = SYS_sigreturn
    0x0008:       0x1000000b int 0x80
    0x000c:              0x0 gs
    0x0010:              0x0 fs
@@ -197,8 +237,10 @@ from __future__ import absolute_import
 import collections
 import copy
 import hashlib
+import itertools
 import os
 import re
+import string
 import sys
 import tempfile

@@ -215,9 +257,9 @@ from pwnlib.rop.call import CurrentStackPointer
 from pwnlib.rop.call import NextGadgetAddress
 from pwnlib.rop.call import StackAdjustment
 from pwnlib.rop.gadgets import Gadget
-from pwnlib.util import cyclic
 from pwnlib.util import lists
 from pwnlib.util import packing
+from pwnlib.util.cyclic import cyclic
 from pwnlib.util.packing import *

 log = getLogger(__name__)
@@ -228,6 +270,8 @@ class Padding(object):
    """
    Placeholder for exactly one pointer-width of padding.
    """
+    def __init__(self, name='<pad>'):
+        self.name = name

 class DescriptiveStack(list):
    """
@@ -306,16 +350,16 @@ class ROP(object):
    >>> r.execve(4, 5, 6)
    >>> print r.dump()
    0x0000:       0x10001234 funcname(1, 2)
-    0x0004:       0x10000003 <adjust: add esp, 0x10; ret>
+    0x0004:       0x10000003 <adjust @0x18> add esp, 0x10; ret
    0x0008:              0x1 arg0
    0x000c:              0x2 arg1
    0x0010:           'eaaa' <pad>
    0x0014:           'faaa' <pad>
    0x0018:       0x10001234 funcname(3)
-    0x001c:       0x10000007 <adjust: pop eax; ret>
+    0x001c:       0x10000007 <adjust @0x24> pop eax; ret
    0x0020:              0x3 arg0
    0x0024:       0x10000007 pop eax; ret
-    0x0028:             0x77
+    0x0028:             0x77 [arg0] eax = SYS_sigreturn
    0x002c:       0x10000000 int 0x80
    0x0030:              0x0 gs
    0x0034:              0x0 fs
@@ -344,16 +388,16 @@ class ROP(object):
    >>> r.execve(4, 5, 6)
    >>> print r.dump()
    0x8048000:       0x10001234 funcname(1, 2)
-    0x8048004:       0x10000003 <adjust: add esp, 0x10; ret>
+    0x8048004:       0x10000003 <adjust @0x8048018> add esp, 0x10; ret
    0x8048008:              0x1 arg0
    0x804800c:              0x2 arg1
    0x8048010:           'eaaa' <pad>
    0x8048014:           'faaa' <pad>
    0x8048018:       0x10001234 funcname(3)
-    0x804801c:       0x10000007 <adjust: pop eax; ret>
+    0x804801c:       0x10000007 <adjust @0x8048024> pop eax; ret
    0x8048020:              0x3 arg0
    0x8048024:       0x10000007 pop eax; ret
-    0x8048028:             0x77
+    0x8048028:             0x77 [arg0] eax = SYS_sigreturn
    0x804802c:       0x10000000 int 0x80
    0x8048030:              0x0 gs
    0x8048034:              0x0 fs
@@ -375,25 +419,24 @@ class ROP(object):
    0x8048074:              0x0 esp_at_signal
    0x8048078:             0x2b ss
    0x804807c:              0x0 fpstate
+
+
+    >>> elf = ELF.from_assembly('ret')
+    >>> r = ROP(elf)
+    >>> r.ret.address == 0x10000000
+    True
+    >>> r = ROP(elf, badchars='\x00')
+    >>> r.gadgets == {}
+    True
+    >>> r.ret is None
+    True
    """
-    #: List of individual ROP gadgets, ROP calls, SROP frames, etc.
-    #: This is intended to be the highest-level abstraction that we can muster.
-    _chain = []
-
-    #: List of ELF files which are available for mining gadgets
-    elfs = []
-
-    #: Stack address where the first byte of the ROP chain lies, if known.
-    base = 0
-
-    #: Whether or not the ROP chain directly sets the stack pointer to a value
-    #: which is not contiguous
-    migrated = False
-
-    def __init__(self, elfs, base = None, **kwargs):
+    def __init__(self, elfs, base = None, badchars = '', **kwargs):
        """
        Arguments:
            elfs(list): List of :class:`.ELF` objects for mining
+            base(int): Stack address where the first byte of the ROP chain lies, if known.
+            badchars(str): Characters which should not appear in ROP gadget addresses.
        """
        import ropgadget

@@ -402,10 +445,24 @@ class ROP(object):
            elfs = [elfs]
        elif isinstance(elfs, (str, unicode)):
            elfs = [ELF(elfs)]
-        self.elfs = elfs
+
+        #: List of individual ROP gadgets, ROP calls, SROP frames, etc.
+        #: This is intended to be the highest-level abstraction that we can muster.
        self._chain = []
+
+        #: List of ELF files which are available for mining gadgets
+        self.elfs = elfs
+
+        #: Stack address where the first byte of the ROP chain lies, if known.
        self.base = base
+
+        #: Whether or not the ROP chain directly sets the stack pointer to a value
+        #: which is not contiguous
        self.migrated = False
+
+        #: Characters which should not appear in ROP gadget addresses.
+        self._badchars = set(badchars)
+
        self.__load()

    @staticmethod
@@ -415,24 +472,93 @@ class ROP(object):

    def setRegisters(self, registers):
        """
-        Returns an OrderedDict of addresses/values which will set the specified
-        register context.
+        Returns an list of addresses/values which will set the specified register context.

        Arguments:
            registers(dict): Dictionary of ``{register name: value}``

        Returns:
-            An OrderedDict of ``{register: sequence of gadgets, values, etc.}``.
+            A list of tuples, ordering the stack.
+
+            Each tuple is in the form of ``(value, name)`` where ``value`` is either a
+            gadget address or literal value to go on the stack, and ``name`` is either
+            a string name or other item which can be "unresolved".
+
+        Note:
+            This is basically an implementation of the Set Cover Problem, which is
+            NP-hard.  This means that we will take polynomial time N**2, where N is
+            the number of gadgets.  We can reduce runtime by discarding useless and
+            inferior gadgets ahead of time.
        """
-        reg_order = collections.OrderedDict()
+        if not registers:
+            return []

-        for reg, value in registers.items():
-            gadget = self.find_gadget(['pop ' + reg, 'ret'])
-            if not gadget:
-                log.error("can't set %r" % reg)
-            reg_order[reg] = [gadget, value]
+        regset = set(registers)

-        return reg_order
+        # Collect all gadgets which use these registers
+        # Also collect the "best" gadget for each combination of registers
+        gadgets = []
+        best_gadgets = {}
+
+        for gadget in self.gadgets.values():
+            # Do not use gadgets which end in e.g. "int 0x80"
+            if gadget.insns[-1] != 'ret':
+                continue
+
+            touched = tuple(regset & set(gadget.regs))
+
+            if not touched:
+                continue
+
+            old = best_gadgets.get(touched, gadget)
+
+            if old is gadget or old.move > gadget.move:
+                best_gadgets[touched] = gadget
+
+
+        winner = None
+        budget = 999999999
+
+        for num_gadgets in range(len(registers)):
+            for combo in itertools.combinations(best_gadgets.values(), 1+num_gadgets):
+                # Is this better than what we can already do?
+                cost = sum((g.move for g in combo))
+                if cost > budget:
+                    continue
+
+                # Does it hit all of the registers we want?
+                coverage = set(sum((g.regs for g in combo), [])) & regset
+
+                if coverage != regset:
+                    continue
+
+                # It is better than what we had, and hits all of the registers.
+                winner = combo
+                budget = cost
+
+        if not winner:
+            log.error("Could not satisfy setRegisters(%r)", registers)
+
+        # We have our set of "winner" gadgets, let's build a stack!
+        stack = []
+
+        for gadget in winner:
+            moved = 8 # Account for the gadget itself
+            goodregs = set(gadget.regs) & regset
+            name = ",".join(goodregs)
+            stack.append((gadget.address, gadget))
+            for r in gadget.regs:
+                moved += 8
+                if r in registers:
+                    stack.append((registers[r], r))
+                else:
+                    stack.append((Padding('<pad %s>' % r), r))
+
+            for slot in range(moved, gadget.move, context.bytes):
+                left = gadget.move - slot
+                stack.append((Padding('<pad %#x>' % left), 'stack padding'))
+
+        return stack

    def resolve(self, resolvable):
        """Resolves a symbol to an address
@@ -486,8 +612,12 @@ class ROP(object):
        ''

        """
+
+        # Ensure we don't generate a cyclic pattern which contains badchars
+        alphabet = ''.join(c for c in string.ascii_lowercase if c not in self._badchars)
+
        if count:
-            return cyclic.cyclic(offset + count)[-count:]
+            return cyclic(offset + count, alphabet=alphabet)[-count:]
        return ''

    def describe(self, object):
@@ -575,13 +705,17 @@ class ROP(object):
                stack.describe(self.describe(slot))

                registers    = dict(zip(slot.abi.register_arguments, slot.args))
-                setRegisters = self.setRegisters(registers)

-                for register, gadgets in setRegisters.items():
-                    value       = registers[register]
-                    description = self.describe(value) or 'arg%i' % slot.args.index(value)
-                    stack.describe('set %s = %s' % (register, description))
-                    stack.extend(gadgets)
+                for value, name in self.setRegisters(registers):
+                    if name in registers:
+                        index = slot.abi.register_arguments.index(name)
+                        description = self.describe(value) or value
+                        stack.describe('[arg%s] %s = %s' % (index, name, description))
+                    elif isinstance(name, Gadget):
+                        stack.describe('; '.join(name.insns))
+                    elif isinstance(name, str):
+                        stack.describe(name)
+                    stack.append(value)

                if address != stack.next:
                    stack.describe(slot.name)
@@ -599,24 +733,33 @@ class ROP(object):
                # properly, but likely also need to adjust the stack past the
                # arguments.
                if slot.abi.returns:
-                    if remaining:
-                        fix_size  = (1 + len(stackArguments))
-                        fix_bytes = fix_size * context.bytes
-                        adjust   = self.search(move = fix_bytes)

-                        if not adjust:
-                            log.error("Could not find gadget to adjust stack by %#x bytes" % fix_bytes)
+                    # Save off the address of the next gadget
+                    if remaining or stackArguments:
+                        nextGadgetAddr = stack.next

-                        nextGadgetAddr = stack.next + adjust.move
+                    # If there were arguments on the stack, we need to stick something
+                    # in the slot where the return address goes.
+                    if len(stackArguments) > 0:
+                        if remaining:
+                            fix_size  = (1 + len(stackArguments))
+                            fix_bytes = fix_size * context.bytes
+                            adjust   = self.search(move = fix_bytes)

-                        stack.describe('<adjust: %s>' % self.describe(adjust))
-                        stack.append(adjust.address)
+                            if not adjust:
+                                log.error("Could not find gadget to adjust stack by %#x bytes" % fix_bytes)

-                        for pad in range(fix_bytes, adjust.move, context.bytes):
-                            stackArguments.append(Padding())
-                    else:
-                        stack.describe('<pad>')
-                        stack.append(Padding())
+                            nextGadgetAddr += adjust.move
+
+                            stack.describe('<adjust @%#x> %s' % (nextGadgetAddr, self.describe(adjust)))
+                            stack.append(adjust.address)
+
+                            for pad in range(fix_bytes, adjust.move, context.bytes):
+                                stackArguments.append(Padding())
+
+                        # We could not find a proper "adjust" gadget, but also didn't need one.
+                        else:
+                            stack.append(Padding("<return address>"))


                for i, argument in enumerate(stackArguments):
@@ -657,7 +800,7 @@ class ROP(object):

            elif isinstance(slot, Padding):
                stack[i] = self.generatePadding(i * context.bytes, context.bytes)
-                stack.describe('<pad>', slot_address)
+                stack.describe(slot.name, slot_address)

            elif isinstance(slot, Gadget):
                stack[i] = slot.address
@@ -835,13 +978,22 @@ class ROP(object):
        """Returns: Raw bytes of the ROP chain"""
        return self.chain()

-    def __get_cachefile_name(self, elf):
-        basename = os.path.basename(elf.file.name)
-        sha256 = hashlib.sha256(elf.get_data()).hexdigest()
+    def __get_cachefile_name(self, files):
+        """Given an ELF or list of ELF objects, return a cache file for the set of files"""
        cachedir = os.path.join(tempfile.gettempdir(), 'pwntools-rop-cache')
        if not os.path.exists(cachedir):
            os.mkdir(cachedir)
-        return os.path.join(cachedir, sha256)
+
+        if isinstance(files, ELF):
+            files = [files]
+
+        hashes = []
+
+        for elf in self.elfs:
+            sha256 = hashlib.sha256(elf.get_data()).hexdigest()
+            hashes.append(sha256)
+
+        return os.path.join(cachedir, '_'.join(hashes))

    def __cache_load(self, elf):
        filename = self.__get_cachefile_name(elf)
@@ -871,7 +1023,7 @@ class ROP(object):
        #

        pop   = re.compile(r'^pop (.{3})')
-        add   = re.compile(r'^add .sp, (\S+)$')
+        add   = re.compile(r'^add [er]sp, (\S+)$')
        ret   = re.compile(r'^ret$')
        leave = re.compile(r'^leave$')
        int80 = re.compile(r'int +0x80')
@@ -950,6 +1102,11 @@ class ROP(object):
        }[context.bytes]

        for addr, insns in gadgets.items():
+
+            # Filter out gadgets by address against badchars
+            if set(pack(addr)) & self._badchars:
+                continue
+
            sp_move = 0
            regs = []
            for insn in insns:
@@ -999,7 +1156,8 @@ class ROP(object):
        regs = set(regs or ())

        for addr, gadget in self.gadgets.items():
-            if gadget.move < move:          continue
+            if gadget.insns[-1] != 'ret':        continue
+            if gadget.move < move:               continue
            if not (regs <= set(gadget.regs)):   continue
            yield gadget

@@ -16,9 +16,9 @@ import subprocess
 import time
 import tty

+from pwnlib import qemu
 from pwnlib.context import context
 from pwnlib.log import getLogger
-from pwnlib.qemu import get_qemu_user
 from pwnlib.timeout import Timeout
 from pwnlib.tubes.tube import tube
 from pwnlib.util.hashes import sha256file
@@ -108,10 +108,6 @@ class process(tube):
        alarm(int):
            Set a SIGALRM alarm timeout on the process.

-    Attributes:
-
-        proc(subprocess)
-
    Examples:

        >>> p = process('python2')
@@ -238,7 +234,7 @@ class process(tube):
                raise TypeError('Must provide argv or set context.binary')


-        #: `subprocess.Popen` object
+        #: :class:`subprocess.Popen` object that backs this process
        self.proc = None

        if not shell:
@@ -436,21 +432,21 @@ class process(tube):

        # Determine what architecture the binary is, and find the
        # appropriate qemu binary to run it.
-        qemu = get_qemu_user(arch=binary.arch)
+        qemu_path = qemu.user_path(arch=binary.arch)

        if not qemu:
            raise exception

-        qemu = which(qemu)
+        qemu_path = which(qemu_path)
        if qemu:
-            self._qemu = qemu
+            self._qemu = qemu_path

-            args = [qemu]
+            args = [qemu_path]
            if self.argv:
                args += ['-0', self.argv[0]]
            args += ['--']

-            return [args, qemu]
+            return [args, qemu_path]

        # If we get here, we couldn't run the binary directly, and
        # we don't have a qemu which can run it.
@@ -957,3 +953,25 @@ class process(tube):
        with open('/proc/%i/mem' % self.pid, 'rb') as mem:
            mem.seek(address)
            return mem.read(count) or None
+
+    @property
+    def stdin(self):
+        """Shorthand for ``self.proc.stdin``
+
+        See: :obj:`.process.proc`
+        """
+        return self.proc.stdin
+    @property
+    def stdout(self):
+        """Shorthand for ``self.proc.stdout``
+
+        See: :obj:`.process.proc`
+        """
+        return self.proc.stdout
+    @property
+    def stderr(self):
+        """Shorthand for ``self.proc.stderr``
+
+        See: :obj:`.process.proc`
+        """
+        return self.proc.stderr
@@ -23,7 +23,7 @@ class remote(sock):
        typ: The string "tcp" or "udp" or an integer to pass to :func:`socket.getaddrinfo`.
        timeout: A positive number, None or the string "default".
        ssl(bool): Wrap the socket with SSL
-        sock(socket): Socket to inherit, rather than connecting
+        sock(socket.socket): Socket to inherit, rather than connecting

    Examples:

@@ -341,7 +341,8 @@ class ssh_process(ssh_channel):
        libs = self.parent.libs(self.executable)

        for lib in libs:
-            if self.executable in lib:
+            # Cannot just check "executable in lib", see issue #1047
+            if lib.endswith(self.executable):
                return pwnlib.elf.elf.ELF(lib)


@@ -2,15 +2,15 @@ from __future__ import absolute_import

 import string

-from pwnlib.context import context
+from pwnlib.context import context, LocalContext
 from pwnlib.log import getLogger
 from pwnlib.util import packing

 log = getLogger(__name__)

 # Taken from https://en.wikipedia.org/wiki/De_Bruijn_sequence but changed to a generator
-def de_bruijn(alphabet = string.ascii_lowercase, n = None):
-    """de_bruijn(alphabet = string.ascii_lowercase, n = 4) -> generator
+def de_bruijn(alphabet = None, n = None):
+    """de_bruijn(alphabet = None, n = None) -> generator

    Generator for a sequence of unique substrings of length `n`. This is implemented using a
    De Bruijn Sequence over the given `alphabet`.
@@ -21,8 +21,10 @@ def de_bruijn(alphabet = string.ascii_lowercase, n = None):
        alphabet: List or string to generate the sequence over.
        n(int): The length of subsequences that should be unique.
    """
+    if alphabet is None:
+        alphabet = context.cyclic_alphabet
    if n is None:
-        n = 4
+        n = context.cyclic_size
    k = len(alphabet)
    a = [0] * k * n
    def db(t, p):
@@ -42,8 +44,8 @@ def de_bruijn(alphabet = string.ascii_lowercase, n = None):

    return db(1,1)

-def cyclic(length = None, alphabet = string.ascii_lowercase, n = None):
-    """cyclic(length = None, alphabet = string.ascii_lowercase, n = 4) -> list/str
+def cyclic(length = None, alphabet = None, n = None):
+    """cyclic(length = None, alphabet = None, n = None) -> list/str

    A simple wrapper over :func:`de_bruijn`. This function returns at most
    `length` elements.
@@ -56,17 +58,68 @@ def cyclic(length = None, alphabet = string.ascii_lowercase, n = None):
        alphabet: List or string to generate the sequence over.
        n(int): The length of subsequences that should be unique.

+    Notes:
+        The maximum length is `len(alphabet)**n`.
+
+        The default values for `alphabet` and `n` restrict the total space to ~446KB.
+
+        If you need to generate a longer cyclic pattern, provide a longer `alphabet`,
+        or if possible a larger `n`.
+
    Example:
-        >>> cyclic(alphabet = "ABC", n = 3)
-        'AAABAACABBABCACBACCBBBCBCCC'
+
+        Cyclic patterns are usually generated by providing a specific `length`.
+
        >>> cyclic(20)
        'aaaabaaacaaadaaaeaaa'
+
+        >>> cyclic(32)
+        'aaaabaaacaaadaaaeaaafaaagaaahaaa'
+
+        The `alphabet` and `n` arguments will control the actual output of the pattern
+
+        >>> cyclic(20, alphabet=string.ascii_uppercase)
+        'AAAABAAACAAADAAAEAAA'
+
+        >>> cyclic(20, n=8)
+        'aaaaaaaabaaaaaaacaaa'
+
+        >>> cyclic(20, n=2)
+        'aabacadaeafagahaiaja'
+
+        The size of `n` and `alphabet` limit the maximum length that can be generated.
+        Without providing `length`, the entire possible cyclic space is generated.
+
+        >>> cyclic(alphabet = "ABC", n = 3)
+        'AAABAACABBABCACBACCBBBCBCCC'
+
+        >>> cyclic(length=512, alphabet = "ABC", n = 3)
+        Traceback (most recent call last):
+        ...
+        PwnlibException: Can't create a pattern length=512 with len(alphabet)==3 and n==3
+
+        The `alphabet` can be set in `context`, which is useful for circumstances
+        when certain characters are not allowed.  See :obj:`.context.cyclic_alphabet`.
+
+        >>> context.cyclic_alphabet = "ABC"
+        >>> cyclic(10)
+        'AAAABAAACA'
+
+        The original values can always be restored with:
+
+        >>> context.clear()
+
+        The following just a test to make sure the length is correct.
+
        >>> alphabet, n = range(30), 3
        >>> len(alphabet)**n, len(cyclic(alphabet = alphabet, n = n))
        (27000, 27000)
    """
    if n is None:
-        n = 4
+        n = context.cyclic_size
+
+    if alphabet is None:
+        alphabet = context.cyclic_alphabet

    if len(alphabet) ** n < length:
        log.error("Can't create a pattern length=%i with len(alphabet)==%i and n==%i" \
@@ -84,8 +137,9 @@ def cyclic(length = None, alphabet = string.ascii_lowercase, n = None):
    else:
        return out

-def cyclic_find(subseq, alphabet = string.ascii_lowercase, n = None):
-    """cyclic_find(subseq, alphabet = string.ascii_lowercase, n = None) -> int
+@LocalContext
+def cyclic_find(subseq, alphabet = None, n = None):
+    """cyclic_find(subseq, alphabet = None, n = None) -> int

    Calculates the position of a substring into a De Bruijn sequence.

@@ -103,25 +157,71 @@ def cyclic_find(subseq, alphabet = string.ascii_lowercase, n = None):
                integer. If an integer is provided it will be packed as a
                little endian integer.
        alphabet: List or string to generate the sequence over.
+                  By default, uses :obj:`.context.cyclic_alphabet`.
        n(int): The length of subsequences that should be unique.
-
+                By default, uses :obj:`.context.cyclic_size`.

    Examples:

+        Let's generate an example cyclic pattern.
+
+        >>> cyclic(16)
+        'aaaabaaacaaadaaa'
+
+        Note that 'baaa' starts at offset 4.  The `cyclic_find` routine shows us this:
+
+        >>> cyclic_find('baaa')
+        4
+
+        The *default* length of a subsequence generated by `cyclic` is `4`.
+        If a longer value is submitted, it is automatically truncated to four bytes.
+
+        >>> cyclic_find('baaacaaa')
+        4
+
+        If you provided e.g. `n=8` to `cyclic` to generate larger subsequences,
+        you must explicitly provide that argument.
+
+        >>> cyclic_find('baaacaaa', n=8)
+        3515208
+
+        We can generate a large cyclic pattern, and grab a subset of it to
+        check a deeper offset.
+
        >>> cyclic_find(cyclic(1000)[514:518])
        514
+
+        Instead of passing in the byte representation of the pattern, you can
+        also pass in the integer value.  Note that this is sensitive to the
+        selected endianness via `context.endian`.
+
        >>> cyclic_find(0x61616162)
        4
-    """
-    if isinstance(subseq, (int, long)):
-        width = 'all' if n is None else n * 8
-        subseq = packing.pack(subseq, width, 'little', False)
+        >>> cyclic_find(0x61616162, endian='big')
+        1

-    if n is None and len(subseq) != 4:
-        log.warn_once("cyclic_find() expects 4-byte subsequences by default, you gave %r\n" % subseq \
-            + "Unless you specified cyclic(..., n=%i), you probably just want the first 4 bytes.\n" % len(subseq) \
-            + "Truncating the data at 4 bytes.  Specify cyclic_find(..., n=%i) to override this." % len(subseq))
-        subseq = subseq[:4]
+        You can use anything for the cyclic pattern, including non-printable
+        characters.
+
+        >>> cyclic_find(0x00000000, alphabet=unhex('DEADBEEF00'))
+        621
+    """
+
+    if n is None:
+        n = context.cyclic_size
+
+    if isinstance(subseq, (int, long)):
+        subseq = packing.pack(subseq, bytes=n)
+
+    if len(subseq) != n:
+        log.warn_once("cyclic_find() expects %i-byte subsequences by default, you gave %r\n"\
+            + "Unless you specified cyclic(..., n=%i), you probably just want the first 4 bytes.\n"\
+            + "Truncating the data at 4 bytes.  Specify cyclic_find(..., n=%i) to override this.",
+            n, subseq, len(subseq), len(subseq))
+        subseq = subseq[:n]
+
+    if alphabet is None:
+        alphabet = context.cyclic_alphabet

    if any(c not in alphabet for c in subseq):
        return -1
@@ -534,8 +534,8 @@ def iter_except(func, exception):
    the end.

    Arguments:
-      func:  The function to call.
-      exception(exception):  The exception that signals the end.  Other
+      func(callable): The function to call.
+      exception(Exception):  The exception that signals the end.  Other
        exceptions will not be caught.

    Returns:
@@ -58,7 +58,7 @@ def pack(number, word_size = None, endianness = None, sign = None, **kwargs):

    Arguments:
        number (int): Number to convert
-        word_size (int): Word size of the converted integer or the string 'all'.
+        word_size (int): Word size of the converted integer or the string 'all' (in bits).
        endianness (str): Endianness of the converted integer ("little"/"big")
        sign (str): Signedness of the converted integer (False/True)
        kwargs: Anything that can be passed to context.local
@@ -171,7 +171,7 @@ def unpack(data, word_size = None):

    Arguments:
        number (int): String to convert
-        word_size (int): Word size of the converted integer or the string "all".
+        word_size (int): Word size of the converted integer or the string "all" (in bits).
        endianness (str): Endianness of the converted integer ("little"/"big")
        sign (str): Signedness of the converted integer (False/True)
        kwargs: Anything that can be passed to context.local
@@ -237,7 +237,7 @@ def unpack_many(data, word_size = None):

    Args
        number (int): String to convert
-        word_size (int): Word size of the converted integers or the string "all".
+        word_size (int): Word size of the converted integers or the string "all" (in bits).
        endianness (str): Endianness of the converted integer ("little"/"big")
        sign (str): Signedness of the converted integer (False/True)
        kwargs: Anything that can be passed to context.local
@@ -366,7 +366,7 @@ def make_packer(word_size = None, sign = None, **kwargs):
    faster to call this function, since it will then use a specialized version.

    Arguments:
-        word_size (int): The word size to be baked into the returned packer or the string all.
+        word_size (int): The word size to be baked into the returned packer or the string all (in bits).
        endianness (str): The endianness to be baked into the returned packer. ("little"/"big")
        sign (str): The signness to be baked into the returned packer. ("unsigned"/"signed")
        kwargs: Additional context flags, for setting by alias (e.g. ``endian=`` rather than index)
@@ -429,7 +429,7 @@ def make_unpacker(word_size = None, endianness = None, sign = None, **kwargs):
    faster to call this function, since it will then use a specialized version.

    Arguments:
-        word_size (int): The word size to be baked into the returned packer.
+        word_size (int): The word size to be baked into the returned packer (in bits).
        endianness (str): The endianness to be baked into the returned packer. ("little"/"big")
        sign (str): The signness to be baked into the returned packer. ("unsigned"/"signed")
        kwargs: Additional context flags, for setting by alias (e.g. ``endian=`` rather than index)
@@ -525,7 +525,7 @@ def flat(*args, **kwargs):
      preprocessor (function): Gets called on every element to optionally
         transform the element before flattening. If :const:`None` is
         returned, then the original value is uded.
-      word_size (int): Word size of the converted integer.
+      word_size (int): Word size of the converted integer (in bits).
      endianness (str): Endianness of the converted integer ("little"/"big").
      sign (str): Signedness of the converted integer (False/True)

@@ -577,7 +577,7 @@ def fit(pieces=None, **kwargs):
      preprocessor (function): Gets called on every element to optionally
         transform the element before flattening. If :const:`None` is
         returned, then the original value is used.
-      word_size (int): Word size of the converted integer.
+      word_size (int): Word size of the converted integer (in bits).
      endianness (str): Endianness of the converted integer ("little"/"big").
      sign (str): Signedness of the converted integer (False/True)

@@ -393,7 +393,7 @@ def sh_string(s):
        return "''"

    chars = set(s)
-    very_good = set(string.ascii_letters + string.digits + "_+.,/")
+    very_good = set(string.ascii_letters + string.digits + "_+.,/-")

    # Alphanumeric can always just be used verbatim.
    if chars <= very_good:
@@ -1 +1 @@
-__version__ = '3.9.2'
+__version__ = '3.11.0'
@@ -86,7 +86,7 @@ except Exception as e:
 setup(
    name                 = 'pwntools',
    packages             = find_packages(),
-    version              = '3.9.2',
+    version              = '3.11.0',
    data_files           = [('',
                             glob.glob('*.md') + glob.glob('*.txt')),
                            ],
@@ -90,7 +90,7 @@ setup_android_emulator()
        elif [[ -n "$TRAVIS_TAG" ]]; then
            echo "TRAVIS_TAG ($TRAVIS_TAG) indicates a new relase"
            echo "Forcing Android Emulator installation"
-        elif (git log --stat "$TRAVIS_COMMIT_RANGE" | grep -iE "android|adb"); then
+        elif (git log --stat "$TRAVIS_COMMIT_RANGE" | grep -iE "android|adb" | grep -v "commit "); then
            echo "Found Android-related commits, forcing Android Emulator installation"
        else
            # In order to avoid running the doctests that require the Android
@@ -119,7 +119,7 @@ setup_android_emulator()
    else
        if [ ! -f android-sdk/android ]; then
            # Install the SDK, which gives us the 'android' and 'emulator' commands
-            wget https://dl.google.com/android/android-sdk_r24.4.1-linux.tgz
+            wget -nv https://dl.google.com/android/android-sdk_r24.4.1-linux.tgz
            tar xf android-sdk_r24.4.1-linux.tgz
            rm  -f android-sdk_r24.4.1-linux.tgz

@@ -137,7 +137,7 @@ setup_android_emulator()
        # Install the NDK, which is required for adb.compile()
        NDK_VERSION=android-ndk-r12b
        if [ ! -f android-ndk/ndk-build ]; then
-            wget   https://dl.google.com/android/repository/$NDK_VERSION-linux-x86_64.zip
+            wget -nv https://dl.google.com/android/repository/$NDK_VERSION-linux-x86_64.zip
            unzip -q android-ndk-*.zip
            rm -f    android-ndk-*.zip
Author	SHA1	Message	Date
Mathias Svensson	d689805690	Release 3.11.0	2018-01-03 08:19:05 +01:00
Zach Riggle	953e73aa37	Update changelog for 3.11 and 3.12	2018-01-03 08:03:28 +01:00
Mathias Svensson	ee9f9fa7de	Merge branch 'stable' into beta	2018-01-03 08:02:50 +01:00
Benno Fünfstück	46a1c44fc9	elf.py: fix variable name typo (fixes dt_rpath undefined var) (#1008 )	2018-01-02 11:36:08 -06:00
Zach Riggle	6944b664e8	Switch server to ftp.ubuntu.org (#1084 ) Currently ftp.debian.org is down on port 21 (cherry picked from commit `ec5185c0c4`)	2017-12-11 17:47:21 -06:00
Mathias Svensson	712295f3c5	Release 3.11.0beta0	2017-10-25 12:06:47 +02:00
Mathias Svensson	cca1083f0b	Release 3.10.0	2017-10-25 12:01:59 +02:00
Mathias Svensson	9dacd1b5d9	Merge branch 'stable' into beta	2017-10-25 11:56:07 +02:00
Zach Riggle	d9f5a5b009	Update CHANGELOG.md	2017-10-23 23:48:53 -05:00
Zach Riggle	74a5876c0d	Fix corefile environment variables and speed up ELF.string() (#1055 ) * Fix corefile environment variables and speed up ELF.string() * Even more doctests * Fix doctest arch * No longer split on = since it isn't necessary * Fix arch again * Delete intermediate corefiles * Do not flag for extended testing when the commit SHA is what matches	2017-10-23 23:45:11 -05:00
Zach Riggle	9d6b4881fe	Fix path checking in ssh_process.corefile for edge cases (#1058 ) Fixes #1047	2017-10-23 15:37:48 -05:00
Zach Riggle	91025408a4	Fix incorrect variable in log statement (#1057 ) Fixes #1056	2017-10-23 14:39:12 -05:00
Zach Riggle	b53e58a840	Use "file /path/to/file" when debugging processes	2017-10-18 14:52:13 -05:00
Zach Riggle	438096f97f	Remove doctest prefixes, these are just examples	2017-10-18 14:41:49 -05:00
Zach Riggle	7bab49e7b9	Update GDB docs and examples	2017-10-15 15:31:39 -05:00
Zach Riggle	412d345964	QEMU-related enhancements (#1052 ) * Add support for finding QEMU's LD_PREFIX * Refactor pwnlib.qemu and add official documentation * Add more documentation, support for sysroot * More docs, binfmt info * Fix variable name overload * Fix missed refactor * Check for empty/null env * Add qemu.rst to index	2017-10-12 21:41:36 -05:00
Zach Riggle	b088f46988	Update CHANGELOG	2017-10-12 11:26:10 -05:00
Zach Riggle	8f3839d69e	Add cyclic default args to context (#1049 ) * Add cyclic default args to context Specifically, this adds context.cyclic_alphabet and context.cyclic_size This helps cut down on writing "n=8" all the time for users who use 8-byte subsequences on 64-bit architectures. Exposing the default alphabet means that users can avoid "bad bytes" in cyclic patterns which are generated by pwntools internally (e.g. ROP padding) Some pwnlib.util.packing docstrings were changed to clarify that word_size is in bits. The doctests in pwnlib.util.cyclic were updated and extended to have more and better doctests, as well as better exaples. * Add an example showing an alphabet with NUL bytes This is important because old code would use pack(..., "all") when an integer value was provided, instead of using the correct width.	2017-10-11 19:08:31 -05:00
Zach Riggle	5a81e71368	Update README	2017-10-11 16:59:37 -05:00
Zach Riggle	1577263334	Overhaul ROP.setRegisters to support more complex cases needed by AMD64 (#1044 ) * Overhaul ROP.setRegisters to support more complex cases needed by AMD64 Additionally, have a separate cache for ROPs composed of multiple ELF files * Better presentation for stack padding slots * Remove unnecessary stack adjustment on amd64 For amd64, it is possible that len(stackArguments) == 0. * Add a doctest for amd64 register dependency stuff * More i386 adjust optimizations * Use nextGadgetAddr as the adjust value * Always use padding for the last "ret" adjustment * Better dump display * Update doctests for new ROP dump() enhancements * Add badchars support for ROP gadgets * More doctest fixes, also ensure ROP.search does not return gadgets ending in e.g. int 0x80	2017-10-11 16:57:38 -05:00
Mathias Svensson	7ad5a4e34e	Merge branch 'beta' into dev	2017-10-05 09:59:45 +02:00
Mathias Svensson	513682373e	Release 3.10.0beta2	2017-10-05 09:44:33 +02:00
Zach Riggle	559ffa7c9e	Merge branch 'beta' into dev	2017-10-01 16:56:31 -05:00
Zach Riggle	2512026b3c	Merge branch 'stable' into beta	2017-10-01 16:56:09 -05:00
Zach Riggle	518ed71488	Better error message on gdb.debug when qemu-user is missing	2017-10-01 12:27:23 -05:00
Zach Riggle	26d60cbf63	Expose icache_flush for the ARM alphanumeric encoder	2017-10-01 12:27:04 -05:00
Zach Riggle	6577f24d3a	Add --shared to shellcraft	2017-09-29 16:15:26 -05:00
Mathias Svensson	4a7ff431b9	Merge branch 'beta' into dev	2017-09-28 11:41:59 +02:00
Mathias Svensson	716c67d993	Release 3.10.0beta1	2017-09-28 11:37:08 +02:00
Zach Riggle	9ed8ab6d4f	Reduce log spam when downloading the Android NDK	2017-09-26 12:22:16 -05:00
Zach Riggle	19f7268701	Reduce log spam when downloading the Android SDK	2017-09-26 12:21:04 -05:00
Zach Riggle	e4d919d6bf	Update README.md for 3.9.1 release	2017-09-26 12:16:57 -05:00
Zach Riggle	93d9a2cb5e	Merge branch 'beta' into dev	2017-09-26 12:13:56 -05:00
Zach Riggle	69f8cc2c47	Merge branch 'stable' into beta	2017-09-26 12:12:20 -05:00
Zach Riggle	47d5cf7da1	Make some docs warnings go away and add some accessor methodds (#1035 )	2017-09-22 16:23:47 -07:00
Zach Riggle	a7263688db	Make 'pwn template' better (#1032 ) * Slightly enhance template and advertise for the feature itself in scripts Add single-dash to the "very good" characters in shell escaping, to avoid the need to single-quote things like --host and --port. * Add --quiet option for pwn template, move gdbscript down a bit * Slightly better docs for 'pwn template' * Add checksec information directly to 'pwn template' output	2017-09-21 18:55:03 -07:00
Zach Riggle	630c4651fa	Synthesize fault_addr on amd64 if it is zero. (#1031 ) * Synthesize fault_addr on amd64 if it is zero. Fixes Gallopsled/pwntool#1018 * Add a doctest for fault_addr on amd64 * Fix doctest	2017-09-21 17:53:32 -07:00
Zach Riggle	54c47b3daa	Try to cut down on the crazy verbosity of logging (#1036 ) * Prevent Sphinx from dumping all level=INFO logging output to stdout. See Gallopsled/pwntools#1036 for more information. * Filter out Paramiko logs too, apply filter to all root handlers during testing * Remove unused import	2017-09-21 17:50:53 -07:00
Zach Riggle	ebbe917713	Fix some unicode problems in readline (#1029 ) Technically this introduces issues with e.g. 'unhex ffdf \| python foo.py' if foo.py uses .interactive(). However, that was already broken for other reasons. Fixes #1004	2017-09-16 20:11:58 -05:00
Mathias Svensson	b62804eff6	Begin working on 3.11.0	2017-09-11 19:13:20 +02:00
Mathias Svensson	ec98dc1246	Release 3.10.0beta0	2017-09-11 18:59:06 +02:00
Zach Riggle	7cd3c7b300	Merge branch 'beta' into dev	2017-09-08 12:41:42 -05:00
Benno Fünfstück	5831796f6f	Add configurable gdbinit (#1007 ) * fix typo in comment * new config option: context.gdbinit This allows setting the default gdbinit for GDB, which is helpful when you want to enable some GDB extension only when using pwntools and not by default.	2017-09-06 12:56:55 -05:00
Mathias Svensson	3e33180655	Update CHANGELOG	2017-07-29 13:26:21 +02:00
Mathias Svensson	e09543c098	Start working on 3.10	2017-07-29 13:02:55 +02:00