Release 3.12.0

* Fix test breakage caused by Sphinx deprecation

* Fix missing import in RST

Closes #1112

CHANGELOG.md

@@ -9,9 +9,12 @@ The table below shows which release corresponds to each branch, and what date th
 
 | Version          | Branch   | Release Date           |
 | ---------------- | -------- | ---------------------- |
-| [3.11.0](#3110)  | `dev`    | Dec 2, 2017 (planned)
-| [3.10.0](#3100)  | `beta`   | Oct 21, 2017 (planned)
-| [3.9.2](#392)    | `stable` | Oct 5, 2017
+| [3.14.0](#3130)  | `dev`    | Jun 12, 2018 (planned)
+| [3.13.0](#3130)  | `beta`   | Mar 31, 2018 (planned)
+| [3.12.0](#3120)  | `stable` | Feb 22, 2018
+| [3.11.0](#3110)  |          | Jan 3, 2018
+| [3.10.0](#3100)  |          | Oct 25, 2017
+| [3.9.2](#392)    |          | Oct 5, 2017
 | [3.9.1](#391)    |          | Sep 28, 2017
 | [3.9.0](#390)    |          | Sep 11, 2017
 | [3.8.0](#380)    |          | Jul 29, 2017
@@ -39,13 +42,80 @@ The table below shows which release corresponds to each branch, and what date th
 | [3.0.0](#300)    |          | Aug 20, 2016
 | [2.2.0](#220)    |          | Jan 5, 2015
 
+## 3.14.0
+
+To be released on Jun 12, 2018.
+
+## 3.13.0
+
+To be released on Mar 31, 2018.
+
+- [#1104][1104] Add `DynELF.dump()` for dumping remote ELF files
+- [#1101][1101] Set `context.os` via `context.binary`, useful for Android exploitation
+- [5fdc08][5fdc08] Work around broken `pidof` on Android
+- [63dfed][63dfed] Print warning when Corefile deletion fails instead of throwing an exception
+- [#1094][1094] Make hexdump output alignment more consistent
+- [#1096][1096] `flat()` and `fit()` are now the same function
+- [#1068][1068] Work around very old OpenSSL versions which don't have sha256 support *AND* don't exit with an error code when trying to use it
+
+[1104]: https://github.com/Gallopsled/pwntools/pull/1104
+[1101]: https://github.com/Gallopsled/pwntools/pull/1101
+[1094]: https://github.com/Gallopsled/pwntools/pull/1094
+[1096]: https://github.com/Gallopsled/pwntools/pull/1096
+[1068]: https://github.com/Gallopsled/pwntools/pull/1068
+[5fdc08]: https://github.com/Gallopsled/pwntools/commit/5fdc08
+[63dfed]: https://github.com/Gallopsled/pwntools/commit/63dfed
+
+## 3.12.0
+
+- [#1083][1083] Better error messages for `gdb` when `LD_PRELOAD` is incorrect
+- [#1085][1085] Add support for extracting Android `BOOTLDR!` images
+- [#1075][1075] Add support for detecting GNU Screen for `run_in_new_terminal`
+- [#1074][1074] Add support for running `pwntools-gdb` wrapper script instead of `gdb`
+- [#1067][1067] Add `pwnlib.tubes.server` module, which adds a reusable `server` listener
+- [#1063][1063] Add support for labels in `fit()`, allowing dynamic contents to be injected.  (This feature is really cool, check out the pull request!)
+
+[1083]: https://github.com/Gallopsled/pwntools/pull/1083
+[1085]: https://github.com/Gallopsled/pwntools/pull/1085
+[1075]: https://github.com/Gallopsled/pwntools/pull/1075
+[1074]: https://github.com/Gallopsled/pwntools/pull/1074
+[1067]: https://github.com/Gallopsled/pwntools/pull/1067
+[1063]: https://github.com/Gallopsled/pwntools/pull/1063
+
 ## 3.11.0
 
-To be released on Dec 2, 2017.
+- [#1044][1044] Enhancements to ROP
+    + Much better support for 64-bit Intel (amd64) ROP
+    + ROP gadget selection is optimized to favor multi-pops instead of multiple single-pop gadgets
+    + Added support for blacklisting byte values in ROP gadget addresses
+- [#1049][1049] Enhancements to `cyclic`
+    + `context` now has two additional attributes, `cyclic_alphabet` and `cyclic_length`, which correspond to the arguments `alphabet` and `n` to `cyclic()` and `cyclic_find()` and related routines.
+    + The motivation for this change is to allow setting the `alphabet` globally, so that any padding / patterns generated internally to pwntools can be controlled.  The specific motivation is blacklisting values in ROP padding.
+- [#1052][1052] Enhancements for detecting `QEMU_LD_PREFIX` used by QEMU user-mode emulation for sysroots
+- [#1035][1035] Minor documentation changes
+- [#1032][1032] Enhancements to `pwn template`
+- [#1031][1031] More accurate `Coredump.fault_addr` on amd64
+- [#1084][1084] Fix broken tests due to `ftp.debian.org` going down
+
+[1044]: https://github.com/Gallopsled/pwntools/pull/1044
+[1049]: https://github.com/Gallopsled/pwntools/pull/1049
+[1052]: https://github.com/Gallopsled/pwntools/pull/1052
+[1035]: https://github.com/Gallopsled/pwntools/pull/1035
+[1032]: https://github.com/Gallopsled/pwntools/pull/1032
+[1031]: https://github.com/Gallopsled/pwntools/pull/1031
+[1084]: https://github.com/Gallopsled/pwntools/pull/1084
 
 ## 3.10.0
 
-To be released on Oct 21, 2017.
+- [#1007][1007] Add support for setting a `gdbinit` file in the context
+- [#1055][1055] Fixes for `Corefile` stack parsing, speed up `ELF.string()`
+- [#1057][1057] Fix a variable name typo in `DynELF` logging which results in an exception being thrown
+- [#1058][1058] Fix an edge case in `ssh_process.exe`
+
+[1007]: https://github.com/Gallopsled/pwntools/pull/1007
+[1055]: https://github.com/Gallopsled/pwntools/pull/1055
+[1057]: https://github.com/Gallopsled/pwntools/pull/1057
+[1058]: https://github.com/Gallopsled/pwntools/pull/1058
 
 ## 3.9.2
 

README.md

@@ -2,7 +2,7 @@
 ![pwntools logo](https://github.com/Gallopsled/pwntools/blob/stable/docs/source/logo.png?raw=true)
 
 [![Docs](https://readthedocs.org/projects/pwntools/badge/?version=stable)](https://docs.pwntools.com/)
-[![PyPI](https://img.shields.io/badge/pypi-v3.9.2-green.svg?style=flat)](https://pypi.python.org/pypi/pwntools/)
+[![PyPI](https://img.shields.io/badge/pypi-v3.12.0-green.svg?style=flat)](https://pypi.python.org/pypi/pwntools/)
 [![Travis](https://travis-ci.org/Gallopsled/pwntools.svg)](https://travis-ci.org/Gallopsled/pwntools)
 [![Coveralls](https://img.shields.io/coveralls/Gallopsled/pwntools/dev.svg)](https://coveralls.io/github/Gallopsled/pwntools?branch=dev)
 [![Twitter](https://img.shields.io/badge/twitter-pwntools-4099FF.svg?style=flat)](https://twitter.com/pwntools)

docs/source/conf.py

@@ -24,6 +24,27 @@ sys.path.insert(0, os.path.abspath('../..'))
 
 import pwnlib
 
+# -- WORK-AROUNDS FOR DEPRECATION ----------------------------------------------
+# Deprecated
+# 1.6b1
+# sphinx.util.compat.Directive class is now deprecated.
+# Please use instead docutils.parsers.rst.Directive
+#
+# Pwntools Note:
+#   Can't just "do the right thing" since we have dependencies that
+#   are also affected by this, specifically sphinxcontrib.autoprogram
+try:
+    import sphinx.util.compat
+except ImportError:
+    import sys
+    import types
+    import sphinx.util
+    import docutils.parsers.rst
+    class compat(types.ModuleType):
+        Directive = docutils.parsers.rst.Directive
+    sphinx.util.compat = compat('sphinx.util.compat')
+    sys.modules['sphinx.util.compat'] = sphinx.util.compat
+
 # -- General configuration -----------------------------------------------------
 
 # If your documentation needs a minimal Sphinx version, state it here.

docs/source/context.rst

@@ -8,7 +8,7 @@
 :mod:`pwnlib.context` --- Setting runtime variables
 =====================================================
 
-Many settings in ``pwntools`` are controlled via the global variable :data:`.context, such as the selected target operating system, architecture, and bit-width.
+Many settings in ``pwntools`` are controlled via the global variable :data:`.context`, such as the selected target operating system, architecture, and bit-width.
 
 In general, exploits will start with something like:
 

docs/source/elf/corefile.rst

@@ -1,5 +1,6 @@
 .. testsetup:: *
 
+   from glob import glob
    from pwn import *
 
 :mod:`pwnlib.elf.corefile` --- Core Files

docs/source/globals.rst

@@ -13,7 +13,7 @@ Which imports a bazillion things into the global namespace to make your life eas
 
 This is a quick list of most of the objects and routines imported, in rough order of importance and frequency of use.
 
-- :obj:`.context`
+- :mod:`pwnlib.context`
     - :data:`pwnlib.context.context`
     - Responsible for most of the pwntools convenience settings
     - Set `context.log_level = 'debug'` when troubleshooting your exploit

docs/source/index.rst

@@ -61,6 +61,7 @@ Each of the ``pwntools`` modules is documented here.
    log
    memleak
    protocols
+   qemu
    replacements
    rop
    rop/*

docs/source/intro.rst

@@ -27,7 +27,7 @@ This exposes a standard interface to talk to processes, sockets, serial ports,
 and all manner of things, along with some nifty helpers for common tasks.
 For example, remote connections via :mod:`pwnlib.tubes.remote`.
 
-    >>> conn = remote('ftp.debian.org',21)
+    >>> conn = remote('ftp.ubuntu.org',21)
     >>> conn.recvline() # doctest: +ELLIPSIS
     '220 ...'
     >>> conn.send('USER anonymous\r\n')

docs/source/qemu.rst

@@ -0,0 +1,10 @@
+.. testsetup:: *
+
+   from pwn import *
+
+
+:mod:`pwnlib.qemu` --- QEMU Utilities
+==========================================
+
+.. automodule:: pwnlib.qemu
+   :members:

docs/source/tubes/sockets.rst

@@ -1,6 +1,7 @@
 .. testsetup:: *
 
    from pwn import *
+   from pwnlib.tubes.server import server
 
 :mod:`pwnlib.tubes.sock` --- Sockets
 ===========================================================
@@ -23,3 +24,9 @@
    .. autoclass:: pwnlib.tubes.listen.listen
       :members:
       :show-inheritance:
+
+.. automodule:: pwnlib.tubes.server
+
+   .. autoclass:: pwnlib.tubes.server.server
+      :members:
+      :show-inheritance:

docs/source/util/cyclic.rst

@@ -1,5 +1,6 @@
 .. testsetup:: *
 
+   from pwn import *
    from pwnlib.util.cyclic import *
 
 

pwn/toplevel.py

@@ -43,6 +43,7 @@ from pwnlib.tubes.listen import listen
 from pwnlib.tubes.process import process, PTY, PIPE, STDOUT
 from pwnlib.tubes.remote import remote, tcp, udp, connect
 from pwnlib.tubes.serialtube import serialtube
+from pwnlib.tubes.server import server
 from pwnlib.tubes.ssh import ssh
 from pwnlib.tubes.tube import tube
 from pwnlib.ui import *

pwnlib/abi.py

@@ -124,8 +124,3 @@ linux_aarch64_sigreturn = SigreturnABI('sp', ['x8'], 16, 0)
 
 windows_i386  = ABI('esp', [], 4, 0)
 windows_amd64 = ABI('rsp', ['rcx','rdx','r8','r9'], 32, 32)
-
-# Fake ABIs used by SROP
-linux_i386_srop = ABI('esp', ['eax'], 4, 0)
-linux_amd64_srop = ABI('rsp', ['rax'], 4, 0)
-linux_arm_srop = ABI('sp', ['r7'], 4, 0)

pwnlib/adb/adb.py

@@ -820,6 +820,7 @@ echo $PATH | while read -d: directory; do
     [ -x "$directory/{name}" ] || continue;
     echo -n "$directory/{name}\\x00";
 done
+[ -x "{name}" ] && echo -n "$PWD/{name}\\x00"
 '''.format(name=name)
 
     which_cmd = which_cmd.strip()

pwnlib/adb/bootloader.py

@@ -0,0 +1,103 @@
+from __future__ import unicode_literals
+
+import ctypes
+import io
+import os
+import sys
+
+class img_info(ctypes.Structure):
+    _fields_ = [
+        ('name', ctypes.c_char * 64),
+        ('size', ctypes.c_uint32)
+    ]
+
+class bootloader_images_header(ctypes.Structure):
+    _fields_ = [
+        ('magic', ctypes.c_char * 8),
+        ('num_images', ctypes.c_uint32),
+        ('start_offset', ctypes.c_uint32),
+        ('bootldr_size', ctypes.c_uint32),
+    ]
+
+    def __init__(self, *a, **kw):
+        super(bootloader_images_header, self).__init__(*a, **kw)
+        if self.magic != self.MAGIC:
+            raise ValueError("Incorrect magic (%r, expected %r)" % (self.magic, self.MAGIC))
+    MAGIC = 'BOOTLDR!'
+
+class BootloaderImage(object):
+    def __init__(self, data):
+        """Android Bootloader image
+
+        Arguments:
+            data(str): Binary data from the image file.
+        """
+        self.data = data
+        self.header = bootloader_images_header.from_buffer_copy(data)
+
+        imgarray = ctypes.ARRAY(img_info, self.header.num_images)
+        self.img_info = imgarray.from_buffer_copy(data, ctypes.sizeof(self.header))
+
+    def extract(self, index_or_name):
+        """extract(index_or_name) -> bytes
+
+        Extract the contents of an image.
+
+        Arguments:
+            index_or_name(str,int): Either an image index or name.
+
+        Returns:
+            Contents of the image.
+        """
+        if isinstance(index_or_name, int):
+            index = index_or_name
+        else:
+            for i in range(len(self.img_info)):
+                if self.img_info[i].name == index_or_name:
+                    index = i
+                    break
+            else:
+                raise ValueError("Invalid img name: %r" % index_or_name)
+
+        if index >= len(self.img_info):
+            raise ValueError("index out of range (%s, max %s)" % (index, len(self.img_info)))
+
+        offset = self.header.start_offset
+
+        for i in range(index):
+            offset += self.img_info[i].size
+
+        return self.data[offset:offset + self.img_info[index].size]
+
+    def extract_all(self, path):
+        """extract_all(path)
+
+        Extracts all images to the provided path.  The filenames are taken
+        from the image name, with '.img' appended.
+        """
+        if not os.path.isdir(path):
+            raise ValueError("%r does not exist or is not a directory" % path)
+
+        for img in self.img_info:
+            imgpath = os.path.join(path, img.name + '.img')
+            with open(imgpath, 'wb+') as f:
+                data = self.extract(img.name)
+                f.write(data)
+
+    def __str__(self):
+        rv = []
+        rv.append("Bootloader")
+        rv.append("  Magic:  %r" % self.header.magic)
+        rv.append("  Offset: %#x" % self.header.start_offset)
+        rv.append("  Size:   %#x" % self.header.bootldr_size)
+        rv.append("  Images: %s" % self.header.num_images)
+        for img in self.img_info:
+            rv.append("    Name: %s" % img.name)
+            rv.append("    Size: %#x" % img.size)
+            rv.append("    Data: %r..." % self.extract(img.name)[:32])
+        return '\n'.join(rv)
+
+if __name__ == '__main__':
+    # Easy sanity checking
+    b = BootloaderImage(open(sys.argv[1]).read())
+    print(b)

pwnlib/commandline/debug.py

@@ -30,13 +30,25 @@ parser.add_argument(
     help = 'The os/architecture/endianness/bits the shellcode will run in (default: linux/i386), choose from: %s' % common.choices,
 )
 parser.add_argument(
-    '--exec', type=file, dest='executable',
+    '--exec',
+
+    # NOTE: Type cannot be "file" because we may be referring to a remote
+    #       file, or a file on an Android device.
+    type=str,
+
+    dest='executable',
     help='File to debug'
 )
 parser.add_argument(
     '--process', metavar='PROCESS_NAME',
     help='Name of the process to attach to (e.g. "bash")'
 )
+parser.add_argument(
+    '--sysroot', metavar='SYSROOT',
+    type=str,
+    default='',
+    help="GDB sysroot path"
+)
 
 def main(args):
     gdbscript = ''
@@ -47,23 +59,41 @@ def main(args):
         context.device = adb.wait_for_device()
 
     if args.executable:
-        context.binary = ELF(args.executable.name)
-        target = context.binary.path
+        if os.path.exists(args.executable):
+            context.binary = ELF(args.executable)
+            target = context.binary.path
+
+        # This path does nothing, but avoids the "print_usage()"
+        # path below.
+        elif context.os == 'android':
+            target = args.executable
     elif args.pid:
         target = int(args.pid)
     elif args.process:
         if context.os == 'android':
-            target = adb.pidof(process)
+            target = adb.pidof(args.process)
         else:
-            target = pidof(process)
+            target = pidof(args.process)
+
+        # pidof() returns a list
+        if not target:
+            log.error("Could not find a PID for %r", args.process)
+        target = target[0]
     else:
         parser.print_usage()
         return 1
 
     if args.pid or args.process:
-        gdb.attach(target, gdbscript=gdbscript)
+        pid = gdb.attach(target, gdbscript=gdbscript, sysroot=args.sysroot)
+
+        # Since we spawned the gdbserver process, and process registers an
+        # atexit handler to close itself, gdbserver will be terminated when
+        # we exit.  This will manifest as a "remote connected ended" or
+        # similar error message.  Hold it open for the user.
+        log.info("GDB connection forwarding will terminate when you press enter")
+        pause()
     else:
-        gdb.debug(target, gdbscript=gdbscript).interactive()
+        gdb.debug(target, gdbscript=gdbscript, sysroot=args.sysroot).interactive()
 
 if __name__ == '__main__':
     pwnlib.commandline.common.main(__file__)

pwnlib/commandline/shellcraft.py

@@ -164,6 +164,12 @@ p.add_argument(
     help='List available shellcodes, optionally provide a filter'
 )
 
+p.add_argument(
+    '-s', '--shared',
+    action='store_true',
+    help='Generated ELF is a shared library'
+)
+
 def get_template(name):
     func = shellcraft
     for attr in name.split('.'):
@@ -296,11 +302,11 @@ def main(args):
 
 
         if not args.avoid:
-            code = read(make_elf_from_assembly(assembly, vma=vma))
+            code = read(make_elf_from_assembly(assembly, vma=vma, shared=args.shared))
         else:
             code = asm(assembly)
             code = encode(code, args.avoid)
-            code = make_elf(code, vma=vma)
+            code = make_elf(code, vma=vma, shared=args.shared)
             # code = read(make_elf(encode(asm(code), args.avoid)))
     else:
         code = encode(asm(assembly), args.avoid)

pwnlib/commandline/template.py

@@ -19,6 +19,7 @@ parser.add_argument('--port', help='Remote port / SSH port', type=int)
 parser.add_argument('--user', help='SSH Username')
 parser.add_argument('--pass', help='SSH Password', dest='password')
 parser.add_argument('--path', help='Remote path of file on SSH server')
+parser.add_argument('--quiet', help='Less verbose template comments', action='store_true')
 
 def main(args):
     cache = None
@@ -49,7 +50,8 @@ def main(args):
                              args.port,
                              args.user,
                              args.password,
-                             args.path)
+                             args.path,
+                             args.quiet)
 
     # Fix Mako formatting bs
     output = re.sub('\n\n\n', '\n\n', output)

pwnlib/context/__init__.py

@@ -150,7 +150,7 @@ class _Tls_DictStack(threading.local, _DictStack):
 
 def _validator(validator):
     """
-    Validator that tis tightly coupled to the implementation
+    Validator that is tightly coupled to the implementation
     of the classes here.
 
     This expects that the object has a ._tls property which
@@ -337,9 +337,12 @@ class ContextType(object):
         'binary': None,
         'bits': 32,
         'buffer_size': 4096,
+        'cyclic_alphabet': string.ascii_lowercase,
+        'cyclic_size': 4,
         'delete_corefiles': False,
         'device': os.getenv('ANDROID_SERIAL', None) or None,
         'endian': 'little',
+        'gdbinit': "",
         'kernel': None,
         'log_level': logging.INFO,
         'log_file': _devnull(),
@@ -1231,6 +1234,49 @@ class ContextType(object):
         """
         return bool(v)
 
+
+    @_validator
+    def gdbinit(self, value):
+        """Path to the gdbinit that is used when running GDB locally.
+
+        This is useful if you want pwntools-launched GDB to include some additional modules,
+        like PEDA but you do not want to have GDB include them by default.
+
+        The setting will only apply when GDB is launched locally since remote hosts may not have
+        the necessary requirements for the gdbinit.
+
+        If set to an empty string, GDB will use the default `~/.gdbinit`.
+
+        Default value is ``""``.
+        """
+        return str(value)
+
+    @_validator
+    def cyclic_alphabet(self, alphabet):
+        """Cyclic alphabet.
+
+        Default value is `string.ascii_lowercase`.
+        """
+
+        # Do not allow multiple occurrences
+        if len(set(alphabet)) != len(alphabet):
+            raise AttributeError("cyclic alphabet cannot contain duplicates")
+
+        return str(alphabet)
+
+    @_validator
+    def cyclic_size(self, size):
+        """Cyclic pattern size.
+
+        Default value is `4`.
+        """
+        size = int(size)
+
+        if size > self.bytes:
+            raise AttributeError("cyclic pattern size cannot be larger than word size")
+
+        return size
+
     #*************************************************************************
     #                               ALIASES
     #*************************************************************************

pwnlib/data/templates/pwnup.mako

@@ -1,13 +1,19 @@
-<%page args="binary, host=None, port=None, user=None, password=None, remote_path=None"/>\
+<%page args="binary, host=None, port=None, user=None, password=None, remote_path=None, quiet=False"/>\
 <%
+import os
+import sys
+
 from pwnlib.context import context as ctx
 from pwnlib.elf.elf import ELF
+from pwnlib.util.sh_string import sh_string
 from elftools.common.exceptions import ELFError
 
-import os
+argv = list(sys.argv)
+argv[0] = os.path.basename(argv[0])
+
 try:
     if binary:
-        ctx.binary = ELF(binary, checksec=False)
+       ctx.binary = ELF(binary, checksec=False)
 except ELFError:
     pass
 
@@ -28,9 +34,15 @@ binary_repr = repr(binary)
 %>\
 #!/usr/bin/env python2
 # -*- coding: utf-8 -*-
+%if not quiet:
+# This exploit template was generated via:
+# $ ${' '.join(map(sh_string, argv))}
+%endif
 from pwn import *
 
+%if not quiet:
 # Set up pwntools for the correct architecture
+%endif
 %if ctx.binary:
 exe = context.binary = ELF(${binary_repr})
 <% binary_repr = 'exe.path' %>
@@ -40,6 +52,7 @@ exe = ${binary_repr}
 <% binary_repr = 'exe' %>
 %endif
 
+%if not quiet:
 # Many built-in settings can be controlled on the command-line and show up
 # in "args".  For example, to dump all data sent/received, and disable ASLR
 # for all created processes...
@@ -47,6 +60,7 @@ exe = ${binary_repr}
 %if host or port or user:
 # ./exploit.py GDB HOST=example.com PORT=4141
 %endif
+%endif
 %if host:
 host = args.HOST or ${repr(host)}
 %endif
@@ -61,10 +75,60 @@ password = args.PASSWORD or ${repr(password)}
 remote_path = ${repr(remote_path)}
 %endif
 
+%if ssh:
+# Connect to the remote SSH server
+shell = None
+if not args.LOCAL:
+    shell = ssh(user, host, port, password)
+    shell.set_working_directory(symlink=True)
+%endif
+
+%if host:
+def local(argv=[], *a, **kw):
+    '''Execute the target binary locally'''
+    if args.GDB:
+        return gdb.debug([${binary_repr}] + argv, gdbscript=gdbscript, *a, **kw)
+    else:
+        return process([${binary_repr}] + argv, *a, **kw)
+
+def remote(argv=[], *a, **kw):
+  %if ssh:
+    '''Execute the target binary on the remote host'''
+    if args.GDB:
+        return gdb.debug([remote_path] + argv, gdbscript=gdbscript, ssh=shell, *a, **kw)
+    else:
+        return shell.process([remote_path] + argv, *a, **kw)
+  %else:
+    '''Connect to the process on the remote host'''
+    io = connect(host, port)
+    if args.GDB:
+        gdb.attach(io, gdbscript=gdbscript)
+    return io
+  %endif
+%endif
+
+%if host:
+def start(argv=[], *a, **kw):
+    '''Start the exploit against the target.'''
+    if args.LOCAL:
+        return local(argv, *a, **kw)
+    else:
+        return remote(argv, *a, **kw)
+%else:
+def start(argv=[], *a, **kw):
+    '''Start the exploit against the target.'''
+    if args.GDB:
+        return gdb.debug([${binary_repr}] + argv, gdbscript=gdbscript, *a, **kw)
+    else:
+        return process([${binary_repr}] + argv, *a, **kw)
+%endif
+
 %if exe or remote_path:
+%if not quiet:
 # Specify your GDB script here for debugging
 # GDB will be launched if the exploit is run via e.g.
 # ./exploit.py GDB
+%endif
 gdbscript = '''
 %if ctx.binary:
   %if 'main' in ctx.binary.symbols:
@@ -77,56 +141,27 @@ continue
 '''.format(**locals())
 %endif
 
-%if ssh:
-# Connect to the remote SSH server
-shell = None
-if not args.LOCAL:
-    shell = ssh(user, host, port, password)
-    shell.set_working_directory(symlink=True)
-%endif
-
-%if host:
-# Execute the target binary locally
-def local(argv=[], *a, **kw):
-    if args.GDB:
-        return gdb.debug([exe.path] + argv, gdbscript=gdbscript, *a, **kw)
-    else:
-        return process([exe.path] + argv, *a, **kw)
-
-%if ssh:
-# Execute the target binary on the remote host
-%else:
-# Connect to the process on the remote host
-%endif
-def remote(argv=[], *a, **kw):
-  %if ssh:
-    if args.GDB:
-        return gdb.debug([remote_path] + argv, gdbscript=gdbscript, ssh=shell, *a, **kw)
-    else:
-        return shell.process([remote_path] + argv, *a, **kw)
-  %else:
-    io = connect(host, port)
-    if args.GDB:
-        gdb.attach(io, gdbscript=gdbscript)
-    return io
-  %endif
-%endif
-
-%if host:
-start = local if args.LOCAL else remote
-%else:
-def start(argv=[], *a, **kw):
-    if args.GDB:
-        return gdb.debug([${binary_repr}] + argv, gdbscript=gdbscript, *a, **kw)
-    else:
-        return process([${binary_repr}] + argv, *a, **kw)
-%endif
 
+%if not quiet:
 #===========================================================
 #                    EXPLOIT GOES HERE
 #===========================================================
+%else:
+# -- Exploit goes here --
+%endif
+%if ctx.binary and not quiet:
+# ${'%-10s%s-%s-%s' % ('Arch:',
+                       ctx.binary.arch,
+                       ctx.binary.bits,
+                       ctx.binary.endian)}
+%for line in ctx.binary.checksec(color=False).splitlines():
+# ${line}
+%endfor
+%endif
+
 io = start()
 
+%if not quiet:
 # shellcode = asm(shellcraft.sh())
 # payload = fit({
 #     32: 0xdeadbeef,
@@ -135,5 +170,6 @@ io = start()
 # io.send(payload)
 # flag = io.recv(...)
 # log.success(flag)
+%endif
 
 io.interactive()

pwnlib/dynelf.py

@@ -447,7 +447,7 @@ class DynELF(object):
             if debug:
                 w.status("r_debug.linkmap")
                 linkmap = self.leak.field(debug, r_debug.r_map)
-                w.status("r_debug.linkmap %#x" % result)
+                w.status("r_debug.linkmap %#x" % linkmap)
 
         if not linkmap:
             w.failure("Could not find DT_PLTGOT or DT_DEBUG")

pwnlib/elf/corefile.py

@@ -389,7 +389,7 @@ class Corefile(ELF):
 
         >>> core.vdso.data[:4]
         '\x7fELF'
-        >>> core.libc # docteset: +ELLIPSIS
+        >>> core.libc # doctest: +ELLIPSIS
         Mapping('/lib/x86_64-linux-gnu/libc-...', ...)
 
         The corefile also contains a :attr:`.Corefile.stack` property, which gives
@@ -421,7 +421,7 @@ class Corefile(ELF):
 
         >>> s = ssh('travis', 'example.pwnme')
         >>> _ = s.set_working_directory()
-        >>> elf = ELF.from_assembly('int3')
+        >>> elf = ELF.from_assembly(shellcraft.trap())
         >>> path = s.upload(elf.path)
         >>> _ =s.chmod('+x', path)
         >>> io = s.process(path)
@@ -429,6 +429,55 @@ class Corefile(ELF):
         -1
         >>> io.corefile.signal == signal.SIGTRAP # doctest: +SKIP
         True
+
+        Make sure fault_addr synthesis works for amd64 on ret.
+
+        >>> context.clear(arch='amd64')
+        >>> elf = ELF.from_assembly('push 1234; ret')
+        >>> io = elf.process()
+        >>> io.wait()
+        >>> io.corefile.fault_addr
+        1234
+
+    Tests:
+
+        These are extra tests not meant to serve as examples.
+
+        Corefile.getenv() works correctly, even if the environment variable's
+        value contains embedded '='. Corefile is able to find the stack, even
+        if the stack pointer doesn't point at the stack.
+
+        >>> elf = ELF.from_assembly(shellcraft.crash())
+        >>> io = elf.process(env={'FOO': 'BAR=BAZ'})
+        >>> io.wait()
+        >>> core = io.corefile
+        >>> core.getenv('FOO')
+        'BAR=BAZ'
+        >>> core.sp == 0
+        True
+        >>> core.sp in core.stack
+        False
+
+        Corefile gracefully handles the stack being filled with garbage, including
+        argc / argv / envp being overwritten.
+
+        >>> context.clear(arch='i386')
+        >>> assembly = '''
+        ... LOOP:
+        ...   mov dword ptr [esp], 0x41414141
+        ...   pop eax
+        ...   jmp LOOP
+        ... '''
+        >>> elf = ELF.from_assembly(assembly)
+        >>> io = elf.process()
+        >>> io.wait()
+        >>> core = io.corefile
+        >>> core.argc, core.argv, core.env
+        (0, [], {})
+        >>> core.stack.data.endswith('AAAA')
+        True
+        >>> core.fault_addr == core.sp
+        True
     """
 
     _fill_gaps = False
@@ -454,6 +503,9 @@ class Corefile(ELF):
         #: variable.
         #:
         #: Note: Use with the :meth:`.ELF.string` method to extract them.
+        #:
+        #: Note: If FOO=BAR is in the environment, self.env['FOO'] is the
+        #:       address of the string "BAR\x00".
         self.env = {}
 
         #: :class:`list`: List of addresses of arguments on the stack.
@@ -664,7 +716,22 @@ class Corefile(ELF):
 
     @property
     def signal(self):
-        """:class:`int`: Signal which caused the core to be dumped."""
+        """:class:`int`: Signal which caused the core to be dumped.
+
+        Example:
+
+            >>> elf = ELF.from_assembly(shellcraft.trap())
+            >>> io = elf.process()
+            >>> io.wait()
+            >>> io.corefile.signal == signal.SIGTRAP
+            True
+
+            >>> elf = ELF.from_assembly(shellcraft.crash())
+            >>> io = elf.process()
+            >>> io.wait()
+            >>> io.corefile.signal == signal.SIGSEGV
+            True
+        """
         if self.siginfo:
             return int(self.siginfo.si_signo)
         if self.prstatus:
@@ -675,11 +742,49 @@ class Corefile(ELF):
         """:class:`int`: Address which generated the fault, for the signals
             SIGILL, SIGFPE, SIGSEGV, SIGBUS.  This is only available in native
             core dumps created by the kernel.  If the information is unavailable,
-            this returns the address of the instruction pointer."""
-        if self.siginfo:
-            return int(self.siginfo.sigfault_addr)
+            this returns the address of the instruction pointer.
 
-        return getattr(self, 'pc', 0)
+
+        Example:
+
+            >>> elf = ELF.from_assembly('mov eax, 0xdeadbeef; jmp eax', arch='i386')
+            >>> io = elf.process()
+            >>> io.wait()
+            >>> io.corefile.fault_addr == io.corefile.eax == 0xdeadbeef
+            True
+        """
+        if not self.siginfo:
+            return getattr(self, 'pc', 0)
+
+        fault_addr = int(self.siginfo.sigfault_addr)
+
+        # The fault_addr is zero if the crash occurs due to a
+        # "protection fault", e.g. a dereference of 0x4141414141414141
+        # because this is technically a kernel address.
+        #
+        # A protection fault does not set "fault_addr" in the siginfo.
+        # (http://elixir.free-electrons.com/linux/v4.14-rc8/source/kernel/signal.c#L1052)
+        #
+        # Since a common use for corefiles is to spray the stack with a
+        # cyclic pattern to find the offset to get control of $PC,
+        # check for a "ret" instruction ("\xc3").
+        #
+        # If we find a RET at $PC, extract the "return address" from the
+        # top of the stack.
+        if fault_addr == 0 and self.siginfo.si_code == 0x80:
+            try:
+                code = self.read(self.pc, 1)
+                RET = '\xc3'
+                if code == RET:
+                    fault_addr = self.unpack(self.sp)
+            except Exception:
+                # Could not read $rsp or $rip
+                pass
+
+        return fault_addr
+
+        # No embedded siginfo structure, so just return the
+        # current instruction pointer.
 
     @property
     def _pc_register(self):
@@ -722,8 +827,8 @@ class Corefile(ELF):
         fields = [
             repr(self.path),
             '%-10s %s' % ('Arch:', gnu_triplet),
-            '%-10s %#x' % ('%s:' % self._pc_register.upper(), getattr(self, 'pc', 0)),
-            '%-10s %#x' % ('%s:' % self._sp_register.upper(), getattr(self, 'sp', 0)),
+            '%-10s %#x' % ('%s:' % self._pc_register.upper(), self.pc or 0),
+            '%-10s %#x' % ('%s:' % self._sp_register.upper(), self.sp or 0),
         ]
 
         if self.exe and self.exe.name:
@@ -848,11 +953,31 @@ class Corefile(ELF):
         # Now we can fill in the environment
         env_pointer_data = stack[start_of_envp:p_last_env_addr+self.bytes]
         for pointer in unpack_many(env_pointer_data):
-            end = stack.find('=', last_env_addr)
 
-            if pointer in stack and end in stack:
-                name = stack[pointer:end]
-                self.env[name] = pointer
+            # If the stack is corrupted, the pointer will be outside of
+            # the stack.
+            if pointer not in stack:
+                continue
+
+            try:
+                name_value = self.string(pointer)
+            except Exception:
+                continue
+
+            name, value = name_value.split('=', 1)
+
+            # "end" points at the byte after the null terminator
+            end = pointer + len(name_value) + 1
+
+            # Do not mark things as environment variables if they point
+            # outside of the stack itself, or we had to cross into a different
+            # mapping (after the stack) to read it.
+            # This may occur when the entire stack is filled with non-NUL bytes,
+            # and we NULL-terminate on a read failure in .string().
+            if end not in stack:
+                continue
+
+            self.env[name] = pointer + len(name) + len('=')
 
         # May as well grab the arguments off the stack as well.
         # argc comes immediately before argv[0] on the stack, but
@@ -904,15 +1029,32 @@ class Corefile(ELF):
 
         Returns:
             :class:`str`: The contents of the environment variable.
+
+        Example:
+
+            >>> elf = ELF.from_assembly(shellcraft.trap())
+            >>> io = elf.process(env={'GREETING': 'Hello!'})
+            >>> io.wait()
+            >>> io.corefile.getenv('GREETING')
+            'Hello!'
         """
         if name not in self.env:
             log.error("Environment variable %r not set" % name)
 
-        return self.string(self.env[name]).split('=',1)[-1]
+        return self.string(self.env[name])
 
     @property
     def registers(self):
-        """:class:`dict`: All available registers in the coredump."""
+        """:class:`dict`: All available registers in the coredump.
+
+        Example:
+
+            >>> elf = ELF.from_assembly('mov eax, 0xdeadbeef;' + shellcraft.trap(), arch='i386')
+            >>> io = elf.process()
+            >>> io.wait()
+            >>> io.corefile.registers['eax'] == 0xdeadbeef
+            True
+        """
         if not self.prstatus:
             return {}
 
@@ -1020,6 +1162,7 @@ class CorefileFinder(object):
             new_path = 'core.%i' % core_pid
             if core_pid > 0 and new_path != self.core_path:
                 write(new_path, self.read(self.core_path))
+                self.unlink(self.core_path)
                 self.core_path = new_path
 
         # Check the PID

pwnlib/elf/elf.py

@@ -60,6 +60,7 @@ from elftools.elf.segments import InterpSegment
 import intervaltree
 
 from pwnlib import adb
+from pwnlib import qemu
 from pwnlib.asm import *
 from pwnlib.context import LocalContext
 from pwnlib.context import context
@@ -68,7 +69,6 @@ from pwnlib.elf.config import parse_kconfig
 from pwnlib.elf.datatypes import constants
 from pwnlib.elf.plt import emulate_plt_instructions
 from pwnlib.log import getLogger
-from pwnlib.qemu import get_qemu_arch
 from pwnlib.term import text
 from pwnlib.tubes.process import process
 from pwnlib.util import misc
@@ -261,6 +261,15 @@ class ELF(ELFFile):
                 self.arch = 'mips64'
                 self.bits = 64
 
+        # Is this a native binary? Should we be checking QEMU?
+        try:
+            with context.local(arch=self.arch):
+                #: Whether this ELF should be able to run natively
+                self.native = context.native
+        except AttributeError:
+            # The architecture may not be supported in pwntools
+            self.native = False
+
         self._address = 0
         if self.elftype != 'DYN':
             for seg in self.iter_segments_by_type('PT_LOAD'):
@@ -409,10 +418,10 @@ class ELF(ELFFile):
         import pwnlib.gdb
         return pwnlib.gdb.debug([self.path] + argv, *a, **kw)
 
-    def _describe(self):
+    def _describe(self, *a, **kw):
         log.info_once('\n'.join((repr(self.path),
                                 '%-10s%s-%s-%s' % ('Arch:', self.arch, self.bits, self.endian),
-                                self.checksec())))
+                                self.checksec(*a, **kw))))
 
     def __repr__(self):
         return "ELF(%r)" % self.path
@@ -630,10 +639,11 @@ class ELF(ELFFile):
                 if os.path.exists(lib):
                     continue
 
-                qemu_lib = '/etc/qemu-binfmt/%s/%s' % (get_qemu_arch(arch=self.arch), lib)
-
-                if os.path.exists(qemu_lib):
-                    libs[os.path.realpath(qemu_lib)] = libs.pop(lib)
+                if not self.native:
+                    ld_prefix = qemu.ld_prefix()
+                    qemu_lib = os.path.exists(os.path.join(ld_prefix, lib))
+                    if qemu_lib:
+                        libs[os.path.realpath(qemu_lib)] = libs.pop(lib)
 
             self.libs = libs
 
@@ -1499,9 +1509,9 @@ class ELF(ELFFile):
         if not dt_runpath:
             return None
 
-        return self.dynamic_string(dt_rpath.entry.d_ptr)
+        return self.dynamic_string(dt_runpath.entry.d_ptr)
 
-    def checksec(self, banner=True):
+    def checksec(self, banner=True, color=True):
         """checksec(banner=True)
 
         Prints out information in the binary, similar to ``checksec.sh``.
@@ -1509,9 +1519,9 @@ class ELF(ELFFile):
         Arguments:
             banner(bool): Whether to print the path to the ELF binary.
         """
-        red    = text.red
-        green  = text.green
-        yellow = text.yellow
+        red    = text.red if color else str
+        green  = text.green if color else str
+        yellow = text.yellow if color else str
 
         res = []
 
@@ -1686,16 +1696,33 @@ class ELF(ELFFile):
         return packing.unpack(self.read(address, context.bytes), *a, **kw)
 
     def string(self, address):
-        """Reads a null-terminated string from the specified ``address``"""
+        """string(address) -> str
+
+        Reads a null-terminated string from the specified ``address``
+
+        Returns:
+            A ``str`` with the string contents (NUL terminator is omitted),
+            or an empty string if no NUL terminator could be found.
+        """
         data = ''
         while True:
-            c = self.read(address, 1)
+            read_size = 0x1000
+            partial_page = address & 0xfff
+
+            if partial_page:
+                read_size -= partial_page
+
+            c = self.read(address, read_size)
+
             if not c:
                 return ''
-            if c == '\x00':
-                return data
+
             data += c
-            address += 1
+
+            if '\x00' in c:
+                return data[:data.index('\x00')]
+
+            address += len(c)
 
     def flat(self, address, *a, **kw):
         """Writes a full array of values to the specified address.

pwnlib/encoders/arm/alphanumeric/__init__.py

@@ -14,11 +14,9 @@ class ArmEncoder(Encoder):
     arch = 'arm'
 
     blacklist  = {chr(c) for c in range(256) if chr(c) in (string.ascii_letters + string.digits)}
+    icache_flush = 1
 
     def __call__(self, input, avoid, pcreg=None):
-
-        icache_flush = 1
-
         # If randomization is disabled, ensure that the seed
         # is always the same for the builder.
         state = random.getstate()
@@ -29,9 +27,9 @@ class ArmEncoder(Encoder):
             b = builder.builder()
 
             enc_data = b.enc_data_builder(input)
-            dec_loop = b.DecoderLoopBuilder(icache_flush)
+            dec_loop = b.DecoderLoopBuilder(self.icache_flush)
             enc_dec_loop = b.encDecoderLoopBuilder(dec_loop)
-            dec = b.DecoderBuilder(dec_loop, icache_flush)
+            dec = b.DecoderBuilder(dec_loop, self.icache_flush)
 
             output,dec = b.buildInit(dec);
 

pwnlib/gdb.py

@@ -100,6 +100,7 @@ import time
 from pwnlib import adb
 from pwnlib import atexit
 from pwnlib import elf
+from pwnlib import qemu
 from pwnlib import tubes
 from pwnlib.asm import _bfdname
 from pwnlib.asm import make_elf
@@ -107,7 +108,6 @@ from pwnlib.asm import make_elf_from_assembly
 from pwnlib.context import LocalContext
 from pwnlib.context import context
 from pwnlib.log import getLogger
-from pwnlib.qemu import get_qemu_user
 from pwnlib.util import misc
 from pwnlib.util import proc
 
@@ -127,10 +127,19 @@ def debug_assembly(asm, gdbscript=None, vma=None):
         asm(str): Assembly code to debug
         gdbscript(str): Script to run in GDB
         vma(int): Base address to load the shellcode at
-        **kwargs: Override any :obj:`.context` values.
+        **kwargs: Override any :obj:`pwnlib.context.context` values.
 
     Returns:
         :class:`.process`
+
+    Example:
+
+        .. code-block:: python
+
+            assembly = shellcraft.echo("Hello world!\n")
+            io = gdb.debug_assembly(assembly)
+            io.recvline()
+            # 'Hello world!'
     """
     tmp_elf = make_elf_from_assembly(asm, vma=vma, extract=False)
     os.chmod(tmp_elf, 0777)
@@ -153,10 +162,20 @@ def debug_shellcode(data, gdbscript=None, vma=None):
         data(str): Assembled shellcode bytes
         gdbscript(str): Script to run in GDB
         vma(int): Base address to load the shellcode at
-        **kwargs: Override any :obj:`.context` values.
+        **kwargs: Override any :obj:`pwnlib.context.context` values.
 
     Returns:
         :class:`.process`
+
+    Example:
+
+        .. code-block:: python
+
+            assembly = shellcraft.echo("Hello world!\n")
+            shellcode = asm(assembly)
+            io = gdb.debug_shellcode(shellcode)
+            io.recvline()
+            # 'Hello world!'
     """
     if isinstance(data, unicode):
         log.error("Shellcode is cannot be unicode.  Did you mean debug_assembly?")
@@ -210,7 +229,7 @@ def _gdbserver_args(pid=None, path=None, args=None, which=None):
 
     orig_args = args
 
-    gdbserver_args = [gdbserver]
+    gdbserver_args = [gdbserver, '--multi']
     if context.aslr:
         gdbserver_args += ['--no-disable-randomization']
     else:
@@ -230,6 +249,12 @@ def _gdbserver_port(gdbserver, ssh):
     # Process /bin/bash created; pid = 14366
     # Listening on port 34816
     process_created = gdbserver.recvline()
+
+    if process_created.startswith('ERROR:'):
+        raise ValueError(
+            'Failed to spawn process under gdbserver. gdbserver error message: %s' % process_created
+        )
+
     gdbserver.pid   = int(process_created.split()[-1], 0)
 
     listening_on = ''
@@ -268,7 +293,7 @@ def _get_runner(ssh=None):
     else:                          return tubes.process.process
 
 @LocalContext
-def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
+def debug(args, gdbscript=None, exe=None, ssh=None, env=None, sysroot=None, **kwargs):
     """debug(args) -> tube
 
     Launch a GDB server with the specified command line,
@@ -280,6 +305,8 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
         exe(str): Path to the executable on disk
         env(dict): Environment to start the binary in
         ssh(:class:`.ssh`): Remote ssh session to use to launch the process.
+        sysroot(str): Foreign-architecture sysroot, used for QEMU-emulated binaries
+            and Android targets.
 
     Returns:
         :class:`.process` or :class:`.ssh_channel`: A tube connected to the target process
@@ -290,14 +317,6 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
         from the very beginning.  This requires that both ``gdb`` and ``gdbserver``
         are installed on your machine.
 
-        .. code-block:: python
-
-            # Create a new process, and stop it at 'main'
-            io = gdb.debug('bash', '''
-            break main
-            continue
-            ''')
-
         When GDB opens via :func:`debug`, it will initially be stopped on the very first
         instruction of the dynamic linker (``ld.so``) for dynamically-linked binaries.
 
@@ -322,6 +341,22 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
             - Generally, you just add a few ``continue`` commands until things are set up
               the way you want it to be.
 
+    Examples:
+
+        .. code-block:: python
+
+            # Create a new process, and stop it at 'main'
+            io = gdb.debug('bash', '''
+            break main
+            continue
+            ''')
+
+            # Send a command to Bash
+            io.sendline("echo hello")
+
+            # Interact with the process
+            io.interactive()
+
         .. code-block:: python
 
             # Create a new process, and stop it at 'main'
@@ -336,6 +371,12 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
             continue
             ''')
 
+            # Send a command to Bash
+            io.sendline("echo hello")
+
+            # Interact with the process
+            io.interactive()
+
         You can use :func:`debug` to spawn new processes on remote machines as well,
         by using the ``ssh=`` keyword to pass in your :class:`.ssh` instance.
 
@@ -345,12 +386,18 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
             shell = ssh('passcode', 'pwnable.kr', 2222, password='guest')
 
             # Start a process on the server
-            bash = gdb.debug(['bash'],
+            io = gdb.debug(['bash'],
                             ssh=shell,
                             gdbscript='''
             break main
             continue
             ''')
+
+            # Send a command to Bash
+            io.sendline("echo hello")
+
+            # Interact with the process
+            io.interactive()
     """
     if isinstance(args, (int, tubes.process.process, tubes.ssh.ssh_channel)):
         log.error("Use gdb.attach() to debug a running process")
@@ -365,6 +412,7 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
 
     runner = _get_runner(ssh)
     which  = _get_which(ssh)
+    gdbscript = gdbscript or ''
 
     if context.noptrace:
         log.warn_once("Skipping debugger since context.noptrace==True")
@@ -374,7 +422,15 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
         args = _gdbserver_args(args=args, which=which)
     else:
         qemu_port = random.randint(1024, 65535)
-        args = [get_qemu_user(), '-g', str(qemu_port)] + args
+        qemu_user = qemu.user_path()
+        sysroot = sysroot or qemu.ld_prefix(env)
+        if not qemu_user:
+            log.error("Cannot debug %s binaries without appropriate QEMU binaries" % context.arch)
+        args = [qemu_user, '-g', str(qemu_port)] + args
+
+    # Use a sane default sysroot for Android
+    if not sysroot and context.os == 'android':
+        sysroot = 'remote:/'
 
     # Make sure gdbserver/qemu is installed
     if not which(args[0]):
@@ -383,6 +439,8 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
     exe = exe or which(orig_args[0])
     if not exe:
         log.error("%s does not exist" % orig_args[0])
+    else:
+        gdbscript = 'file %s\n%s' % (exe, gdbscript)
 
     # Start gdbserver/qemu
     # (Note: We override ASLR here for the gdbserver process itself.)
@@ -401,7 +459,7 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, **kwargs):
     if not ssh and context.os == 'android':
         host = context.adb_host
 
-    attach((host, port), exe=exe, gdbscript=gdbscript, need_ptrace_scope = False, ssh=ssh)
+    attach((host, port), exe=exe, gdbscript=gdbscript, need_ptrace_scope = False, ssh=ssh, sysroot=sysroot)
 
     # gdbserver outputs a message when a client connects
     garbage = gdbserver.recvline(timeout=1)
@@ -425,8 +483,13 @@ def binary():
 
     Returns:
         str: Path to the appropriate ``gdb`` binary to use.
+
+    Example:
+
+        >>> gdb.binary() # doctest: +SKIP
+        '/usr/bin/gdb'
     """
-    gdb = misc.which('gdb')
+    gdb = misc.which('pwntools-gdb') or misc.which('gdb')
 
     if not context.native:
         multiarch = misc.which('gdb-multiarch')
@@ -443,7 +506,7 @@ def binary():
     return gdb
 
 @LocalContext
-def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_args = None, ssh = None):
+def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_args = None, ssh = None, sysroot = None):
     """attach(target, gdbscript = None, exe = None, arch = None, ssh = None) -> None
 
     Start GDB in a new terminal and attach to `target`.
@@ -455,6 +518,8 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
         arch(str): Architechture of the target binary.  If `exe` known GDB will
           detect the architechture automatically (if it is supported).
         gdb_args(list): List of additional arguments to pass to GDB.
+        sysroot(str): Foreign-architecture sysroot, used for QEMU-emulated binaries
+            and Android targets.
 
     Returns:
         PID of the GDB process (or the window which it is running in).
@@ -479,6 +544,8 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
             This will use the GDB installed on the remote machine.
             If a password is required to connect, the ``sshpass`` program must be installed.
 
+    Examples:
+
         .. code-block:: python
 
             # Attach directly to pid 1234
@@ -552,11 +619,17 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
     if gdbscript and not gdbscript.endswith('\n'):
         gdbscript += '\n'
 
+    # Use a sane default sysroot for Android
+    if not sysroot and context.os == 'android':
+        sysroot = 'remote:/'
+
     # gdb script to run before `gdbscript`
     pre = ''
     if not context.native:
         pre += 'set endian %s\n' % context.endian
         pre += 'set architecture %s\n' % get_gdb_arch()
+        if sysroot:
+            pre += 'set sysroot %s\n' % sysroot
 
         if context.os == 'android':
             pre += 'set gnutarget ' + _bfdname() + '\n'
@@ -611,9 +684,18 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
         pid = pids[0]
     elif isinstance(target, tubes.process.process):
         pid = proc.pidof(target)[0]
+        exe = exe or target.executable
     elif isinstance(target, tuple) and len(target) == 2:
         host, port = target
-        pre += 'target remote %s:%d\n' % (host, port)
+
+        if context.os != 'android':
+            pre += 'target remote %s:%d\n' % (host, port)
+        else:
+            # Android debugging is done over gdbserver, which can't follow
+            # new inferiors (tldr; follow-fork-mode child) unless it is run
+            # in extended-remote mode.
+            pre += 'target extended-remote %s:%d\n' % (host, port)
+            pre += 'set detach-on-fork off\n'
 
         def findexe():
             for spid in proc.pidof(target):
@@ -640,12 +722,20 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
     if not pid and not exe:
         log.error('could not find target process')
 
+    if exe:
+        # The 'file' statement should go first
+        pre = 'file %s\n%s' % (exe, pre)
+
     cmd = binary()
 
     if gdb_args:
         cmd += ' '
         cmd += ' '.join(gdb_args)
 
+    if context.gdbinit:
+        cmd += ' -nh '                     # ignore ~/.gdbinit
+        cmd += ' -x %s ' % context.gdbinit # load custom gdbinit
+
     cmd += ' -q '
 
     if exe and context.native:
@@ -666,7 +756,11 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
         gdbserver = runner(gdb_cmd)
         port    = _gdbserver_port(gdbserver, None)
         host    = context.adb_host
-        pre    += 'target remote %s:%i' % (context.adb_host, port)
+        pre    += 'target extended-remote %s:%i\n' % (context.adb_host, port)
+
+        # gdbserver on Android sets 'detach-on-fork on' which breaks things
+        # when you're trying to debug anything that forks.
+        pre += 'set detach-on-fork off\n'
 
     gdbscript = pre + (gdbscript or '')
 
@@ -694,7 +788,7 @@ def ssh_gdb(ssh, argv, gdbscript = None, arch = None, **kwargs):
         argv = [argv]
 
     exe = argv[0]
-    argv = ["gdbserver", "127.0.0.1:0"] + argv
+    argv = ["gdbserver", "--multi", "127.0.0.1:0"] + argv
 
     # Download the executable
     local_exe = os.path.basename(exe)

pwnlib/qemu.py

@@ -1,5 +1,81 @@
+"""Run foreign-architecture binaries
+
+Overview
+--------
+
+So you want to exploit ARM binaries on your Intel PC?
+
+Pwntools has a good level of integration with QEMU user-mode emulation,
+in order to run, debug, and pwn foreign architecture binaries.
+
+In general, everything magic happens "behind the scenes", and pwntools
+attempts to make your life easier.
+
+When using :class:`.process.process`, pwntools will attempt to blindly
+execute the binary, in case your system is configured to use ``binfmt-misc``.
+
+If this fails, pwntools will attempt to manually launch the binary under
+qemu user-mode emulation.  Preference is given to statically-linked variants,
+i.e. ``qemu-arm-static`` will be selected before ``qemu-arm``.
+
+Debugging
+~~~~~~~~~
+
+When debugging binaries with :func:`.gdb.debug`, pwntools automatically adds
+the appropriate command-line flags to QEMU to start its GDB stub, and
+automatically informs GDB of the correct architecture and sysroot.
+
+Sysroot
+~~~~~~~
+You can override the default sysroot by setting the ``QEMU_LD_PREFIX``
+environment variable.  This affects where ``qemu`` will look for files when
+``open()`` is called, e.g. when the linker is attempting to resolve ``libc.so``.
+
+Required Setup
+--------------
+
+For Ubuntu 16.04 and newer, the setup is relatively straightforward for most
+architectures.
+
+First, install the QEMU emulator itself.  If your binary is statically-linked,
+thsi is sufficient.
+
+    $ sudo apt-get install qemu-user
+
+If your binary is dynamically linked, you need to install libraries like libc.
+Generally, this package is named ``libc6-$ARCH-cross``, e.g. ``libc-mips-cross``.
+ARM comes in both soft-float and hard-float variants, e.g. ``armhf``.
+
+    $ sudo apt-get install libc6-arm64-cross
+
+If your binary relies on additional libraries, you can generally find them
+easily with ``apt-cache search``.  For example, if it's a C++ binary it
+may require ``libstdc++``.
+
+    $ apt-cache search 'libstdc++' | grep arm64
+
+Any other libraries that you require you'll have to find some other way.
+
+Telling QEMU Where Libraries Are
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The libraries are now installed on your system at e.g. ``/usr/aarch64-linux-gnu``.
+
+QEMU does not know where they are, and expects them to be at e.g. ``/etc/qemu-binfmt/aarch64``.
+If you try to run your library now, you'll probably see an error about ``libc.so.6`` missing.
+
+Create the ``/etc/qemu-binfmt`` directory if it does not exist, and create a symlink to
+the appropriate path.
+
+    $ sudo mkdir /etc/qemu-binfmt
+    $ sudo ln -s /usr/aarch64-linux-gnu /etc/qemu-binfmt/aarch64
+
+Now QEMU should be able to run the libraries.
+"""
 from __future__ import absolute_import
 
+import os
+
 from pwnlib.context import LocalContext
 from pwnlib.context import context
 from pwnlib.log import getLogger
@@ -8,14 +84,14 @@ from pwnlib.util import misc
 log = getLogger(__name__)
 
 @LocalContext
-def get_qemu_arch():
+def archname():
     """
     Returns the name which QEMU uses for the currently selected
     architecture.
 
-    >>> get_qemu_arch()
+    >>> pwnlib.qemu.archname()
     'i386'
-    >>> get_qemu_arch(arch='powerpc')
+    >>> pwnlib.qemu.archname(arch='powerpc')
     'ppc'
     """
     return {
@@ -31,17 +107,17 @@ def get_qemu_arch():
     }.get((context.arch, context.endian), context.arch)
 
 @LocalContext
-def get_qemu_user():
+def user_path():
     """
     Returns the path to the QEMU-user binary for the currently
     selected architecture.
 
-    >>> get_qemu_user()
+    >>> pwnlib.qemu.user_path()
     'qemu-i386-static'
-    >>> get_qemu_user(arch='thumb')
+    >>> pwnlib.qemu.user_path(arch='thumb')
     'qemu-arm-static'
     """
-    arch   = get_qemu_arch()
+    arch   = archname()
     normal = 'qemu-' + arch
     static = normal + '-static'
 
@@ -52,3 +128,32 @@ def get_qemu_user():
         return normal
 
     log.warn_once("Neither %r nor %r are available" % (normal, static))
+
+@LocalContext
+def ld_prefix(path=None, env=None):
+    """Returns the linker prefix for the selected qemu-user binary
+
+    >>> pwnlib.qemu.ld_prefix(arch='arm')
+    '/etc/qemu-binfmt/arm'
+    """
+    if path is None:
+        path = user_path()
+
+    # Did we explicitly specify the path in an environment variable?
+    if env and 'QEMU_LD_PREFIX' in env:
+        return env['QEMU_LD_PREFIX']
+
+    if 'QEMU_LD_PREFIX' in os.environ:
+        return os.environ['QEMU_LD_PREFIX']
+
+    # Cyclic imports!
+    from pwnlib.tubes.process import process
+
+    with context.quiet:
+        with process([path, '--help'], env=env) as io:
+            line = io.recvline_regex('QEMU_LD_PREFIX *=')
+
+    name, libpath = line.split('=', 1)
+
+    return libpath.strip()
+

pwnlib/rop/call.py

@@ -233,4 +233,12 @@ class Call(object):
                 args.extend(map(repr, arg.values))
             else:
                 args.append(arg)
-        return '%s(%s)' % (self.name or fmt % self.target, ', '.join(map(str, args)))
+
+        name = self.name or fmt & self.target
+        arg_str = []
+        for arg in args:
+            if isinstance(arg, (int,long)) and arg > 0x100:
+                arg_str.append(hex(arg))
+            else:
+                arg_str.append(str(arg))
+        return '%s(%s)' % (name, ', '.join(arg_str))

pwnlib/rop/rop.py

@@ -40,7 +40,7 @@ standard Linux ABIs.
     0x0004:       0x64636261
     0x0008:              0x2
     0x000c:       0xdeadbeef read(4, 5, 6)
-    0x0010:           'eaaa' <pad>
+    0x0010:           'eaaa' <return address>
     0x0014:              0x4 arg0
     0x0018:              0x5 arg1
     0x001c:              0x6 arg2
@@ -55,19 +55,18 @@ The stack is automatically adjusted for the next frame
     0x0004:       0x64636261
     0x0008:              0x2
     0x000c:       0xdeadbeef read(4, 5, 6)
-    0x0010:       0x10000000 <adjust: add esp, 0x10; ret>
+    0x0010:       0x10000000 <adjust @0x24> add esp, 0x10; ret
     0x0014:              0x4 arg0
     0x0018:              0x5 arg1
     0x001c:              0x6 arg2
     0x0020:           'iaaa' <pad>
     0x0024:       0xdecafbad write(7, 8, 9)
-    0x0028:       0x10000000 <adjust: add esp, 0x10; ret>
+    0x0028:       0x10000000 <adjust @0x3c> add esp, 0x10; ret
     0x002c:              0x7 arg0
     0x0030:              0x8 arg1
     0x0034:              0x9 arg2
     0x0038:           'oaaa' <pad>
     0x003c:       0xfeedface exit()
-    0x0040:           'qaaa' <pad>
 
 ROP Example
 -------------------
@@ -106,20 +105,19 @@ Finally, let's build our ROP stack
     >>> rop.write(c.STDOUT_FILENO, binary.symbols['flag'], 8)
     >>> rop.exit()
     >>> print rop.dump()
-    0x0000:       0x10000012 write(STDOUT_FILENO, 268435494, 8)
-    0x0004:       0x1000000e <adjust: add esp, 0x10; ret>
+    0x0000:       0x10000012 write(STDOUT_FILENO, 0x10000026, 8)
+    0x0004:       0x1000000e <adjust @0x18> add esp, 0x10; ret
     0x0008:              0x1 arg0
     0x000c:       0x10000026 flag
     0x0010:              0x8 arg2
     0x0014:           'faaa' <pad>
     0x0018:       0x1000002f exit()
-    0x001c:           'haaa' <pad>
 
 The raw data from the ROP stack is available via `str`.
 
     >>> raw_rop = str(rop)
     >>> print enhex(raw_rop)
-    120000100e000010010000002600001008000000666161612f00001068616161
+    120000100e000010010000002600001008000000666161612f000010
 
 Let's try it out!
 
@@ -128,6 +126,48 @@ Let's try it out!
     >>> print p.recvall(timeout=5)
     The flag
 
+ROP Example (amd64)
+-------------------
+
+For amd64 binaries, the registers are loaded off the stack.  Pwntools can do
+basic reasoning about simple "pop; pop; add; ret"-style gadgets, and satisfy
+requirements so that everything "just works".
+
+    >>> context.clear(arch='amd64')
+    >>> assembly = 'pop rdx; pop rdi; pop rsi; add rsp, 0x20; ret; target: ret'
+    >>> binary = ELF.from_assembly(assembly)
+    >>> rop = ROP(binary)
+    >>> rop.target(1,2,3)
+    >>> print rop.dump()
+    0x0000:       0x10000000 pop rdx; pop rdi; pop rsi; add rsp, 0x20; ret
+    0x0008:              0x3 [arg2] rdx = 3
+    0x0010:              0x1 [arg0] rdi = 1
+    0x0018:              0x2 [arg1] rsi = 2
+    0x0020:       'iaaajaaa' <pad 0x20>
+    0x0028:       'kaaalaaa' <pad 0x18>
+    0x0030:       'maaanaaa' <pad 0x10>
+    0x0038:       'oaaapaaa' <pad 0x8>
+    0x0040:       0x10000008 target
+    >>> rop.target(1)
+    >>> print rop.dump()
+    0x0000:       0x10000000 pop rdx; pop rdi; pop rsi; add rsp, 0x20; ret
+    0x0008:              0x3 [arg2] rdx = 3
+    0x0010:              0x1 [arg0] rdi = 1
+    0x0018:              0x2 [arg1] rsi = 2
+    0x0020:       'iaaajaaa' <pad 0x20>
+    0x0028:       'kaaalaaa' <pad 0x18>
+    0x0030:       'maaanaaa' <pad 0x10>
+    0x0038:       'oaaapaaa' <pad 0x8>
+    0x0040:       0x10000008 target
+    0x0048:       0x10000001 pop rdi; pop rsi; add rsp, 0x20; ret
+    0x0050:              0x1 [arg0] rdi = 1
+    0x0058:       'waaaxaaa' <pad rsi>
+    0x0060:       'yaaazaab' <pad 0x20>
+    0x0068:       'baabcaab' <pad 0x18>
+    0x0070:       'daabeaab' <pad 0x10>
+    0x0078:       'faabgaab' <pad 0x8>
+    0x0080:       0x10000008 target
+
 ROP + Sigreturn
 -----------------------
 
@@ -160,7 +200,7 @@ That's all there is to it.
 
     >>> print rop.dump()
     0x0000:       0x1000000e pop eax; ret
-    0x0004:             0x77
+    0x0004:             0x77 [arg0] eax = SYS_sigreturn
     0x0008:       0x1000000b int 0x80
     0x000c:              0x0 gs
     0x0010:              0x0 fs
@@ -197,8 +237,10 @@ from __future__ import absolute_import
 import collections
 import copy
 import hashlib
+import itertools
 import os
 import re
+import string
 import sys
 import tempfile
 
@@ -215,9 +257,9 @@ from pwnlib.rop.call import CurrentStackPointer
 from pwnlib.rop.call import NextGadgetAddress
 from pwnlib.rop.call import StackAdjustment
 from pwnlib.rop.gadgets import Gadget
-from pwnlib.util import cyclic
 from pwnlib.util import lists
 from pwnlib.util import packing
+from pwnlib.util.cyclic import cyclic
 from pwnlib.util.packing import *
 
 log = getLogger(__name__)
@@ -228,6 +270,8 @@ class Padding(object):
     """
     Placeholder for exactly one pointer-width of padding.
     """
+    def __init__(self, name='<pad>'):
+        self.name = name
 
 class DescriptiveStack(list):
     """
@@ -306,16 +350,16 @@ class ROP(object):
     >>> r.execve(4, 5, 6)
     >>> print r.dump()
     0x0000:       0x10001234 funcname(1, 2)
-    0x0004:       0x10000003 <adjust: add esp, 0x10; ret>
+    0x0004:       0x10000003 <adjust @0x18> add esp, 0x10; ret
     0x0008:              0x1 arg0
     0x000c:              0x2 arg1
     0x0010:           'eaaa' <pad>
     0x0014:           'faaa' <pad>
     0x0018:       0x10001234 funcname(3)
-    0x001c:       0x10000007 <adjust: pop eax; ret>
+    0x001c:       0x10000007 <adjust @0x24> pop eax; ret
     0x0020:              0x3 arg0
     0x0024:       0x10000007 pop eax; ret
-    0x0028:             0x77
+    0x0028:             0x77 [arg0] eax = SYS_sigreturn
     0x002c:       0x10000000 int 0x80
     0x0030:              0x0 gs
     0x0034:              0x0 fs
@@ -344,16 +388,16 @@ class ROP(object):
     >>> r.execve(4, 5, 6)
     >>> print r.dump()
     0x8048000:       0x10001234 funcname(1, 2)
-    0x8048004:       0x10000003 <adjust: add esp, 0x10; ret>
+    0x8048004:       0x10000003 <adjust @0x8048018> add esp, 0x10; ret
     0x8048008:              0x1 arg0
     0x804800c:              0x2 arg1
     0x8048010:           'eaaa' <pad>
     0x8048014:           'faaa' <pad>
     0x8048018:       0x10001234 funcname(3)
-    0x804801c:       0x10000007 <adjust: pop eax; ret>
+    0x804801c:       0x10000007 <adjust @0x8048024> pop eax; ret
     0x8048020:              0x3 arg0
     0x8048024:       0x10000007 pop eax; ret
-    0x8048028:             0x77
+    0x8048028:             0x77 [arg0] eax = SYS_sigreturn
     0x804802c:       0x10000000 int 0x80
     0x8048030:              0x0 gs
     0x8048034:              0x0 fs
@@ -375,25 +419,24 @@ class ROP(object):
     0x8048074:              0x0 esp_at_signal
     0x8048078:             0x2b ss
     0x804807c:              0x0 fpstate
+
+
+    >>> elf = ELF.from_assembly('ret')
+    >>> r = ROP(elf)
+    >>> r.ret.address == 0x10000000
+    True
+    >>> r = ROP(elf, badchars='\x00')
+    >>> r.gadgets == {}
+    True
+    >>> r.ret is None
+    True
     """
-    #: List of individual ROP gadgets, ROP calls, SROP frames, etc.
-    #: This is intended to be the highest-level abstraction that we can muster.
-    _chain = []
-
-    #: List of ELF files which are available for mining gadgets
-    elfs = []
-
-    #: Stack address where the first byte of the ROP chain lies, if known.
-    base = 0
-
-    #: Whether or not the ROP chain directly sets the stack pointer to a value
-    #: which is not contiguous
-    migrated = False
-
-    def __init__(self, elfs, base = None, **kwargs):
+    def __init__(self, elfs, base = None, badchars = '', **kwargs):
         """
         Arguments:
             elfs(list): List of :class:`.ELF` objects for mining
+            base(int): Stack address where the first byte of the ROP chain lies, if known.
+            badchars(str): Characters which should not appear in ROP gadget addresses.
         """
         import ropgadget
 
@@ -402,10 +445,24 @@ class ROP(object):
             elfs = [elfs]
         elif isinstance(elfs, (str, unicode)):
             elfs = [ELF(elfs)]
-        self.elfs = elfs
+
+        #: List of individual ROP gadgets, ROP calls, SROP frames, etc.
+        #: This is intended to be the highest-level abstraction that we can muster.
         self._chain = []
+
+        #: List of ELF files which are available for mining gadgets
+        self.elfs = elfs
+
+        #: Stack address where the first byte of the ROP chain lies, if known.
         self.base = base
+
+        #: Whether or not the ROP chain directly sets the stack pointer to a value
+        #: which is not contiguous
         self.migrated = False
+
+        #: Characters which should not appear in ROP gadget addresses.
+        self._badchars = set(badchars)
+
         self.__load()
 
     @staticmethod
@@ -415,24 +472,93 @@ class ROP(object):
 
     def setRegisters(self, registers):
         """
-        Returns an OrderedDict of addresses/values which will set the specified
-        register context.
+        Returns an list of addresses/values which will set the specified register context.
 
         Arguments:
             registers(dict): Dictionary of ``{register name: value}``
 
         Returns:
-            An OrderedDict of ``{register: sequence of gadgets, values, etc.}``.
+            A list of tuples, ordering the stack.
+
+            Each tuple is in the form of ``(value, name)`` where ``value`` is either a
+            gadget address or literal value to go on the stack, and ``name`` is either
+            a string name or other item which can be "unresolved".
+
+        Note:
+            This is basically an implementation of the Set Cover Problem, which is
+            NP-hard.  This means that we will take polynomial time N**2, where N is
+            the number of gadgets.  We can reduce runtime by discarding useless and
+            inferior gadgets ahead of time.
         """
-        reg_order = collections.OrderedDict()
+        if not registers:
+            return []
 
-        for reg, value in registers.items():
-            gadget = self.find_gadget(['pop ' + reg, 'ret'])
-            if not gadget:
-                log.error("can't set %r" % reg)
-            reg_order[reg] = [gadget, value]
+        regset = set(registers)
 
-        return reg_order
+        # Collect all gadgets which use these registers
+        # Also collect the "best" gadget for each combination of registers
+        gadgets = []
+        best_gadgets = {}
+
+        for gadget in self.gadgets.values():
+            # Do not use gadgets which end in e.g. "int 0x80"
+            if gadget.insns[-1] != 'ret':
+                continue
+
+            touched = tuple(regset & set(gadget.regs))
+
+            if not touched:
+                continue
+
+            old = best_gadgets.get(touched, gadget)
+
+            if old is gadget or old.move > gadget.move:
+                best_gadgets[touched] = gadget
+
+
+        winner = None
+        budget = 999999999
+
+        for num_gadgets in range(len(registers)):
+            for combo in itertools.combinations(best_gadgets.values(), 1+num_gadgets):
+                # Is this better than what we can already do?
+                cost = sum((g.move for g in combo))
+                if cost > budget:
+                    continue
+
+                # Does it hit all of the registers we want?
+                coverage = set(sum((g.regs for g in combo), [])) & regset
+
+                if coverage != regset:
+                    continue
+
+                # It is better than what we had, and hits all of the registers.
+                winner = combo
+                budget = cost
+
+        if not winner:
+            log.error("Could not satisfy setRegisters(%r)", registers)
+
+        # We have our set of "winner" gadgets, let's build a stack!
+        stack = []
+
+        for gadget in winner:
+            moved = 8 # Account for the gadget itself
+            goodregs = set(gadget.regs) & regset
+            name = ",".join(goodregs)
+            stack.append((gadget.address, gadget))
+            for r in gadget.regs:
+                moved += 8
+                if r in registers:
+                    stack.append((registers[r], r))
+                else:
+                    stack.append((Padding('<pad %s>' % r), r))
+
+            for slot in range(moved, gadget.move, context.bytes):
+                left = gadget.move - slot
+                stack.append((Padding('<pad %#x>' % left), 'stack padding'))
+
+        return stack
 
     def resolve(self, resolvable):
         """Resolves a symbol to an address
@@ -486,8 +612,12 @@ class ROP(object):
         ''
 
         """
+
+        # Ensure we don't generate a cyclic pattern which contains badchars
+        alphabet = ''.join(c for c in string.ascii_lowercase if c not in self._badchars)
+
         if count:
-            return cyclic.cyclic(offset + count)[-count:]
+            return cyclic(offset + count, alphabet=alphabet)[-count:]
         return ''
 
     def describe(self, object):
@@ -575,13 +705,17 @@ class ROP(object):
                 stack.describe(self.describe(slot))
 
                 registers    = dict(zip(slot.abi.register_arguments, slot.args))
-                setRegisters = self.setRegisters(registers)
 
-                for register, gadgets in setRegisters.items():
-                    value       = registers[register]
-                    description = self.describe(value) or 'arg%i' % slot.args.index(value)
-                    stack.describe('set %s = %s' % (register, description))
-                    stack.extend(gadgets)
+                for value, name in self.setRegisters(registers):
+                    if name in registers:
+                        index = slot.abi.register_arguments.index(name)
+                        description = self.describe(value) or value
+                        stack.describe('[arg%s] %s = %s' % (index, name, description))
+                    elif isinstance(name, Gadget):
+                        stack.describe('; '.join(name.insns))
+                    elif isinstance(name, str):
+                        stack.describe(name)
+                    stack.append(value)
 
                 if address != stack.next:
                     stack.describe(slot.name)
@@ -599,24 +733,33 @@ class ROP(object):
                 # properly, but likely also need to adjust the stack past the
                 # arguments.
                 if slot.abi.returns:
-                    if remaining:
-                        fix_size  = (1 + len(stackArguments))
-                        fix_bytes = fix_size * context.bytes
-                        adjust   = self.search(move = fix_bytes)
 
-                        if not adjust:
-                            log.error("Could not find gadget to adjust stack by %#x bytes" % fix_bytes)
+                    # Save off the address of the next gadget
+                    if remaining or stackArguments:
+                        nextGadgetAddr = stack.next
 
-                        nextGadgetAddr = stack.next + adjust.move
+                    # If there were arguments on the stack, we need to stick something
+                    # in the slot where the return address goes.
+                    if len(stackArguments) > 0:
+                        if remaining:
+                            fix_size  = (1 + len(stackArguments))
+                            fix_bytes = fix_size * context.bytes
+                            adjust   = self.search(move = fix_bytes)
 
-                        stack.describe('<adjust: %s>' % self.describe(adjust))
-                        stack.append(adjust.address)
+                            if not adjust:
+                                log.error("Could not find gadget to adjust stack by %#x bytes" % fix_bytes)
 
-                        for pad in range(fix_bytes, adjust.move, context.bytes):
-                            stackArguments.append(Padding())
-                    else:
-                        stack.describe('<pad>')
-                        stack.append(Padding())
+                            nextGadgetAddr += adjust.move
+
+                            stack.describe('<adjust @%#x> %s' % (nextGadgetAddr, self.describe(adjust)))
+                            stack.append(adjust.address)
+
+                            for pad in range(fix_bytes, adjust.move, context.bytes):
+                                stackArguments.append(Padding())
+
+                        # We could not find a proper "adjust" gadget, but also didn't need one.
+                        else:
+                            stack.append(Padding("<return address>"))
 
 
                 for i, argument in enumerate(stackArguments):
@@ -657,7 +800,7 @@ class ROP(object):
 
             elif isinstance(slot, Padding):
                 stack[i] = self.generatePadding(i * context.bytes, context.bytes)
-                stack.describe('<pad>', slot_address)
+                stack.describe(slot.name, slot_address)
 
             elif isinstance(slot, Gadget):
                 stack[i] = slot.address
@@ -835,13 +978,22 @@ class ROP(object):
         """Returns: Raw bytes of the ROP chain"""
         return self.chain()
 
-    def __get_cachefile_name(self, elf):
-        basename = os.path.basename(elf.file.name)
-        sha256 = hashlib.sha256(elf.get_data()).hexdigest()
+    def __get_cachefile_name(self, files):
+        """Given an ELF or list of ELF objects, return a cache file for the set of files"""
         cachedir = os.path.join(tempfile.gettempdir(), 'pwntools-rop-cache')
         if not os.path.exists(cachedir):
             os.mkdir(cachedir)
-        return os.path.join(cachedir, sha256)
+
+        if isinstance(files, ELF):
+            files = [files]
+
+        hashes = []
+
+        for elf in self.elfs:
+            sha256 = hashlib.sha256(elf.get_data()).hexdigest()
+            hashes.append(sha256)
+
+        return os.path.join(cachedir, '_'.join(hashes))
 
     def __cache_load(self, elf):
         filename = self.__get_cachefile_name(elf)
@@ -871,7 +1023,7 @@ class ROP(object):
         #
 
         pop   = re.compile(r'^pop (.{3})')
-        add   = re.compile(r'^add .sp, (\S+)$')
+        add   = re.compile(r'^add [er]sp, (\S+)$')
         ret   = re.compile(r'^ret$')
         leave = re.compile(r'^leave$')
         int80 = re.compile(r'int +0x80')
@@ -950,6 +1102,11 @@ class ROP(object):
         }[context.bytes]
 
         for addr, insns in gadgets.items():
+
+            # Filter out gadgets by address against badchars
+            if set(pack(addr)) & self._badchars:
+                continue
+
             sp_move = 0
             regs = []
             for insn in insns:
@@ -999,7 +1156,10 @@ class ROP(object):
         regs = set(regs or ())
 
         for addr, gadget in self.gadgets.items():
-            if gadget.move < move:          continue
+            addr_bytes = set(pack(gadget.address))
+            if addr_bytes & self._badchars:     continue
+            if gadget.insns[-1] != 'ret':        continue
+            if gadget.move < move:               continue
             if not (regs <= set(gadget.regs)):   continue
             yield gadget
 

pwnlib/shellcraft/templates/aarch64/linux/open.asm

@@ -1,7 +1,7 @@
 <%
   from pwnlib import shellcraft
 %>
-<%page args="filename, mode='O_RDONLY'"/>
+<%page args="filename, flags='O_RDONLY', mode=0"/>
 <%docstring>
 Opens a file
 </%docstring>
@@ -9,4 +9,4 @@ Opens a file
   AT_FDCWD=-100
 %>
     ${shellcraft.pushstr(filename)}
-    ${shellcraft.syscall('SYS_openat', AT_FDCWD, 'sp', mode, 0)}
+    ${shellcraft.syscall('SYS_openat', AT_FDCWD, 'sp', flags, mode)}

pwnlib/shellcraft/templates/amd64/linux/cat.asm

@@ -8,5 +8,5 @@ Opens a file and writes its contents to the specified file descriptor.
 </%docstring>
 
     ${pushstr(filename)}
-    ${syscall('SYS_open', 'rsp', 0, 'O_RDONLY')}
+    ${syscall('SYS_open', 'rsp', 'O_RDONLY', 0)}
     ${syscall('SYS_sendfile', fd, 'rax', 0, 0x7fffffff)}

pwnlib/shellcraft/templates/amd64/linux/dup.asm

@@ -10,7 +10,7 @@ Args: [sock (imm/reg) = rbp]
   after     = common.label("after")
 %>
 
-
+    /* dup() file descriptor ${sock} into stdin/stdout/stderr */
 ${dup}:
     ${amd64.mov('rbp', sock)}
 

pwnlib/shellcraft/templates/arm/linux/cat.asm

@@ -15,5 +15,5 @@ Example:
 
 </%docstring>
     ${arm.pushstr(filename)}
-    ${arm.linux.open('sp', 0, int(constants.O_RDONLY))}
+    ${arm.linux.open('sp', int(constants.O_RDONLY), 0)}
     ${arm.linux.sendfile(fd, 'r0', 0, 0x7fffffff)}

pwnlib/shellcraft/templates/i386/linux/cat.asm

@@ -19,5 +19,5 @@ label = common.label("sendfile_loop")
 %>
 
     ${sc.pushstr(filename)}
-    ${sc.open('esp', 0, 'O_RDONLY')}
+    ${sc.open('esp', 'O_RDONLY', 0)}
     ${sc.sendfile(fd, 'eax', 0, 0x7fffffff)}

pwnlib/shellcraft/templates/i386/linux/dupio.asm

@@ -11,6 +11,7 @@ Args: [sock (imm/reg) = ebp]
   looplabel = common.label("loop")
 %>
 
+    /* dup() file descriptor ${sock} into stdin/stdout/stderr */
 ${dup}:
     ${mov('ebx', sock)}
     ${mov('ecx', 3)}

pwnlib/shellcraft/templates/mips/linux/cat.asm

@@ -22,5 +22,5 @@ label = common.label("sendfile_loop")
 %>
 
     ${mips.pushstr(filename)}
-    ${mips.open('$sp', 0, int(constants.O_RDONLY))}
+    ${mips.open('$sp', int(constants.O_RDONLY), 0)}
     ${mips.sendfile(fd, '$v0', 0, 0x7fffffff)}

pwnlib/shellcraft/templates/thumb/linux/cat.asm

@@ -20,6 +20,6 @@ label = common.label("sendfile_loop")
 %>
 
     ${thumb.pushstr(filename)}
-    ${thumb.linux.open('sp', 0, constants.O_RDONLY)}
+    ${thumb.linux.open('sp', constants.O_RDONLY, 0)}
     ${thumb.mov('r5', 'r0')}
     ${thumb.linux.sendfile(fd, 'r5', 0, 0x7fffffff)}

pwnlib/shellcraft/templates/thumb/linux/dup.asm

@@ -9,6 +9,7 @@ Args: [sock (imm/reg) = r6]
   dup       = common.label("dup")
   looplabel = common.label("loop")
 %>
+    /* dup() file descriptor ${sock} into stdin/stdout/stderr */
 ${dup}:
         ${mov('r1', 2)}
         ${mov('r7', 'SYS_dup2')}

pwnlib/tubes/__init__.py

@@ -15,8 +15,9 @@ from pwnlib.tubes import listen
 from pwnlib.tubes import process
 from pwnlib.tubes import remote
 from pwnlib.tubes import serialtube
+from pwnlib.tubes import server
 from pwnlib.tubes import sock
 from pwnlib.tubes import ssh
 from pwnlib.tubes import tube
 
-__all__ = ['tube', 'sock', 'remote', 'listen', 'process', 'serialtube', 'ssh']
+__all__ = ['tube', 'sock', 'remote', 'listen', 'process', 'serialtube', 'server', 'ssh']

pwnlib/tubes/process.py

@@ -16,9 +16,9 @@ import subprocess
 import time
 import tty
 
+from pwnlib import qemu
 from pwnlib.context import context
 from pwnlib.log import getLogger
-from pwnlib.qemu import get_qemu_user
 from pwnlib.timeout import Timeout
 from pwnlib.tubes.tube import tube
 from pwnlib.util.hashes import sha256file
@@ -108,10 +108,6 @@ class process(tube):
         alarm(int):
             Set a SIGALRM alarm timeout on the process.
 
-    Attributes:
-
-        proc(subprocess)
-
     Examples:
 
         >>> p = process('python2')
@@ -238,7 +234,7 @@ class process(tube):
                 raise TypeError('Must provide argv or set context.binary')
 
 
-        #: `subprocess.Popen` object
+        #: :class:`subprocess.Popen` object that backs this process
         self.proc = None
 
         if not shell:
@@ -436,21 +432,21 @@ class process(tube):
 
         # Determine what architecture the binary is, and find the
         # appropriate qemu binary to run it.
-        qemu = get_qemu_user(arch=binary.arch)
+        qemu_path = qemu.user_path(arch=binary.arch)
 
         if not qemu:
             raise exception
 
-        qemu = which(qemu)
+        qemu_path = which(qemu_path)
         if qemu:
-            self._qemu = qemu
+            self._qemu = qemu_path
 
-            args = [qemu]
+            args = [qemu_path]
             if self.argv:
                 args += ['-0', self.argv[0]]
             args += ['--']
 
-            return [args, qemu]
+            return [args, qemu_path]
 
         # If we get here, we couldn't run the binary directly, and
         # we don't have a qemu which can run it.
@@ -569,7 +565,7 @@ class process(tube):
         #
 
         # Create a duplicate so we can modify it safely
-        env = dict(os.environ if env is None else env)
+        env = (os.environ if env is None else env).copy()
 
         for k,v in env.items():
             if not isinstance(k, (str, unicode)):
@@ -957,3 +953,25 @@ class process(tube):
         with open('/proc/%i/mem' % self.pid, 'rb') as mem:
             mem.seek(address)
             return mem.read(count) or None
+
+    @property
+    def stdin(self):
+        """Shorthand for ``self.proc.stdin``
+
+        See: :obj:`.process.proc`
+        """
+        return self.proc.stdin
+    @property
+    def stdout(self):
+        """Shorthand for ``self.proc.stdout``
+
+        See: :obj:`.process.proc`
+        """
+        return self.proc.stdout
+    @property
+    def stderr(self):
+        """Shorthand for ``self.proc.stderr``
+
+        See: :obj:`.process.proc`
+        """
+        return self.proc.stderr

pwnlib/tubes/remote.py

@@ -23,7 +23,7 @@ class remote(sock):
         typ: The string "tcp" or "udp" or an integer to pass to :func:`socket.getaddrinfo`.
         timeout: A positive number, None or the string "default".
         ssl(bool): Wrap the socket with SSL
-        sock(socket): Socket to inherit, rather than connecting
+        sock(socket.socket): Socket to inherit, rather than connecting
 
     Examples:
 

pwnlib/tubes/server.py

@@ -0,0 +1,162 @@
+from __future__ import absolute_import
+
+import errno
+import socket
+import threading
+
+from pwnlib.context import context
+from pwnlib.log import getLogger
+from pwnlib.tubes.sock import sock
+from pwnlib.tubes.remote import remote
+
+log = getLogger(__name__)
+
+class server(sock):
+    r"""Creates an TCP or UDP-server to listen for connections. It supports
+    both IPv4 and IPv6.
+
+    Arguments:
+        port(int): The port to connect to.
+            Defaults to a port auto-selected by the operating system.
+        bindaddr(str): The address to bind to.
+            Defaults to ``0.0.0.0`` / `::`.
+        fam: The string "any", "ipv4" or "ipv6" or an integer to pass to :func:`socket.getaddrinfo`.
+        typ: The string "tcp" or "udp" or an integer to pass to :func:`socket.getaddrinfo`.
+        callback: A function to be started on incoming connections. It should take a :class:`pwnlib.tubes.remote` as its only argument.
+
+    Examples:
+
+        >>> s = server(8888)
+        >>> client_conn = remote('localhost', s.lport)
+        >>> server_conn = s.next_connection()
+        >>> client_conn.sendline('Hello')
+        >>> server_conn.recvline()
+        'Hello\n'
+        >>> def cb(r):
+        ...     client_input = r.readline()
+        ...     r.send(client_input[::-1])
+        ...
+        >>> t = server(8889, callback=cb)
+        >>> client_conn = remote('localhost', t.lport)
+        >>> client_conn.sendline('callback')
+        >>> client_conn.recv()
+        '\nkcabllac'
+    """
+
+    #: Local port
+    lport = 0
+
+    #: Local host
+    lhost = None
+
+    #: Socket type (e.g. socket.SOCK_STREAM)
+    type = None
+
+    #: Socket family
+    family = None
+
+    #: Socket protocol
+    protocol = None
+
+    #: Canonical name of the listening interface
+    canonname = None
+
+    #: Sockaddr structure that is being listened on
+    sockaddr = None
+
+    _accepter = None
+
+    def __init__(self, port=0, bindaddr = "0.0.0.0", fam = "any", typ = "tcp",
+                 callback = None, blocking = False, *args, **kwargs):
+        super(server, self).__init__(*args, **kwargs)
+
+        port = int(port)
+        fam  = {socket.AF_INET: 'ipv4',
+                socket.AF_INET6: 'ipv6'}.get(fam, fam)
+
+        fam = self._get_family(fam)
+        typ = self._get_type(typ)
+
+        if fam == socket.AF_INET6 and bindaddr == '0.0.0.0':
+            bindaddr = '::'
+
+        h = self.waitfor('Trying to bind to %s on port %d' % (bindaddr, port))
+
+        for res in socket.getaddrinfo(bindaddr, port, fam, typ, 0, socket.AI_PASSIVE):
+            self.family, self.type, self.proto, self.canonname, self.sockaddr = res
+
+            if self.type not in [socket.SOCK_STREAM, socket.SOCK_DGRAM]:
+                continue
+
+            h.status("Trying %s" % self.sockaddr[0])
+            listen_sock = socket.socket(self.family, self.type, self.proto)
+            listen_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+            listen_sock.bind(self.sockaddr)
+            self.lhost, self.lport = listen_sock.getsockname()[:2]
+            if self.type == socket.SOCK_STREAM:
+                listen_sock.listen(1)
+            break
+        else:
+            h.failure()
+            self.error("Could not bind to %s on port %d" % (bindaddr, port))
+
+        h.success()
+
+        self.sock = listen_sock
+        self.connections_waiting = threading.Event()
+        self.connections = []
+        def accepter():
+            while True:
+                h = self.waitfor('Waiting for connections on %s:%s' % (self.lhost, self.lport))
+                while True:
+                    try:
+                        if self.type == socket.SOCK_STREAM:
+                            sock, rhost = listen_sock.accept()
+                        else:
+                            data, rhost = listen_sock.recvfrom(4096)
+                            listen_sock.connect(rhost)
+                            sock = listen_sock
+                            sock.unrecv(data)
+                        sock.settimeout(self.timeout)
+                        break
+                    except socket.error as e:
+                        if e.errno == errno.EINTR:
+                            continue
+                        h.failure()
+                        self.exception("Socket failure while waiting for connection")
+                        sock = None
+                        return
+
+                self.rhost, self.rport = rhost[:2]
+                r = remote(self.rhost, self.rport, sock = sock)
+                h.success('Got connection from %s on port %d' % (self.rhost, self.rport))
+                if callback:
+                    if not blocking:
+                        t = context.Thread(target = callback, args = (r,))
+                        t.daemon = True
+                        t.start()
+                    else:
+                        callback(r)
+                else:
+                    self.connections.append(r)
+                    if not self.connections_waiting.is_set():
+                        self.connections_waiting.set()
+
+        self._accepter = context.Thread(target = accepter)
+        self._accepter.daemon = True
+        self._accepter.start()
+
+    def next_connection(self):
+        if not self.connections_waiting.is_set():
+            self.connections_waiting.wait()
+        conn = self.connections.pop(0)
+        if not self.connections:
+            self.connections_waiting.clear()
+        return conn
+
+    def close(self):
+        # since `close` is scheduled to run on exit we must check that we got
+        # a connection or the program will hang in the `join` call above
+        if self._accepter and self._accepter.is_alive():
+            return
+        super(server, self).close()

pwnlib/tubes/ssh.py

@@ -341,7 +341,8 @@ class ssh_process(ssh_channel):
         libs = self.parent.libs(self.executable)
 
         for lib in libs:
-            if self.executable in lib:
+            # Cannot just check "executable in lib", see issue #1047
+            if lib.endswith(self.executable):
                 return pwnlib.elf.elf.ELF(lib)
 
 
@@ -873,6 +874,8 @@ os.chdir(%(cwd)r)
 if env is not None:
     os.environ.clear()
     os.environ.update(env)
+else:
+    env = os.environ
 
 def is_exe(path):
     return os.path.isfile(path) and os.access(path, os.X_OK)
@@ -960,7 +963,7 @@ except Exception:
 %(func_src)s
 apply(%(func_name)s, %(func_args)r)
 
-os.execve(exe, argv, os.environ)
+os.execve(exe, argv, env)
 """ % locals()
 
         script = script.strip()
@@ -1301,7 +1304,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))
         return misc.parse_ldd_output(data)
 
     def _get_fingerprint(self, remote):
-        cmd = '(openssl sha256 || sha256 || sha256sum) 2>/dev/null < '
+        cmd = '(sha256 || sha256sum || openssl sha256) 2>/dev/null < '
         cmd = cmd + sh_string(remote)
         data, status = self.run_to_end(cmd)
 

pwnlib/util/cyclic.py

@@ -2,15 +2,15 @@ from __future__ import absolute_import
 
 import string
 
-from pwnlib.context import context
+from pwnlib.context import context, LocalContext
 from pwnlib.log import getLogger
 from pwnlib.util import packing
 
 log = getLogger(__name__)
 
 # Taken from https://en.wikipedia.org/wiki/De_Bruijn_sequence but changed to a generator
-def de_bruijn(alphabet = string.ascii_lowercase, n = None):
-    """de_bruijn(alphabet = string.ascii_lowercase, n = 4) -> generator
+def de_bruijn(alphabet = None, n = None):
+    """de_bruijn(alphabet = None, n = None) -> generator
 
     Generator for a sequence of unique substrings of length `n`. This is implemented using a
     De Bruijn Sequence over the given `alphabet`.
@@ -21,8 +21,10 @@ def de_bruijn(alphabet = string.ascii_lowercase, n = None):
         alphabet: List or string to generate the sequence over.
         n(int): The length of subsequences that should be unique.
     """
+    if alphabet is None:
+        alphabet = context.cyclic_alphabet
     if n is None:
-        n = 4
+        n = context.cyclic_size
     k = len(alphabet)
     a = [0] * k * n
     def db(t, p):
@@ -42,8 +44,8 @@ def de_bruijn(alphabet = string.ascii_lowercase, n = None):
 
     return db(1,1)
 
-def cyclic(length = None, alphabet = string.ascii_lowercase, n = None):
-    """cyclic(length = None, alphabet = string.ascii_lowercase, n = 4) -> list/str
+def cyclic(length = None, alphabet = None, n = None):
+    """cyclic(length = None, alphabet = None, n = None) -> list/str
 
     A simple wrapper over :func:`de_bruijn`. This function returns at most
     `length` elements.
@@ -56,17 +58,68 @@ def cyclic(length = None, alphabet = string.ascii_lowercase, n = None):
         alphabet: List or string to generate the sequence over.
         n(int): The length of subsequences that should be unique.
 
+    Notes:
+        The maximum length is `len(alphabet)**n`.
+
+        The default values for `alphabet` and `n` restrict the total space to ~446KB.
+
+        If you need to generate a longer cyclic pattern, provide a longer `alphabet`,
+        or if possible a larger `n`.
+
     Example:
-        >>> cyclic(alphabet = "ABC", n = 3)
-        'AAABAACABBABCACBACCBBBCBCCC'
+
+        Cyclic patterns are usually generated by providing a specific `length`.
+
         >>> cyclic(20)
         'aaaabaaacaaadaaaeaaa'
+
+        >>> cyclic(32)
+        'aaaabaaacaaadaaaeaaafaaagaaahaaa'
+
+        The `alphabet` and `n` arguments will control the actual output of the pattern
+
+        >>> cyclic(20, alphabet=string.ascii_uppercase)
+        'AAAABAAACAAADAAAEAAA'
+
+        >>> cyclic(20, n=8)
+        'aaaaaaaabaaaaaaacaaa'
+
+        >>> cyclic(20, n=2)
+        'aabacadaeafagahaiaja'
+
+        The size of `n` and `alphabet` limit the maximum length that can be generated.
+        Without providing `length`, the entire possible cyclic space is generated.
+
+        >>> cyclic(alphabet = "ABC", n = 3)
+        'AAABAACABBABCACBACCBBBCBCCC'
+
+        >>> cyclic(length=512, alphabet = "ABC", n = 3)
+        Traceback (most recent call last):
+        ...
+        PwnlibException: Can't create a pattern length=512 with len(alphabet)==3 and n==3
+
+        The `alphabet` can be set in `context`, which is useful for circumstances
+        when certain characters are not allowed.  See :obj:`.context.cyclic_alphabet`.
+
+        >>> context.cyclic_alphabet = "ABC"
+        >>> cyclic(10)
+        'AAAABAAACA'
+
+        The original values can always be restored with:
+
+        >>> context.clear()
+
+        The following just a test to make sure the length is correct.
+
         >>> alphabet, n = range(30), 3
         >>> len(alphabet)**n, len(cyclic(alphabet = alphabet, n = n))
         (27000, 27000)
     """
     if n is None:
-        n = 4
+        n = context.cyclic_size
+
+    if alphabet is None:
+        alphabet = context.cyclic_alphabet
 
     if len(alphabet) ** n < length:
         log.error("Can't create a pattern length=%i with len(alphabet)==%i and n==%i" \
@@ -84,8 +137,9 @@ def cyclic(length = None, alphabet = string.ascii_lowercase, n = None):
     else:
         return out
 
-def cyclic_find(subseq, alphabet = string.ascii_lowercase, n = None):
-    """cyclic_find(subseq, alphabet = string.ascii_lowercase, n = None) -> int
+@LocalContext
+def cyclic_find(subseq, alphabet = None, n = None):
+    """cyclic_find(subseq, alphabet = None, n = None) -> int
 
     Calculates the position of a substring into a De Bruijn sequence.
 
@@ -103,25 +157,71 @@ def cyclic_find(subseq, alphabet = string.ascii_lowercase, n = None):
                 integer. If an integer is provided it will be packed as a
                 little endian integer.
         alphabet: List or string to generate the sequence over.
+                  By default, uses :obj:`.context.cyclic_alphabet`.
         n(int): The length of subsequences that should be unique.
-
+                By default, uses :obj:`.context.cyclic_size`.
 
     Examples:
 
+        Let's generate an example cyclic pattern.
+
+        >>> cyclic(16)
+        'aaaabaaacaaadaaa'
+
+        Note that 'baaa' starts at offset 4.  The `cyclic_find` routine shows us this:
+
+        >>> cyclic_find('baaa')
+        4
+
+        The *default* length of a subsequence generated by `cyclic` is `4`.
+        If a longer value is submitted, it is automatically truncated to four bytes.
+
+        >>> cyclic_find('baaacaaa')
+        4
+
+        If you provided e.g. `n=8` to `cyclic` to generate larger subsequences,
+        you must explicitly provide that argument.
+
+        >>> cyclic_find('baaacaaa', n=8)
+        3515208
+
+        We can generate a large cyclic pattern, and grab a subset of it to
+        check a deeper offset.
+
         >>> cyclic_find(cyclic(1000)[514:518])
         514
+
+        Instead of passing in the byte representation of the pattern, you can
+        also pass in the integer value.  Note that this is sensitive to the
+        selected endianness via `context.endian`.
+
         >>> cyclic_find(0x61616162)
         4
-    """
-    if isinstance(subseq, (int, long)):
-        width = 'all' if n is None else n * 8
-        subseq = packing.pack(subseq, width, 'little', False)
+        >>> cyclic_find(0x61616162, endian='big')
+        1
 
-    if n is None and len(subseq) != 4:
-        log.warn_once("cyclic_find() expects 4-byte subsequences by default, you gave %r\n" % subseq \
-            + "Unless you specified cyclic(..., n=%i), you probably just want the first 4 bytes.\n" % len(subseq) \
-            + "Truncating the data at 4 bytes.  Specify cyclic_find(..., n=%i) to override this." % len(subseq))
-        subseq = subseq[:4]
+        You can use anything for the cyclic pattern, including non-printable
+        characters.
+
+        >>> cyclic_find(0x00000000, alphabet=unhex('DEADBEEF00'))
+        621
+    """
+
+    if n is None:
+        n = context.cyclic_size
+
+    if isinstance(subseq, (int, long)):
+        subseq = packing.pack(subseq, bytes=n)
+
+    if len(subseq) != n:
+        log.warn_once("cyclic_find() expects %i-byte subsequences by default, you gave %r\n"\
+            + "Unless you specified cyclic(..., n=%i), you probably just want the first 4 bytes.\n"\
+            + "Truncating the data at 4 bytes.  Specify cyclic_find(..., n=%i) to override this.",
+            n, subseq, len(subseq), len(subseq))
+        subseq = subseq[:n]
+
+    if alphabet is None:
+        alphabet = context.cyclic_alphabet
 
     if any(c not in alphabet for c in subseq):
         return -1

pwnlib/util/iters.py

@@ -534,8 +534,8 @@ def iter_except(func, exception):
     the end.
 
     Arguments:
-      func:  The function to call.
-      exception(exception):  The exception that signals the end.  Other
+      func(callable): The function to call.
+      exception(Exception):  The exception that signals the end.  Other
         exceptions will not be caught.
 
     Returns:

pwnlib/util/misc.py

@@ -190,6 +190,8 @@ def run_in_new_terminal(command, terminal = None, args = None):
           variable), ``x-terminal-emulator`` is used.
         - If tmux is detected (by the presence of the ``$TMUX`` environment
           variable), a new pane will be opened.
+        - If GNU Screen is detected (by the presence of the ``$STY`` environment
+          variable), a new screen will be opened.
 
     Arguments:
         command (str): The command to run.
@@ -219,6 +221,9 @@ def run_in_new_terminal(command, terminal = None, args = None):
         elif 'TMUX' in os.environ and which('tmux'):
             terminal = 'tmux'
             args     = ['splitw']
+        elif 'STY' in os.environ and which('screen'):
+            terminal = 'screen'
+            args     = ['-t','pwntools-gdb','bash','-c']
 
     if not terminal:
         log.error('Could not find a terminal binary to use. Set context.terminal to your terminal.')

pwnlib/util/packing.py

@@ -58,7 +58,7 @@ def pack(number, word_size = None, endianness = None, sign = None, **kwargs):
 
     Arguments:
         number (int): Number to convert
-        word_size (int): Word size of the converted integer or the string 'all'.
+        word_size (int): Word size of the converted integer or the string 'all' (in bits).
         endianness (str): Endianness of the converted integer ("little"/"big")
         sign (str): Signedness of the converted integer (False/True)
         kwargs: Anything that can be passed to context.local
@@ -171,7 +171,7 @@ def unpack(data, word_size = None):
 
     Arguments:
         number (int): String to convert
-        word_size (int): Word size of the converted integer or the string "all".
+        word_size (int): Word size of the converted integer or the string "all" (in bits).
         endianness (str): Endianness of the converted integer ("little"/"big")
         sign (str): Signedness of the converted integer (False/True)
         kwargs: Anything that can be passed to context.local
@@ -237,7 +237,7 @@ def unpack_many(data, word_size = None):
 
     Args
         number (int): String to convert
-        word_size (int): Word size of the converted integers or the string "all".
+        word_size (int): Word size of the converted integers or the string "all" (in bits).
         endianness (str): Endianness of the converted integer ("little"/"big")
         sign (str): Signedness of the converted integer (False/True)
         kwargs: Anything that can be passed to context.local
@@ -366,7 +366,7 @@ def make_packer(word_size = None, sign = None, **kwargs):
     faster to call this function, since it will then use a specialized version.
 
     Arguments:
-        word_size (int): The word size to be baked into the returned packer or the string all.
+        word_size (int): The word size to be baked into the returned packer or the string all (in bits).
         endianness (str): The endianness to be baked into the returned packer. ("little"/"big")
         sign (str): The signness to be baked into the returned packer. ("unsigned"/"signed")
         kwargs: Additional context flags, for setting by alias (e.g. ``endian=`` rather than index)
@@ -429,7 +429,7 @@ def make_unpacker(word_size = None, endianness = None, sign = None, **kwargs):
     faster to call this function, since it will then use a specialized version.
 
     Arguments:
-        word_size (int): The word size to be baked into the returned packer.
+        word_size (int): The word size to be baked into the returned packer (in bits).
         endianness (str): The endianness to be baked into the returned packer. ("little"/"big")
         sign (str): The signness to be baked into the returned packer. ("unsigned"/"signed")
         kwargs: Additional context flags, for setting by alias (e.g. ``endian=`` rather than index)
@@ -525,7 +525,7 @@ def flat(*args, **kwargs):
       preprocessor (function): Gets called on every element to optionally
          transform the element before flattening. If :const:`None` is
          returned, then the original value is uded.
-      word_size (int): Word size of the converted integer.
+      word_size (int): Word size of the converted integer (in bits).
       endianness (str): Endianness of the converted integer ("little"/"big").
       sign (str): Signedness of the converted integer (False/True)
 
@@ -577,7 +577,7 @@ def fit(pieces=None, **kwargs):
       preprocessor (function): Gets called on every element to optionally
          transform the element before flattening. If :const:`None` is
          returned, then the original value is used.
-      word_size (int): Word size of the converted integer.
+      word_size (int): Word size of the converted integer (in bits).
       endianness (str): Endianness of the converted integer ("little"/"big").
       sign (str): Signedness of the converted integer (False/True)
 

pwnlib/util/sh_string.py

@@ -393,7 +393,7 @@ def sh_string(s):
         return "''"
 
     chars = set(s)
-    very_good = set(string.ascii_letters + string.digits + "_+.,/")
+    very_good = set(string.ascii_letters + string.digits + "_+.,/-")
 
     # Alphanumeric can always just be used verbatim.
     if chars <= very_good:

pwnlib/version.py

@@ -1 +1 @@
-__version__ = '3.9.2'
+__version__ = '3.12.0'

setup.py

@@ -86,7 +86,7 @@ except Exception as e:
 setup(
     name                 = 'pwntools',
     packages             = find_packages(),
-    version              = '3.9.2',
+    version              = '3.12.0',
     data_files           = [('',
                              glob.glob('*.md') + glob.glob('*.txt')),
                             ],

travis/install.sh

@@ -90,7 +90,7 @@ setup_android_emulator()
         elif [[ -n "$TRAVIS_TAG" ]]; then
             echo "TRAVIS_TAG ($TRAVIS_TAG) indicates a new relase"
             echo "Forcing Android Emulator installation"
-        elif (git log --stat "$TRAVIS_COMMIT_RANGE" | grep -iE "android|adb"); then
+        elif (git log --stat "$TRAVIS_COMMIT_RANGE" | grep -iE "android|adb" | grep -v "commit "); then
             echo "Found Android-related commits, forcing Android Emulator installation"
         else
             # In order to avoid running the doctests that require the Android
@@ -119,7 +119,7 @@ setup_android_emulator()
     else
         if [ ! -f android-sdk/android ]; then
             # Install the SDK, which gives us the 'android' and 'emulator' commands
-            wget https://dl.google.com/android/android-sdk_r24.4.1-linux.tgz
+            wget -nv https://dl.google.com/android/android-sdk_r24.4.1-linux.tgz
             tar xf android-sdk_r24.4.1-linux.tgz
             rm  -f android-sdk_r24.4.1-linux.tgz
 
@@ -137,7 +137,7 @@ setup_android_emulator()
         # Install the NDK, which is required for adb.compile()
         NDK_VERSION=android-ndk-r12b
         if [ ! -f android-ndk/ndk-build ]; then
-            wget   https://dl.google.com/android/repository/$NDK_VERSION-linux-x86_64.zip
+            wget -nv https://dl.google.com/android/repository/$NDK_VERSION-linux-x86_64.zip
             unzip -q android-ndk-*.zip
             rm -f    android-ndk-*.zip