Release 4.6.0

Merge branch 'stable' into beta
[4.5.1] Update setup.py
2026-11-11 02:02:17 -10:00 · 2021-07-12 23:00:40 +02:00 · 2021-05-31 15:55:40 -05:00 · 2021-05-31 15:44:49 -05:00 · 2021-05-31 20:29:49 +00:00 · 2021-05-31 15:20:35 -05:00
79 changed files with 960 additions and 418 deletions
@@ -0,0 +1,60 @@
+name: Android Tests
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    strategy:
+      matrix:
+        python-version: [3.8]
+        os: [ubuntu-latest]
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 30
+    steps:
+    - uses: actions/checkout@v2
+
+    - name: Cache for pip
+      uses: actions/cache@v1
+      id: cache-pip
+      with:
+        path: ~/.cache/pip
+        key: ${{ matrix.os }}-cache-pip
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v1
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Install Linux dependencies
+      run: |
+        sudo apt-get update
+        sudo apt-get install -y --no-install-recommends -o Acquire::Retries=3 \
+          gdb gdbserver socat \
+          qemu-user-static \
+          binutils-aarch64-linux-gnu \
+          binutils-arm-linux-gnueabihf \
+          libc6-dbg \
+          openjdk-8-jre-headless
+
+    - name: Install Android AVD
+      run: |
+        USER=travis source travis/setup_avd.sh
+        sed -i 's/skip_android = True/skip_android = False/' docs/source/conf.py
+        set | egrep '^(ANDROID|PATH)' >.android.env
+
+    - name: Install dependencies
+      run: |
+        pip install --upgrade pip
+        python setup.py egg_info
+        pip install --upgrade --editable .
+
+    - name: Sanity checks
+      run:  PWNLIB_NOTERM=1 python -c 'from pwn import *; print(pwnlib.term.term_mode)'
+
+    - name: Install documentation dependencies
+      run:  pip install -r docs/requirements.txt
+
+    - name: Coverage Doctests (Android Only)
+      run: |
+        source .android.env
+        PWNLIB_NOTERM=1 coverage run -m sphinx -b doctest docs/source docs/build/doctest docs/source/adb.rst
@@ -4,7 +4,7 @@ env:
  GITHUB_BASE: origin/${{ github.event.pull_request.base.ref }}

 jobs:
-  build:
+  check:
    strategy:
      matrix:
        os: [ubuntu-latest]
@@ -20,7 +20,7 @@ jobs:
        if git log --stat ${GITHUB_BASE}..HEAD | grep CHANGELOG.md; then
           echo 'Changelog updated :D'
        else
-           if git log --stat ${GITHUB_BASE}..HEAD | grep '++\|--'; then
+           if git log --stat ${GITHUB_BASE}..HEAD pwnlib pwn | grep '++\|--'; then
              echo 'Major changes detected, changelog required!'
              false
           else
@@ -86,7 +86,7 @@ jobs:
        pip install --upgrade --editable .

    - name: Sanity checks
-      run:  PWNLIB_NOTERM=1 python -c 'from pwn import *; print(pwnlib.term.term_mode)'
+      run:  PWNLIB_NOTERM=1 python -bb -c 'from pwn import *; print(pwnlib.term.term_mode)'

    - name: Install documentation dependencies
      run:  pip install -r docs/requirements.txt
@@ -94,29 +94,29 @@ jobs:
    - name: Manually install non-broken Unicorn
      run:  pip install unicorn==1.0.2rc3

-    - name: Version Check
-      run:  PWNLIB_NOTERM=1 pwn version
+    - name: Disable yama ptrace_scope
+      run: |
+        echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope # required by some gdb doctests

    - name: Coverage doctests
      run: |
-        echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope # required by some gdb doctests
-        PWNLIB_NOTERM=1 coverage run -m sphinx -b doctest docs/source docs/build/doctest
+        PWNLIB_NOTERM=1 python -bb -m coverage run -m sphinx -b doctest docs/source docs/build/doctest

    - name: Coverage running examples
      run: |
        export TERM=linux
        set -x
-        python travis/coverage_chdir.py examples/fmtstr examples/fmtstr/exploit.py
-        python travis/coverage_chdir.py examples/fmtstr examples/fmtstr/exploit2.py || : # can fail randomly?
-        coverage run examples/asm.py
-        coverage run examples/asm.py
-        coverage run examples/text.py
+        python -bb travis/coverage_chdir.py examples/fmtstr examples/fmtstr/exploit.py
+        python -bb travis/coverage_chdir.py examples/fmtstr examples/fmtstr/exploit2.py || : # can fail randomly?
+        python -bb -m coverage run examples/asm.py
+        python -bb -m coverage run examples/asm.py
+        python -bb -m coverage run examples/text.py
        # for f in examples/sigreturn_corefile_*.py; do coverage run "$f"; done # XXX something is wrong

    - name: Coverage running commandline tools
      run: |
        export TERM=linux
-        pwn() { ( set +x; cmd=$1; shift; PYTHONUNBUFFERED=1 exec coverage run -m pwnlib.commandline."$cmd" "$@" ) }
+        pwn() { ( set +x; cmd=$1; shift; PYTHONUNBUFFERED=1 exec python -bb -m coverage run -m pwnlib.commandline."$cmd" "$@" ) }
        set -x

        pwn cyclic 32
@@ -17,3 +17,6 @@ venv
 *.core
 .coverage
 .coverage.*
+.idea
+__pycache__
+!.github
@@ -9,10 +9,12 @@ The table below shows which release corresponds to each branch, and what date th

 | Version          | Branch   | Release Date           |
 | ---------------- | -------- | ---------------------- |
-| [4.7.0](#470)    | `dev`    | May 29, 2020 (planned)
-| [4.6.0](#460)    | `beta`   | May 29, 2020 (planned)
-| [4.5.0](#450)    | `stable` | Apr 30, 2020
-| [4.4.0](#440)    |          | Mar 29, 2020
+| [4.8.0](#480)    | `dev`    |
+| [4.7.0](#470)    | `beta`   |
+| [4.6.0](#460)    | `stable` | Jul 12, 2021
+| [4.5.1](#451)    |          | May 30, 2021
+| [4.5.0](#450)    |          | Apr 30, 2021
+| [4.4.0](#440)    |          | Mar 29, 2021
 | [4.3.1](#431)    |          | Nov 29, 2020
 | [4.3.0](#430)    |          | Oct 20, 2020
 | [4.2.0](#420)    |          | Jul 3, 2020
@@ -59,7 +61,37 @@ The table below shows which release corresponds to each branch, and what date th
 | [3.0.0](#300)    |          | Aug 20, 2016
 | [2.2.0](#220)    |          | Jan 5, 2015

-## 4.7.0 (`dev`)
+## 4.8.0 (`dev`)
+
+
+
+## 4.7.0 (`beta`)
+
+- [#1733][1733] Update libc headers -> more syscalls available!
+- [#1876][1876] add `self.message` and change `sys.exc_type` to `sys.exec_info()` in PwnlibException
+- [#1877][1877] encoders error message handles when `avoid` is bytes in python3
+- [#1891][1891] Keep ROP gadgets when setting registers via setattr/call
+- [#1892][1892] Silence SIGPIPE error for "pwn phd"
+- [#1893][1893] Fix bytes warning in "pwn cyclic"
+- [#1897][1897] Add basic support for RISC-V
+- [#1903][1903] Add zsh completion script
+- [#1904][1904] Add bash completion script
+- [#1906][1906] Defer import of several modules to save on startup time
+- [#1921][1921] Add basic support for the bare-metal ARM specific toolchain
+
+[1733]: https://github.com/Gallopsled/pwntools/pull/1733
+[1876]: https://github.com/Gallopsled/pwntools/pull/1876
+[1877]: https://github.com/Gallopsled/pwntools/pull/1877
+[1891]: https://github.com/Gallopsled/pwntools/pull/1891
+[1892]: https://github.com/Gallopsled/pwntools/pull/1892
+[1893]: https://github.com/Gallopsled/pwntools/pull/1893
+[1897]: https://github.com/Gallopsled/pwntools/pull/1897
+[1903]: https://github.com/Gallopsled/pwntools/pull/1903
+[1904]: https://github.com/Gallopsled/pwntools/pull/1904
+[1906]: https://github.com/Gallopsled/pwntools/pull/1906
+[1921]: https://github.com/Gallopsled/pwntools/pull/1921
+
+## 4.6.0 (`stable`)

 - [#1429][1429] Add a mechanism for ret2csu (originally #1138)
 - [#1566][1566] Add `ignore_config` argument to `pwnlib.tubes.ssh` and improve `allow_agent` implementation
@@ -81,29 +113,13 @@ The table below shows which release corresponds to each branch, and what date th
 [1776]: https://github.com/Gallopsled/pwntools/pull/1776
 [1846]: https://github.com/Gallopsled/pwntools/pull/1846

-## 4.6.0 (`beta`)
+## 4.5.1

- [#1429][1429] Add a mechanism for ret2csu (originally #1138)
- [#1566][1566] Add `ignore_config` argument to `pwnlib.tubes.ssh` and improve `allow_agent` implementation
- [#1652][1652] Add `process.readmem` and `process.writemem`
- [#1739][1739] Add/fix shellcraft.linux.kill() / shellcraft.linux.killparent()
- [#1746][1746] Prefer Python3 over Python2 for spawning remote processes over SSH
- [#1751][1751] Fix process() with executable relative to cwd
- [#1753][1753] major change: less unconditional imports in pwnlib
- [#1776][1776] mips: do not use $t0 temporary variable in dupio
- [#1846][1846] support launching GDB in more different terminals
+- [#1902][1902] Always specify -F and -P for tmux in `run_in_new_terminal`

-[1429]: https://github.com/Gallopsled/pwntools/pull/1429
-[1566]: https://github.com/Gallopsled/pwntools/pull/1566
-[1652]: https://github.com/Gallopsled/pwntools/pull/1652
-[1739]: https://github.com/Gallopsled/pwntools/pull/1739
-[1746]: https://github.com/Gallopsled/pwntools/pull/1746
-[1751]: https://github.com/Gallopsled/pwntools/pull/1751
-[1753]: https://github.com/Gallopsled/pwntools/pull/1753
-[1776]: https://github.com/Gallopsled/pwntools/pull/1776
-[1846]: https://github.com/Gallopsled/pwntools/pull/1846
+[1902]: https://github.com/Gallopsled/pwntools/pull/1902

-## 4.5.0 (`stable`)
+## 4.5.0

 - [#1261][1261] Misc `run_in_new_terminal` improvements (notably gdb terminated by default)
 - [#1695][1695] Allow using GDB Python API
@@ -121,6 +137,7 @@ The table below shows which release corresponds to each branch, and what date th
 - [#1839][1839] run_in_new_terminal now creates a runner script if given a list or tuple
 - [#1833][1833] Add pwnlib.filesystem module
 - [#1852][1852] Fix `atexit` on Python 3
+- [#1883][1883] ROP gadget verifier accounts for 2 character registers

 [1261]: https://github.com/Gallopsled/pwntools/pull/1261
 [1695]: https://github.com/Gallopsled/pwntools/pull/1695
@@ -137,6 +154,7 @@ The table below shows which release corresponds to each branch, and what date th
 [1839]: https://github.com/Gallopsled/pwntools/pull/1839
 [1833]:  https://github.com/Gallopsled/pwntools/pull/1833
 [1852]: https://github.com/Gallopsled/pwntools/pull/1852
+[1883]: https://github.com/Gallopsled/pwntools/pull/1883

 ## 4.4.0

@@ -25,7 +25,7 @@ build_dash = tags.has('dash')
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 sys.path.insert(0, os.path.abspath('../..'))

-import pwnlib
+import pwnlib.update
 pwnlib.update.disabled = True

 # If your documentation needs a minimal Sphinx version, state it here.
@@ -71,8 +71,9 @@ doctest_global_setup = '''
 import sys, os
 os.environ['PWNLIB_NOTERM'] = '1'
 os.environ['PWNLIB_RANDOMIZE'] = '0'
-
-import pwnlib, logging
+import pwnlib.update
+import pwnlib.util.fiddling
+import logging
 pwnlib.update.disabled = True
 pwnlib.context.context.reset_local()
 pwnlib.context.ContextType.defaults['log_level'] = logging.ERROR
@@ -412,7 +413,14 @@ class Py2OutputChecker(_DummyClass, doctest.OutputChecker):
        if sup(rly_want, got, optionflags):
            return True
        rly_want = ' '.join(x[:2].replace('b"',' "').replace("b'"," '")+x[2:] for x in want.replace('\n','\n ').split(' ')).replace('\n ','\n')
-        return sup(rly_want, got, optionflags)
+        if sup(rly_want, got, optionflags):
+            return True
+        for wantl, gotl in six.moves.zip_longest(want.splitlines(), got.splitlines(), fillvalue=''):
+            rly_want1 = '['.join(x[:2].replace('b"','"').replace("b'","'")+x[2:] for x in wantl.split('['))
+            rly_want2 = ' '.join(x[:2].replace('b"',' "').replace("b'"," '")+x[2:] for x in wantl.split(' '))
+            if not sup(rly_want1, gotl, optionflags) and not sup(rly_want2, gotl, optionflags):
+                return False
+        return True

 def py2_doctest_init(self, checker=None, verbose=None, optionflags=0):
    if checker is None:
@@ -426,3 +434,15 @@ if 'doctest' in sys.argv:
    if sys.version_info[:1] < (3,):
        import sphinx.ext.doctest
        sphinx.ext.doctest.SphinxDocTestRunner.__init__ = py2_doctest_init
+    else:
+        # monkey patching paramiko due to https://github.com/paramiko/paramiko/pull/1661
+        import paramiko.client
+        import binascii
+        paramiko.client.hexlify = lambda x: binascii.hexlify(x).decode()
+        paramiko.util.safe_string = lambda x: '' # function result never *actually used*
+    class EndlessLoop(Exception): pass
+    def alrm_handler(sig, frame):
+        signal.alarm(180) # three minutes
+        raise EndlessLoop()
+    signal.signal(signal.SIGALRM, alrm_handler)
+    signal.alarm(600) # ten minutes
@@ -92,3 +92,12 @@ Each of the ``pwntools`` modules is documented here.
   * :ref:`genindex`
   * :ref:`modindex`
   * :ref:`search`
+
+Bytes
+-----
+
+The bytes vs text distinction is so important that it even made it to this main page.
+See the pwntools-tutorial_ repo for the latest tutorial finally
+explaining the difference once and for all (hopefully).
+
+.. _pwntools-tutorial: https://github.com/Gallopsled/pwntools-tutorial/blob/master/bytes.md
@@ -7,7 +7,6 @@ pwnlib.args.initialize()
 pwnlib.log.install_default_handler()
 pwnlib.config.initialize()

-log = pwnlib.log.getLogger('pwnlib.exploit')
 args = pwnlib.args.args

 if not platform.architecture()[0].startswith('64'):
@@ -78,6 +78,7 @@ from pwnlib.util.web import *
 from six.moves import cPickle as pickle, cStringIO as StringIO
 from six import BytesIO

+log = getLogger("pwnlib.exploit")
 error   = log.error
 warning = log.warning
 warn    = log.warning
@@ -1,12 +1,11 @@
 from __future__ import absolute_import

-import importlib
-
-from pwnlib.version import __version__
+from .version import __version__

 version = __version__

 __all__ = [
+    'adb',
    'args',
    'asm',
    'atexception',
@@ -35,9 +34,8 @@ __all__ = [
    'ui',
    'useragents',
    'util',
-    'adb',
    'update',
+    'version',
 ]

-for module in __all__:
-    importlib.import_module('.%s' % module, 'pwnlib')
+from . import args
@@ -68,6 +68,7 @@ from pwnlib.context import context
 from pwnlib.device import Device
 from pwnlib.log import getLogger
 from pwnlib.protocols.adb import AdbClient
+from pwnlib.util.packing import _decode
 from pwnlib.util import misc

 log = getLogger(__name__)
@@ -120,8 +121,8 @@ def current_device(any=False):
       :skipif: skip_android

        >>> device = adb.current_device(any=True)
-        >>> device
-        AdbDevice(serial='emulator-5554', type='device', port='emulator', product='sdk_google_phone_armv7', model='sdk google phone armv7', device='generic')
+        >>> device  # doctest: +ELLIPSIS
+        AdbDevice(serial='emulator-5554', type='device', port='emulator', product='sdk_...phone_armv7', model='sdk ...phone armv7', device='generic')
        >>> device.port
        'emulator'
    """
@@ -256,8 +257,8 @@ class AdbDevice(Device):
        32
        >>> device.os
        'android'
-        >>> device.product
-        'sdk_google_phone_armv7'
+        >>> device.product  # doctest: +ELLIPSIS
+        'sdk_...phone_armv7'
        >>> device.serial
        'emulator-5554'
    """
@@ -895,7 +896,7 @@ def which(name, all = False, *a, **kw):

    which_cmd = which_cmd.strip()
    data = process(['sh','-c', which_cmd], *a, **kw).recvall()
-    data = context._decode(data)
+    data = _decode(data)
    result = []

    for path in data.split('\x00'):
@@ -1012,12 +1013,12 @@ def getprop(name=None):
    with context.quiet:
        if name:
            result = process(['getprop', name]).recvall().strip()
-            result = context._decode(result)
+            result = _decode(result)
            return result

        result = process(['getprop']).recvall()

-    result = context._decode(result)
+    result = _decode(result)
    expr = r'\[([^\]]+)\]: \[(.*)\]'

    props = {}
@@ -1591,3 +1592,4 @@ def version():
    """Returns rthe platform version as a tuple."""
    prop = getprop('ro.build.version.release')
    return [int(v) for v in prop.split('.')]
+
@@ -299,6 +299,7 @@ def _arch_header():
    prefix  = ['.section .shellcode,"awx"',
                '.global _start',
                '.global __start',
+                '.p2align 2',
                '_start:',
                '__start:']
    headers = {
@@ -367,7 +368,7 @@ def _bfdarch():
    return arch

 def _run(cmd, stdin = None):
-    log.debug(subprocess.list2cmdline(cmd))
+    log.debug('%s', subprocess.list2cmdline(cmd))
    try:
        proc = subprocess.Popen(
            cmd,
@@ -380,17 +381,20 @@ def _run(cmd, stdin = None):
        exitcode = proc.wait()
    except OSError as e:
        if e.errno == errno.ENOENT:
-            log.exception('Could not run %r the program' % cmd[0])
+            log.exception('Could not run %r the program', cmd[0])
        else:
            raise

    if (exitcode, stderr) != (0, ''):
-        msg = 'There was an error running %s:\n' % repr(cmd)
+        msg = 'There was an error running %r:\n'
+        args = cmd,
        if exitcode != 0:
-            msg += 'It had the exitcode %d.\n' % exitcode
+            msg += 'It had the exitcode %d.\n'
+            args += exitcode,
        if stderr != '':
-            msg += 'It had this on stdout:\n%s\n' % stderr
-        log.error(msg)
+            msg += 'It had this on stdout:\n%s\n'
+            args += stderr,
+        log.error(msg, *args)

    return stdout

@@ -5,7 +5,7 @@ from __future__ import division
 import argparse
 import sys

-import pwnlib
+import pwnlib.args
 pwnlib.args.free_form = False

 from pwn import *
@@ -6,7 +6,7 @@ import argparse
 import functools
 import re

-import pwnlib
+import pwnlib.args
 pwnlib.args.free_form = False

 from pwn import *
@@ -7,7 +7,7 @@ import six
 import string
 import sys

-import pwnlib
+import pwnlib.args
 pwnlib.args.free_form = False

 from pwn import *
@@ -23,7 +23,7 @@ parser.add_argument(
    '-a', '--alphabet',
    metavar = 'alphabet',
    default = string.ascii_lowercase.encode(),
-    type = six.ensure_binary,
+    type = packing._encode,
    help = 'The alphabet to use in the cyclic pattern (defaults to all lower case letters)',
 )

@@ -7,7 +7,7 @@ import argparse
 import string
 import sys

-import pwnlib
+import pwnlib.args
 pwnlib.args.free_form = False

 from pwn import *
@@ -8,7 +8,7 @@ from subprocess import CalledProcessError
 from subprocess import check_output
 from tempfile import NamedTemporaryFile

-import pwnlib
+import pwnlib.args
 pwnlib.args.free_form = False

 from pwn import *
@@ -4,7 +4,7 @@ from __future__ import division

 import sys

-import pwnlib
+import pwnlib.args
 pwnlib.args.free_form = False

 from pwn import *
@@ -7,7 +7,7 @@ import os
 import sys
 import io

-import pwnlib
+import pwnlib.args
 pwnlib.args.free_form = False

 from pwn import *
@@ -3,7 +3,7 @@ from __future__ import division

 import argparse

-import pwnlib
+import pwnlib.args
 pwnlib.args.free_form = False

 from pwn import *
@@ -4,7 +4,7 @@ from __future__ import division
 import argparse
 import sys

-import pwnlib
+import pwnlib.args
 pwnlib.args.free_form = False

 from pwn import *
@@ -8,7 +8,7 @@ import six
 import sys
 import types

-import pwnlib
+import pwnlib.args
 pwnlib.args.free_form = False

 from pwn import *
@@ -2,8 +2,6 @@
 from __future__ import absolute_import
 from __future__ import division

-import re
-
 from pwn import *
 from pwnlib.commandline import common

@@ -25,14 +23,9 @@ parser.add_argument('--quiet', help='Less verbose template comments', action='st
 parser.add_argument('--color', help='Print the output in color', choices=['never', 'always', 'auto'], default='auto')

 def main(args):
-    cache = None
-
-    if cache:
-        cache = os.path.join(context.cache_dir, 'mako')
-
    lookup = TemplateLookup(
        directories      = [os.path.join(pwnlib.data.path, 'templates')],
-        module_directory = cache
+        module_directory = None
    )

    # For the SSH scenario, check that the binary is at the
@@ -44,10 +37,10 @@ def main(args):
        s = ssh(args.user, args.host, args.port or 22, args.password or None)

        try:
-            remote = args.path or args.exe
-            s.download(remote)
+            remote_file = args.path or args.exe
+            s.download(remote_file)
        except Exception:
-            log.warning("Could not download file %r, opening a shell", remote)
+            log.warning("Could not download file %r, opening a shell", remote_file)
            s.interactive()
            return

@@ -4,7 +4,7 @@ from __future__ import division

 import subprocess

-import pwnlib
+import pwnlib.args
 pwnlib.args.free_form = False

 from pwn import *
@@ -5,7 +5,7 @@ from __future__ import division
 import os
 import subprocess

-import pwnlib
+import pwnlib.args
 pwnlib.args.free_form = False

 from pwn import *
@@ -901,27 +901,6 @@ class ContextType(object):

        return charset

-    def _encode(self, s):
-        if isinstance(s, (bytes, bytearray)):
-            return s   # already bytes
-
-        if self.encoding == 'auto':
-            try:
-                return s.encode('latin1')
-            except UnicodeEncodeError:
-                return s.encode('utf-8', 'surrogateescape')
-        return s.encode(self.encoding)
-
-    def _decode(self, b):
-        if self.encoding == 'auto':
-            try:
-                return b.decode('utf-8')
-            except UnicodeDecodeError:
-                return b.decode('latin1')
-            except AttributeError:
-                return b
-        return b.decode(self.encoding)
-
    @_validator
    def endian(self, endianness):
        """
@@ -1362,7 +1341,9 @@ class ContextType(object):
        This configures the newline emitted by e.g. ``sendline`` or that is used
        as a delimiter for e.g. ``recvline``.
        """
-        return six.ensure_binary(v)
+        # circular imports
+        from pwnlib.util.packing import _need_bytes
+        return _need_bytes(v)


    @_validator
@@ -32,7 +32,7 @@ remote_path = remote_path or exe
 password = password or 'secret1234'
 binary_repr = repr(binary)
 %>\
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 %if not quiet:
 # This exploit template was generated via:
@@ -84,14 +84,14 @@ if not args.LOCAL:
 %endif

 %if host:
-def local(argv=[], *a, **kw):
+def start_local(argv=[], *a, **kw):
    '''Execute the target binary locally'''
    if args.GDB:
        return gdb.debug([${binary_repr}] + argv, gdbscript=gdbscript, *a, **kw)
    else:
        return process([${binary_repr}] + argv, *a, **kw)

-def remote(argv=[], *a, **kw):
+def start_remote(argv=[], *a, **kw):
  %if ssh:
    '''Execute the target binary on the remote host'''
    if args.GDB:
@@ -111,9 +111,9 @@ def remote(argv=[], *a, **kw):
 def start(argv=[], *a, **kw):
    '''Start the exploit against the target.'''
    if args.LOCAL:
-        return local(argv, *a, **kw)
+        return start_local(argv, *a, **kw)
    else:
-        return remote(argv, *a, **kw)
+        return start_remote(argv, *a, **kw)
 %else:
 def start(argv=[], *a, **kw):
    '''Start the exploit against the target.'''
@@ -62,7 +62,7 @@ from pwnlib.elf import constants
 from pwnlib.log import getLogger
 from pwnlib.memleak import MemLeak
 from pwnlib.util.fiddling import enhex
-from pwnlib.util.packing import unpack
+from pwnlib.util.packing import unpack, _encode
 from pwnlib.util.web import wget

 log    = getLogger(__name__)
@@ -75,7 +75,7 @@ def sysv_hash(symbol):
    """
    h = 0
    g = 0
-    for c in bytearray(context._encode(symbol)):
+    for c in bytearray(_encode(symbol)):
        h = (h << 4) + c
        g = h & 0xf0000000
        h ^= (g >> 24)
@@ -87,7 +87,7 @@ def gnu_hash(s):

    Function used to generated GNU-style hashes for strings.
    """
-    s = bytearray(context._encode(s))
+    s = bytearray(_encode(s))
    h = 5381
    for c in s:
        h = h * 33 + c
@@ -539,7 +539,7 @@ class DynELF(object):
            lib = 'libc.so'

        if symb:
-            symb = context._encode(symb)
+            symb = _encode(symb)

        #
        # Get a pretty name for the symbol to show the user
@@ -640,7 +640,7 @@ class DynELF(object):
        while leak.field(cur, LinkMap.l_prev):
            cur = leak.field(cur, LinkMap.l_prev)

-        libname = context._encode(libname)
+        libname = _encode(libname)

        while cur:
            self.status("link_map entry %#x" % cur)
@@ -392,7 +392,7 @@ class Corefile(ELF):
        >>> env = dict(os.environ)
        >>> env['HELLO'] = 'WORLD'
        >>> io = process(elf.path, env=env)
-        >>> io.sendline('echo hello')
+        >>> io.sendline(b'echo hello')
        >>> io.recvline()
        b'hello\n'

@@ -1569,7 +1569,7 @@ class CorefileFinder(object):
            path = os.path.join(binfmt_misc, entry)

            try:
-                data = self.read(path)
+                data = self.read(path).decode()
            except Exception:
                continue

@@ -745,7 +745,7 @@ class ELF(ELFFile):
                    self._libs[qemu_lib] = address

    def _patch_elf_and_read_maps(self):
-        """patch_elf_and_read_maps(self) -> dict
+        r"""patch_elf_and_read_maps(self) -> dict

        Read ``/proc/self/maps`` as if the ELF were executing.

@@ -771,7 +771,7 @@ class ELF(ELFFile):
            ...     sc += shellcraft.exit()
            ...     sc = asm(sc)
            ...     sc = enhex(sc)
-            ...     assert sc == CAT_PROC_MAPS_EXIT[arch]
+            ...     assert sc == CAT_PROC_MAPS_EXIT[arch], (arch, sc)
        """

        # Get our shellcode
@@ -816,7 +816,7 @@ class ELF(ELFFile):
        try:
            with context.silent:
                io = process(path)
-                data = context._decode(io.recvall(timeout=2))
+                data = packing._decode(io.recvall(timeout=2))
        except Exception:
            log.warn_once("Injected /proc/self/maps code did not execute correctly")
            return {}
@@ -17,7 +17,7 @@ CAT_PROC_MAPS_EXIT = {
        '000020e00170a0e3000000ef',
    'thumb':
        '004f01e0617073ff4fea07274fea172780b4dff8047001e06c662f6d80b4dff8047001e0632f736580b4dff8047001e02f70726f80b4684681ea010182ea02024ff0050741df05464ff00100294682ea02026ff000434ff0bb0741df'
-        '80ea00004ff0010741df',
+        '80ea00004ff0010741df00bf',
    'mips':
        '726f093c2f702935f0ffa9af7365093c632f2935f4ffa9af2f6d093c6c662935f8ffa9af8cff193c9e8f393727482003fcffa9aff0ffbd272020a003ffff0528ffff0628a50f02340c010101feff192427202003fcffa2affcffa58fffff0628ff7f073cffffe7346f1002340c010101'
        'ffff0428a10f02340c010101',
@@ -9,11 +9,11 @@ import struct
 # |                       RANDOM NUMBERS FUNCTIONS                         |
 # +------------------------------------------------------------------------+

-# get a random integer i (0<=i<max)
-# =================================
-def random_get_int(max):
-   return random.randint(0, max-1)
-   return struct.unpack("I", os.urandom(4))[0] % max
+# get a random integer i (0<=i<maxv)
+# ==================================
+def random_get_int(maxv):
+   return random.randrange(0, maxv)
+

 def randel(arr):
   return arr[random_get_int(len(arr))]
@@ -12,6 +12,7 @@ import time

 from pwnlib.context import context
 from pwnlib.util.misc import read, write
+from pwnlib.util.packing import _decode

 if six.PY3:
    from pathlib import *
@@ -86,7 +87,7 @@ class SSHPath(PosixPath):

        # We also don't want binary
        if isinstance(other, six.binary_type):
-            return str(context._decode(other))
+            return str(_decode(other))

    def _new(self, path, *a, **kw):
        kw['ssh'] = self.ssh
@@ -782,12 +783,12 @@ class SSHPath(PosixPath):
 #----------------------------- PWNTOOLS ADDITIONS -----------------------------
    @classmethod
    def mktemp(cls):
-        temp = context._decode(context.ssh_session.mktemp())
+        temp = _decode(context.ssh_session.mktemp())
        return SSHPath(temp, ssh=context.ssh_session)

    @classmethod
    def mkdtemp(self):
-        temp = context._decode(context.ssh_session.mkdtemp())
+        temp = _decode(context.ssh_session.mkdtemp())
        return SSHPath(temp, ssh=context.ssh_session)

 __all__ = ['SSHPath']
@@ -58,7 +58,7 @@ def submit_flag(flag,
                      exploit,
                      target,
                      team,
-                      ''])
+                      '']).encode('ascii')

    if os.path.exists(env_file):
        write(env_file, data)
@@ -69,4 +69,4 @@ def submit_flag(flag,
            r.send(data)
            return r.recvall(timeout=1)
    except Exception:
-        log.warn("Could not submit flag %r to %s:%s" % (flag, server, port))
+        log.warn("Could not submit flag %r to %s:%s", flag, server, port)
@@ -144,6 +144,7 @@ from __future__ import division
 from contextlib import contextmanager
 import os
 import platform
+import psutil
 import random
 import re
 import shlex
@@ -154,9 +155,6 @@ import tempfile
 from threading import Event
 import time

-from rpyc import BgServingThread
-from rpyc.utils.factory import unix_connect
-
 from pwnlib import adb
 from pwnlib import atexit
 from pwnlib import elf
@@ -170,6 +168,7 @@ from pwnlib.context import context
 from pwnlib.log import getLogger
 from pwnlib.timeout import Timeout
 from pwnlib.util import misc
+from pwnlib.util import packing
 from pwnlib.util import proc

 log = getLogger(__name__)
@@ -320,13 +319,13 @@ def _gdbserver_port(gdbserver, ssh):

    if process_created.startswith(b'ERROR:'):
        raise ValueError(
-            'Failed to spawn process under gdbserver. gdbserver error message: %s' % process_created
+            'Failed to spawn process under gdbserver. gdbserver error message: %r' % process_created
        )

    try:
        gdbserver.pid   = int(process_created.split()[-1], 0)
    except ValueError:
-        log.error('gdbserver did not output its pid (maybe chmod +x?): %s', six.ensure_str(process_created))
+        log.error('gdbserver did not output its pid (maybe chmod +x?): %r', process_created)

    listening_on = b''
    while b'Listening' not in listening_on:
@@ -425,7 +424,7 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, sysroot=None, api=

        Send a command to Bash

-        >>> io.sendline("echo hello")
+        >>> io.sendline(b"echo hello")
        >>> io.recvline()
        b'hello\n'

@@ -449,7 +448,7 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, sysroot=None, api=

        Send a command to Bash

-        >>> io.sendline("echo hello")
+        >>> io.sendline(b"echo hello")
        >>> io.recvline()
        b'hello\n'

@@ -494,7 +493,7 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, sysroot=None, api=
        Connect to the SSH server and start a process on the server

        >>> shell = ssh('travis', 'example.pwnme', password='demopass')
-        >>> io = gdb.debug(['bash'],
+        >>> io = gdb.debug(['whoami'],
        ...                 ssh = shell,
        ...                 gdbscript = '''
        ... break main
@@ -503,7 +502,7 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, sysroot=None, api=

        Send a command to Bash

-        >>> io.sendline("echo hello")
+        >>> io.sendline(b"echo hello")

        Interact with the process
        >>> io.interactive() # doctest: +SKIP
@@ -583,7 +582,7 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, sysroot=None, api=
    garbage = gdbserver.recvline(timeout=1)

    # Some versions of gdbserver output an additional message
-    garbage2 = gdbserver.recvline_startswith(b"Remote debugging from host ", timeout=1)
+    garbage2 = gdbserver.recvline_startswith(b"Remote debugging from host ", timeout=2)

    return gdbserver

@@ -773,7 +772,7 @@ def attach(target, gdbscript = '', exe = None, gdb_args = None, ssh = None, sysr
        ... ''')
        >>> io.recvline()
        b'Hello from process debugger!\n'
-        >>> io.sendline('echo Hello from bash && exit')
+        >>> io.sendline(b'echo Hello from bash && exit')
        >>> io.recvall()
        b'Hello from bash\n'

@@ -803,7 +802,7 @@ def attach(target, gdbscript = '', exe = None, gdb_args = None, ssh = None, sysr

            Interact with the program in a regular way

-            >>> io.sendline('echo Hello from bash && exit')
+            >>> io.sendline(b'echo Hello from bash && exit')

            Observe the results

@@ -824,7 +823,7 @@ def attach(target, gdbscript = '', exe = None, gdb_args = None, ssh = None, sysr
        ... ''')
        >>> io.recvline()
        b'Hello from remote debugger!\n'
-        >>> io.sendline('echo Hello from bash && exit')
+        >>> io.sendline(b'echo Hello from bash && exit')
        >>> io.recvall()
        b'Hello from bash\n'

@@ -840,7 +839,7 @@ def attach(target, gdbscript = '', exe = None, gdb_args = None, ssh = None, sysr
        ... ''')
        >>> io.recvline(timeout=5)  # doctest: +SKIP
        b'Hello from ssh debugger!\n'
-        >>> io.sendline('This will be echoed back')
+        >>> io.sendline(b'This will be echoed back')
        >>> io.recvline()
        b'This will be echoed back\n'
        >>> io.close()
@@ -891,7 +890,7 @@ def attach(target, gdbscript = '', exe = None, gdb_args = None, ssh = None, sysr

        pids = list(pidof(target))
        if not pids:
-            log.error('No such process: %s' % target)
+            log.error('No such process: %s', target)
        pid = pids[0]
        log.info('Attaching to youngest process "%s" (PID = %d)' %
                 (target, pid))
@@ -914,12 +913,9 @@ def attach(target, gdbscript = '', exe = None, gdb_args = None, ssh = None, sysr
            cmd = ['sshpass', '-p', shell.password] + cmd
        if shell.keyfile:
            cmd += ['-i', shell.keyfile]
-        exefile = target.executable
-        cmd += ['gdb -q %s %s -x "%s"' % (exefile,
-                                       target.pid,
-                                       tmpfile)]
+        cmd += ['gdb', '-q', target.executable, target.pid, '-x', tmpfile]

-        misc.run_in_new_terminal(' '.join(cmd))
+        misc.run_in_new_terminal(cmd)
        return

    elif isinstance(target, tubes.sock.sock):
@@ -971,7 +967,7 @@ def attach(target, gdbscript = '', exe = None, gdb_args = None, ssh = None, sysr
    elif isinstance(target, elf.corefile.Corefile):
        pre += 'target core %s\n' % target.path
    else:
-        log.error("don't know how to attach to target: %r" % target)
+        log.error("don't know how to attach to target: %r", target)

    # if we have a pid but no exe, just look it up in /proc/
    if pid and not exe:
@@ -984,30 +980,29 @@ def attach(target, gdbscript = '', exe = None, gdb_args = None, ssh = None, sysr
        log.error('could not find target process')

    gdb_binary = binary()
-    cmd = gdb_binary
+    cmd = [gdb_binary]

    if gdb_args:
-        cmd += ' '
-        cmd += ' '.join(gdb_args)
+        cmd += gdb_args

    if context.gdbinit:
-        cmd += ' -nh '                     # ignore ~/.gdbinit
-        cmd += ' -x %s ' % context.gdbinit # load custom gdbinit
+        cmd += ['-nh']                  # ignore ~/.gdbinit
+        cmd += ['-x', context.gdbinit]  # load custom gdbinit

-    cmd += ' -q '
+    cmd += ['-q']

    if exe and context.native:
        if not ssh and not os.path.isfile(exe):
-            log.error('No such file: %s' % exe)
-        cmd += ' "%s"' % exe
+            log.error('No such file: %s', exe)
+        cmd += [exe]

    if pid and not context.os == 'android':
-        cmd += ' %d' % pid
+        cmd += [str(pid)]

    if context.os == 'android' and pid:
        runner  = _get_runner()
        which   = _get_which()
-        gdb_cmd = _gdbserver_args(pid=pid, which=which, env=env)
+        gdb_cmd = _gdbserver_args(pid=pid, which=which)
        gdbserver = runner(gdb_cmd)
        port    = _gdbserver_port(gdbserver, None)
        host    = context.adb_host
@@ -1033,14 +1028,14 @@ def attach(target, gdbscript = '', exe = None, gdb_args = None, ssh = None, sysr
    if gdbscript:
        tmp = tempfile.NamedTemporaryFile(prefix = 'pwn', suffix = '.gdb',
                                          delete = False, mode = 'w+')
-        log.debug('Wrote gdb script to %r\n%s' % (tmp.name, gdbscript))
+        log.debug('Wrote gdb script to %r\n%s', tmp.name, gdbscript)
        gdbscript = 'shell rm %s\n%s' % (tmp.name, gdbscript)

        tmp.write(gdbscript)
        tmp.close()
-        cmd += ' -x %s' % (tmp.name)
+        cmd += ['-x', tmp.name]

-    log.info('running in new terminal: %s' % cmd)
+    log.info('running in new terminal: %s', cmd)

    if api:
        # prevent gdb_faketerminal.py from messing up api doctests
@@ -1057,6 +1052,8 @@ def attach(target, gdbscript = '', exe = None, gdb_args = None, ssh = None, sysr
        return gdb_pid

    # connect to the GDB Python API bridge
+    from rpyc import BgServingThread
+    from rpyc.utils.factory import unix_connect
    if six.PY2:
        retriable = socket.error
    else:
@@ -1215,23 +1212,23 @@ def find_module_addresses(binary, ssh=None, ulimit=False):

    with runner(cmd) as gdb:
        if context.aslr:
-            gdb.sendline('set disable-randomization off')
+            gdb.sendline(b'set disable-randomization off')

-        gdb.send("""
+        gdb.send(b"""\
        set prompt
        catch load
        run
        """)
-        gdb.sendline('info sharedlibrary')
-        lines = context._decode(gdb.recvrepeat(2))
+        gdb.sendline(b'info sharedlibrary')
+        lines = packing._decode(gdb.recvrepeat(2))

        for line in lines.splitlines():
            m = expr.match(line)
            if m:
                libs[m.group(2)] = int(m.group(1),16)
-        gdb.sendline('kill')
-        gdb.sendline('y')
-        gdb.sendline('quit')
+        gdb.sendline(b'kill')
+        gdb.sendline(b'y')
+        gdb.sendline(b'quit')

    #
    # Fix up all of the addresses against the .text address
@@ -1296,11 +1293,11 @@ def corefile(process):
    gdb_args = ['-batch',
                '-q',
                '-nx',
-                '-ex', '"set pagination off"',
-                '-ex', '"set height 0"',
-                '-ex', '"set width 0"',
-                '-ex', '"set use-coredump-filter on"',
-                '-ex', '"generate-core-file %s"' % corefile_path,
+                '-ex', 'set pagination off',
+                '-ex', 'set height 0',
+                '-ex', 'set width 0',
+                '-ex', 'set use-coredump-filter on',
+                '-ex', 'generate-core-file %s' % corefile_path,
                '-ex', 'detach']

    with context.local(terminal = ['sh', '-c']):
@@ -1308,8 +1305,8 @@ def corefile(process):
            pid = attach(process, gdb_args=gdb_args)
            log.debug("Got GDB pid %d", pid)
            try:
-                os.waitpid(pid, 0)
-            except Exception:
+                psutil.Process(pid).wait()
+            except psutil.Error:
                pass

    if not os.path.exists(corefile_path):
@@ -185,7 +185,7 @@ class Progress(object):

    def _log(self, status, args, kwargs, msgtype):
        # Logs are strings, not bytes.  Handle Python3 bytes() objects.
-        status = six.ensure_text(status)
+        status = _need_text(status)

        # this progress logger is stopped, so don't generate any more records
        if self._stopped:
@@ -289,7 +289,7 @@ class Logger(object):

    def _log(self, level, msg, args, kwargs, msgtype, progress = None):
        # Logs are strings, not bytes.  Handle Python3 bytes() objects.
-        msg = six.ensure_text(msg)
+        msg = _need_text(msg)

        extra = kwargs.get('extra', {})
        extra.setdefault('pwnlib_msgtype', msgtype)
@@ -630,6 +630,12 @@ class Formatter(logging.Formatter):
        msg = self.nlindent.join(msg.splitlines())
        return msg

+def _need_text(s):
+    # circular import wrapper
+    global _need_text
+    from pwnlib.util.packing import _need_text
+    return _need_text(s, 2)
+
 # we keep a dictionary of loggers such that multiple calls to `getLogger` with
 # the same name will return the same logger
 def getLogger(name):
@@ -46,7 +46,7 @@ class Message(object):
    def __str__(self):
        return self.__flat__()
    def __flat__(self):
-        return ('%04x' % len(self.string)).encode('ascii') + self.string
+        return b'%04x' % len(self.string) + self.string

 class Connection(remote):
    """Connection to the ADB server"""
@@ -434,7 +434,7 @@ class AdbClient(Logger):
    def _list(self, path):
        if isinstance(path, six.text_type):
            path = path.encode('utf-8')
-        self.c.flat32('LIST', len(path), path)
+        self.c.flat32(b'LIST', len(path), path)
        files = {}
        while True:
            response = self.c.recvn(4)
@@ -488,7 +488,7 @@ class AdbClient(Logger):
        """
        if isinstance(path, six.text_type):
            path = path.encode('utf-8')
-        self.c.flat32('STAT', len(path), path)
+        self.c.flat32(b'STAT', len(path), path)
        if self.c.recvn(4) != b'STAT':
            self.error("An error occurred while attempting to STAT a file.")

@@ -530,14 +530,17 @@ class AdbClient(Logger):
    @_with_transport
    @_sync
    def _write(self, path, data, mode=0o755, timestamp=None, callback=None):
-        path += ',' + str(mode)
-        self.c.flat32('SEND', len(path), path)
+        if isinstance(path, six.text_type):
+            path = path.encode('utf-8')
+        path += b',%d' % mode
+
+        self.c.flat32(b'SEND', len(path), path)

        sent = 0

        # Data needs to be broken up into chunks!
        for chunk in group(0x10000, data):
-            self.c.flat32('DATA', len(chunk), chunk)
+            self.c.flat32(b'DATA', len(chunk), chunk)
            if callback:
                callback(path, data[:sent], len(data), chunk, len(chunk))
            sent += len(chunk)
@@ -545,7 +548,7 @@ class AdbClient(Logger):
        # Send completion notification and timestamp
        if timestamp is None:
            timestamp = int(time.time())
-        self.c.flat32('DONE', timestamp)
+        self.c.flat32(b'DONE', timestamp)

        result = self.c.recvn(4)
        if result != OKAY:
@@ -69,13 +69,13 @@ class AppendedArgument(Unresolved):

        >>> context.clear()
        >>> context.arch = 'amd64'
-        >>> u = AppendedArgument([1,2,'hello',3])
+        >>> u = AppendedArgument([1,2,b'hello',3])
        >>> len(u)
        32
        >>> u.resolve()
        [1, 2, b'hello\x00$$', 3]

-        >>> u = AppendedArgument([1,2,['hello'],3])
+        >>> u = AppendedArgument([1,2,[b'hello'],3])
        >>> u.resolve()
        [1, 2, 32, 3, b'hello\x00$$']
        >>> u.resolve(10000)
@@ -84,7 +84,7 @@ class AppendedArgument(Unresolved):
        >>> u.resolve()
        [1, 2, 20032, 3, b'hello\x00$$']

-        >>> u = AppendedArgument([[[[[[[[['pointers!']]]]]]]]], 1000)
+        >>> u = AppendedArgument([[[[[[[[[b'pointers!']]]]]]]]], 1000)
        >>> u.resolve()
        [1008, 1016, 1024, 1032, 1040, 1048, 1056, 1064, b'pointers!\x00$$$$$$']
    """
@@ -115,6 +115,8 @@ class AppendedArgument(Unresolved):
            if isinstance(v, (list, tuple)):
                self.size += context.bytes
            else:
+                if isinstance(v, six.text_type):
+                    v = packing._need_bytes(v)
                try:
                    self.size += align(context.bytes, len(v))
                except TypeError: # no 'len'
@@ -156,9 +158,9 @@ class AppendedArgument(Unresolved):

        return LocalAddress()

-    def resolve(self, addr = None):
+    def resolve(self, addr=None):
        """
-        Return a flat list of ``int`` or ``str`` objects which can be
+        Return a flat list of ``int`` or ``bytes`` objects which can be
        passed to :func:`.flat`.

        Arguments:
@@ -173,15 +175,15 @@ class AppendedArgument(Unresolved):
            for i, value in enumerate(self.values):
                if isinstance(value, six.integer_types):
                    rv[i] = value
-                if isinstance(value, six.text_type):
-                    value = context._encode(value)
+                elif isinstance(value, six.text_type):
+                    value = packing._need_bytes(value)
                if isinstance(value, (bytes, bytearray)):
                    value += b'\x00'
                    while len(value) % context.bytes:
                        value += b'$'

                    rv[i] = value
-                if isinstance(value, Unresolved):
+                elif isinstance(value, Unresolved):
                    rv[i] = value.address
                    rv.extend(value.resolve())
                assert rv[i] is not None
@@ -212,8 +214,8 @@ class Call(object):

    Example:

-        >>> Call('system', 0xdeadbeef, [1, 2, '/bin/sh'])
-        Call('system', 0xdeadbeef, [1, 2, AppendedArgument(['/bin/sh'], 0x0)])
+        >>> Call('system', 0xdeadbeef, [1, 2, b'/bin/sh'])
+        Call('system', 0xdeadbeef, [1, 2, AppendedArgument([b'/bin/sh'], 0x0)])
    """
    #: Pretty name of the call target, e.g. 'system'
    name = None
@@ -0,0 +1,94 @@
+from __future__ import absolute_import
+from __future__ import division
+
+from capstone import *
+from capstone.x86 import *
+
+from .rop import Padding
+from ..log import getLogger
+from ..util.packing import p64
+
+
+log = getLogger(__name__)
+
+
+def ret2csu(rop, elf, edi, rsi, rdx, rbx, rbp, r12, r13, r14, r15, call=None):
+    """Build a ret2csu ROPchain
+
+    Arguments:
+        edi, rsi, rdx: Three primary registers to populate
+        rbx, rbp, r12, r13, r14, r15: Optional registers to populate
+        call: Pointer to the address of a function to call during
+            second gadget. If None then use the address of _fini in the
+            .dynamic section. .got.plt entries are a good target. Required
+            for PIE binaries.
+    """
+
+    # Prepare capstone
+    md = Cs(CS_ARCH_X86, CS_MODE_64)
+    md.detail = True
+    md.skipdata = True
+
+    # Resolve __libc_csu_ symbols if candidate binary is stripped
+    if '__libc_csu_init' not in elf.symbols:
+        if elf.pie:
+            for insn in md.disasm(elf.section('.text'),
+                                  elf.offset_to_vaddr(elf.get_section_by_name('.text').header['sh_offset'])):
+                if insn.mnemonic == 'lea' and insn.operands[0].reg == X86_REG_R8:
+                    elf.sym['__libc_csu_fini'] = insn.address + insn.size + insn.disp
+                if insn.mnemonic == 'lea' and insn.operands[0].reg == X86_REG_RCX:
+                    elf.sym['__libc_csu_init'] = insn.address + insn.size + insn.disp
+                    break
+        else:
+            for insn in md.disasm(elf.section('.text'), elf.get_section_by_name('.text').header['sh_addr']):
+                if insn.mnemonic == 'mov' and insn.operands[0].reg == X86_REG_R8:
+                    elf.sym['__libc_csu_fini'] = insn.operands[1].imm
+                if insn.mnemonic == 'mov' and insn.operands[0].reg == X86_REG_RCX:
+                    elf.sym['__libc_csu_init'] = insn.operands[1].imm
+                    break
+
+    # Resolve location of _fini address if required
+    if not elf.pie and not call:
+        fini = next(elf.search(p64(elf.dynamic_by_tag('DT_FINI')['d_ptr'])))
+    elif elf.pie and not call:
+        log.error('No non-PIE binaries in [elfs], \'call\' parameter is required')
+
+    csu_function = elf.read(elf.sym['__libc_csu_init'], elf.sym['__libc_csu_fini'] - elf.sym['__libc_csu_init'])
+
+    # 1st gadget: Populate registers in preparation for 2nd gadget
+    for insn in md.disasm(csu_function, elf.sym['__libc_csu_init']):
+        if insn.mnemonic == 'pop' and insn.operands[0].reg == X86_REG_RBX:
+            rop.raw(insn.address)
+            break
+    # rbx and rbp must be equal after 'add rbx, 1'
+    rop.raw(0x00)  # pop rbx
+    rop.raw(0x01)  # pop rbp
+    if call:
+        rop.raw(call)  # pop r12
+    else:
+        rop.raw(fini)  # pop r12
+
+    # Older versions of gcc use r13 to populate rdx then r15d to populate edi, newer versions use the reverse
+    # Account for this when the binary was linked against a glibc that was built with a newer gcc
+    for insn in md.disasm(csu_function, elf.sym['__libc_csu_init']):
+        if insn.mnemonic == 'mov' and insn.operands[0].reg == X86_REG_RDX and insn.operands[1].reg == X86_REG_R13:
+            rop.raw(rdx)  # pop r13
+            rop.raw(rsi)  # pop r14
+            rop.raw(edi)  # pop r15
+            rop.raw(insn.address)
+            break
+        elif insn.mnemonic == 'mov' and insn.operands[0].reg == X86_REG_RDX and insn.operands[1].reg == X86_REG_R15:
+            rop.raw(edi)  # pop r13
+            rop.raw(rsi)  # pop r14
+            rop.raw(rdx)  # pop r15
+            rop.raw(insn.address)
+            break
+
+    # 2nd gadget: Populate edi, rsi & rdx. Populate optional registers
+    rop.raw(Padding('<add rsp, 8>'))  # add rsp, 8
+    rop.raw(rbx)  # pop rbx
+    rop.raw(rbp)  # pop rbp
+    rop.raw(r12)  # pop r12
+    rop.raw(r13)  # pop r13
+    rop.raw(r14)  # pop r14
+    rop.raw(r15)  # pop r15
@@ -71,6 +71,7 @@ from copy import deepcopy
 from pwnlib.context import context
 from pwnlib.log import getLogger
 from pwnlib.util.packing import *
+from pwnlib.util.packing import _encode
 from pwnlib.util.misc import align

 log = getLogger(__name__)
@@ -229,7 +230,7 @@ class Ret2dlresolvePayload(object):
        self.symtab = elf.dynamic_value_by_tag("DT_SYMTAB") + self.elf_load_address_fixup
        self.jmprel = elf.dynamic_value_by_tag("DT_JMPREL") + self.elf_load_address_fixup
        self.versym = elf.dynamic_value_by_tag("DT_VERSYM") + self.elf_load_address_fixup
-        self.symbol = context._encode(symbol)
+        self.symbol = _encode(symbol)
        self.args = args
        self.real_args = self._format_args()
        self.unreliable = False
@@ -251,7 +252,7 @@ class Ret2dlresolvePayload(object):
        def aux(args):
            for i, arg in enumerate(args):
                if isinstance(arg, (str,bytes)):
-                    args[i] = context._encode(args[i]) + b"\x00"
+                    args[i] = _encode(args[i]) + b"\x00"
                elif isinstance(arg, (list, tuple)):
                    aux(arg)

@@ -71,9 +71,9 @@ The stack is automatically adjusted for the next frame
 You can also append complex arguments onto stack when the stack pointer is known.

    >>> rop = ROP(binary, base=0x7fffe000)
-    >>> rop.call('execve', ['/bin/sh', [['/bin/sh'], ['-p'], ['-c'], ['ls']], 0])
+    >>> rop.call('execve', [b'/bin/sh', [[b'/bin/sh'], [b'-p'], [b'-c'], [b'ls']], 0])
    >>> print(rop.dump())
-    0x7fffe000:       0xcafebabe execve(['/bin/sh'], [['/bin/sh'], ['-p'], ['-c'], ['ls']], 0)
+    0x7fffe000:       0xcafebabe execve([b'/bin/sh'], [[b'/bin/sh'], [b'-p'], [b'-c'], [b'ls']], 0)
    0x7fffe004:          b'baaa' <return address>
    0x7fffe008:       0x7fffe014 arg0 (+0xc)
    0x7fffe00c:       0x7fffe01c arg1 (+0x10)
@@ -310,6 +310,7 @@ import re
 import shutil
 import six
 import string
+import struct
 import sys
 import tempfile

@@ -652,12 +653,12 @@ class ROP(object):
        stack = []

        for gadget in winner:
-            moved = 8 # Account for the gadget itself
+            moved = context.bytes # Account for the gadget itself
            goodregs = set(gadget.regs) & regset
            name = ",".join(goodregs)
            stack.append((gadget.address, gadget))
            for r in gadget.regs:
-                moved += 8
+                moved += context.bytes
                if r in registers:
                    stack.append((registers[r], r))
                else:
@@ -1105,7 +1106,7 @@ class ROP(object):
        sure that any given string is aligned!

        Arguments:
-            data(int/str): The raw value to put onto the rop chain.
+            data(int/bytes): The raw value to put onto the rop chain.

        >>> context.clear(arch='i386')
        >>> rop = ROP([])
@@ -1146,6 +1147,12 @@ class ROP(object):
        """Returns: Raw bytes of the ROP chain"""
        return self.chain()

+    def __flat__(self):
+        return self.chain()
+
+    def __flat_at__(self, address):
+        return self.chain(address)
+
    def __get_cachefile_name(self, files):
        """Given an ELF or list of ELF objects, return a cache file for the set of files"""
        cachedir = os.path.join(context.cache_dir, 'rop-cache')
@@ -1194,7 +1201,7 @@ class ROP(object):
        # https://github.com/JonathanSalwan/ROPgadget/issues/53
        #

-        pop   = re.compile(r'^pop (.{3})')
+        pop   = re.compile(r'^pop (.{2,3})')
        add   = re.compile(r'^add [er]sp, ((?:0[xX])?[0-9a-fA-F]+)$')
        ret   = re.compile(r'^ret$')
        leave = re.compile(r'^leave$')
@@ -1379,6 +1386,84 @@ class ROP(object):

        return result

+    def ret2csu(self, edi=Padding('edi'), rsi=Padding('rsi'),
+                rdx=Padding('rdx'), rbx=Padding('rbx'), rbp=Padding('rbp'),
+                r12=Padding('r12'), r13=Padding('r13'), r14=Padding('r14'),
+                r15=Padding('r15'), call=None):
+        """Build a ret2csu ROPchain
+
+        Arguments:
+            edi, rsi, rdx: Three primary registers to populate
+            rbx, rbp, r12, r13, r14, r15: Optional registers to populate
+            call: Pointer to the address of a function to call during
+                second gadget. If None then use the address of _fini in the
+                .dynamic section. .got.plt entries are a good target. Required
+                for PIE binaries.
+        Test:
+            >>> context.clear(binary=pwnlib.data.elf.ret2dlresolve.get("amd64"))
+            >>> r = ROP(context.binary)
+            >>> r.ret2csu(1, 2, 3, 4, 5, 6, 7, 8, 9)
+            >>> r.call(0xdeadbeef)
+            >>> print(r.dump())
+            0x0000:         0x40058a
+            0x0008:              0x0
+            0x0010:              0x1
+            0x0018:         0x600e48
+            0x0020:              0x1
+            0x0028:              0x2
+            0x0030:              0x3
+            0x0038:         0x400570
+            0x0040:      b'qaaaraaa' <add rsp, 8>
+            0x0048:              0x4
+            0x0050:              0x5
+            0x0058:              0x6
+            0x0060:              0x7
+            0x0068:              0x8
+            0x0070:              0x9
+            0x0078:       0xdeadbeef 0xdeadbeef()
+            >>> open('core','w').close(); os.unlink('core')  # remove any old core file for the tests
+            >>> p = process()
+            >>> p.send(fit({64+context.bytes: r}))
+            >>> p.wait(0.5)
+            >>> core = p.corefile
+            >>> hex(core.pc)
+            '0xdeadbeef'
+            >>> core.rdi, core.rsi, core.rdx, core.rbx, core.rbp, core.r12, core.r13, core.r14, core.r15
+            (1, 2, 3, 4, 5, 6, 7, 8, 9)
+        """
+        if self.migrated:
+            log.error('Cannot append to a migrated chain')
+
+        # Ensure 'edi' argument is packable
+        try:
+            packing.p32(edi)
+        except struct.error:
+            log.error('edi must be a 32bit value')
+
+        # Find an appropriate, non-library ELF.
+        # Prioritise non-PIE binaries so we can use _fini
+        exes = (elf for elf in self.elfs if not elf.library and elf.bits == 64)
+        if not exes:
+            log.error('No non-library binaries in [elfs]')
+
+        nonpie = csu = None
+        for elf in exes:
+            if not elf.pie:
+                if '__libc_csu_init' in elf.symbols:
+                    break
+                nonpie = elf
+            elif '__libc_csu_init' in elf.symbols:
+                csu = elf
+
+        if elf.pie:
+            if nonpie:
+                elf = nonpie
+            elif csu:
+                elf = csu
+
+        from .ret2csu import ret2csu
+        ret2csu(self, elf, edi, rsi, rdx, rbx, rbp, r12, r13, r14, r15, call)
+
    def ret2dlresolve(self, dlresolve):
        elf = next(elf for elf in self.elfs if elf.get_section_by_name(".plt"))
        elf_base = elf.address if elf.pie else 0
@@ -1449,11 +1534,10 @@ class ROP(object):

                    return Gadget(addr, [jmp_sp], [], context.bytes)
            return None
-
-        if attr in ('int80', 'syscall', 'sysenter'):
-            mapping = {'int80': 'int 0x80',
-             'syscall': 'syscall',
-             'sysenter': 'sysenter'}
+        mapping = {'int80': 'int 0x80',
+            'syscall': 'syscall',
+            'sysenter': 'sysenter'}
+        if attr in mapping:
            for each in self.gadgets:
                if self.gadgets[each]['insns'][0] == mapping[attr]:
                    return gadget(each, self.gadgets[each])
@@ -0,0 +1 @@
+../../common/linux/kill.asm
@@ -0,0 +1,16 @@
+<%
+    from pwnlib.shellcraft.aarch64.linux import getppid, kill
+    from pwnlib.shellcraft.common import label
+%>
+<%docstring>
+Kills its parent process until whatever the parent is (probably init)
+cannot be killed any longer.
+</%docstring>
+<%
+    killparent_loop = label('killparent')
+%>
+${killparent_loop}:
+    ${getppid()}
+    ${kill('x0')}
+    cmp x0,x0
+    beq ${killparent_loop}
@@ -1,8 +1,9 @@
 <%
    from pwnlib import shellcraft
-    from pwnlib.context import context as ctx
+    from pwnlib.shellcraft import pretty
    from pwnlib.util.iters import group
-    from six import text_type, binary_type, ensure_binary
+    from pwnlib.util.packing import _encode
+    from six import text_type, binary_type
 %>
 <%docstring>
 Pushes an array/envp-style array of pointers onto the stack.
@@ -26,7 +27,7 @@ if isinstance(array, (binary_type, text_type)):
    array = [array]

 # Convert all items to strings
-array = [ensure_binary(x) for x in array]
+array = [_encode(x) for x in array]

 # Normalize line endings for each item
 array = [arg.rstrip(b'\x00') + b'\x00' for arg in array]
@@ -50,7 +51,7 @@ if len(array) * 8 > 4095:
 %for i, value in enumerate(reversed(array)):
   ${shellcraft.mov(register1, (i+1)*8 + string.index(value))}
   add ${register1}, sp, ${register1}
-   str ${register1}, [sp, #-8]! /* ${array[-i]} */
+   str ${register1}, [sp, #-8]! /* ${pretty(array[-i], False)} */
 %endfor

    /* set ${reg} to the current top of the stack */
@@ -0,0 +1,11 @@
+<% from pwnlib.shellcraft import common %>
+<%docstring>Returns code to switch from amd64 to i386 mode.
+
+Note that you most surely want to set up some stack (and place this code)
+in low address space before (or afterwards).</%docstring>
+.code64
+    call $+4
+    .byte 0xc0  /* inc eax */
+    mov byte ptr [rsp+4], 0x23  /* This is the segment we want to go to */
+    retfd
+.code32
@@ -1,5 +0,0 @@
-<% from pwnlib.shellcraft import amd64 %>
-<%page args="pid, signal = 'SIGKILL'"/>
-<%docstring>Writes a string to a file descriptor</%docstring>
-
-    ${amd64.linux.syscall('SYS_kill', pid, signal)}
@@ -0,0 +1 @@
+../../common/linux/kill.asm
@@ -1,6 +1,5 @@
 <%
    from pwnlib.shellcraft.amd64.linux import getppid, kill
-    from pwnlib.constants import SIGKILL
    from pwnlib.shellcraft.common import label
 %>
 <%docstring>
@@ -12,6 +11,6 @@ cannot be killed any longer.
 %>
 ${killparent_loop}:
    ${getppid()}
-    ${kill('eax', SIGKILL)}
+    ${kill('eax')}
    test eax, eax
    jz ${killparent_loop}
@@ -0,0 +1 @@
+../../common/linux/kill.asm
@@ -1,6 +1,5 @@
 <%
    from pwnlib.shellcraft.arm.linux import getppid, kill
-    from pwnlib.constants import SIGKILL
    from pwnlib.shellcraft.common import label
 %>
 <%docstring>
@@ -12,6 +11,6 @@ cannot be killed any longer.
 %>
 ${killparent_loop}:
    ${getppid()}
-    ${kill('r0', SIGKILL)}
+    ${kill('r0')}
    cmp r0, r0
    beq ${killparent_loop}
@@ -0,0 +1,16 @@
+<% from pwnlib import shellcraft %>
+<%page args="pid, signal = 'SIGKILL'"/>
+<%docstring>kill(pid, sig) -> str
+
+Invokes the syscall kill.
+
+See 'man 2 kill' for more information.
+
+Arguments:
+    pid(pid_t): pid
+    sig(int): sig
+Returns:
+    int
+</%docstring>
+
+    ${shellcraft.linux.syscall('SYS_kill', pid, signal)}
@@ -1,12 +1,8 @@
 <% from pwnlib.shellcraft import common %>
 <%docstring>Returns code to switch from i386 to amd64 mode.</%docstring>
-<% helper, end = common.label("helper"), common.label("end") %>
 .code32
    push 0x33 /*  This is the segment we want to go to */
+    /* "db 0xff; sub al,0x24" is "jmp far [esp]" by chance */
    call $+4
-${helper}:
-    .byte 0xc0
-    add dword ptr [esp], ${end} - ${helper}
-    jmp far [esp]
-${end}:
+    sub al, 0x24
 .code64
@@ -0,0 +1 @@
+../../common/linux/kill.asm
@@ -1,6 +1,5 @@
 <%
    from pwnlib.shellcraft.i386.linux import getppid, kill
-    from pwnlib.constants import SIGKILL
    from pwnlib.shellcraft.common import label
 %>
 <%docstring>
@@ -12,6 +11,6 @@ cannot be killed any longer.
 %>
 ${killparent_loop}:
    ${getppid()}
-    ${kill('eax', SIGKILL)}
+    ${kill('eax')}
    test eax, eax
    jz ${killparent_loop}
@@ -63,7 +63,10 @@ Args:
 </%docstring>
 <%
 original = string
-string   = packing.flat(string)
+if isinstance(string, six.text_type):
+    string   = packing._encode(string)
+else:
+    string = packing.flat(string)

 if append_null:
    string += b'\x00'
@@ -13,8 +13,8 @@ Args: [sock (imm/reg) = s0]

    /* dup() file descriptor ${sock} into stdin/stdout/stderr */
 ${dup}:
-    ${mov('$t0',2)}
+    ${mov('$v0',2)}
 ${looplabel}:
-    ${dup2(sock,'$t0')}
-    bgtz $t0,${looplabel}
-    addi $t0,-1
+    ${dup2(sock,'$v0')}
+    bgtz $v0, ${looplabel}
+    addi $v0, -1
@@ -0,0 +1 @@
+../../common/linux/kill.asm
@@ -1,6 +1,5 @@
 <%
    from pwnlib.shellcraft.mips.linux import getppid, kill
-    from pwnlib.constants import SIGKILL
    from pwnlib.shellcraft.common import label
 %>
 <%docstring>
@@ -12,5 +11,5 @@ cannot be killed any longer.
 %>
 ${killparent_loop}:
    ${getppid()}
-    ${kill('$v0', SIGKILL)}
+    ${kill('$v0')}
    beq $v0, $zero, ${killparent_loop}
@@ -0,0 +1 @@
+../../common/linux/kill.asm
@@ -1,6 +1,5 @@
 <%
    from pwnlib.shellcraft.thumb.linux import getppid, kill
-    from pwnlib.constants import SIGKILL
    from pwnlib.shellcraft.common import label
 %>
 <%docstring>
@@ -12,6 +11,6 @@ cannot be killed any longer.
 %>
 ${killparent_loop}:
    ${getppid()}
-    ${kill('eax', SIGKILL)}
-    test eax, eax
-    jz ${killparent_loop}
+    ${kill('r0')}
+    cmp r0,r0
+    beq ${killparent_loop}
@@ -19,6 +19,7 @@ on your version of binutils.

    >>> enhex(asm(shellcraft.pushstr('Hello\nWorld!', True))) in [
    ... '87ea070780b4dff8047001e0726c642180b4dff8047001e06f0a576f80b4dff8047001e048656c6c80b4',
+    ... '87ea070780b4dff8047001e0726c642180b4dff8047001e06f0a576f80b4dff8047001e048656c6c80b400bf',
    ... '87ea070780b4dff8067000f002b8726c642180b4dff8047000f002b86f0a576f80b4014f00f002b848656c6c80b4']
    True
    >>> print(shellcraft.pushstr('abc').rstrip()) #doctest: +ELLIPSIS
@@ -30,7 +31,7 @@ on your version of binutils.
        lsl r7, #8
        lsr r7, #8
        push {r7}
-    >>> print(enhex(asm(shellcraft.pushstr('\x00', False))))
+    >>> print(enhex(asm(shellcraft.pushstr('\x00', False)).rstrip(b'\x00\xbf')))
    87ea070780b4

 </%docstring>
@@ -201,6 +201,11 @@ class process(tube):

        >>> binary = ELF.from_assembly('nop', arch='mips')
        >>> p = process(binary.path)
+        >>> binary_dir, binary_name = os.path.split(binary.path)
+        >>> p = process('./{}'.format(binary_name), cwd=binary_dir)
+        >>> p = process(binary.path, cwd=binary_dir)
+        >>> p = process('./{}'.format(binary_name), cwd=os.path.relpath(binary_dir))
+        >>> p = process(binary.path, cwd=os.path.relpath(binary_dir))
    """

    STDOUT = STDOUT
@@ -357,16 +362,14 @@ class process(tube):
            fcntl.fcntl(fd, fcntl.F_SETFL, fl | os.O_NONBLOCK)

        # Save off information about whether the binary is setuid / setgid
-        self.uid = os.getuid()
-        self.gid = os.getgid()
-        self.suid = -1
-        self.sgid = -1
+        self.suid = self.uid = os.getuid()
+        self.sgid = self.gid = os.getgid()
        st = os.stat(self.executable)
        if self._setuid:
            if (st.st_mode & stat.S_ISUID):
-                self.setuid = st.st_uid
+                self.suid = st.st_uid
            if (st.st_mode & stat.S_ISGID):
-                self.setgid = st.st_gid
+                self.sgid = st.st_gid

    def __preexec_fn(self):
        """
@@ -507,6 +510,7 @@ class process(tube):
        Mostly to make Python happy, but also to prevent common pitfalls.
        """

+        orig_cwd = cwd
        cwd = cwd or os.path.curdir

        #
@@ -571,6 +575,12 @@ class process(tube):
            executable = os.path.join(cwd, executable)
            self.warn_once("Could not find executable %r in $PATH, using %r instead" % (tmp, executable))

+        # There is a path component and user specified a working directory,
+        # it must be relative to that directory. For example, 'bar/baz' with
+        # cwd='foo' or './baz' with cwd='foo/bar'
+        elif orig_cwd:
+            executable = os.path.join(orig_cwd, executable)
+
        if not os.path.exists(executable):
            self.error("%r does not exist"  % executable)
        if not os.path.isfile(executable):
@@ -996,6 +1006,52 @@ class process(tube):
            mem.seek(address)
            return mem.read(count) or None

+    readmem = leak
+
+    def writemem(self, address, data):
+        r"""Writes memory within the process at the specified address.
+
+        Arguments:
+            address(int): Address to write memory
+            data(bytes): Data to write to the address
+
+        Example:
+        
+            Let's write data to  the beginning of the mapped memory of the  ELF.
+
+            >>> context.clear(arch='i386')
+            >>> address = 0x100000
+            >>> data = cyclic(32)
+            >>> assembly = shellcraft.nop() * len(data)
+
+            Wait for one byte of input, then write the data to stdout
+
+            >>> assembly += shellcraft.write(1, address, 1)
+            >>> assembly += shellcraft.read(0, 'esp', 1)
+            >>> assembly += shellcraft.write(1, address, 32)
+            >>> assembly += shellcraft.exit()
+            >>> asm(assembly)[32:]
+            b'j\x01[\xb9\xff\xff\xef\xff\xf7\xd1\x89\xdaj\x04X\xcd\x801\xdb\x89\xe1j\x01Zj\x03X\xcd\x80j\x01[\xb9\xff\xff\xef\xff\xf7\xd1j Zj\x04X\xcd\x801\xdbj\x01X\xcd\x80'
+
+            Assemble the binary and test it
+
+            >>> elf = ELF.from_assembly(assembly, vma=address)
+            >>> io = elf.process()
+            >>> _ = io.recvuntil(b'\x90')
+            >>> _ = io.writemem(address, data)
+            >>> io.send(b'X')
+            >>> io.recvall()
+            b'aaaabaaacaaadaaaeaaafaaagaaahaaa'
+        """
+
+        if 'qemu-' in os.path.realpath('/proc/%i/exe' % self.pid):
+            self.error("Cannot use leaker on binaries under QEMU.")
+
+        with open('/proc/%i/mem' % self.pid, 'wb') as mem:
+            mem.seek(address)
+            return mem.write(data)
+
+
    @property
    def stdin(self):
        """Shorthand for ``self.proc.stdin``
@@ -79,7 +79,7 @@ class remote(sock):
            except socket.gaierror as e:
                if e.errno != socket.EAI_NONAME:
                    raise
-                self.error('Could not resolve hostname: %r' % host)
+                self.error('Could not resolve hostname: %r', host)
        if self.sock:
            self.settimeout(self.timeout)
            self.lhost, self.lport = self.sock.getsockname()[:2]
@@ -104,7 +104,7 @@ class remote(sock):
                if self.type not in [socket.SOCK_STREAM, socket.SOCK_DGRAM]:
                    continue

-                h.status("Trying %s" % sockaddr[0])
+                h.status("Trying %s", sockaddr[0])

                sock = socket.socket(self.family, self.type, self.proto)

@@ -121,7 +121,7 @@ class remote(sock):
                    raise
                except socket.error:
                    pass
-            self.error("Could not connect to %s on port %d" % (self.rhost, self.rport))
+            self.error("Could not connect to %s on port %d", self.rhost, self.rport)

    @classmethod
    def fromsocket(cls, socket):
@@ -41,15 +41,13 @@ class sock(tube):
            except socket.timeout:
                return None
            except IOError as e:
-                if e.errno == errno.EAGAIN:
+                if e.errno in (errno.EAGAIN, errno.ETIMEDOUT) or 'timed out' in e.strerror:
                    return None
                elif e.errno in (errno.ECONNREFUSED, errno.ECONNRESET):
                    self.shutdown("recv")
                    raise EOFError
                elif e.errno == errno.EINTR:
                    continue
-                elif 'timed out' in e.message:
-                    return None
                else:
                    raise

@@ -175,7 +173,7 @@ class sock(tube):
        self._close_msg()

    def _close_msg(self):
-        self.info('Closed connection to %s port %d' % (self.rhost, self.rport))
+        self.info('Closed connection to %s port %d', self.rhost, self.rport)

    def fileno(self):
        if not self.sock:
@@ -24,6 +24,7 @@ from pwnlib.timeout import Timeout
 from pwnlib.tubes.sock import sock
 from pwnlib.util import hashes
 from pwnlib.util import misc
+from pwnlib.util import packing
 from pwnlib.util import safeeval
 from pwnlib.util.sh_string import sh_string

@@ -69,25 +70,25 @@ class ssh_channel(sock):
        self.process = process
        self.cwd  = wd or '.'
        if isinstance(wd, six.text_type):
-            wd = context._encode(wd)
+            wd = packing._encode(wd)

        env = env or {}
        msg = 'Opening new channel: %r' % (process or 'shell')

        if isinstance(process, (list, tuple)):
-            process = b' '.join(context._encode(sh_string(s)) for s in process)
+            process = b' '.join(packing._encode(sh_string(s)) for s in process)
        if isinstance(process, six.text_type):
-            process = context._encode(process)
+            process = packing._encode(process)

        if process and wd:
            process = b'cd ' + sh_string(wd) + b' >/dev/null 2>&1; ' + process

        if process and env:
            for name, value in env.items():
-                nameb = context._encode(name)
+                nameb = packing._encode(name)
                if not re.match(b'^[a-zA-Z_][a-zA-Z0-9_]*$', nameb):
                    self.error('run(): Invalid environment key %r' % name)
-                export = b'export %s=%s;' % (nameb, sh_string(context._encode(value)))
+                export = b'export %s=%s;' % (nameb, sh_string(packing._encode(value)))
                process = export + process

        if process and tty:
@@ -389,7 +390,7 @@ class ssh_process(ssh_channel):
        """
        argv0 = self.argv[0]

-        variable = context._encode(variable)
+        variable = packing._encode(variable)

        script = ';'.join(('from ctypes import *',
                           'import os',
@@ -552,7 +553,7 @@ class ssh(Timeout, Logger):

    def __init__(self, user=None, host=None, port=22, password=None, key=None,
                 keyfile=None, proxy_command=None, proxy_sock=None,
-                 level=None, cache=True, ssh_agent=False, *a, **kw):
+                 level=None, cache=True, ssh_agent=False, ignore_config=False, *a, **kw):
        """Creates a new ssh connection.

        Arguments:
@@ -568,6 +569,7 @@ class ssh(Timeout, Logger):
            level: Log level
            cache: Cache downloaded files (by hash/size/timestamp)
            ssh_agent: If :const:`True`, enable usage of keys via ssh-agent
+            ignore_config: If :const:`True`, disable usage of ~/.ssh/config and ~/.ssh/authorized_keys

        NOTE: The proxy_command and proxy_sock arguments is only available if a
        fairly new version of paramiko is used.
@@ -606,19 +608,13 @@ class ssh(Timeout, Logger):

        misc.mkdir_p(self._cachedir)

-        # This is a dirty hack to make my Yubikey shut up.
-        # If anybody has a problem with this, please open a bug and I'll
-        # figure out a better workaround.
-        if not ssh_agent:
-            os.environ.pop('SSH_AUTH_SOCK', None)
-
        import paramiko

        # Make a basic attempt to parse the ssh_config file
        try:
            config_file = os.path.expanduser('~/.ssh/config')

-            if os.path.exists(config_file):
+            if not ignore_config and os.path.exists(config_file):
                ssh_config  = paramiko.SSHConfig()
                ssh_config.parse(open(config_file))
                host_config = ssh_config.lookup(host)
@@ -640,9 +636,10 @@ class ssh(Timeout, Logger):
            self.client = paramiko.SSHClient()
            self.client.set_missing_host_key_policy(paramiko.AutoAddPolicy())

-            known_hosts = os.path.expanduser('~/.ssh/known_hosts')
-            if os.path.exists(known_hosts):
-                self.client.load_host_keys(known_hosts)
+            if not ignore_config:
+                known_hosts = os.path.expanduser('~/.ssh/known_hosts')
+                if os.path.exists(known_hosts):
+                    self.client.load_host_keys(known_hosts)

            has_proxy = bool(proxy_sock or proxy_command)
            if has_proxy:
@@ -658,7 +655,7 @@ class ssh(Timeout, Logger):
                proxy_sock = None

            try:
-                self.client.connect(host, port, user, password, key, keyfiles, self.timeout, compress = True, sock = proxy_sock)
+                self.client.connect(host, port, user, password, key, keyfiles, self.timeout, allow_agent=ssh_agent, compress=True, sock=proxy_sock, look_for_keys=not ignore_config)
            except paramiko.BadHostKeyException as e:
                self.error("Remote host %(host)s is using a different key than stated in known_hosts\n"
                           "    To remove the existing entry from your known_hosts and trust the new key, run the following commands:\n"
@@ -674,7 +671,7 @@ class ssh(Timeout, Logger):

        if self.sftp:
            with context.quiet:
-                self.cwd = context._decode(self.pwd())
+                self.cwd = packing._decode(self.pwd())
        else:
            self.cwd = '.'

@@ -863,7 +860,7 @@ class ssh(Timeout, Logger):
            >>> print(s.process('false', preexec_fn=uses_globals).recvall().strip().decode()) # doctest: +ELLIPSIS
            Traceback (most recent call last):
            ...
-            NameError: ... name 'bar' is not defined
+            NameError: ...name 'bar' is not defined

            >>> s.process('echo hello', shell=True).recvall()
            b'hello\n'
@@ -931,7 +928,7 @@ class ssh(Timeout, Logger):
        # Validate, since failures on the remote side will suck.
        if not isinstance(executable, (six.text_type, six.binary_type, bytearray)):
            self.error("executable / argv[0] must be a string: %r" % executable)
-        executable = context._decode(executable)
+        executable = bytearray(packing._encode(executable))

        # Allow passing in sys.stdin/stdout/stderr objects
        handles = {sys.stdin: 0, sys.stdout:1, sys.stderr:2}
@@ -953,7 +950,7 @@ class ssh(Timeout, Logger):

        func_src  = inspect.getsource(func).strip()
        setuid = True if setuid is None else bool(setuid)
-        
+
        script = r"""
 #!/usr/bin/env python
 import os, sys, ctypes, resource, platform, stat
@@ -962,25 +959,27 @@ try:
    integer_types = int, long
 except NameError:
    integer_types = int,
-exe   = %(executable)r
+exe   = bytes(%(executable)r)
 argv  = [bytes(a) for a in %(argv)r]
 env   = %(env)r

 os.chdir(%(cwd)r)

+environ = getattr(os, 'environb', os.environ)
+
 if env is not None:
    env = OrderedDict((bytes(k), bytes(v)) for k,v in env)
    os.environ.clear()
-    getattr(os, 'environb', os.environ).update(env)
+    environ.update(env)
 else:
    env = os.environ

 def is_exe(path):
    return os.path.isfile(path) and os.access(path, os.X_OK)

-PATH = os.environ.get('PATH','').split(os.pathsep)
+PATH = environ.get(b'PATH',b'').split(os.pathsep.encode())

-if os.path.sep not in exe and not is_exe(exe):
+if os.path.sep.encode() not in exe and not is_exe(exe):
    for path in PATH:
        test_path = os.path.join(path, exe)
        if is_exe(test_path):
@@ -989,7 +988,7 @@ if os.path.sep not in exe and not is_exe(exe):

 if not is_exe(exe):
    sys.stderr.write('3\n')
-    sys.stderr.write("{} is not executable or does not exist in $PATH: {}".format(exe,PATH))
+    sys.stderr.write("{!r} is not executable or does not exist in $PATH: {!r}".format(exe,PATH))
    sys.exit(-1)

 if not %(setuid)r:
@@ -1026,7 +1025,7 @@ if sys.argv[-1] == 'check':
    sys.stdout.write(str(os.getgid()) + "\n")
    sys.stdout.write(str(suid) + "\n")
    sys.stdout.write(str(sgid) + "\n")
-    sys.stdout.write(os.path.realpath(exe) + '\x00')
+    getattr(sys.stdout, 'buffer', sys.stdout).write(os.path.realpath(exe) + b'\x00')
    sys.stdout.flush()

 for fd, newfd in {0: %(stdin)r, 1: %(stdout)r, 2:%(stderr)r}.items():
@@ -1097,16 +1096,20 @@ os.execve(exe, argv, env)

        with self.progress(msg) as h:

-            script = 'for py in python2.7 python2 python; do test -x "$(which $py 2>&1)" && exec $py -c %s check; done; echo 2' % sh_string(script)
+            script = 'echo PWNTOOLS; for py in python3 python2.7 python2 python; do test -x "$(which $py 2>&1)" && echo $py && exec $py -c %s check; done; echo 2' % sh_string(script)
            with context.quiet:
                python = ssh_process(self, script, tty=True, raw=True, level=self.level, timeout=timeout)

            try:
-                result = safeeval.const(python.recvline())
+                python.recvline_contains(b'PWNTOOLS')        # Magic flag so that any sh/bash initialization errors are swallowed
+                python.recvline()                           # Python interpreter that was selected
+                result = safeeval.const(python.recvline())  # Status flag from the Python script
            except (EOFError, ValueError):
                h.failure("Process creation failed")
-                self.warn_once('Could not find a Python interpreter on %s\n' % self.host \
-                               + "Use ssh.run() instead of ssh.process()")
+                self.warn_once('Could not find a Python interpreter on %s\n' % self.host
+                               + "Use ssh.run() instead of ssh.process()\n"
+                                 "The original error message:\n"
+                               + python.recvall().decode())
                return None

            # If an error occurred, try to grab as much output
@@ -1129,7 +1132,7 @@ os.execve(exe, argv, env)
            python.suid = safeeval.const(python.recvline())
            python.sgid = safeeval.const(python.recvline())
            python.argv = argv
-            python.executable = context._decode(python.recvuntil(b'\x00')[:-1])
+            python.executable = packing._decode(python.recvuntil(b'\x00')[:-1])

            h.success('pid %i' % python.pid)

@@ -1346,7 +1349,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))
            if len(args) == 1 and isinstance(args[0], (list, tuple)):
                command = [attr] + args[0]
            else:
-                command = ' '.join((attr,) + tuple(map(six.ensure_str, args)))
+                command = b' '.join((packing._encode(attr),) + tuple(map(packing._encode, args)))

            return self.run(command).recvall().strip()
        return runner
@@ -1390,7 +1393,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))
            self.error('Unable to find libraries for %r' % remote)
            return {}

-        return misc.parse_ldd_output(context._decode(data))
+        return misc.parse_ldd_output(packing._decode(data))

    def _get_fingerprint(self, remote):
        cmd = '(sha256 || sha256sum || openssl sha256) 2>/dev/null < '
@@ -1611,7 +1614,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))
            >>> print(open('/tmp/upload_bar').read())
            Hello, world
        """
-        data = context._encode(data)
+        data = packing._encode(data)
        # If a relative path was provided, prepend the cwd
        if os.path.normpath(remote) == os.path.basename(remote):
            remote = os.path.join(self.cwd, remote)
@@ -1759,7 +1762,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))

        libs = self._libs_remote(remote)

-        remote = context._decode(self.readlink('-f',remote).strip())
+        remote = packing._decode(self.readlink('-f',remote).strip())
        libs[remote] = 0

        if directory is None:
@@ -1834,7 +1837,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))
            >>> cwd = s.set_working_directory()
            >>> s.ls()
            b''
-            >>> context._decode(s.pwd()) == cwd
+            >>> packing._decode(s.pwd()) == cwd
            True

            >>> s =  ssh(host='example.pwnme')
@@ -48,13 +48,13 @@ class tube(Timeout, Logger):
        r'''Character sent with methods like sendline() or used for recvline().

            >>> t = tube()
-            >>> t.newline = 'X'
-            >>> t.unrecv('A\nB\nCX')
+            >>> t.newline = b'X'
+            >>> t.unrecv(b'A\nB\nCX')
            >>> t.recvline()
            b'A\nB\nCX'

            >>> t = tube()
-            >>> context.newline = '\r\n'
+            >>> context.newline = b'\r\n'
            >>> t.newline
            b'\r\n'

@@ -67,7 +67,7 @@ class tube(Timeout, Logger):

    @newline.setter
    def newline(self, newline):
-        self._newline = six.ensure_binary(newline)
+        self._newline = packing._need_bytes(newline)

    # Functions based on functions from subclasses
    def recv(self, numb = None, timeout = default):
@@ -124,7 +124,7 @@ class tube(Timeout, Logger):
            >>> t.recv()
            b'hello'
        """
-        data = context._encode(data)
+        data = packing._need_bytes(data)
        self.buffer.unget(data)

    def _fillbuffer(self, timeout = default):
@@ -316,9 +316,9 @@ class tube(Timeout, Logger):

        """
        # Convert string into singleton tupple
-        if isinstance(delims, (bytes, six.text_type)):
+        if isinstance(delims, (bytes, bytearray, six.text_type)):
            delims = (delims,)
-        delims = tuple(map(context._encode, delims))
+        delims = tuple(map(packing._need_bytes, delims))

        # Longest delimiter for tracking purposes
        longest = max(map(len, delims))
@@ -434,7 +434,7 @@ class tube(Timeout, Logger):
            >>> t.recvlinesS(3)
            ['Foo', 'Bar', 'Baz']
        """
-        return [context._decode(x) for x in self.recvlines(numlines, keepends, timeout)]
+        return [packing._decode(x) for x in self.recvlines(numlines, keepends, timeout)]

    def recvlinesb(self, numlines=2**20, keepends=False, timeout=default):
        r"""recvlinesb(numlines, keepends=False, timeout=default) -> bytearray list
@@ -563,9 +563,9 @@ class tube(Timeout, Logger):
            >>> t.recvline_contains((b'car', b'train'))
            b'bicycle car train'
        """
-        if isinstance(items, (bytes, six.text_type)):
+        if isinstance(items, (bytes, bytearray, six.text_type)):
            items = (items,)
-        items = tuple(map(context._encode, items))
+        items = tuple(map(packing._need_bytes, items))

        def pred(line):
            return any(d in line for d in items)
@@ -601,9 +601,9 @@ class tube(Timeout, Logger):
            b'World'
        """
        # Convert string into singleton tupple
-        if isinstance(delims, (bytes, six.text_type)):
+        if isinstance(delims, (bytes, bytearray, six.text_type)):
            delims = (delims,)
-        delims = tuple(map(context._encode, delims))
+        delims = tuple(map(packing._need_bytes, delims))

        return self.recvline_pred(lambda line: any(map(line.startswith, delims)),
                                  keepends=keepends,
@@ -632,10 +632,10 @@ class tube(Timeout, Logger):
            b'Kaboodle'
        """
        # Convert string into singleton tupple
-        if isinstance(delims, (bytes, six.text_type)):
+        if isinstance(delims, (bytes, bytearray, six.text_type)):
            delims = (delims,)

-        delims = tuple(context._encode(delim) + self.newline for delim in delims)
+        delims = tuple(packing._need_bytes(delim) + self.newline for delim in delims)

        return self.recvline_pred(lambda line: any(map(line.endswith, delims)),
                                  keepends=keepends,
@@ -654,8 +654,8 @@ class tube(Timeout, Logger):
        all data is buffered and an empty string (``''``) is returned.
        """

-        if isinstance(regex, (bytes, six.text_type)):
-            regex = context._encode(regex)
+        if isinstance(regex, (bytes, bytearray, six.text_type)):
+            regex = packing._need_bytes(regex)
            regex = re.compile(regex)

        if exact:
@@ -678,8 +678,8 @@ class tube(Timeout, Logger):
        all data is buffered and an empty string (``''``) is returned.
        """

-        if isinstance(regex, (bytes, six.text_type)):
-            regex = context._encode(regex)
+        if isinstance(regex, (bytes, bytearray, six.text_type)):
+            regex = packing._need_bytes(regex)
            regex = re.compile(regex)

        if exact:
@@ -763,7 +763,7 @@ class tube(Timeout, Logger):
            b'hello'
        """

-        data = context._encode(data)
+        data = packing._need_bytes(data)

        if self.isEnabledFor(logging.DEBUG):
            self.debug('Sent %#x bytes:' % len(data))
@@ -793,12 +793,13 @@ class tube(Timeout, Logger):
            b'hello\r\n'
        """

-        line = context._encode(line)
+        line = packing._need_bytes(line)

        self.send(line + self.newline)

    def sendlines(self, lines=[]):
        for line in lines:
+            line = packing._need_bytes(line)
            self.sendline(line)

    def sendafter(self, delim, data, timeout = default):
@@ -807,6 +808,7 @@ class tube(Timeout, Logger):
        A combination of ``recvuntil(delim, timeout=timeout)`` and ``send(data)``.
        """

+        data = packing._need_bytes(data)
        res = self.recvuntil(delim, timeout=timeout)
        self.send(data)
        return res
@@ -816,6 +818,7 @@ class tube(Timeout, Logger):

        A combination of ``recvuntil(delim, timeout=timeout)`` and ``sendline(data)``."""

+        data = packing._need_bytes(data)
        res = self.recvuntil(delim, timeout=timeout)
        self.sendline(data)
        return res
@@ -825,6 +828,7 @@ class tube(Timeout, Logger):

        A combination of ``send(data)`` and ``recvuntil(delim, timeout=timeout)``."""

+        data = packing._need_bytes(data)
        self.send(data)
        return self.recvuntil(delim, timeout=timeout)

@@ -833,6 +837,7 @@ class tube(Timeout, Logger):

        A combination of ``sendline(data)`` and ``recvuntil(delim, timeout=timeout)``."""

+        data = packing._need_bytes(data)
        self.sendline(data)
        return self.recvuntil(delim, timeout=timeout)

@@ -1399,7 +1404,7 @@ class tube(Timeout, Logger):
        def wrapperb(self, *a, **kw):
            return bytearray(func(self, *a, **kw))
        def wrapperS(self, *a, **kw):
-            return context._decode(func(self, *a, **kw))
+            return packing._decode(func(self, *a, **kw))
        wrapperb.__doc__ = 'Same as :meth:`{func.__name__}`, but returns a bytearray'.format(func=func)
        wrapperb.__name__ = func.__name__ + 'b'
        wrapperS.__doc__ = 'Same as :meth:`{func.__name__}`, but returns a str, ' \
@@ -92,7 +92,7 @@ def yesno(prompt, default=None):
       :skipif: branch_dev

        >>> p = testpwnproc("print(yesno('is it ok??'))")
-        >>> b"is it ok" in p.recvuntil("??")
+        >>> b"is it ok" in p.recvuntil(b"??")
        True
        >>> p.sendline(b"x\nny")
        >>> b"True" in p.recvall()
@@ -1,19 +1,5 @@
 from __future__ import absolute_import

-from pwnlib.util import crc
-from pwnlib.util import cyclic
-from pwnlib.util import fiddling
-from pwnlib.util import hashes
-from pwnlib.util import iters
-from pwnlib.util import lists
-from pwnlib.util import misc
-from pwnlib.util import net
-from pwnlib.util import packing
-from pwnlib.util import proc
-from pwnlib.util import safeeval
-from pwnlib.util import sh_string
-from pwnlib.util import web
-
 __all__ = [
    'crc', 'cyclic', 'fiddling', 'hashes', 'iters',
    'lists', 'misc', 'net', 'packing', 'proc', 'safeeval',
@@ -209,7 +209,7 @@ def cyclic_find(subseq, alphabet = None, n = None):

    if isinstance(subseq, six.integer_types):
        subseq = packing.pack(subseq, bytes=n)
-    subseq = context._encode(subseq)
+    subseq = packing._encode(subseq)

    if len(subseq) != n:
        log.warn_once("cyclic_find() expects %i-byte subsequences by default, you gave %r\n"
@@ -220,7 +220,7 @@ def cyclic_find(subseq, alphabet = None, n = None):

    if alphabet is None:
        alphabet = context.cyclic_alphabet
-    alphabet = context._encode(alphabet)
+    alphabet = packing._encode(alphabet)

    if any(c not in alphabet for c in subseq):
        return -1
@@ -541,8 +541,10 @@ def isprint(c):
    """isprint(c) -> bool

    Return True if a character is printable"""
-    t = (string.ascii_letters + string.digits + string.punctuation + ' ').encode()
-    return c in set(map(packing._p8lu, bytearray(t)))|set(map(chr, bytearray(t)))
+    if isinstance(c, six.text_type):
+        c = ord(c)
+    t = bytearray(string.ascii_letters + string.digits + string.punctuation + ' ', 'ascii')
+    return c in t


 def hexii(s, width = 16, skip = True):
@@ -800,8 +802,7 @@ def hexdump(s, width=16, skip=True, hexii=False, begin=0, style=None,
        dead0018  .g  .a  .a  .a   .h  .a  .a  .a  │
        dead0020

-        >>> import struct
-        >>> print(hexdump(list(map(struct.Struct("B").pack, range(256)))))
+        >>> print(hexdump(bytearray(range(256))))
        00000000  00 01 02 03  04 05 06 07  08 09 0a 0b  0c 0d 0e 0f  │····│····│····│····│
        00000010  10 11 12 13  14 15 16 17  18 19 1a 1b  1c 1d 1e 1f  │····│····│····│····│
        00000020  20 21 22 23  24 25 26 27  28 29 2a 2b  2c 2d 2e 2f  │ !"#│$%&'│()*+│,-./│
@@ -820,7 +821,7 @@ def hexdump(s, width=16, skip=True, hexii=False, begin=0, style=None,
        000000f0  f0 f1 f2 f3  f4 f5 f6 f7  f8 f9 fa fb  fc fd fe ff  │····│····│····│····│
        00000100

-        >>> print(hexdump(list(map(struct.Struct("B").pack, range(256))), hexii=True))
+        >>> print(hexdump(bytearray(range(256)), hexii=True))
        00000000      01  02  03   04  05  06  07   08  09  0a  0b   0c  0d  0e  0f  │
        00000010  10  11  12  13   14  15  16  17   18  19  1a  1b   1c  1d  1e  1f  │
        00000020  20  .!  ."  .#   .$  .%  .&  .'   .(  .)  .*  .+   .,  .-  ..  ./  │
@@ -934,14 +935,14 @@ def hexdump(s, width=16, skip=True, hexii=False, begin=0, style=None,
        00000010  41 41 41 41 41 41 41 41                          │AAAAAAAA│
        00000018

-        >>> print(hexdump('A'*24, width=16, total=False))
+        >>> print(hexdump(b'A'*24, width=16, total=False))
        00000000  41 41 41 41  41 41 41 41  41 41 41 41  41 41 41 41  │AAAA│AAAA│AAAA│AAAA│
        00000010  41 41 41 41  41 41 41 41                            │AAAA│AAAA│
-        >>> print(hexdump('A'*24, width=16, groupsize=8, total=False))
+        >>> print(hexdump(b'A'*24, width=16, groupsize=8, total=False))
        00000000  41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  │AAAAAAAA│AAAAAAAA│
        00000010  41 41 41 41 41 41 41 41                           │AAAAAAAA│
    """
-    s = packing.flat(s)
+    s = packing.flat(s, stacklevel=1)
    return '\n'.join(hexdump_iter(BytesIO(s),
                                  width,
                                  skip,
@@ -229,7 +229,7 @@ def run_in_new_terminal(command, terminal=None, args=None, kill_at_exit=True, pr
            args     = []
        elif 'TMUX' in os.environ and which('tmux'):
            terminal = 'tmux'
-            args     = ['splitw', '-F' '#{pane_pid}', '-P']
+            args     = ['splitw']
        elif 'STY' in os.environ and which('screen'):
            terminal = 'screen'
            args     = ['-t','pwntools-gdb','bash','-c']
@@ -256,6 +256,12 @@ def run_in_new_terminal(command, terminal=None, args=None, kill_at_exit=True, pr
    if isinstance(args, tuple):
        args = list(args)

+    # When not specifying context.terminal explicitly, we used to set these flags above.
+    # However, if specifying terminal=['tmux', 'splitw', '-h'], we would be lacking these flags.
+    # Instead, set them here and hope for the best.
+    if terminal == 'tmux':
+        args += ['-F' '#{pane_pid}', '-P']
+
    argv = [which(terminal)] + args

    if isinstance(command, six.string_types):
@@ -37,6 +37,7 @@ import collections
 import six
 import struct
 import sys
+import warnings

 from six.moves import range

@@ -50,7 +51,7 @@ mod = sys.modules[__name__]
 log = getLogger(__name__)

 def pack(number, word_size = None, endianness = None, sign = None, **kwargs):
-    """pack(number, word_size = None, endianness = None, sign = None, **kwargs) -> str
+    r"""pack(number, word_size = None, endianness = None, sign = None, **kwargs) -> str

    Packs arbitrary-sized integer.

@@ -80,23 +81,23 @@ def pack(number, word_size = None, endianness = None, sign = None, **kwargs):
        >>> pack(0x414243, 24, 'little', True)
        b'CBA'
        >>> pack(0x814243, 24, 'big', False)
-        b'\\x81BC'
+        b'\x81BC'
        >>> pack(0x814243, 24, 'big', True)
        Traceback (most recent call last):
           ...
        ValueError: pack(): number does not fit within word_size
        >>> pack(0x814243, 25, 'big', True)
-        b'\\x00\\x81BC'
+        b'\x00\x81BC'
        >>> pack(-1, 'all', 'little', True)
-        b'\\xff'
+        b'\xff'
        >>> pack(-256, 'all', 'big', True)
-        b'\\xff\\x00'
+        b'\xff\x00'
        >>> pack(0x0102030405, 'all', 'little', True)
-        b'\\x05\\x04\\x03\\x02\\x01'
+        b'\x05\x04\x03\x02\x01'
        >>> pack(-1)
-        b'\\xff\\xff\\xff\\xff'
+        b'\xff\xff\xff\xff'
        >>> pack(0x80000000, 'all', 'big', True)
-        b'\\x00\\x80\\x00\\x00\\x00'
+        b'\x00\x80\x00\x00\x00'
 """
    if sign is None and number < 0:
        sign = True
@@ -165,7 +166,7 @@ def pack(number, word_size = None, endianness = None, sign = None, **kwargs):

@LocalNoarchContext
 def unpack(data, word_size = None):
-    """unpack(data, word_size = None, endianness = None, sign = None, **kwargs) -> int
+    r"""unpack(data, word_size = None, endianness = None, sign = None, **kwargs) -> int

    Packs arbitrary-sized integer.

@@ -188,17 +189,17 @@ def unpack(data, word_size = None):
        The unpacked number.

    Examples:
-        >>> hex(unpack(b'\\xaa\\x55', 16, endian='little', sign=False))
+        >>> hex(unpack(b'\xaa\x55', 16, endian='little', sign=False))
        '0x55aa'
-        >>> hex(unpack(b'\\xaa\\x55', 16, endian='big', sign=False))
+        >>> hex(unpack(b'\xaa\x55', 16, endian='big', sign=False))
        '0xaa55'
-        >>> hex(unpack(b'\\xaa\\x55', 16, endian='big', sign=True))
+        >>> hex(unpack(b'\xaa\x55', 16, endian='big', sign=True))
        '-0x55ab'
-        >>> hex(unpack(b'\\xaa\\x55', 15, endian='big', sign=True))
+        >>> hex(unpack(b'\xaa\x55', 15, endian='big', sign=True))
        '0x2a55'
-        >>> hex(unpack(b'\\xff\\x02\\x03', 'all', endian='little', sign=True))
+        >>> hex(unpack(b'\xff\x02\x03', 'all', endian='little', sign=True))
        '0x302ff'
-        >>> hex(unpack(b'\\xff\\x02\\x03', 'all', endian='big', sign=True))
+        >>> hex(unpack(b'\xff\x02\x03', 'all', endian='big', sign=True))
        '-0xfdfd'
    """

@@ -206,6 +207,7 @@ def unpack(data, word_size = None):
    word_size  = word_size  or context.word_size
    endianness = context.endianness
    sign       = context.sign
+    data = _need_bytes(data, 2)

    # Verify that word_size make sense
    if word_size == 'all':
@@ -289,13 +291,19 @@ def unpack_many(data, word_size = None):
 #
 # Make individual packers, e.g. _p8lu
 #
-ops   = {'p': struct.pack, 'u': lambda *a: struct.unpack(*(
-                                             x.encode('latin1') if not hasattr(x, 'decode') else x
-                                             for x in a))[0]}
+ops   = ['p','u']
 sizes = {8:'b', 16:'h', 32:'i', 64:'q'}
 ends  = ['b','l']
 signs = ['s','u']

+return_types     = {'p': 'bytes', 'u': 'int'}
+op_verbs         = {'p': 'pack', 'u': 'unpack'}
+arg_doc          = {'p': 'number (int): Number to convert',
+                    'u': 'data (bytes): Byte string to convert'}
+rv_doc           = {'p': 'The packed number as a byte string',
+                    'u': 'The unpacked number'}
+
+
 def make_single(op,size,end,sign):
    name = '_%s%s%s%s' % (op, size, end, sign)
    fmt  = sizes[size]
@@ -305,22 +313,23 @@ def make_single(op,size,end,sign):
        fmt = fmt.upper()
    fmt = end+fmt

-    def routine(data):
-        return ops[op](fmt,data)
+    struct_op = getattr(struct.Struct(fmt), op_verbs[op])
+    if op == 'u':
+        def routine(data, stacklevel=1):
+            data = _need_bytes(data, stacklevel)
+            return struct_op(data)[0]
+    else:
+        def routine(data, stacklevel=None):
+            return struct_op(data)
    routine.__name__ = routine.__qualname__ = name

    return name, routine

+
 for op,size,end,sign in iters.product(ops, sizes, ends, signs):
    name, routine = make_single(op,size,end,sign)
    setattr(mod, name, routine)

-return_types     = {'p': 'str', 'u': 'int'}
-op_verbs         = {'p': 'pack', 'u': 'unpack'}
-arg_doc          = {'p': 'number (int): Number to convert',
-                    'u': 'data (str): String to convert'}
-rv_doc           = {'p': 'The packed number as a string',
-                    'u': 'The unpacked number'}

 #
 # Make normal user-oriented packers, e.g. p8
@@ -341,7 +350,7 @@ def make_multi(op, size):
        return {("little", True  ): ls,
                ("little", False):  lu,
                ("big",    True  ): bs,
-                ("big",    False):  bu}[endian, signed](number)
+                ("big",    False):  bu}[endian, signed](number, 3)

    routine.__name__ = name
    routine.__doc__  = """%s%s(number, sign, endian, ...) -> %s
@@ -421,7 +430,7 @@ def make_packer(word_size = None, sign = None, **kwargs):
                (64, 0, 1): _p64ls,
                (64, 1, 0): _p64bu,
                (64, 1, 1): _p64bs,
-            }.get((word_size, {'big': 1, 'little': 0}[endianness], sign), None)
+            }.get((word_size, {'big': 1, 'little': 0}[endianness], sign))

            if packer:
                return packer
@@ -452,9 +461,9 @@ def make_unpacker(word_size = None, endianness = None, sign = None, **kwargs):
        >>> u = make_unpacker(32, endian='little', sign='unsigned')
        >>> u
        <function _u32lu at 0x...>
-        >>> hex(u('/bin'))
+        >>> hex(u(b'/bin'))
        '0x6e69622f'
-        >>> u('abcde')
+        >>> u(b'abcde')
        Traceback (most recent call last):
            ...
        error: unpack requires a string argument of length 4
@@ -489,7 +498,7 @@ def make_unpacker(word_size = None, endianness = None, sign = None, **kwargs):
    else:
        return lambda number: unpack(number, word_size, endianness, sign)

-def _fit(pieces, preprocessor, packer, filler):
+def _fit(pieces, preprocessor, packer, filler, stacklevel=1):

    # Pulls bytes from `filler` and adds them to `pad` until it ends in `key`.
    # Returns the index of `key` in `pad`.
@@ -511,10 +520,8 @@ def _fit(pieces, preprocessor, packer, filler):
        if isinstance(k, six.integer_types):
            if k >= large_key:
                k = fill(pack(k))
-        elif isinstance(k, six.text_type):
-            k = fill(k.encode('utf8'))
-        elif isinstance(k, (bytearray, bytes)):
-            k = fill(k)
+        elif isinstance(k, (six.text_type, bytearray, bytes)):
+            k = fill(_encode(k))
        else:
            raise TypeError("flat(): offset must be of type int or str, but got '%s'" % type(k))
        if k in pieces_:
@@ -545,7 +552,7 @@ def _fit(pieces, preprocessor, packer, filler):
            out += p8(next(filler))

        # Recursively flatten data
-        out += _flat([v], preprocessor, packer, filler)
+        out += _flat([v], preprocessor, packer, filler, stacklevel + 1)

    # Now do negative indices
    out_negative = b''
@@ -562,11 +569,11 @@ def _fit(pieces, preprocessor, packer, filler):
                out_negative += p8(next(filler))

            # Recursively flatten data
-            out_negative += _flat([v], preprocessor, packer, filler)
+            out_negative += _flat([v], preprocessor, packer, filler, stacklevel + 1)

    return filler, out_negative + out

-def _flat(args, preprocessor, packer, filler):
+def _flat(args, preprocessor, packer, filler, stacklevel=1):
    out = []
    for arg in args:

@@ -578,13 +585,13 @@ def _flat(args, preprocessor, packer, filler):
        if hasattr(arg, '__flat__'):
            val = arg.__flat__()
        elif isinstance(arg, (list, tuple)):
-            val = _flat(arg, preprocessor, packer, filler)
+            val = _flat(arg, preprocessor, packer, filler, stacklevel + 1)
        elif isinstance(arg, dict):
-            filler, val = _fit(arg, preprocessor, packer, filler)
+            filler, val = _fit(arg, preprocessor, packer, filler, stacklevel + 1)
        elif isinstance(arg, bytes):
            val = arg
        elif isinstance(arg, six.text_type):
-            val = arg.encode('utf8')
+            val = _need_bytes(arg, stacklevel + 1)
        elif isinstance(arg, six.integer_types):
            val = packer(arg)
        elif isinstance(arg, bytearray):
@@ -662,13 +669,13 @@ def flat(*args, **kwargs):
        b'X'
        >>> flat([1,2,3])
        b'\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00'
-        >>> flat({4:'X'})
+        >>> flat({4:b'X'})
        b'aaaaX'

        :meth:`.flat` flattens all of the values provided, and allows nested lists
        and dictionaries.

-        >>> flat([{4:'X'}] * 2)
+        >>> flat([{4:b'X'}] * 2)
        b'aaaaXaaacX'
        >>> flat([[[[[[[[[1]]]], 2]]]]])
        b'\x01\x00\x00\x00\x02\x00\x00\x00'
@@ -676,20 +683,20 @@ def flat(*args, **kwargs):
        You can also provide additional arguments like endianness, word-size, and
        whether the values are treated as signed or not.

-        >>> flat(1, "test", [[["AB"]*2]*3], endianness = 'little', word_size = 16, sign = False)
+        >>> flat(1, b"test", [[[b"AB"]*2]*3], endianness = 'little', word_size = 16, sign = False)
        b'\x01\x00testABABABABABAB'

        A preprocessor function can be provided in order to modify the values in-flight.
-        This example converts increments each value by 1, then converts to a string.
+        This example converts increments each value by 1, then converts to a byte string.

-        >>> flat([1, [2, 3]], preprocessor = lambda x: str(x+1))
+        >>> flat([1, [2, 3]], preprocessor = lambda x: str(x+1).encode())
        b'234'

        Using dictionaries is a fast way to get specific values at specific offsets,
        without having to do ``data += "foo"`` repeatedly.

        >>> flat({12: 0x41414141,
-        ...       24: 'Hello',
+        ...       24: b'Hello',
        ...      })
        b'aaaabaaacaaaAAAAeaaafaaaHello'

@@ -699,32 +706,32 @@ def flat(*args, **kwargs):

        The cyclic pattern can be provided as either the text or hexadecimal offset.

-        >>> flat({ 0x61616162: 'X'})
+        >>> flat({ 0x61616162: b'X'})
        b'aaaaX'
-        >>> flat({'baaa': 'X'})
+        >>> flat({'baaa': b'X'})
        b'aaaaX'

        Fields do not have to be in linear order, and can be freely mixed.
        This also works with cyclic offsets.

-        >>> flat({2: 'A', 0:'B'})
+        >>> flat({2: b'A', 0:b'B'})
        b'BaA'
-        >>> flat({0x61616161:'x', 0x61616162:'y'})
+        >>> flat({0x61616161: b'x', 0x61616162: b'y'})
        b'xaaay'
-        >>> flat({0x61616162:'y', 0x61616161:'x'})
+        >>> flat({0x61616162: b'y', 0x61616161: b'x'})
        b'xaaay'

        Fields do not have to be in order, and can be freely mixed.

-        >>> flat({'caaa': 'XXXX', 16: '\x41', 20: 0xdeadbeef})
+        >>> flat({'caaa': b'XXXX', 16: b'\x41', 20: 0xdeadbeef})
        b'aaaabaaaXXXXdaaaAaaa\xef\xbe\xad\xde'
-        >>> flat({ 8: [0x41414141, 0x42424242], 20: 'CCCC'})
+        >>> flat({ 8: [0x41414141, 0x42424242], 20: b'CCCC'})
        b'aaaabaaaAAAABBBBeaaaCCCC'
        >>> fit({
-        ...     0x61616161: 'a',
-        ...     1: 'b',
-        ...     0x61616161+2: 'c',
-        ...     3: 'd',
+        ...     0x61616161: b'a',
+        ...     1: b'b',
+        ...     0x61616161+2: b'c',
+        ...     3: b'd',
        ... })
        b'abadbaaac'

@@ -732,22 +739,22 @@ def flat(*args, **kwargs):
        You can customize this by providing an iterable or method for the ``filler``
        argument.

-        >>> flat({12: 'XXXX'}, filler = b'_', length = 20)
+        >>> flat({12: b'XXXX'}, filler = b'_', length = 20)
        b'____________XXXX____'
-        >>> flat({12: 'XXXX'}, filler = b'AB', length = 20)
+        >>> flat({12: b'XXXX'}, filler = b'AB', length = 20)
        b'ABABABABABABXXXXABAB'

        Nested dictionaries also work as expected.

-        >>> flat({4: {0: 'X', 4: 'Y'}})
+        >>> flat({4: {0: b'X', 4: b'Y'}})
        b'aaaaXaaaY'
-        >>> fit({4: {4: 'XXXX'}})
+        >>> fit({4: {4: b'XXXX'}})
        b'aaaabaaaXXXX'

        Negative indices are also supported, though this only works for integer
        keys.
    
-        >>> flat({-4: 'x', -1: 'A', 0: '0', 4:'y'})
+        >>> flat({-4: b'x', -1: b'A', 0: b'0', 4: b'y'})
        b'xaaA0aaay'
    """
    # HACK: To avoid circular imports we need to delay the import of `cyclic`
@@ -756,15 +763,16 @@ def flat(*args, **kwargs):
    preprocessor = kwargs.pop('preprocessor', lambda x: None)
    filler       = kwargs.pop('filler', cyclic.de_bruijn())
    length       = kwargs.pop('length', None)
+    stacklevel   = kwargs.pop('stacklevel', 0)

-    if isinstance(filler, str):
-        filler = bytearray(six.ensure_binary(filler))
+    if isinstance(filler, (str, six.text_type)):
+        filler = bytearray(_need_bytes(filler))

    if kwargs != {}:
        raise TypeError("flat() does not support argument %r" % kwargs.popitem()[0])

    filler = iters.cycle(filler)
-    out = _flat(args, preprocessor, make_packer(), filler)
+    out = _flat(args, preprocessor, make_packer(), filler, stacklevel + 2)

    if length:
        if len(out) > length:
@@ -775,6 +783,7 @@ def flat(*args, **kwargs):

 def fit(*args, **kwargs):
    """Legacy alias for :func:`flat`"""
+    kwargs['stacklevel'] = kwargs.get('stacklevel', 0) + 1
    return flat(*args, **kwargs)

 """
@@ -1000,5 +1009,65 @@ def dd(dst, src, count = 0, skip = 0, seek = 0, truncate = False):

    return dst

+def _need_bytes(s, level=1):
+    if isinstance(s, (bytes, bytearray)):
+        return s   # already bytes
+
+    encoding = context.encoding
+    errors = 'strict'
+    if encoding == 'auto':
+        worst = max(map(ord, s))
+        if worst > 255:
+            encoding = 'UTF-8'
+            errors = 'surrogateescape'
+        elif worst > 127:
+            encoding = 'ISO-8859-1'
+        else:
+            encoding = 'ASCII'
+
+    warnings.warn("Text is not bytes; assuming {}, no guarantees. See https://docs.pwntools.com/#bytes"
+                  .format(encoding), BytesWarning, level + 2)
+    return s.encode(encoding, errors)
+
+def _need_text(s, level=1):
+    if isinstance(s, (str, six.text_type)):
+        return s   # already text
+
+    encoding = context.encoding
+    errors = 'strict'
+    if encoding == 'auto':
+        for encoding in 'ASCII', 'UTF-8', 'ISO-8859-1':
+            try:
+                s = s.decode(encoding)
+            except UnicodeDecodeError:
+                pass
+            else:
+                break
+
+    warnings.warn("Bytes is not text; assuming {}, no guarantees. See https://docs.pwntools.com/#bytes"
+                  .format(encoding), BytesWarning, level + 2)
+    return s.decode(encoding, errors)
+
+def _encode(s):
+    if isinstance(s, (bytes, bytearray)):
+        return s   # already bytes
+
+    if context.encoding == 'auto':
+        try:
+            return s.encode('latin1')
+        except UnicodeEncodeError:
+            return s.encode('utf-8', 'surrogateescape')
+    return s.encode(context.encoding)
+
+def _decode(b):
+    if context.encoding == 'auto':
+        try:
+            return b.decode('utf-8')
+        except UnicodeDecodeError:
+            return b.decode('latin1')
+        except AttributeError:
+            return b
+    return b.decode(context.encoding)
+
 del op, size, end, sign
 del name, routine, mod
@@ -333,7 +333,7 @@ def wait_for_debugger(pid, debugger_pid=None):
    t = Timeout()
    with t.countdown(timeout=15):
        with log.waitfor('Waiting for debugger') as l:
-            if debugger_pid:
+            while debugger_pid:
                debugger = psutil.Process(debugger_pid)
                while t.timeout and tracer(pid) is None:
                    try:
@@ -341,12 +341,15 @@ def wait_for_debugger(pid, debugger_pid=None):
                    except psutil.TimeoutExpired:
                        pass
                    else:
-                        l.failure("debugger exited! (maybe check /proc/sys/kernel/yama/ptrace_scope)")
+                        debugger_pid = 0
+                        break
            else:
                while t.timeout and tracer(pid) is None:
                    time.sleep(0.01)

        if tracer(pid):
            l.success()
+        elif debugger_pid == 0:
+            l.failure("debugger exited! (maybe check /proc/sys/kernel/yama/ptrace_scope)")
        else:
            l.failure('Debugger did not attach to pid %d within 15 seconds', pid)
@@ -37,7 +37,7 @@ def wget(url, save=None, timeout=5, **kwargs):
    """
    import requests

-    with log.progress("Downloading '%s'" % url, rate=0.1) as w:
+    with log.progress("Downloading %r" % url, rate=0.1) as w:
        w.status("Making request...")

        response = requests.get(url, stream=True, timeout=timeout, **kwargs)
@@ -1 +1 @@
-__version__ = '4.5.0'
+__version__ = '4.6.0'
@@ -88,7 +88,7 @@ setup(
    name                 = 'pwntools',
    python_requires      = '>=2.7',
    packages             = find_packages(),
-    version              = '4.5.0',
+    version              = '4.6.0',
    data_files           = [('pwntools-doc',
                             glob.glob('*.md') + glob.glob('*.txt')),
                            ],
@@ -33,5 +33,5 @@ echo > ~/pwntools/docs/source/adb.rst
 echo > ~/pwntools/docs/source/protocols.rst

 export PWNLIB_NOTERM=1 
-coverage-2.7 run -m sphinx -b doctest $HOME/pwntools/docs/source $HOME/pwntools/docs/build/doctest $TARGET
+python2.7 -bb -m coverage run -m sphinx -b doctest $HOME/pwntools/docs/source $HOME/pwntools/docs/build/doctest $TARGET

@@ -31,5 +31,5 @@ echo > ~/pwntools/docs/source/adb.rst
 echo > ~/pwntools/docs/source/protocols.rst

 export PWNLIB_NOTERM=1 
-coverage3 run -m sphinx -b doctest $HOME/pwntools/docs/source $HOME/pwntools/docs/build/doctest $TARGET
+python3 -bb -m coverage run -m sphinx -b doctest $HOME/pwntools/docs/source $HOME/pwntools/docs/build/doctest $TARGET

@@ -73,6 +73,9 @@ setup_gdbserver()
 {
    # https://docs.improbable.io/reference/14.3/shared/debug-cloud-workers#common-issues
    # wget http://archive.ubuntu.com/ubuntu/pool/main/g/gdb/gdbserver_8.3-0ubuntu1_amd64.deb
+    if [[ "$(gdbserver --version|grep -Eo '[0-9]+\.[0-9]' |head -1 |cut -d. -f1)" -gt 8 ]]; then
+        return
+    fi
    wget https://launchpad.net/ubuntu/+source/gdb/8.3-0ubuntu1/+build/16807407/+files/gdbserver_8.3-0ubuntu1_amd64.deb
    sudo apt-get install ./gdbserver_8.3-0ubuntu1_amd64.deb
 }
@@ -0,0 +1,96 @@
+#!/bin/sh
+
+set -ex
+
+# If we are running on Travis CI, and there were no changes to Android
+# or ADB code, then we do not need the emulator
+if [ -n "$TRAVIS" ]; then
+if [ -z "$TRAVIS_COMMIT_RANGE" ]; then
+    echo "TRAVIS_COMMIT_RANGE is empty, forcing Android Emulator installation"
+elif ! (git show "$TRAVIS_COMMIT_RANGE" >/dev/null) ; then
+    echo "TRAVIS_COMMIT_RANGE is invalid, forcing Android Emulator installation"
+elif [[ "$TRAVIS_BRANCH" =~ "staging" ]]; then
+    echo "TRAVIS_BRANCH ($TRAVIS_BRANCH) indicates a branch we care about"
+    echo "Forcing Android Emulator installation"
+elif [[ -n "$TRAVIS_TAG" ]]; then
+    echo "TRAVIS_TAG ($TRAVIS_TAG) indicates a new relase"
+    echo "Forcing Android Emulator installation"
+elif (git log --stat "$TRAVIS_COMMIT_RANGE" | grep -iE "android|adb" | grep -v "commit "); then
+    echo "Found Android-related commits, forcing Android Emulator installation"
+else
+    # In order to avoid running the doctests that require the Android
+    # emulator, while still leaving the code intact, we remove the
+    # RST file that Sphinx searches.
+    rm -f 'docs/source/adb.rst'
+    rm -f 'docs/source/protocols/adb.rst'
+
+    # However, the file needs to be present or else things break.
+    touch 'docs/source/adb.rst'
+    touch 'docs/source/protocols/adb.rst' || true
+
+    echo "Skipping Android emulator install, Android tests disabled."
+    return
+fi
+fi
+
+
+if ! which java; then
+echo "OpenJDK-8-JRE is required for Android stuff"
+exit 1
+fi
+
+if (uname | grep -i Darwin &>/dev/null); then
+brew install android-sdk android-ndk
+else
+if [ ! -f android-sdk/tools/bin/sdkmanager ]; then
+    # Install the SDK, which gives us the 'android' and 'emulator' commands
+    wget -nv -O sdk-tools-linux.zip https://dl.google.com/android/repository/commandlinetools-linux-6858069_latest.zip
+    unzip -q sdk-tools-linux.zip
+    rm    -f sdk-tools-linux.zip
+
+    # Travis caching causes this to exist already
+    rm -rf android-sdk
+
+    mkdir android-sdk
+    mv cmdline-tools android-sdk/
+    file android-sdk/cmdline-tools/bin/sdkmanager
+fi
+
+export PATH="$PWD/android-sdk/cmdline-tools:$PATH"
+export PATH="$PWD/android-sdk/cmdline-tools/bin:$PATH"
+export PATH="$PWD/android-sdk/platform-tools:$PATH"
+export ANDROID_SDK_ROOT="$PWD/android-sdk"
+export ANDROID_HOME="$PWD/android-sdk"
+which sdkmanager
+fi
+
+# Grab prerequisites
+# Valid ABIs:
+# - armeabi-v7a
+# - arm64-v8a
+# - x86
+# - x86_64
+ANDROID_ABI='armeabi-v7a'
+ANDROIDV=android-24
+yes | sdkmanager --sdk_root="$ANDROID_HOME" --install platform-tools 'extras;android;m2repository' emulator ndk-bundle \
+  "platforms;$ANDROIDV" "system-images;$ANDROIDV;default;$ANDROID_ABI"
+yes | sdkmanager --sdk_root="$ANDROID_HOME" --licenses
+
+# enable NDK for adb.compile()
+for d in "$PWD/android-sdk/ndk-bundle/"*/; do
+export PATH="$PATH:$d"
+done
+
+# Create our emulator Android Virtual Device (AVD)
+# --snapshot flag is deprecated, see bitrise-steplib/steps-create-android-emulator#18
+echo no | avdmanager --silent create avd --name android-$ANDROID_ABI --force --package "system-images;$ANDROIDV;default;$ANDROID_ABI"
+
+# The emulator is crazy and does not even respect its own paths
+sed -i "s@=android-sdk@=$PWD/android-sdk@" ~/.android/avd/android-$ANDROID_ABI.avd/config.ini
+
+# In the future, it would be nice to be able to use snapshots.
+# However, I haven't gotten them to work nicely.
+android-sdk/emulator/emulator -avd android-$ANDROID_ABI -no-window -no-boot-anim -read-only -no-audio -no-window -no-snapshot &
+adb wait-for-device
+adb shell id
+adb shell getprop
Author	SHA1	Message	Date
Arusekk	56d43680f1	Release 4.6.0	2021-07-12 23:00:40 +02:00
heapcrash	d7139997dd	Merge branch 'stable' into beta	2021-05-31 15:55:40 -05:00
heapcrash	5b0bec0c09	[4.5.1] Update setup.py	2021-05-31 15:44:49 -05:00
Heap Crash	2d460e5738	[4.5.1] Update version.py (#1908 )	2021-05-31 20:29:49 +00:00
Heap Crash	dff64e2366	For tmux, always specify -F and -P (#1902 ) * For tmux, always specify -F and -P Fixes #1898 * [4.5.1] Update CHANGELOG	2021-05-31 15:20:35 -05:00
Arusekk	fcf008b4fd	Merge branch 'stable' into beta	2021-05-13 14:01:07 +02:00
tirefire	b2c83a7cd0	Change the regex for the rop verifier to handle 2 character registers (#1884 ) * Change the regex for the rop verifier to handle 2 character registers as well * Update changelog	2021-05-13 06:52:48 +00:00
Arusekk	882fd6a926	[testing] Google strikes back once again	2021-05-07 15:59:04 +02:00
Arusekk	f16cead9d7	[testing] Google strikes back	2021-05-07 10:46:01 +02:00
Arusekk	d7e7ecce3f	Merge branch 'stable' into beta	2021-05-07 10:39:35 +02:00
Arusekk	3914b212ae	Get a sane CHANGELOG	2021-05-07 10:38:33 +02:00
Arusekk	6780c890cd	Update CHANGELOG.md	2021-04-30 18:14:36 +02:00
Arusekk	bf023053fc	Release 4.6.0beta0	2021-04-30 18:11:16 +02:00
Arusekk	c8681de1ee	Merge branch 'beta' into dev	2021-04-30 18:10:57 +02:00
Saleem RashidandArusekk	8926425aed	Add `ignore_config` argument to `pwnlib.tubes.ssh` and improve `allow_agent` implementation (#1566 ) * ssh: Use paramiko allow_agent parameter * ssh: Add ignore_config parameter to opt out of using ~/.ssh * [changelog] Add `ignore_config` argument to `pwnlib.tubes.ssh` and improve `allow_agent` implementation (#1566) Co-authored-by: Arusekk <[email protected]>	2021-04-29 19:20:14 +02:00
mephi42andArusekk	e27209469e	Fix process() with executable relative to cwd (#1751 ) * Add PyCharm project directory to .gitignore * Fix process() with executable relative to cwd * Update CHANGELOG.md Co-authored-by: Arusekk <[email protected]>	2021-04-29 19:13:22 +02:00
Heap CrashandArusekk	797c3ee178	[process] Add writemem with tests, alias readmem == leak (#1652 ) * [process] Add writemem with tests, alias readmem == leak * Update CHANGELOG * Explicitly show result for easier debugging on CI * Explicitly set arch to i386 * Troubleshooting and avoid overwriting top-level asm * Update pwnlib/tubes/process.py * Fix nasty race condition Co-authored-by: Arusekk <[email protected]>	2021-04-29 19:10:21 +02:00
Arusekk	e3bd4212de	Respect context.bytes when counting ROP padding Closes #1862	2021-04-29 16:26:27 +02:00
Heap CrashandArusekk	a7b34f7d79	[android] Split Android AVD testing into its own Workflow (#1594 ) * [android] Split Android AVD testing into its own Workflow * [android] Modify to trigger tests * [android] Make paths more succinct and explicit * Remove environment variable * Rename to test * Always run Android jobs for debugging * Enhance Android workflow * Disable gdbserver download * Make the workflow actually work * Travis missing gdbserver * Try to fix travis * Add good ol' SIGALRM-based debugs for tests * Revert changes made to Travis * But fix GHA * Actually enable the android tests * Actually enable * Not necessary anymore * Actually run the tests with AVD installation Also fix incorrect text-vs-bytes usages in protocols.adb. Co-authored-by: Arusekk <[email protected]>	2021-04-29 15:56:18 +02:00
Arusekk	4b196c7d96	Cleanup unconditional imports (#1753 ) * Cleanup unconditional imports This may aid #1725 * Import explicitly * Add good ol' SIGALRM-based debugs for tests * Fix py2 GDB tests somehow XXX: see why this matters * Update CHANGELOG.md	2021-04-29 11:23:09 +02:00
Heap CrashandArusekk	ad251a8285	Make gdb.attach work with arrays instead of string concatenation (#1846 ) * Make gdb.attach work with arrays instead of string concatenation This results in a simpler set of code that is more resilient to issues with quoting and files with spaces in their name. * Remove detach_and_quit for now, this was for Docker CI testing * Fix typo * Fix lint errors * shlex.quote is unavailable on Python3 * Update CHANGELOG.md * With no escaping comes no escaping Co-authored-by: Arusekk <[email protected]>	2021-04-28 22:11:29 +02:00
ArusekkandCptGibbon	603e5ba0f1	Another attempt for rop.ret2csu (#1429 ) * Add rop.ret2csu() * Removed magic numbers * Replaced if elf.sym.has_key(u'__libc_csu_init') with if u'__libc_csu_init' in elf.symbols * Consistently used unicode strings to search for symbols * Used the Capstone SKIPDATA option to deal with endbr64 instructions * Added comment to clarify why we set rbp and rbx * Account for newer versions of gcc populating registers in a different order * Code style issues * Add padding functionality * Split files and add tests at last * Implement flat interface in ROP * Make linters happy * Update changelog * Add a more detailed doctest * Actually set uid and hopefully solve all the corefile problems * Show me the bug * More verbosity BTW fix py3 sockets incosistency * The most debug * Should finally work * It must work now Co-authored-by: CptGibbon <[email protected]>	2021-04-28 22:10:46 +02:00
Heap CrashandArusekk	cb24f376ce	Prefer Python3 for the remotely-executed Python script (#1746 ) * Prefer Python3 for the remotely-executed Python script This should be done as part of our general transition to Python3, but should also be done because some systems with e.g. Python2.6 do not support OrderedDict, but DO have a Python3 version that works. * Make the script actually work in py2 and py3 * Fix & update CHANGELOG.md This actually is a behavior change, and can be a breaking change for some, so must be stated prominently in the changelog. Co-authored-by: Arusekk <[email protected]>	2021-04-28 22:00:53 +02:00
Arusekk	f3b08ed289	Merge branch 'beta' into dev	2021-04-19 10:21:02 +02:00
Arusekk	835bc6ad56	Address code review from #1861 (#1866 ) * Address code review from #1861 Review by @heapcrash * Fix regression	2021-04-19 09:06:46 +02:00
Robert ReithandArusekk	2b71971821	fixed wrong description for shellcraft.amd64.linux.kill()... (#1739 ) * fixed wrong description for shellcraft.amd64.linux.kill(). new desciption copied from pwntools/pwnlib/shellcraft/templates/common/linux/syscalls/kill.asm * updated changelog with PR #1739 * Fix more syscalls and generify Co-authored-by: Arusekk <[email protected]>	2021-04-15 13:01:35 +02:00
Kasper Føns Heap Crash Arusekk	084890f739	mips: do not use $t0 temporary variable in dupio (#1776 ) * mips: do not use $t0 temporary variable in dupio dupio currently uses $t0 which is a temporary callee saved register. The dup2 call is thus allowed to trash the register. Use $v0 instead. $v0 is the return value register. Either dup2 returns the provided $v0 (newfd) or -1 if it fails, and therefore $v0 can be used safely here. Reference: https://courses.cs.washington.edu/courses/cse410/09sp/examples/MIPSCallingConventionsSummary.pdf * Update changelog * Update dupio.asm Make a small change to kick Github Actions * Fix CHANGELOG Co-authored-by: Heap Crash <[email protected]> Co-authored-by: Arusekk <[email protected]>	2021-04-15 12:05:25 +02:00
Arusekk	3121f68e39	Try to catch more errors by using python -bb (#1861 ) * Try to catch more errors by using python -bb * Apply changes to travis dockerfiles as well * Introduce context._need_bytes and context._need_text ...and resolve many issues	2021-04-15 12:04:56 +02:00
martinclauss	9066a68879	fix #1803 and #1860 (#1864 ) * fix #1803 and #1860 * bring back Python2 shebang and imports	2021-04-15 12:03:20 +02:00
Arusekk	7e03bfd5a6	Fix unreliable doctest	2021-04-09 18:19:37 +02:00
Arusekk	cf7ee859c9	Prevent CHANGELOG checks on non-code changes	2021-04-09 18:18:01 +02:00
Arusekk	7ececc6bce	Support forking terminals (#1717 ) * Support forking terminals Closes #1677 * Update proc.py	2021-04-09 17:52:41 +02:00
Arusekk	b3fe1eb6ed	Enhance x86 bitsmode-switching templates (#1756 )	2021-04-09 16:38:10 +02:00
Heap Crash	5d54e0d66d	Work around bash complaining about setlocale on some servers (#1745 ) Fixes #1744	2021-04-09 16:15:26 +02:00
Arusekk	3e0e0a5833	Merge branch 'beta' into dev	2021-03-31 23:10:34 +02:00
Arusekk	c6fd41b733	Begin working on 4.6.0	2021-03-29 12:02:45 +02:00