Release 4.1.0

Fix bytes in flag.py
Re-add recommends for cross-gcc
2026-11-12 14:20:24 -10:00 · 2020-05-08 13:54:09 +02:00 · 2020-04-07 00:59:22 +02:00 · 2020-04-07 00:41:17 +02:00 · 2020-04-07 00:27:34 +02:00 · 2020-04-07 00:26:55 +02:00
69 changed files with 1363 additions and 489 deletions
@@ -1,8 +1,10 @@
 [run]
 branch = True
+parallel = True
 omit =
    */constants/*
 source =
    pwn
    pwnlib
-    ~/.pwntools-cache/
+    ~/.pwntools-cache-2.7/
+    ~/.pwntools-cache-3.8/
@@ -0,0 +1,107 @@
+name: Continuous Integration
+on: [push, pull_request]
+
+jobs:
+  build:
+    strategy:
+      matrix:
+        python-version: [2.7, 3.8]
+        os: [ubuntu-latest]
+    runs-on: ${{ matrix.os }}
+    steps:
+    - uses: actions/checkout@v2
+    - name: Cache for pip
+      uses: actions/cache@v1
+      id: cache-pip
+      with:
+        path: ~/.cache/pip
+        key: ${{ matrix.os }}-cache-pip
+
+    - name: Cache for dependencies
+      uses: actions/cache@v1
+      id: cache-deps
+      with:
+        path: android-?dk
+        key: ${{ matrix.os }}-cache-deps
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v1
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Lint
+      run: |
+        pip install flake8
+        flake8 . --count --select=E9,F63,F7 --show-source --statistics --exclude=android-?dk  # TODO: Add F82
+        flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics --exclude=pwnlib/constants,android-?dk,.git,__pycache__
+
+    - name: Install Linux dependencies
+      run: |
+        sudo apt-get update
+        sudo apt-get install -y --no-install-recommends -o Acquire::Retries=3 \
+          ash bash-static dash ksh mksh zsh \
+          pandoc gdb socat sshpass \
+          binutils-multiarch qemu-user-static \
+          binutils-aarch64-linux-gnu \
+          binutils-arm-linux-gnueabihf \
+          binutils-mips-linux-gnu \
+          binutils-powerpc-linux-gnu \
+          gcc-multilib \
+          openjdk-8-jre-headless
+        sudo apt-get install -y -o Acquire::Retries=3 \
+          gcc-aarch64-linux-gnu \
+          gcc-arm-linux-gnueabihf
+
+    - name: Install android avd
+      if:  steps.cache-deps.outputs.cache-hit != 'true'
+      run: |
+        USER=travis source travis/install.sh
+        adb emu kill
+        set | egrep '^(ANDROID|PATH)' >android-sdk/.android.env
+
+    - name: Set up SSH
+      run: |
+        chmod og-rw ~ # see https://stackoverflow.com/a/60367309/3869724
+        ssh-keygen -t ed25519 -f ~/.ssh/pwntools-ci -N ''
+        cat > ~/.ssh/config <<EOF
+        Host example.pwnme
+            User $USER
+            HostName 127.0.0.1
+            IdentityFile ~/.ssh/pwntools-ci
+        EOF
+        echo -n 'from="127.0.0.1" ' | cat - ~/.ssh/pwntools-ci.pub > ~/.ssh/authorized_keys
+        ssh -o 'StrictHostKeyChecking no' example.pwnme id
+
+    - name: Install dependencies
+      run: |
+        pip install --upgrade pip
+        pip install --upgrade flake8 appdirs
+        python setup.py egg_info
+        pip install --upgrade --editable .
+
+    - name: Sanity checks
+      run:  PWNLIB_NOTERM=1 python -c 'from pwn import *; print(pwnlib.term.term_mode)'
+
+    - name: Install documentation dependencies
+      run:  pip install -r docs/requirements.txt
+
+    - name: Coverage doctests
+      run: |
+        source android-sdk/.android.env
+        android-sdk/emulator/emulator -avd android-$ANDROID_ABI -no-window -no-boot-anim -read-only -no-audio -no-window -no-snapshot &
+        adb wait-for-device
+        echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope # required by some gdb doctests
+        PWNLIB_NOTERM=1 coverage run -m sphinx -b doctest docs/source docs/build/doctest
+        coverage combine
+
+    - name: Build source and wheel distributions
+      run: |
+        python setup.py sdist
+        python setup.py bdist_wheel --universal
+
+    - uses: actions/upload-artifact@v2-preview
+      with:
+        path: dist/*
+
+    - name: Upload coverage to coveralls.io
+      run:  COVERALLS_REPO_TOKEN=PP20MEgztXIQJJTguQwe2jeCh6Bm4lkbv coveralls
@@ -16,4 +16,3 @@ venv
 *.egg-info
 *.core
 .coverage
-.tox
@@ -1,4 +1,3 @@
-dist: xenial
 language: python
 addons:
  apt:
@@ -13,6 +12,8 @@ addons:
    - zsh
    - pandoc
    - gdb
+    - socat
+    - sshpass
    - binutils
    - qemu-user-static
    - binutils-multiarch
@@ -26,9 +27,12 @@ cache:
        - usr
        - /home/travis/virtualenv/python2.7.15/lib/python2.7/site-packages/
        - /home/travis/virtualenv/python2.7.15/bin/
+        - /home/travis/virtualenv/python3.8.*/lib/python3.8/site-packages/
+        - /home/travis/virtualenv/python3.8.*/bin/
+        - docs/build/doctest/.doctrees
 python:
-  - "2.7.15"
-  - "3.6"
+  - "2.7"
+  - "3.8"
 before_install:
  - echo $-
  - source travis/install.sh
@@ -36,7 +40,7 @@ before_install:
  - echo $-
 install:
  - pip install --upgrade pip
-  - pip install --upgrade appdirs # https://github.com/ActiveState/appdirs/issues/89#issuecomment-282326570
+  - pip install --upgrade flake8 appdirs # https://github.com/ActiveState/appdirs/issues/89#issuecomment-282326570
  - python setup.py egg_info      # pypa/pip#6275
  - pip install --upgrade --editable .
  - pip install --upgrade --requirement docs/requirements.txt
@@ -45,8 +49,14 @@ before_script:
  - PWNLIB_NOTERM=1 python -c 'from pwn import *; print(pwnlib.term.term_mode)'
  - PWNLIB_NOTERM=1 python -c 'from pwn import *; print(pwnlib.term.term_mode)'
  - PWNLIB_NOTERM=1 python -c 'from pwn import *; print(pwnlib.term.term_mode)'
+  - sudo sh -c "echo 0 > /proc/sys/kernel/yama/ptrace_scope"
 script:
+  # stop the build if there are Python syntax errors or undefined names
+  - flake8 . --count --select=E9,F63,F7 --show-source --statistics --exclude=android-?dk  # TODO: Add F82
+  # exit-zero treats all errors as warnings.  The GitHub editor is 127 chars wide
+  - flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics --exclude=pwnlib/constants,android-?dk,.git,__pycache__
  - PWNLIB_NOTERM=1 coverage run -m sphinx -b doctest docs/source docs/build/doctest
+  - coverage combine
 after_success:
  - coveralls
  - source travis/update_demo.sh
@@ -9,9 +9,11 @@ The table below shows which release corresponds to each branch, and what date th

 | Version          | Branch   | Release Date           |
 | ---------------- | -------- | ---------------------- |
-| [4.2.0](#420)    | `dev`    | Feb 10, 2020 (planned)
-| [4.1.0](#410)    | `beta`   | Jan 30, 2020 (planned)
-| [4.0.0](#400)    | `stable` | Jan 09, 2020
+| [4.3.0](#430)    | `dev`    | Jun 30, 2020 (planned)
+| [4.2.0](#420)    | `beta`   | May 30, 2020 (planned)
+| [4.1.0](#410)    | `stable` | May 8, 2020
+| [4.0.1](#401)    |          | Jan 22, 2020
+| [4.0.0](#400)    |          | Jan 09, 2020
 | [3.13.0](#3130)  |          | Nov 5, 2019
 | [3.12.1](#3121)  |          | Sept 17, 2018
 | [3.12.0](#3120)  |          | Feb 22, 2018
@@ -45,13 +47,23 @@ The table below shows which release corresponds to each branch, and what date th
 | [3.0.0](#300)    |          | Aug 20, 2016
 | [2.2.0](#220)    |          | Jan 5, 2015

-## 4.2.0 (`dev`)
+## 4.3.0 (`dev`)

-To be released on Feb 10, 2020.
+To be released on Jun 30, 2020.

-## 4.1.0 (`beta`)
+## 4.2.0 (`beta`)

-To be released on Jan 30, 2020.
+To be released on May 30, 2020.
+
+- #1436 Add ret2dlresolve automation
+- [fecf9f] tubes.ssh.process() no longer requires python 2 installed on remote (still requires python, though)
+- Miscellanous improvements to DynElf and fmtstr leaker (see examples/fmtstr/exploit2.py)
+- #1454 Support for windows console colors
+
+[fecf9f]: http://github.com/Gallopsled/pwntols/commit/fecf9f
+
+
+## 4.1.0 (`stable`)

 - [#1316][1316] Fix connect shellcraft in python 3
 - [#1323][1323] Fix issues related with debugging
@@ -63,7 +75,14 @@ To be released on Jan 30, 2020.
 [1241]: https://github.com/Gallopsled/pwntools/pulls/1218
 [1218]: https://github.com/Gallopsled/pwntools/pulls/1218

-## 4.0.0 (`stable`)
+## 4.0.1
+
+- [#1412][1412] `recvline_pred()` and similar do not reorder data
+- Bypass unicorn-engine/unicorn#1100 and unicorn-engine/unicorn#1170 requiring unstable package
+
+[1412]: https://github.com/Gallopsled/pwntools/pulls/1412
+
+## 4.0.0

 - **Python 3 support! <3**
 - [#1402][1402] Fix serialtube in python 3
@@ -2,7 +2,7 @@
 ![pwntools logo](https://github.com/Gallopsled/pwntools/blob/stable/docs/source/logo.png?raw=true)

 [![Docs](https://readthedocs.org/projects/pwntools/badge/?version=stable)](https://docs.pwntools.com/)
-[![PyPI](https://img.shields.io/badge/pypi-v3.13.0-green.svg?style=flat)](https://pypi.python.org/pypi/pwntools/)
+[![PyPI](https://img.shields.io/pypi/v/pwntools?style=flat)](https://pypi.python.org/pypi/pwntools/)
 [![Travis](https://travis-ci.org/Gallopsled/pwntools.svg)](https://travis-ci.org/Gallopsled/pwntools)
 [![Coveralls](https://img.shields.io/coveralls/Gallopsled/pwntools/dev.svg)](https://coveralls.io/github/Gallopsled/pwntools?branch=dev)
 [![Twitter](https://img.shields.io/badge/twitter-pwntools-4099FF.svg?style=flat)](https://twitter.com/pwntools)
@@ -7,6 +7,7 @@ paramiko>=1.15.2
 pip>=6.0.8
 pyelftools>=0.2.3
 pygments>=2.0
+pypandoc
 pyserial>=2.7
 requests>=2.5.1
 ropgadget>=5.3
@@ -14,4 +15,3 @@ sphinx
 sphinx_rtd_theme
 sphinxcontrib-napoleon
 sphinxcontrib-autoprogram
-tox>=1.8.1
@@ -13,6 +13,7 @@

 import os
 import doctest
+import signal
 import six
 import subprocess
 import sys
@@ -350,7 +351,7 @@ def linkcode_resolve(domain, info):
        if isinstance(val, (types.ModuleType, types.MethodType, types.FunctionType, types.TracebackType, types.FrameType, types.CodeType) + six.class_types):
            try:
                lines, first = inspect.getsourcelines(val)
-                filename += '#L%d-%d' % (first, first + len(lines) - 1)
+                filename += '#L%d-L%d' % (first, first + len(lines) - 1)
            except (IOError, TypeError):
                pass

@@ -374,42 +375,11 @@ if build_dash:

 # -- Customization to Sphinx autodoc generation --------------------------------------------
 import sphinx.ext.autodoc
-from sphinx.util.inspect import safe_getmembers, safe_getattr

 # Test hidden members (e.g. def _foo(...))
 def dont_skip_any_doctests(app, what, name, obj, skip, options):
    return False

-def get_object_members_all(self, want_all):
-    if want_all:
-        # if not hasattr(self.object, '__all__'):
-        #     for implicit module members, check __module__ to avoid
-        #     documenting imported objects
-        return True, safe_getmembers(self.object)
-    # else:
-    #     memberlist = self.object.__all__
-    #     # Sometimes __all__ is broken...
-    #     if not isinstance(memberlist, (list, tuple)) or not \
-        #        all(isinstance(entry, string_types) for entry in memberlist):
-    #         self.directive.warn(
-    #             '__all__ should be a list of strings, not %r '
-    #             '(in module %s) -- ignoring __all__' %
-    #             (memberlist, self.fullname))
-    #         # fall back to all members
-    #         return True, safe_getmembers(self.object)
-    else:
-        memberlist = self.options.members or []
-        ret = []
-        for mname in memberlist:
-            try:
-                ret.append((mname, safe_getattr(self.object, mname)))
-            except AttributeError:
-                self.directive.warn(
-                    'missing attribute mentioned in :members: or __all__: '
-                    'module %s, attribute %s' % (
-                        safe_getattr(self.object, '__name__', '???'), mname))
-        return False, ret
-
 class _DummyClass(object): pass

 class Py2OutputChecker(_DummyClass, doctest.OutputChecker):
@@ -437,6 +407,13 @@ def py2_doctest_init(self, checker=None, verbose=None, optionflags=0):
        checker = Py2OutputChecker()
    doctest.DocTestRunner.__init__(self, checker, verbose, optionflags)

+class EndlessLoop(Exception): pass
+def alrm_handler(sig, frame):
+    signal.alarm(180) # three minutes
+    raise EndlessLoop()
+signal.signal(signal.SIGALRM, alrm_handler)
+signal.alarm(600) # ten minutes
+
 if 'doctest' in sys.argv:
    def setup(app):
        app.connect('autodoc-skip-member', dont_skip_any_doctests)
@@ -444,4 +421,3 @@ if 'doctest' in sys.argv:
    if sys.version_info[:1] < (3,):
        import sphinx.ext.doctest
        sphinx.ext.doctest.SphinxDocTestRunner.__init__ = py2_doctest_init
-    sphinx.ext.autodoc.ModuleDocumenter.get_object_members = get_object_members_all
@@ -0,0 +1,9 @@
+.. testsetup:: *
+
+   from pwnlib.filepointer import *
+
+:mod:`pwnlib.filepointer` --- `FILE*` structure exploitation
+============================================================
+
+.. automodule:: pwnlib.filepointer
+   :members:
@@ -1,6 +1,8 @@
 .. testsetup:: *

    from pwn import *
+    context.arch = 'amd64'
+    context.terminal = [os.path.join(os.path.dirname(pwnlib.__file__), 'gdb_faketerminal.py')]

 :mod:`pwnlib.gdb` --- Working with GDB
 ======================================
@@ -54,6 +54,7 @@ Each of the ``pwntools`` modules is documented here.
   encoders
   elf
   exception
+   filepointer
   flag
   fmtstr
   gdb
@@ -2,6 +2,7 @@

   from pwn import *
   from pwnlib.protocols.adb import AdbClient
+   AdbClient().wait_for_device()

 :mod:`pwnlib.protocols.adb` --- ADB Protocol Implementation
 ===========================================================
@@ -0,0 +1,10 @@
+.. testsetup:: *
+
+   from pwnlib.term.readline import *
+   import io
+
+:mod:`pwnlib.term.readline` --- Terminal nice readline
+======================================================
+
+.. automodule:: pwnlib.term.readline
+   :members:
@@ -1,3 +1,8 @@
+.. testsetup:: *
+
+   from pwn import *
+   import io
+
 :mod:`pwnlib.ui` --- Functions for user interaction
 ===================================================

@@ -26,6 +26,10 @@ Try running this example with::
 ```
 Example showing the interface to `pwnlib.asm.asm` and `pwnlib.shellcraft`.
 ```
+* `attach.py`
+```
+Example showing `pwnlib.gdb.attach()`
+```
 * `clean_and_log.py`
 ```
 Use case for `pwnlib.tubes.tube.clean_and_log`.
@@ -65,8 +69,7 @@ Example showing `pwnlib.ui.options()`
 ```
 * `port_forward.py`
 ```
-A very simple port forwarder using `pwnlib.tubes.tube.connect_both()`.  Notice
-that `<>` is just a shorthand.
+A very simple port forwarder using `pwnlib.tubes.tube.connect_both()`.
 ```
 * `readline_completers.py`
 ```
@@ -74,6 +77,10 @@ Example showing pwnlib's readline implementation and a few completers.  This
 part of pwnlib will probably see some major changes soon, but we wanted to show
 off some proof-of-concepts.
 ```
+* `remote.py`
+```
+Example showing how to use the remote class.
+```
 * `remote_gdb_debugging.py`
 ```
 Simple example showing how to use the remote
@@ -103,7 +110,7 @@ and::

  $ TERM=xterm-256color python text.py
 ```
-* `remote.py`
+* `yesno.py`
 ```
-Example showing how to use the remote class.
+Example showing `pwnlib.ui.yesno()`
 ```
@@ -1,22 +1,25 @@
-from pwn import *
+from __future__ import print_function
+
+import gdb
+from pwn import adb, context

 # Tell pwntools that the target is an Android device
-context.os='android'
-context.arch='aarch64' # or 'arm'
+context.os = "android"
+context.arch = "aarch64"  # or 'arm'

 # Optionally, set the remote ADB server address
-context.adb_host='172.16.110.1'
+context.adb_host = "172.16.110.1"

 # Wait for a device to become available
-print adb.wait_for_device()
+print(adb.wait_for_device())

 # Who am I?
-print adb.process('id').recvall().strip()
+print(adb.process("id").recvall().strip())

 # Interactive sessions!
 io = adb.shell()
-io.sendline('echo Hello, world; exit')
-print io.recvall().replace('\r\n', '\n').strip()
+io.sendline("echo Hello, world; exit")
+print(io.recvall().replace("\r\n", "\n").strip())

 # Debugging!
-gdb.debug('sh').interactive()
+gdb.debug("sh").interactive()
@@ -14,5 +14,5 @@ Try running this example with::

 from pwn import *

-print args['RHOST'] or 'RHOST is not set'
-print args['RPORT'] or 'RPORT is not set'
+print(args['RHOST'] or 'RHOST is not set')
+print(args['RPORT'] or 'RPORT is not set')
@@ -9,5 +9,5 @@ context(arch='i386', os='linux')
 shellcode = shellcraft.i386_to_amd64()
 shellcode_asm = asm(shellcode)

-print enhex(shellcode_asm)
-print disasm(shellcode_asm)
+print(enhex(shellcode_asm))
+print(disasm(shellcode_asm))
@@ -5,7 +5,7 @@ Example showing `pwnlib.gdb.attach()`
 from pwn import *

 bash = process('/bin/bash')
-gdb.attach(bash, execute = '''
+gdb.attach(bash, gdbscript = '''
 p "hello from pwnlib"
 c
 ''')
@@ -25,4 +25,4 @@ atexit.register(r.clean_and_log)

 while True:
    line = r.recvline()
-    print re.findall('^prefix (\S+)$', line)[0]
+    print(re.findall('^prefix (\S+)$', line)[0])
@@ -16,7 +16,7 @@ def exploit(binary):
        payload = fmtstr_payload(offset, {addr: p32(0x1337babe)})
        p.sendline(payload)
        p.recvuntil("DONE")
-        print hex(u32(p.recv(4)))
+        print(hex(u32(p.recv(4))))

 binaries = [
    "printf.mips",
@@ -24,7 +24,7 @@ def append_example(_arg, top, names):
            continue
        path = os.path.join(top, name)[2:] # strip './'
        log.info('-> %s' % path)
-        data = read(path).strip()
+        data = read(path).strip().decode()
        if data[0:3] not in ('"""', "'''"):
            log.warning('  Has no docstring!')
            continue
@@ -37,6 +37,7 @@ def append_example(_arg, top, names):
        out += '* `%s`\n' % path
        out += '```%s```\n' % doc

-os.path.walk('.', append_example, None)
+for path, dirs, files in os.walk('.', onerror=None):
+    append_example(dirs, path, sorted(files))

 write('README.md', out)
@@ -5,4 +5,4 @@ Example showing `pwnlib.ui.options()`
 from pwn import *

 opts = [string.letters[x] for x in range(10)]
-print 'You choose "%s"' % opts[options('Pick one:', opts)]
+print('You choose "%s"' % opts[options('Pick one:', opts)])
@@ -1,11 +1,10 @@
 """
-A very simple port forwarder using `pwnlib.tubes.tube.connect_both()`.  Notice
-that `<>` is just a shorthand.
+A very simple port forwarder using `pwnlib.tubes.tube.connect_both()`.
 """

 from pwn import *

 while True:
-    listen(1337).wait_for_connection() <> remote('google.com', 80)
+    listen(1337).wait_for_connection().connect_both(remote('google.com', 80))

 # now point your browser (or curl(1)) to http://localhost:1337
@@ -19,13 +19,13 @@ c1 = LongestPrefixCompleter([
 c2 = PathCompleter(mask = '*.py')

 with c1:
-    print 'type "exit" to exit'
+    print('type "exit" to exit')
    while True:
        s = term.readline.readline(prompt = '> ').strip()
        if s in ('exit', 'enough!'):
            break
-        print 'You wrote', s
+        print('You wrote', s)
 with c2:
-    print 'choose a file'
+    print('choose a file')
    s = term.readline.readline(prompt = text.bold_green('$ ')).strip()
-    print 'You picked', s
+    print('You picked', s)
@@ -6,6 +6,6 @@ from pwn import *

 sock = remote('127.0.0.1', 9001)

-print sock.recvline()
+print(sock.recvline())
 sock.send('foo')
 sock.interactive()
@@ -8,10 +8,10 @@ import getpass
 from pwn import *

 s = ssh(getpass.getuser(), '127.0.0.1', port = 22, keyfile = "~/.ssh/id_rsa")
-c = gdb.ssh_gdb(s, '/bin/sh', execute = '''
+c = gdb.ssh_gdb(s, '/bin/sh', gdbscript = '''
 p/x $pc
 c''')

 c.sendline('ls -la')
 c.sendline('exit')
-print c.recvall()
+print(c.recvall())
@@ -10,7 +10,7 @@ n = 1
 h = log.waitfor('spinners running', status = str(n))

 hs = []
-print 'type "q" to quit'
+print('type "q" to quit')
 while True:
    s = raw_input('> ')
    if s == 'q':
@@ -14,7 +14,7 @@ log.info("pwd: %s" % shell.pwd())
 tube = shell.run('cat')
 tube.send("Hello, cat")
 tube.shutdown("out")
-print tube.recvall()
+print(tube.recvall())

 # Show automatic working directories
 shell.set_working_directory()
@@ -29,12 +29,12 @@ int main() {

 shell.gcc(['example.c','-o','example'])

-print shell['./example']
+print(shell['./example'])

 # Show the different styles of calling
-print shell.echo("single string")
-print shell.echo(["list","of","strings"])
-print shell["echo single statement"]
+print(shell.echo("single string"))
+print(shell.echo(["list","of","strings"]))
+print(shell["echo single statement"])

 # Show off the interactive shell
 shell.interactive()
@@ -12,8 +12,8 @@ and::
 from pwn import *

 s = 'hello from pwntools'
-print text.black_on_green(s)
-print text.black_on_bright_green(s)
-print text.green_on_black(s)
-print text.bright_green_on_black(s)
-print text.bold_green_on_black(s)
+print(text.black_on_green(s))
+print(text.black_on_bright_green(s))
+print(text.green_on_black(s))
+print(text.bright_green_on_black(s))
+print(text.bold_green_on_black(s))
@@ -5,6 +5,6 @@ Example showing `pwnlib.ui.yesno()`
 from pwn import *

 if ui.yesno('Do you like Pwntools?'):
-    print ':D'
+    print(':D')
 else:
-    print ':('
+    print(':(')
@@ -29,6 +29,7 @@ from pwnlib.elf.elf import ELF, load
 from pwnlib.encoders import *
 from pwnlib.exception import PwnlibException
 from pwnlib.gdb import attach, debug, debug_assembly, debug_shellcode
+from pwnlib.filepointer import *
 from pwnlib.flag import *
 from pwnlib.fmtstr import FmtStr, fmtstr_payload
 from pwnlib.log import getLogger
@@ -115,7 +115,7 @@ def current_device(any=False):

        >>> device = adb.current_device(any=True)
        >>> device
-        AdbDevice(serial='emulator-5554', type='device', port='emulator', product='sdk_phone_armv7', model='sdk phone armv7', device='generic')
+        AdbDevice(serial='emulator-5554', type='device', port='emulator', product='sdk_google_phone_armv7', model='sdk google phone armv7', device='generic')
        >>> device.port
        'emulator'
    """
@@ -228,7 +228,7 @@ class AdbDevice(Device):
        >>> device.os
        'android'
        >>> device.product
-        'sdk_phone_armv7'
+        'sdk_google_phone_armv7'
        >>> device.serial
        'emulator-5554'
    """
@@ -277,7 +277,7 @@ class AdbDevice(Device):
            return

        with context.local(device=self.serial):
-            abi = str(properties.ro.product.cpu.abi)
+            abi = getprop('ro.product.cpu.abi')
            context.clear()
            context.arch = str(abi)
            self._arch = context.arch
@@ -680,7 +680,7 @@ def exists(path):

        >>> adb.exists('/')
        True
-        >>> adb.exists('/init')
+        >>> adb.exists('/etc/hosts')
        True
        >>> adb.exists('/does/not/exist')
        False
@@ -975,17 +975,17 @@ def fastboot(args, *a, **kw):
@with_device
 def fingerprint():
    """Returns the device build fingerprint."""
-    return str(properties.ro.build.fingerprint)
+    return getprop('ro.build.fingerprint')

@with_device
 def product():
    """Returns the device product identifier."""
-    return str(properties.ro.build.product)
+    return getprop('ro.build.product')

@with_device
 def build():
    """Returns the Build ID of the device."""
-    return str(properties.ro.build.id)
+    return getprop('ro.build.id')

@with_device
@no_emulator
@@ -995,9 +995,33 @@ def unlock_bootloader():
    Note:
        This requires physical interaction with the device.
    """
-    AdbClient().reboot_bootloader()
-    fastboot(['oem', 'unlock'])
-    fastboot(['continue'])
+    w = log.waitfor("Unlocking bootloader")
+    with w:
+        if getprop('ro.oem_unlock_supported') == '0':
+            log.error("Bootloader cannot be unlocked: ro.oem_unlock_supported=0")
+
+        if getprop('ro.boot.oem_unlock_support') == '0':
+            log.error("Bootloader cannot be unlocked: ro.boot.oem_unlock_support=0")
+
+        if getprop('sys.oem_unlock_allowed') == '0':
+            log.error("Bootloader cannot be unlocked: Enable OEM Unlock in developer settings first", context.device)
+
+        AdbClient().reboot_bootloader()
+
+        # Check to see if it's unlocked before attempting unlock
+        unlocked = fastboot(['getvar', 'unlocked'])
+        if 'unlocked: yes' in unlocked:
+            w.success("Already unlocked")
+            fastboot(['continue'])
+            return
+
+        fastboot(['oem', 'unlock'])
+        unlocked = fastboot(['getvar', 'unlocked'])
+
+        fastboot(['continue'])
+
+        if 'unlocked: yes' not in unlocked:
+            log.error("Unlock failed")

 class Kernel(object):
    _kallsyms = None
@@ -1054,7 +1078,7 @@ class Kernel(object):

    def enable_uart(self):
        """Reboots the device with kernel logging to the UART enabled."""
-        model = str(properties.ro.product.model)
+        model = getprop('ro.product.model')

        known_commands = {
            'Nexus 4': None,
@@ -1102,15 +1126,15 @@ kernel = Kernel()

 class Property(object):
    def __init__(self, name=None):
+        # Need to avoid overloaded setattr() so we go through __dict__
        self.__dict__['_name'] = name

    def __str__(self):
        return str(getprop(self._name)).strip()

-    def __repr__(self):
-        return repr(str(self))
-
    def __getattr__(self, attr):
+        if attr.startswith('_'):
+            raise AttributeError(attr)
        if self._name:
            attr = '%s.%s' % (self._name, attr)
        return Property(attr)
@@ -1123,6 +1147,17 @@ class Property(object):
            attr = '%s.%s' % (self._name, attr)
        setprop(attr, value)

+    def __eq__(self, other):
+        # Allow simple comparison, e.g.:
+        # adb.properties.ro.oem_unlock_supported == "1"
+        if isinstance(other, six.string_types):
+            return str(self) == other
+        return super(Property, self).__eq__(other)
+
+    def __hash__(self, other):
+        # Allow hash indices matching on the property
+        return hash(self._name)
+
 properties = Property()

 def _build_date():
@@ -1225,8 +1260,8 @@ def compile(source):

        # If we have an attached device, use its settings.
        if context.device:
-            abi = str(properties.ro.product.cpu.abi)
-            sdk = str(properties.ro.build.version.sdk)
+            abi = getprop('ro.product.cpu.abi')
+            sdk = getprop('ro.build.version.sdk')

        if abi is None:
            log.error("Unknown CPU ABI")
@@ -1322,9 +1357,10 @@ class Partitions(object):
        for name in listdir(self.by_name_dir):
            yield name

-    @context.quietfunc
-    @with_device
    def __getattr__(self, attr):
+        if name.startswith("_"):
+            raise AttributeError(attr)
+
        for name in self:
            if name == attr:
                break
@@ -1333,19 +1369,20 @@ class Partitions(object):

        path = os.path.join(self.by_name_dir, name)

-        # Find the actual path of the device
-        devpath = readlink(path)
-        devname = os.path.basename(devpath)
+        with context.quiet:
+            # Find the actual path of the device
+            devpath = readlink(path)
+            devname = os.path.basename(devpath)

-        # Get the size of the partition
-        for line in read('/proc/partitions').splitlines():
-            if not line.strip():
-                continue
-            major, minor, blocks, name = line.split(None, 4)
-            if devname == name:
-                break
-        else:
-            log.error("Could not find size of partition %r" % name)
+            # Get the size of the partition
+            for line in read('/proc/partitions').splitlines():
+                if not line.strip():
+                    continue
+                major, minor, blocks, name = line.split(None, 4)
+                if devname == name:
+                    break
+            else:
+                log.error("Could not find size of partition %r" % name)

        return Partition(devpath, attr, int(blocks))

@@ -1397,3 +1434,9 @@ def packages():
    """Returns a list of packages installed on the system"""
    packages = process(['pm', 'list', 'packages']).recvall()
    return [line.split('package:', 1)[-1] for line in packages.splitlines()]
+
+@context.quietfunc
+def version():
+    """Returns rthe platform version as a tuple."""
+    prop = getprop('ro.build.version.release')
+    return [int(v) for v in prop.split('.')]
@@ -576,7 +576,7 @@ def make_elf(data,
        _run(assembler + ['-o', step2, step1])

        linker_options = ['-z', 'execstack']
-        if vma:
+        if vma is not None:
            linker_options += ['--section-start=.shellcode=%#x' % vma,
                               '--entry=%#x' % vma]
        elif shared:
@@ -130,7 +130,7 @@ def main(args):
            output = output.encode('ascii')
        args.output.write(output)

-    if tty and fmt is not 'raw':
+    if tty and fmt != 'raw':
        args.output.write(b'\n')

 if __name__ == '__main__':
@@ -685,6 +685,7 @@ class ContextType(object):
                     ('x86_64', 'amd64'),
                     ('x86', 'i386'),
                     ('i686', 'i386'),
+                     ('armv7l', 'arm'),
                     ('armeabi', 'arm'),
                     ('arm64', 'aarch64')]
        for k, v in transform:
@@ -857,7 +857,7 @@ class DynELF(object):
            if self.leak.compare(address + 0xC, "GNU\x00"):
                return enhex(b''.join(self.leak.raw(address + 0x10, 20)))
            else:
-                self.status("Magic did not match")
+                self.status("Build ID not found at offset %#x" % offset)
                pass

    def _make_absolute_ptr(self, ptr_or_offset):
@@ -381,13 +381,15 @@ class Corefile(ELF):
        This requires GDB to be installed, and can only be done with native
        processes.  Getting a "complete" corefile requires GDB 7.11 or better.

-        >>> elf = ELF('/bin/bash')
+        >>> elf = ELF('/bin/bash-static')
        >>> context.clear(binary=elf)
        >>> io = process(elf.path, env={'HELLO': 'WORLD'})
        >>> core = io.corefile

        Data can also be extracted directly from the corefile.

+        >>> elf.address > 0
+        True
        >>> core.exe[elf.address:elf.address+4]
        b'\x7fELF'
        >>> core.exe.data[:4]
@@ -429,13 +431,13 @@ class Corefile(ELF):

        Corefiles can also be pulled from remote machines via SSH!

-        >>> s = ssh('travis', 'example.pwnme')
+        >>> s = ssh(host='example.pwnme')
        >>> _ = s.set_working_directory()
        >>> elf = ELF.from_assembly(shellcraft.trap())
        >>> path = s.upload(elf.path)
        >>> _ =s.chmod('+x', path)
        >>> io = s.process(path)
-        >>> io.wait()
+        >>> io.wait(1)
        -1
        >>> io.corefile.signal == signal.SIGTRAP # doctest: +SKIP
        True
@@ -445,7 +447,7 @@ class Corefile(ELF):
        >>> context.clear(arch='amd64')
        >>> elf = ELF.from_assembly('push 1234; ret')
        >>> io = elf.process()
-        >>> io.wait()
+        >>> io.wait(1)
        >>> io.corefile.fault_addr
        1234

@@ -459,7 +461,7 @@ class Corefile(ELF):

        >>> elf = ELF.from_assembly(shellcraft.crash())
        >>> io = elf.process(env={'FOO': 'BAR=BAZ'})
-        >>> io.wait()
+        >>> io.wait(1)
        >>> core = io.corefile
        >>> core.getenv('FOO')
        b'BAR=BAZ'
@@ -480,7 +482,7 @@ class Corefile(ELF):
        ... '''
        >>> elf = ELF.from_assembly(assembly)
        >>> io = elf.process()
-        >>> io.wait()
+        >>> io.wait(2)
        >>> core = io.corefile
        [!] End of the stack is corrupted, skipping stack parsing (got: 4141414141414141)
        >>> core.argc, core.argv, core.env
@@ -747,13 +749,13 @@ class Corefile(ELF):

            >>> elf = ELF.from_assembly(shellcraft.trap())
            >>> io = elf.process()
-            >>> io.wait()
+            >>> io.wait(1)
            >>> io.corefile.signal == signal.SIGTRAP
            True

            >>> elf = ELF.from_assembly(shellcraft.crash())
            >>> io = elf.process()
-            >>> io.wait()
+            >>> io.wait(1)
            >>> io.corefile.signal == signal.SIGSEGV
            True
        """
@@ -774,7 +776,7 @@ class Corefile(ELF):

            >>> elf = ELF.from_assembly('mov eax, 0xdeadbeef; jmp eax', arch='i386')
            >>> io = elf.process()
-            >>> io.wait()
+            >>> io.wait(1)
            >>> io.corefile.fault_addr == io.corefile.eax == 0xdeadbeef
            True
        """
@@ -1071,7 +1073,7 @@ class Corefile(ELF):

            >>> elf = ELF.from_assembly(shellcraft.trap())
            >>> io = elf.process(env={'GREETING': 'Hello!'})
-            >>> io.wait()
+            >>> io.wait(1)
            >>> io.corefile.getenv('GREETING')
            b'Hello!'
        """
@@ -1090,7 +1092,7 @@ class Corefile(ELF):

            >>> elf = ELF.from_assembly('mov eax, 0xdeadbeef;' + shellcraft.trap(), arch='i386')
            >>> io = elf.process()
-            >>> io.wait()
+            >>> io.wait(1)
            >>> io.corefile.registers['eax'] == 0xdeadbeef
            True
        """
@@ -1020,7 +1020,7 @@ class ELF(ELFFile):
                                                section.data(),
                                                inv_symbols)

-                for address, target in reversed(sorted(res.items())):
+                for address, target in sorted(res.items()):
                    self.plt[inv_symbols[target]] = address

        for a,n in sorted({v:k for k,v in self.plt.items()}.items()):
@@ -0,0 +1,338 @@
+# -*- coding: utf-8 -*-
+
+r"""
+File Structure Exploitation
+
+struct FILE (_IO_FILE) is the structure for File Streams. This offers various targets for exploitation on an existing bug in the code. Examples - _IO_buf_base and _IO_buf_end for reading data to arbitrary location.
+
+Remembering the offsets of various structure members while faking a FILE structure can be difficult, so this python class helps you with that. Example-
+
+>>> context.clear(arch='amd64')
+>>> fileStr = FileStructure(null=0xdeadbeef)
+>>> fileStr.vtable = 0xcafebabe
+>>> payload = bytes(fileStr)
+>>> payload
+b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\xef\xbe\xad\xde\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\xef\xbe\xad\xde\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xbe\xba\xfe\xca\x00\x00\x00\x00'
+
+Now payload contains the FILE structure with its vtable pointer pointing to 0xcafebabe
+
+Currently only 'amd64' and 'i386' architectures are supported
+"""
+
+from __future__ import absolute_import
+from __future__ import division
+
+from pwnlib.context import context
+from pwnlib.log import getLogger
+from pwnlib.util.packing import pack
+
+log = getLogger(__name__)
+
+length=0
+size='size'
+name='name'
+
+variables={
+    0:{name:'flags',size:length},
+    1:{name:'_IO_read_ptr',size:length},
+    2:{name:'_IO_read_end',size:length},
+    3:{name:'_IO_read_base',size:length},
+    4:{name:'_IO_write_base',size:length},
+    5:{name:'_IO_write_ptr',size:length},
+    6:{name:'_IO_write_end',size:length},
+    7:{name:'_IO_buf_base',size:length},
+    8:{name:'_IO_buf_end',size:length},
+    9:{name:'_IO_save_base',size:length},
+    10:{name:'_IO_backup_base',size:length},
+    11:{name:'_IO_save_end',size:length},
+    12:{name:'markers',size:length},
+    13:{name:'chain',size:length},
+    14:{name:'fileno',size:4},
+    15:{name:'_flags2',size:4},
+    16:{name:'_old_offset',size:length},
+    17:{name:'_cur_column',size:2},
+    18:{name:'_vtable_offset',size:1},
+    19:{name:'_shortbuf',size:1},
+    20:{name:'unknown1',size:-4},
+    21:{name:'_lock',size:length},
+    22:{name:'_offset',size:8},
+    23:{name:'_codecvt',size:length},
+    24:{name:'_wide_data',size:length},
+    25:{name:'unknown2',size:length},
+    26:{name:'vtable',size:length}
+}
+
+
+def update_var(l):
+    r"""
+    Since different members of the file structure have different sizes, we need to keep track of the sizes. The following function is used by the FileStructure class to initialise the lengths of the various fields.
+
+    Arguments:
+        l(int)
+            l=8 for 'amd64' architecture and l=4 for 'i386' architecture
+
+    Return Value:
+        Returns a dictionary in which each field is mapped to its corresponding length according to the architecture set
+
+    Examples:
+
+        >>> update_var(8)
+        {'flags': 8, '_IO_read_ptr': 8, '_IO_read_end': 8, '_IO_read_base': 8, '_IO_write_base': 8, '_IO_write_ptr': 8, '_IO_write_end': 8, '_IO_buf_base': 8, '_IO_buf_end': 8, '_IO_save_base': 8, '_IO_backup_base': 8, '_IO_save_end': 8, 'markers': 8, 'chain': 8, 'fileno': 4, '_flags2': 4, '_old_offset': 8, '_cur_column': 2, '_vtable_offset': 1, '_shortbuf': 1, 'unknown1': 4, '_lock': 8, '_offset': 8, '_codecvt': 8, '_wide_data': 8, 'unknown2': 48, 'vtable': 8}
+    """
+    var={}
+    for i in variables:
+        var[variables[i]['name']]=variables[i]['size']
+    for i in var:
+        if var[i]<=0:
+            var[i]+=l
+    if l==4:
+        var['unknown2']=56
+    else:
+        var['unknown2']=48
+    return var
+
+class FileStructure(object):
+    r"""
+    Crafts a FILE structure, with default values for some fields, like _lock which should point to null ideally, set.
+
+    Arguments:
+        null(int)
+            A pointer to NULL value in memory. This pointer can lie in any segment (stack, heap, bss, libc etc)
+
+    Examples:
+
+        FILE structure with flags as 0xfbad1807 and _IO_buf_base and _IO_buf_end pointing to 0xcafebabe and 0xfacef00d
+
+        >>> context.clear(arch='amd64')
+        >>> fileStr = FileStructure(null=0xdeadbeeef)
+        >>> fileStr.flags = 0xfbad1807
+        >>> fileStr._IO_buf_base = 0xcafebabe
+        >>> fileStr._IO_buf_end = 0xfacef00d
+        >>> payload = bytes(fileStr)
+        >>> payload
+        b'\x07\x18\xad\xfb\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xbe\xba\xfe\xca\x00\x00\x00\x00\r\xf0\xce\xfa\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\xef\xee\xdb\xea\r\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\xef\xee\xdb\xea\r\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+
+        Check the length of the FileStructure
+
+        >>> len(fileStr)
+        224
+
+        The defination for __repr__ orders the structure members and displays then in a dictionary format. It's useful when viewing a structure objet in python/IPython shell
+
+        >>> q=FileStructure(0xdeadbeef)
+        >>> q
+        { flags: 0x0
+         _IO_read_ptr: 0x0
+         _IO_read_end: 0x0
+         _IO_read_base: 0x0
+         _IO_write_base: 0x0
+         _IO_write_ptr: 0x0
+         _IO_write_end: 0x0
+         _IO_buf_base: 0x0
+         _IO_buf_end: 0x0
+         _IO_save_base: 0x0
+         _IO_backup_base: 0x0
+         _IO_save_end: 0x0
+         markers: 0x0
+         chain: 0x0
+         fileno: 0x0
+         _flags2: 0x0
+         _old_offset: 0xffffffffffffffff
+         _cur_column: 0x0
+         _vtable_offset: 0x0
+         _shortbuf: 0x0
+         unknown1: 0x0
+         _lock: 0xdeadbeef
+         _offset: 0xffffffffffffffff
+         _codecvt: 0x0
+         _wide_data: 0xdeadbeef
+         unknown2: 0x0
+         vtable: 0x0}
+    """
+
+    vars_=[]
+    length={}
+
+    def __init__(self, null=0):
+            self.vars_ = [variables[i]['name'] for i in sorted(variables.keys())]
+            self.setdefault(null)
+            self.length = update_var(context.bytes)
+            self._old_offset = (1 << context.bits) - 1
+
+    def __setattr__(self,item,value):
+        if item in FileStructure.__dict__ or item in self.vars_:
+            object.__setattr__(self,item,value)
+        else:
+            log.error("Unknown variable %r" % item)
+
+    def __repr__(self):
+        structure=[]
+        for i in self.vars_:
+            structure.append(" %s: %s" % (i,hex(self.__getattr__(i))))
+        return "{"+ "\n".join(structure)+"}"
+
+    def __getattr__(self,item):
+        if item in FileStructure.__dict__ or item in self.vars_:
+            return object.__getattribute__(self,item)
+        log.error("Unknown variable %r" % item)
+
+    def __len__(self):
+        return len(bytes(self))
+
+    def __bytes__(self):
+        structure = b''
+        for val in self.vars_:
+            if isinstance(self.__getattr__(val), bytes):
+                structure += self.__getattr__(val).ljust(context.bytes, b'\x00')
+            else:
+                structure += pack(self.__getattr__(val), self.length[val]*8)
+        return structure
+    __str__ = __bytes__
+
+    def struntil(self,v):
+        r"""
+        Payload for stuff till 'v' where 'v' is a structure member. This payload includes 'v' as well.
+
+        Arguments:
+            v(string)
+                The name of the field uptil which the payload should be created.
+
+        Example:
+
+            Payload for data uptil _IO_buf_end
+
+            >>> context.clear(arch='amd64')
+            >>> fileStr = FileStructure(0xdeadbeef)
+            >>> payload = fileStr.struntil("_IO_buf_end")
+            >>> payload
+            b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+        """
+        if v not in self.vars_:
+            return b''
+        structure = b''
+        for val in self.vars_:
+            if isinstance(self.__getattr__(val), bytes):
+                structure += self.__getattr__(val).ljust(context.bytes, b'\x00')
+            else:
+                structure += pack(self.__getattr__(val), self.length[val]*8)
+            if val == v:
+                break
+        return structure[:-1]
+
+    def setdefault(self,null):
+            self.flags=0
+            self._IO_read_ptr=0
+            self._IO_read_end=0
+            self._IO_read_base=0
+            self._IO_write_base=0
+            self._IO_write_ptr=0
+            self._IO_write_end=0
+            self._IO_buf_base=0
+            self._IO_buf_end=0
+            self._IO_save_base=0
+            self._IO_backup_base=0
+            self._IO_save_end=0
+            self.markers=0
+            self.chain=0
+            self.fileno=0
+            self._flags2=0
+            self._old_offset=0
+            self._cur_column=0
+            self._vtable_offset=0
+            self._shortbuf=0
+            self.unknown1=0
+            self._lock=null
+            self._offset=0xffffffffffffffff
+            self._codecvt=0
+            self._wide_data=null
+            self.unknown2=0
+            self.vtable=0
+
+    def write(self,addr=0,size=0):
+        r"""
+        Writing data out from arbitrary memory address.
+
+        Arguments:
+            addr(int)
+                The address from which data is to be printed to stdout
+            size(int)
+                The size, in bytes, of the data to be printed
+
+        Example:
+
+            Payload for writing 100 bytes to stdout from the address 0xcafebabe
+
+            >>> context.clear(arch='amd64')
+            >>> fileStr = FileStructure(0xdeadbeef)
+            >>> payload = fileStr.write(addr=0xcafebabe, size=100)
+            >>> payload
+            b'\x00\x08\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xbe\xba\xfe\xca\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xbe\xba\xfe\xca\x00\x00\x00\x00"\xbb\xfe\xca\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00'
+        """
+        self.flags &=~8
+        self.flags |=0x800
+        self._IO_write_base = addr
+        self._IO_write_ptr = addr+size
+        self._IO_read_end = addr
+        self.fileno = 1
+        return self.struntil('fileno')
+
+    def read(self,addr=0,size=0):
+        r"""
+        Reading data into arbitrary memory location.
+
+        Arguments:
+            addr(int)
+                The address into which data is to be written from stdin
+            size(int)
+                The size, in bytes, of the data to be written
+
+        Example:
+
+            Payload for reading 100 bytes from stdin into the address 0xcafebabe
+
+            >>> context.clear(arch='amd64')
+            >>> fileStr = FileStructure(0xdeadbeef)
+            >>> payload = fileStr.read(addr=0xcafebabe, size=100)
+            >>> payload
+            b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xbe\xba\xfe\xca\x00\x00\x00\x00"\xbb\xfe\xca\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+        """
+        self.flags &=~4
+        self._IO_read_base = 0
+        self._IO_read_ptr = 0
+        self._IO_buf_base = addr
+        self._IO_buf_end = addr+size
+        self.fileno = 0
+        return self.struntil('fileno')
+
+    def orange(self,io_list_all,vtable):
+        r"""
+        Perform a House of Orange (https://github.com/shellphish/how2heap/blob/master/glibc_2.25/house_of_orange.c), provided you have libc leaks.
+
+        Arguments:
+            io_list_all(int)
+                Address of _IO_list_all in libc.
+            vtable(int)
+                Address of the fake vtable in memory
+
+        Example:
+
+            Example payload if address of _IO_list_all is 0xfacef00d and fake vtable is at 0xcafebabe -
+
+            >>> context.clear(arch='amd64')
+            >>> fileStr = FileStructure(0xdeadbeef)
+            >>> payload = fileStr.orange(io_list_all=0xfacef00d, vtable=0xcafebabe)
+            >>> payload
+            b'/bin/sh\x00a\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xfd\xef\xce\xfa\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\xef\xbe\xad\xde\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\xef\xbe\xad\xde\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xbe\xba\xfe\xca\x00\x00\x00\x00'
+        """
+        if context.bits == 64:
+            self.flags = b'/bin/sh\x00'
+            self._IO_read_ptr = 0x61
+            self._IO_read_base = io_list_all-0x10
+        elif context.bits == 32:
+            self.flags = b'sh\x00'
+            self._IO_read_ptr = 0x121
+            self._IO_read_base = io_list_all-0x8
+        self._IO_write_base = 0
+        self._IO_write_ptr = 1
+        self.vtable = vtable
+        return self.__bytes__()
@@ -50,7 +50,7 @@ def submit_flag(flag,
        >>> _ = submit_flag('flag', server='localhost', port=l.lport)
        >>> c = l.wait_for_connection()
        >>> c.recvall().split()
-        ['flag', 'unnamed-exploit', 'unknown-target', 'unknown-team']
+        [b'flag', b'unnamed-exploit', b'unknown-target', b'unknown-team']
    """
    flag = flag.strip()

@@ -31,8 +31,9 @@ Examples:
    ...        write(1, my_var, sizeof(int));
    ...        return 0;
    ... }''')
-    >>> cmdline = ["gcc", source, "-Wno-format-security", "-m32", "-o", program]
-    >>> process(cmdline).wait_for_close()
+    >>> gcc = process(["gcc", source, "-Wno-format-security", "-m32", "-o", program])
+    >>> gcc.poll(True) and gcc.recvall()
+    0
    >>> def exec_fmt(payload):
    ...     p = process(program)
    ...     p.sendline(payload)
@@ -136,12 +136,10 @@ def debug_assembly(asm, gdbscript=None, vma=None):

    Example:

-        .. code-block:: python
-
-            assembly = shellcraft.echo("Hello world!\n")
-            io = gdb.debug_assembly(assembly)
-            io.recvline()
-            # 'Hello world!'
+    >>> assembly = shellcraft.echo("Hello world!\n")
+    >>> io = gdb.debug_assembly(assembly)
+    >>> io.recvline()
+    b'Hello world!\n'
    """
    tmp_elf = make_elf_from_assembly(asm, vma=vma, extract=False)
    os.chmod(tmp_elf, 0o777)
@@ -157,7 +155,7 @@ def debug_assembly(asm, gdbscript=None, vma=None):

@LocalContext
 def debug_shellcode(data, gdbscript=None, vma=None):
-    r"""
+    r"""debug_shellcode(data, gdbscript=None, vma=None) -> tube
    Creates an ELF file, and launches it under a debugger.

    Arguments:
@@ -171,13 +169,11 @@ def debug_shellcode(data, gdbscript=None, vma=None):

    Example:

-        .. code-block:: python
-
-            assembly = shellcraft.echo("Hello world!\n")
-            shellcode = asm(assembly)
-            io = gdb.debug_shellcode(shellcode)
-            io.recvline()
-            # 'Hello world!'
+    >>> assembly = shellcraft.echo("Hello world!\n")
+    >>> shellcode = asm(assembly)
+    >>> io = gdb.debug_shellcode(shellcode)
+    >>> io.recvline()
+    b'Hello world!\n'
    """
    if isinstance(data, six.text_type):
        log.error("Shellcode is cannot be unicode.  Did you mean debug_assembly?")
@@ -193,8 +189,8 @@ def debug_shellcode(data, gdbscript=None, vma=None):

    return debug(tmp_elf, gdbscript=gdbscript, arch=context.arch)

-def _gdbserver_args(pid=None, path=None, args=None, which=None):
-    """_gdbserver_args(pid=None, path=None) -> list
+def _gdbserver_args(pid=None, path=None, args=None, which=None, env=None):
+    """_gdbserver_args(pid=None, path=None, args=None, which=None, env=None) -> list

    Sets up a listening gdbserver, to either connect to the specified
    PID, or launch the specified binary by its full path.
@@ -240,6 +236,14 @@ def _gdbserver_args(pid=None, path=None, args=None, which=None):
    if pid:
        gdbserver_args += ['--once', '--attach']

+    if env:
+        env_args = []
+        for key in tuple(env):
+            if key.startswith('LD_'): # LD_PRELOAD / LD_LIBRARY_PATH etc.
+                env_args.append('{}={}'.format(key, env.pop(key)))
+        if env_args:
+            gdbserver_args += ['--wrapper', 'env'] + env_args + ['--']
+
    gdbserver_args += ['localhost:0']
    gdbserver_args += args

@@ -296,7 +300,7 @@ def _get_runner(ssh=None):

@LocalContext
 def debug(args, gdbscript=None, exe=None, ssh=None, env=None, sysroot=None, **kwargs):
-    """debug(args) -> tube
+    r"""debug(args) -> tube

    Launch a GDB server with the specified command line,
    and launches GDB to attach to it.
@@ -345,68 +349,58 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, sysroot=None, **kw

    Examples:

-        .. code-block:: python
+    >>> # Create a new process, and stop it at 'main'
+    >>> io = gdb.debug('bash', '''
+    ... break main
+    ... continue
+    ... ''')
+    >>> io.sendline("echo hello")
+    >>> io.recvline()
+    b'hello\n'
+    >>> io.interactive() # doctest: +SKIP
+    >>> io.close()

-            # Create a new process, and stop it at 'main'
-            io = gdb.debug('bash', '''
-            break main
-            continue
-            ''')
+    >>> # Create a new process, and stop it at '_start'
+    >>> io = gdb.debug('bash', '''
+    ... # Wait until we hit the main executable's entry point
+    ... break _start
+    ... continue
+    ...
+    ... # Now set breakpoint on shared library routines
+    ... break malloc
+    ... break free
+    ... continue
+    ... ''')
+    >>> io.sendline("echo hello")
+    >>> io.recvline()
+    b'hello\n'
+    >>> io.interactive() # doctest: +SKIP
+    >>> io.close()

-            # Send a command to Bash
-            io.sendline("echo hello")
+    You can use :func:`debug` to spawn new processes on remote machines as well,
+    by using the ``ssh=`` keyword to pass in your :class:`.ssh` instance.

-            # Interact with the process
-            io.interactive()
+    >>> # Connect to the SSH server

-        .. code-block:: python
+    >>> # Start a process on the server
+    >>> shell = ssh('runner', 'example.pwnme', password='demopass')
+    >>> io = gdb.debug(['bash'],
+    ...                 ssh = shell,
+    ...                 gdbscript = '''
+    ... break main
+    ... continue
+    ... ''')

-            # Create a new process, and stop it at 'main'
-            io = gdb.debug('bash', '''
-            # Wait until we hit the main executable's entry point
-            break _start
-            continue
+    >>> # Send a command to Bash
+    >>> io.sendline("echo hello")

-            # Now set breakpoint on shared library routines
-            break malloc
-            break free
-            continue
-            ''')
-
-            # Send a command to Bash
-            io.sendline("echo hello")
-
-            # Interact with the process
-            io.interactive()
-
-        You can use :func:`debug` to spawn new processes on remote machines as well,
-        by using the ``ssh=`` keyword to pass in your :class:`.ssh` instance.
-
-        .. code-block:: python
-
-            # Connect to the SSH server
-            shell = ssh('passcode', 'pwnable.kr', 2222, password='guest')
-
-            # Start a process on the server
-            io = gdb.debug(['bash'],
-                            ssh=shell,
-                            gdbscript='''
-            break main
-            continue
-            ''')
-
-            # Send a command to Bash
-            io.sendline("echo hello")
-
-            # Interact with the process
-            io.interactive()
+    >>> # Interact with the process
+    >>> io.interactive() # doctest: +SKIP
+    >>> io.close()
    """
    if isinstance(args, (int, tubes.process.process, tubes.ssh.ssh_channel)):
        log.error("Use gdb.attach() to debug a running process")

-    if env is None:
-        env = os.environ
-
    if isinstance(args, (bytes, six.text_type)):
        args = [args]

@@ -421,14 +415,17 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, sysroot=None, **kw
        return runner(args, executable=exe, env=env)

    if ssh or context.native or (context.os == 'android'):
-        args = _gdbserver_args(args=args, which=which)
+        args = _gdbserver_args(args=args, which=which, env=env)
    else:
        qemu_port = random.randint(1024, 65535)
        qemu_user = qemu.user_path()
        sysroot = sysroot or qemu.ld_prefix(env=env)
        if not qemu_user:
            log.error("Cannot debug %s binaries without appropriate QEMU binaries" % context.arch)
-        args = [qemu_user, '-g', str(qemu_port)] + args
+        qemu_args = [qemu_user, '-g', str(qemu_port)]
+        if sysroot:
+            qemu_args += ['-L', sysroot]
+        args = qemu_args + args

    # Use a sane default sysroot for Android
    if not sysroot and context.os == 'android':
@@ -438,18 +435,17 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, sysroot=None, **kw
    if not which(args[0]):
        log.error("%s is not installed" % args[0])

-    exe = exe or which(orig_args[0])
-    if not exe:
-        log.error("%s does not exist" % orig_args[0])
-    else:
-        gdbscript = 'file "%s"\n%s' % (exe, gdbscript)
+    if not ssh:
+        exe = exe or which(orig_args[0])
+        if not (exe and os.path.exists(exe)):
+            log.error("%s does not exist" % exe)

    # Start gdbserver/qemu
    # (Note: We override ASLR here for the gdbserver process itself.)
    gdbserver = runner(args, env=env, aslr=1, **kwargs)

    # Set the .executable on the process object.
-    gdbserver.executable = which(orig_args[0])
+    gdbserver.executable = exe

    # Find what port we need to connect to
    if context.native or (context.os == 'android'):
@@ -465,7 +461,7 @@ def debug(args, gdbscript=None, exe=None, ssh=None, env=None, sysroot=None, **kw

    # gdbserver outputs a message when a client connects
    garbage = gdbserver.recvline(timeout=1)
-    
+
    # Some versions of gdbserver output an additional message
    garbage2 = gdbserver.recvline_startswith(b"Remote debugging from host ", timeout=1)

@@ -509,8 +505,8 @@ def binary():
    return gdb

@LocalContext
-def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_args = None, ssh = None, sysroot = None):
-    """attach(target, gdbscript = None, exe = None, arch = None, ssh = None) -> None
+def attach(target, gdbscript = '', exe = None, need_ptrace_scope = True, gdb_args = None, ssh = None, sysroot = None):
+    r"""attach(target, gdbscript = None, exe = None, arch = None, ssh = None) -> None

    Start GDB in a new terminal and attach to `target`.

@@ -549,64 +545,68 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a

    Examples:

-        .. code-block:: python
+    >>> # Attach directly to pid 1234
+    >>> gdb.attach(1234) # doctest: +SKIP

-            # Attach directly to pid 1234
-            gdb.attach(1234)

-        .. code-block:: python
+    >>> # Attach to the youngest "bash" process
+    >>> gdb.attach('bash') # doctest: +SKIP

-            # Attach to the youngest "bash" process
-            gdb.attach('bash')

-        .. code-block:: python
+    >>> # Start a process
+    >>> bash = process('bash')

-            # Start a process
-            bash = process('bash')
+    >>> # Attach the debugger
+    >>> pid = gdb.attach(bash, '''
+    ... set follow-fork-mode child
+    ... break execve
+    ... continue
+    ... ''')

-            # Attach the debugger
-            gdb.attach(bash, '''
-            set follow-fork-mode child
-            break execve
-            continue
-            ''')
+    >>> # Interact with the process
+    >>> bash.sendline("whoami")
+    >>> bash.recvline()
+    b'runner\n'
+    >>> bash.close()

-            # Interact with the process
-            bash.sendline('whoami')
+    >>> # Start a forking server
+    >>> server = process(['socat', 'tcp-listen:12345,fork,reuseaddr', 'exec:/bin/bash,nofork'])
+    >>> sleep(1)

-        .. code-block:: python
+    >>> # Connect to the server
+    >>> io = remote('127.0.0.1', 12345)

-            # Start a forking server
-            server = process(['socat', 'tcp-listen:1234,fork,reuseaddr', 'exec:/bin/sh'])
+    >>> # Connect the debugger to the server-spawned process
+    >>> pid = gdb.attach(io, '''
+    ... break exit
+    ... continue
+    ... ''', exe = '/bin/bash')

-            # Connect to the server
-            io = remote('localhost', 1234)
+    >>> # Talk to the spawned 'sh'
+    >>> io.sendline("echo hello")
+    >>> io.recvline()
+    b'hello\n'
+    >>> io.sendline("exit")

-            # Connect the debugger to the server-spawned process
-            gdb.attach(io, '''
-            break exit
-            continue
-            ''')
+    >>> io.close()

-            # Talk to the spawned 'sh'
-            io.sendline('exit')
+    >>> # Connect to the SSH server
+    >>> shell = ssh('runner', 'example.pwnme', password='demopass')

-        .. code-block:: python
+    >>> # Start a process on the server
+    >>> cat = shell.process(['cat'])

-            # Connect to the SSH server
-            shell = ssh('bandit0', 'bandit.labs.overthewire.org', password='bandit0', port=2220)
+    >>> # Attach a debugger to it
+    >>> gdb.attach(cat, '''
+    ... break exit
+    ... continue
+    ... ''')

-            # Start a process on the server
-            cat = shell.process(['cat'])
-
-            # Attach a debugger to it
-            gdb.attach(cat, '''
-            break exit
-            continue
-            ''')
-
-            # Cause `cat` to exit
-            cat.close()
+    >>> cat.sendline("hello")
+    >>> cat.recvline()
+    b'hello\n'
+    >>> # Cause `cat` to exit
+    >>> cat.close()
    """
    if context.noptrace:
        log.warn_once("Skipping debug attach since context.noptrace==True")
@@ -662,8 +662,10 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
        shell = target.parent

        tmpfile = shell.mktemp()
+        if six.PY3:
+            tmpfile = tmpfile.decode()
        gdbscript = 'shell rm %s\n%s' % (tmpfile, gdbscript)
-        shell.upload_data(context._encode(gdbscript), tmpfile)
+        shell.upload_data(gdbscript or '', tmpfile)

        cmd = ['ssh', '-C', '-t', '-p', str(shell.port), '-l', shell.user, shell.host]
        if shell.password:
@@ -672,7 +674,10 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
            cmd = ['sshpass', '-p', shell.password] + cmd
        if shell.keyfile:
            cmd += ['-i', shell.keyfile]
-        cmd += ['gdb -q %r %s -x "%s"' % (target.executable,
+        exefile = target.executable
+        if six.PY3:
+            exefile = exefile.decode()
+        cmd += ['gdb -q %s %s -x "%s"' % (exefile,
                                       target.pid,
                                       tmpfile)]

@@ -684,7 +689,15 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
        if not pids:
            log.error('could not find remote process (%s:%d) on this machine' %
                      target.sock.getpeername())
-        pid = pids[0]
+        waiting = True
+        if exe:
+            while waiting:
+                for pid in pids:
+                    if proc.exe(pid) == exe:
+                        waiting = False
+                        break
+                else:
+                    time.sleep(0.01)
    elif isinstance(target, tubes.process.process):
        pid = proc.pidof(target)[0]
        exe = exe or target.executable
@@ -722,13 +735,9 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
            exe_fn = adb.proc_exe
        exe = exe_fn(pid)

-    if not pid and not exe:
+    if not pid and not exe and not ssh:
        log.error('could not find target process')

-    if exe:
-        # The 'file' statement should go first
-        pre = 'file "%s"\n%s' % (exe, pre)
-
    cmd = binary()

    if gdb_args:
@@ -752,7 +761,7 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
    if context.os == 'android' and pid:
        runner  = _get_runner()
        which   = _get_which()
-        gdb_cmd = _gdbserver_args(pid=pid, which=which)
+        gdb_cmd = _gdbserver_args(pid=pid, which=which, env=env)
        gdbserver = runner(gdb_cmd)
        port    = _gdbserver_port(gdbserver, None)
        host    = context.adb_host
@@ -779,7 +788,7 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
    gdb_pid = misc.run_in_new_terminal(cmd)

    if pid and context.native:
-        proc.wait_for_debugger(pid)
+        proc.wait_for_debugger(pid, gdb_pid)

    return gdb_pid

@@ -856,10 +865,10 @@ def find_module_addresses(binary, ssh=None, ulimit=False):

    Example:

-    >>> with context.local(log_level=9999): # doctest: +SKIP
-    ...     shell = ssh(host='bandit.labs.overthewire.org',user='bandit0',password='bandit0', port=2220)
+    >>> with context.local(log_level=9999):
+    ...     shell =  ssh(host='example.pwnme', user='runner', password='demopass')
    ...     bash_libs = gdb.find_module_addresses('/bin/bash', shell)
-    >>> os.path.basename(bash_libs[0].path) # doctest: +SKIP
+    >>> os.path.basename(bash_libs[0].path)
    'libc.so.6'
    >>> hex(bash_libs[0].symbols['system']) # doctest: +SKIP
    '0x7ffff7634660'
@@ -878,31 +887,29 @@ def find_module_addresses(binary, ssh=None, ulimit=False):
        local_elf  = elf.ELF(binary)
        local_libs = local_elf.libs

-    entry      = local_elf.header.e_entry
-
    #
    # Get the addresses from GDB
    #
    libs = {}
-    cmd  = "gdb -q --args %s" % (binary)
+    cmd  = "gdb -q -nh --args %s | cat" % (binary) # pipe through cat to disable colored output on GDB 9+
    expr = re.compile(r'(0x\S+)[^/]+(.*)')

    if ulimit:
-        cmd = 'sh -c "(ulimit -s unlimited; %s)"' % cmd
-
-    cmd = shlex.split(cmd)
+        cmd = ['sh', '-c', "(ulimit -s unlimited; %s)" % cmd]
+    else:
+        cmd = ['sh', '-c', cmd]

    with runner(cmd) as gdb:
        if context.aslr:
            gdb.sendline('set disable-randomization off')
+
        gdb.send("""
        set prompt
-        break *%#x
+        catch load
        run
-        """ % entry)
-        gdb.clean(2)
+        """)
        gdb.sendline('info sharedlibrary')
-        lines = gdb.recvrepeat(2)
+        lines = context._decode(gdb.recvrepeat(2))

        for line in lines.splitlines():
            m = expr.match(line)
@@ -992,7 +999,7 @@ def version(program='gdb'):

    Example:

-        >>> (7,0) <= gdb.version() <= (8,0)
+        >>> (7,0) <= gdb.version() <= (10,0)
        True
    """
    program = misc.which(program)
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+from pwnlib.tubes.process import process
+from time import sleep
+from sys import argv
+sleep(1)
+sh = process(argv[1], shell=True)
+sh.sendline('set prompt (gdb)')
+res = sh.sendlineafter('(gdb)', 'c')
+while b'The program is not being run.' not in res:
+    res = sh.sendlineafter('(gdb)', 'c')
+sh.close()
@@ -56,7 +56,7 @@ def search_by_hash(hex_encoded_id, hash_type='build_id'):
    data   = b""
    while not data.startswith(b'\x7fELF'):
        log.debug("Downloading data from LibcDB: %s", url)
-        data = wget(url)
+        data = wget(url, timeout=20)

        if not data:
            log.warn_once("Could not fetch libc for build_id %s", hex_encoded_id)
@@ -197,7 +197,7 @@ def get_build_id_offsets():
    # $ check_arch "x86-64"
    #       6 Displaying notes found at file offset 0x00000174 with length 0x00000024:
    #      82 Displaying notes found at file offset 0x00000270 with length 0x00000024:
-        'amd64': [0x270, 0x174],
+        'amd64': [0x270, 0x174, 0x2e0],
    # $ check_arch "PowerPC or cisco"
    #      88 Displaying notes found at file offset 0x00000174 with length 0x00000024:
        'powerpc': [0x174],
@@ -107,6 +107,7 @@ class AdbClient(Logger):
                and self.port == context.defaults['adb_port']:
                    log.warn("Could not connect to ADB server, trying to start it")
                    process(context.adb + ['start-server']).recvall()
+                    time.sleep(0.3)
                else:
                    log.exception('Could not connect to ADB server (%s:%s)' % \
                                    (self.host, self.port))
@@ -168,6 +169,7 @@ class AdbClient(Logger):

        >>> c.version() > (4,0)
        True
+        >>> c.wait_for_device() # ensure doctests alive
        """
        try:
            self.send('host:kill')
@@ -223,8 +225,8 @@ class AdbClient(Logger):
                l = l.decode('utf-8')
            yield l

-    def transport(self, serial=None):
-        """Sets the Transport on the rmeote device.
+    def transport(self, serial=None, try_again=True):
+        """Sets the Transport on the remote device.

        Examples:

@@ -244,10 +246,14 @@ class AdbClient(Logger):
            msg = 'host:transport-any'

        if self.send(msg) == FAIL:
+            err = self.recvl().decode('utf-8')
+            if err == 'device offline' and try_again:
+                self.wait_for_device(serial)
+                return self.transport(serial, try_again=False)
            if serial:
-                self.error("Could not set transport to %r" % serial)
+                self.error("Could not set transport to %r (%s)" % (serial, err))
            else:
-                self.error("Could not set transport 'any'")
+                self.error("Could not set transport 'any' (%s)" % err)

    @_autoclose
    @_with_transport
@@ -289,7 +295,9 @@ class AdbClient(Logger):
    @_with_transport
    def root(self):
        self.send('root:')
-        return self.c.recvall().decode('utf-8')
+        rv = self.c.recvall().decode('utf-8')
+        time.sleep(0.1)
+        return rv

    @_autoclose
    @_with_transport
@@ -336,22 +344,25 @@ class AdbClient(Logger):

        # The first OKAY is that the command was understood
        if response != OKAY:
-            self.error("An error occurred while waiting for device with serial %r" % serial)
+            if response == FAIL:
+                response = self.recvl().decode('utf-8')
+            self.error("An error occurred while trying to wait for device with serial %r (%r)" % (serial, response))

        # The second OKAY is that the device is available
        response = self.c.recvn(4)
        if response != OKAY:
-            self.error("An error occurred while waiting for device with serial %r" % serial)
+            if response == FAIL:
+                response = self.recvl().decode('utf-8')
+            self.error("An error occurred while waiting for device with serial %r (%r)" % (serial, response))

    def _sync(fn):
        """Decorator which enters 'sync:' mode to the selected transport,
        then invokes the decorated funciton."""
        @functools.wraps(fn)
        def wrapper(self, *a, **kw):
-            rv = None
-            if FAIL != self.send('sync:'):
-                rv = fn(self, *a, **kw)
-            return rv
+            if self.send('sync:') == FAIL:
+                self.error("An error occurred while trying to use SYNC API (%r)" % self.recvl().decode('utf-8'))
+            return fn(self, *a, **kw)
        return wrapper

    def list(self, path):
@@ -381,8 +392,10 @@ class AdbClient(Logger):

        Examples:

+            >>> _ = AdbClient().root()
+            >>> AdbClient().wait_for_device()
            >>> pprint(AdbClient().list('/data/user'))
-            {'0': {'mode': 41471, 'size': 11, 'time': ...}}
+            {'0': {'mode': 41471, 'size': 10, 'time': ...}}
            >>> AdbClient().list('/does/not/exist')
            Traceback (most recent call last):
            ...
@@ -277,6 +277,7 @@ import hashlib
 import itertools
 import os
 import re
+import shutil
 import six
 import string
 import sys
@@ -880,7 +881,7 @@ class ROP(object):


    def find_stack_adjustment(self, slots):
-        self.search(move=slots * context.arch)
+        self.search(move=slots * context.bytes)

    def chain(self):
        """Build the ROP chain
@@ -1060,13 +1061,19 @@ class ROP(object):

        return os.path.join(cachedir, sha256.hexdigest())

+    @staticmethod
+    def clear_cache():
+        """Clears the ROP gadget cache"""
+        cachedir = os.path.join(tempfile.gettempdir(), 'pwntools-rop-cache')
+        shutil.rmtree(cachedir)
+
    def __cache_load(self, elf):
        filename = self.__get_cachefile_name(elf)
        if not os.path.exists(filename):
            return None
-        log.info_once('Loaded cached gadgets for %r' % elf.file.name)
        gadgets = eval(open(filename).read())
        gadgets = {k - elf.load_addr + elf.address:v for k, v in gadgets.items()}
+        log.info_once('Loaded %s cached gadgets for %r', len(gadgets), elf.file.name)
        return gadgets

    def __cache_save(self, elf, data):
@@ -17,18 +17,19 @@ Example:
 The following doctest is commented out because it doesn't work on Travis
 for reasons I cannot diagnose.  However, it should work just fine :-)

-    # >>> gcc = process(['aarch64-linux-gnu-gcc','-xc','-static','-Wl,-Ttext-segment=0x20000000','-'])
-    # >>> gcc.write(b'''
-    # ... int main() {
-    # ...     printf("Hello, %s!\\n", "world");
-    # ... }
-    # ... ''')
-    # >>> gcc.shutdown('send')
-    # >>> gcc.poll(True)
-    # 0
-    # >>> sc = shellcraft.loader_append('a.out')
-    # >>> run_assembly(sc).recvline()
-    # 'Hello, world!\n'
+    >>> gcc = process(['aarch64-linux-gnu-gcc','-xc','-static','-Wl,-Ttext-segment=0x20000000','-'])
+    >>> gcc.write(b'''
+    ... #include <stdio.h>
+    ... int main() {
+    ...     printf("Hello, %s!\\n", "world");
+    ... }
+    ... ''')
+    >>> gcc.shutdown('send')
+    >>> gcc.poll(True) and gcc.recvall()
+    0
+    >>> sc = shellcraft.loader_append('a.out')
+    >>> run_assembly(sc).recvline()
+    b'Hello, world!\n'

 </%docstring>
 <%page args="data = None"/>
@@ -0,0 +1,19 @@
+<% from pwnlib.shellcraft import aarch64, pretty, common %>
+<%docstring>Copies memory.
+
+Args:
+    dest: Destination address
+    src: Source address
+    n: Number of bytes
+</%docstring>
+<%page args="dest, src, n"/>
+<%
+memcpy_loop = common.label("memcpy_loop")
+%>
+    /* memcpy(${pretty(dest)}, ${pretty(src)}, ${pretty(n)}) */
+    ${aarch64.setregs({'x0': dest, 'x1': src, 'x2': n})}
+${memcpy_loop}:
+    ldrb w3, [x1], #1
+    strb w3, [x0], #1
+    subs x2, x2, #1
+    bge ${memcpy_loop}
@@ -16,20 +16,21 @@ Example:

    >>> gcc = process(['gcc','-m64','-xc','-static','-Wl,-Ttext-segment=0x20000000','-'])
    >>> gcc.write(b'''
+    ... #include <stdio.h>
    ... int main() {
    ...     printf("Hello, %s!\\n", "amd64");
    ... }
    ... ''')
    >>> gcc.shutdown('send')
-    >>> gcc.poll(True)
+    >>> gcc.poll(True) and gcc.recvall()
    0
    >>> sc = shellcraft.loader_append('a.out')

 The following doctest is commented out because it doesn't work on Travis
 for reasons I cannot diagnose.  However, it should work just fine :-)

-    # >>> run_assembly(sc).recvline() == b'Hello, amd64!\n'
-    # True
+    >>> run_assembly(sc).recvline() == b'Hello, amd64!\n'
+    True

 </%docstring>
 <%page args="data = None"/>
@@ -104,6 +104,9 @@ PT_LOAD  = 1
    /* AT_RANDOM */
    push esp
    push 25
+    /* AT_SECURE */
+    push ebx
+    push 23

    push ebx /* envp */
    push ebx /* argv */
@@ -16,20 +16,21 @@ Example:

    >>> gcc = process(['gcc','-m32','-xc','-static','-Wl,-Ttext-segment=0x20000000','-'])
    >>> gcc.write(b'''
+    ... #include <stdio.h>
    ... int main() {
    ...     printf("Hello, %s!\\n", "i386");
    ... }
    ... ''')
    >>> gcc.shutdown('send')
-    >>> gcc.poll(True)
+    >>> gcc.poll(True) and gcc.recvall()
    0
    >>> sc = shellcraft.loader_append('a.out')

 The following doctest is commented out because it doesn't work on Travis
 for reasons I cannot diagnose.  However, it should work just fine :-)

-    # >>> run_assembly(sc).recvline() == b'Hello, i386!\n'
-    # True
+    >>> run_assembly(sc).recvline()
+    b'Hello, i386!\n'

 </%docstring>
 <%page args="data = None"/>
@@ -9,4 +9,4 @@ Example:
 </%docstring>
    pop {r0-r12,lr}
    ldr sp, [sp, 64]
-    bx  r1
+    pop {pc}
@@ -17,18 +17,19 @@ Example:
 The following doctest is commented out because it doesn't work on Travis
 for reasons I cannot diagnose.  However, it should work just fine :-)

-    # >>> gcc = process(['arm-linux-gnueabihf-gcc','-xc','-static','-Wl,-Ttext-segment=0x20000000','-'])
-    # >>> gcc.write(b'''
-    # ... int main() {
-    # ...     printf("Hello, %s!\\n", "world");
-    # ... }
-    # ... ''')
-    # >>> gcc.shutdown('send')
-    # >>> gcc.poll(True)
-    # 0
-    # >>> sc = shellcraft.loader_append('a.out')
-    # >>> run_assembly(sc).recvline()
-    # 'Hello, world!\n'
+    >>> gcc = process(['arm-linux-gnueabihf-gcc','-xc','-static','-Wl,-Ttext-segment=0x20000000','-'])
+    >>> gcc.write(b'''
+    ... #include <stdio.h>
+    ... int main() {
+    ...     printf("Hello, %s!\\n", "world");
+    ... }
+    ... ''')
+    >>> gcc.shutdown('send')
+    >>> gcc.poll(True) and gcc.recvall()
+    0
+    >>> sc = shellcraft.loader_append('a.out')
+    >>> run_assembly(sc).recvline()
+    b'Hello, world!\n'

 </%docstring>
 <%page args="data = None"/>
@@ -378,7 +378,7 @@ def readline(_size=None, prompt='', float=True, priority=10):
    # XXX circular imports
    from pwnlib.term import term_mode
    if not term_mode:
-        print(prompt, end='', flush=True)
+        six.print_(prompt, end='', flush=True)
        return sys.stdin.readline(_size).rstrip('\n')
    show_suggestions = False
    eof = False
@@ -435,6 +435,19 @@ def raw_input(prompt='', float=True):
    """
    return readline(None, prompt, float)

+def str_input(prompt='', float=True):
+    r"""str_input(prompt='', float=True)
+
+    Replacement for the built-in ``input`` in python3 using ``pwnlib`` readline
+    implementation.
+
+    Arguments:
+        prompt(str): The prompt to show to the user.
+        float(bool): If set to `True`, prompt and input will float to the
+                     bottom of the screen when `term.term_mode` is enabled.
+    """
+    return readline(None, prompt, float).decode()
+
 def eval_input(prompt='', float=True):
    """eval_input(prompt='', float=True)

@@ -450,11 +463,12 @@ def eval_input(prompt='', float=True):
    Example:

        >>> try:
-        ...     saved_stdin = sys.stdin
-        ...     sys.stdin = io.StringIO("{'a':20}")
+        ...     saved = sys.stdin, pwnlib.term.term_mode
+        ...     pwnlib.term.term_mode = False
+        ...     sys.stdin = io.StringIO(u"{'a': 20}")
        ...     eval_input("Favorite object? ")['a']
        ... finally:
-        ...     sys.stdin = saved_stdin
+        ...     sys.stdin, pwnlib.term.term_mode = saved
        Favorite object? 20
    """
    from pwnlib.util import safeeval
@@ -480,4 +494,4 @@ def init():
        builtins.raw_input = raw_input
        builtins.input = eval_input
    else:
-        builtins.input = raw_input
+        builtins.input = str_input
@@ -460,11 +460,13 @@ def render_from(i, force = False, clear_after = False):
 def redraw():
    for i in reversed(range(len(cells))):
        row = cells[i].start[0]
-        if row - scroll + height - 1 < 0:
+        if row - scroll + height <= 0:
+            # XXX: remove this line when render_cell is fixed
+            i += 1
            break
-    # XXX: remove this line when render_cell is fixed
-    if cells[i].start[0] - scroll + height <= 0:
-        i += 1
+    else:
+        if not cells:
+            return
    render_from(i, force = True, clear_after = True)

 lock = threading.Lock()
@@ -1,4 +1,5 @@
 from __future__ import division
+from __future__ import print_function

 __all__ = ['get']
 import curses
@@ -48,7 +49,8 @@ def init():
        # Fix for BPython
        try:
            curses.setupterm()
-        except:
-            pass
+        except curses.error as e:
+            import traceback
+            print('Warning:', ''.join(traceback.format_exception_only(e.__class__, e)), file=sys.stderr)

    cache = {}
@@ -242,10 +242,6 @@ class process(tube):
        if not shell:
            executable, argv, env = self._validate(cwd, executable, argv, env)

-        # Permit invocation as process('sh') and process(['sh'])
-        if isinstance(argv, (bytes, six.text_type)):
-            argv = [argv]
-
        # Avoid the need to have to deal with the STDOUT magic value.
        if stderr is STDOUT:
            stderr = stdout
@@ -308,13 +304,15 @@ class process(tube):
            # and binfmt is not installed (e.g. when running on
            # Travis CI), re-try with qemu-XXX if we get an
            # 'Exec format error'.
-            prefixes = [([], executable)]
-            executables = [executable]
+            prefixes = [([], self.executable)]
            exception = None

            for prefix, executable in prefixes:
                try:
-                    self.proc = subprocess.Popen(args = prefix + argv,
+                    args = argv
+                    if prefix:
+                        args = prefix + args
+                    self.proc = subprocess.Popen(args = args,
                                                 shell = shell,
                                                 executable = executable,
                                                 cwd = cwd,
@@ -509,11 +507,14 @@ class process(tube):
        # - Must be a list/tuple of strings
        # - Each string must not contain '\x00'
        #
-        if isinstance(argv, (bytes, six.text_type)):
+        if isinstance(argv, (six.text_type, six.binary_type)):
            argv = [argv]

-        if not all(isinstance(arg, (bytes, six.text_type)) for arg in argv):
-            self.error("argv must be strings: %r" % argv)
+        if not isinstance(argv, (list, tuple)):
+            self.error('argv must be a list or tuple: %r' % argv)
+
+        if not all(isinstance(arg, (six.text_type, six.binary_type)) for arg in argv):
+            self.error("argv must be strings or bytes: %r" % argv)

        # Create a duplicate so we can modify it
        argv = list(argv or [])
@@ -950,7 +951,7 @@ class process(tube):

        Example:

-            >>> e = ELF('/bin/bash')
+            >>> e = ELF('/bin/bash-static')
            >>> p = process(e.path)

            In order to make sure there's not a race condition against
@@ -21,7 +21,6 @@ from pwnlib.log import Logger
 from pwnlib.log import getLogger
 from pwnlib.term import text
 from pwnlib.timeout import Timeout
-from pwnlib.tubes.process import process
 from pwnlib.tubes.sock import sock
 from pwnlib.util import hashes
 from pwnlib.util import misc
@@ -172,7 +171,8 @@ class ssh_channel(sock):

        return data

-    def wait(self):
+    def wait(self, timeout=sock.default):
+        # TODO: deal with timeouts
        return self.poll(block=True)

    def poll(self, block=False):
@@ -536,9 +536,9 @@ class ssh(Timeout, Logger):
    #: PID of the remote ``sshd`` process servicing this connection.
    pid = None

-    def __init__(self, user, host, port = 22, password = None, key = None,
-                 keyfile = None, proxy_command = None, proxy_sock = None,
-                 level = None, cache = True, ssh_agent = False, *a, **kw):
+    def __init__(self, user=None, host=None, port=22, password=None, key=None,
+                 keyfile=None, proxy_command=None, proxy_sock=None,
+                 level=None, cache=True, ssh_agent=False, *a, **kw):
        """Creates a new ssh connection.

        Arguments:
@@ -561,11 +561,11 @@ class ssh(Timeout, Logger):
        Example proxying:

            >>> s1 = ssh(host='example.pwnme',
-            ...          user='travis',
+            ...          user='runner',
            ...          password='demopass')
            >>> r1 = s1.remote('localhost', 22)
            >>> s2 = ssh(host='example.pwnme',
-            ...          user='travis',
+            ...          user='runner',
            ...          password='demopass',
            ...          proxy_sock=r1.sock)
            >>> r2 = s2.remote('localhost', 22) # and so on...
@@ -702,7 +702,7 @@ class ssh(Timeout, Logger):

        Examples:
            >>> s =  ssh(host='example.pwnme',
-            ...         user='travis',
+            ...         user='runner',
            ...         password='demopass')
            >>> sh = s.shell('/bin/sh')
            >>> sh.sendline(b'echo Hello; exit')
@@ -781,7 +781,7 @@ class ssh(Timeout, Logger):

        Examples:
            >>> s = ssh(host='example.pwnme',
-            ...         user='travis',
+            ...         user='runner',
            ...         password='demopass')
            >>> sh = s.process('/bin/sh', env={'PS1':''})
            >>> sh.sendline(b'echo Hello; exit')
@@ -796,14 +796,14 @@ class ssh(Timeout, Logger):
            >>> s.process(['LOLOLOL\x00', '/proc/self/cmdline'], executable='cat').recvall()
            b'LOLOLOL\x00/proc/self/cmdline\x00'
            >>> sh = s.process(executable='/bin/sh')
-            >>> sh.pid in pidof('sh') # doctest: +SKIP
+            >>> str(sh.pid).encode() in s.pidof('sh') # doctest: +SKIP
            True
            >>> s.process(['pwd'], cwd='/tmp').recvall()
            b'/tmp\n'
-            >>> p = s.process(['python','-c','import os; print(os.read(2, 1024))'], stderr=0)
+            >>> p = s.process(['python','-c','import os; os.write(1, os.read(2, 1024))'], stderr=0)
            >>> p.send(b'hello')
            >>> p.recv()
-            b'hello\n'
+            b'hello'
            >>> s.process(['/bin/echo', 'hello']).recvall()
            b'hello\n'
            >>> s.process(['/bin/echo', 'hello'], stdout='/dev/null').recvall()
@@ -845,11 +845,17 @@ class ssh(Timeout, Logger):
        if not isinstance(argv, (list, tuple)):
            self.error('argv must be a list or tuple')

+        if not all(isinstance(arg, (six.text_type, six.binary_type)) for arg in argv):
+            self.error("argv must be strings or bytes: %r" % argv)
+
        if shell:
            if len(argv) != 1:
                self.error('Cannot provide more than 1 argument if shell=True')
            argv = ['/bin/sh', '-c'] + argv

+        # Create a duplicate so we can modify it
+        argv = list(argv or [])
+
        # Python doesn't like when an arg in argv contains '\x00'
        # -> execve() arg 2 must contain only strings
        for i, oarg in enumerate(argv):
@@ -882,12 +888,8 @@ class ssh(Timeout, Logger):
        # Validate, since failures on the remote side will suck.
        if not isinstance(executable, (six.text_type, six.binary_type)):
            self.error("executable / argv[0] must be a string: %r" % executable)
-        if not isinstance(argv, (list, tuple)):
-            self.error("argv must be a list or tuple: %r" % argv)
        if env is not None and not isinstance(env, dict) and env != os.environ:
            self.error("env must be a dict: %r" % env)
-        if not all(isinstance(s, (six.text_type, six.binary_type)) for s in argv):
-            self.error("argv must only contain strings: %r" % argv)

        # Allow passing in sys.stdin/stdout/stderr objects
        handles = {sys.stdin: 0, sys.stdout:1, sys.stderr:2}
@@ -1125,7 +1127,7 @@ os.execve(exe, argv, env)

        Examples:
            >>> s =  ssh(host='example.pwnme',
-            ...         user='travis',
+            ...         user='runner',
            ...         password='demopass')
            >>> py = s.run('python -i')
            >>> _ = py.recvuntil(b'>>> ')
@@ -1186,7 +1188,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))

        Examples:
            >>> s =  ssh(host='example.pwnme',
-            ...         user='travis',
+            ...         user='runner',
            ...         password='demopass')
            >>> print(s.run_to_end('echo Hello; exit 17'))
            (b'Hello\n', 17)
@@ -1211,10 +1213,10 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))
            >>> from pwn import *
            >>> l = listen()
            >>> s =  ssh(host='example.pwnme',
-            ...         user='travis',
+            ...         user='runner',
            ...         password='demopass')
            >>> a = s.connect_remote(s.host, l.lport)
-            >>> b = l.wait_for_connection()
+            >>> a=a; b = l.wait_for_connection()  # a=a; prevents hangs
            >>> a.sendline(b'Hello')
            >>> print(repr(b.recvline()))
            b'Hello\n'
@@ -1236,11 +1238,11 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))

            >>> from pwn import *
            >>> s =  ssh(host='example.pwnme',
-            ...         user='travis',
+            ...         user='runner',
            ...         password='demopass')
            >>> l = s.listen_remote()
            >>> a = remote(s.host, l.port)
-            >>> b = l.wait_for_connection()
+            >>> a=a; b = l.wait_for_connection()  # a=a; prevents hangs
            >>> a.sendline(b'Hello')
            >>> print(repr(b.recvline()))
            b'Hello\n'
@@ -1256,7 +1258,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))
        Examples:

            >>> s =  ssh(host='example.pwnme',
-            ...         user='travis',
+            ...         user='runner',
            ...         password='demopass')
            >>> print(repr(s['echo hello']))
            b'hello'
@@ -1269,7 +1271,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))
        Examples:

            >>> s =  ssh(host='example.pwnme',
-            ...         user='travis',
+            ...         user='runner',
            ...         password='demopass')
            >>> print(repr(s('echo hello')))
            b'hello'
@@ -1282,12 +1284,12 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))
        Examples:

            >>> s =  ssh(host='example.pwnme',
-            ...         user='travis',
+            ...         user='runner',
            ...         password='demopass')
            >>> s.echo('hello')
            b'hello'
            >>> s.whoami()
-            b'travis'
+            b'runner'
            >>> s.echo(['huh','yay','args'])
            b'huh yay args'
        """
@@ -1315,7 +1317,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))
        Example:

            >>> s =  ssh(host='example.pwnme',
-            ...         user='travis',
+            ...         user='runner',
            ...         password='demopass')
            >>> s.connected()
            True
@@ -1350,7 +1352,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))
            self.error('Unable to find libraries for %r' % remote)
            return {}

-        return misc.parse_ldd_output(data)
+        return misc.parse_ldd_output(context._decode(data))

    def _get_fingerprint(self, remote):
        cmd = '(sha256 || sha256sum || openssl sha256) 2>/dev/null < '
@@ -1473,7 +1475,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))
            >>> with open('/tmp/bar','w+') as f:
            ...     _ = f.write('Hello, world')
            >>> s =  ssh(host='example.pwnme',
-            ...         user='travis',
+            ...         user='runner',
            ...         password='demopass',
            ...         cache=False)
            >>> s.download_data('/tmp/bar')
@@ -1563,7 +1565,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))

        Example:
            >>> s =  ssh(host='example.pwnme',
-            ...         user='travis',
+            ...         user='runner',
            ...         password='demopass')
            >>> s.upload_data(b'Hello, world', '/tmp/upload_foo')
            >>> print(open('/tmp/upload_foo').read())
@@ -1574,6 +1576,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))
            >>> print(open('/tmp/upload_bar').read())
            Hello, world
        """
+        data = context._encode(data)
        # If a relative path was provided, prepend the cwd
        if os.path.normpath(remote) == os.path.basename(remote):
            remote = os.path.join(self.cwd, remote)
@@ -1721,7 +1724,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))

        libs = self._libs_remote(remote)

-        remote = self.readlink('-f',remote).strip()
+        remote = context._decode(self.readlink('-f',remote).strip())
        libs[remote] = 0

        if directory == None:
@@ -1793,7 +1796,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))

        Examples:
            >>> s =  ssh(host='example.pwnme',
-            ...         user='travis',
+            ...         user='runner',
            ...         password='demopass')
            >>> cwd = s.set_working_directory()
            >>> s.ls()
@@ -1802,7 +1805,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))
            True

            >>> s =  ssh(host='example.pwnme',
-            ...         user='travis',
+            ...         user='runner',
            ...         password='demopass')
            >>> homedir = s.pwd()
            >>> _=s.touch('foo')
@@ -1972,7 +1975,7 @@ from ctypes import *; libc = CDLL('libc.so.6'); print(libc.getenv(%r))

        Example:

-            >>> s = ssh("travis", "example.pwnme")
+            >>> s = ssh("runner", "example.pwnme")
            >>> s.aslr
            True
        """
@@ -498,11 +498,11 @@ class tube(Timeout, Logger):
                try:
                    line = self.recvline(keepends=True)
                except Exception:
-                    self.buffer.add(tmpbuf)
+                    self.buffer.unget(tmpbuf)
                    raise

                if not line:
-                    self.buffer.add(tmpbuf)
+                    self.buffer.unget(tmpbuf)
                    return b''

                if pred(line):
@@ -1106,11 +1106,14 @@ class tube(Timeout, Logger):
        """
        self << other << self

-    def wait_for_close(self):
+    def wait_for_close(self, timeout=default):
        """Waits until the tube is closed."""

-        while self.connected():
-            time.sleep(0.05)
+        with self.countdown(timeout):
+            while self.countdown_active():
+                if not self.connected():
+                    return
+                time.sleep(min(self.timeout, 0.05))

    wait = wait_for_close

@@ -1,19 +1,53 @@
 from __future__ import absolute_import
 from __future__ import division

+import fcntl
+import os
+import signal
 import six
+import string
+import struct
+import subprocess
+import sys
+import termios
 import time
 import types

 from pwnlib import term
 from pwnlib.log import getLogger
 from pwnlib.term.readline import raw_input
+from pwnlib.tubes.process import process

 log = getLogger(__name__)

-def yesno(prompt, default = None):
-    """Presents the user with prompt (typically in the form of question) which
-    the user must answer yes or no.
+def testpwnproc(cmd):
+    env = dict(os.environ)
+    env.pop("PWNLIB_NOTERM", None)
+    env["TERM"] = "xterm-256color"
+    def handleusr1(sig, frame):
+        s = p.stderr.read()
+        log.error("child process failed:\n%s", s.decode())
+    signal.signal(signal.SIGUSR1, handleusr1)
+    cmd = """
+from pwn import *
+import signal
+atexception.register(lambda:os.kill(os.getppid(), signal.SIGUSR1))
+""" + cmd
+    if "coverage" in sys.modules:
+        cmd = "import coverage; coverage.process_startup()\n" + cmd
+        env.setdefault("COVERAGE_PROCESS_START", ".coveragerc")
+    p = process([sys.executable, "-c", cmd], env=env, stderr=subprocess.PIPE)
+    try:
+        p.recvuntil(b"\33[6n")
+    except EOFError:
+        raise EOFError("process terminated with code: %r (%r)" % (p.poll(True), p.stderr.read()))
+    fcntl.ioctl(p.stdout.fileno(), termios.TIOCSWINSZ, struct.pack("hh", 80, 80))
+    p.stdout.write(b"\x1b[1;1R")
+    return p
+
+def yesno(prompt, default=None):
+    r"""Presents the user with prompt (typically in the form of question)
+    which the user must answer yes or no.

    Arguments:
      prompt (str): The prompt to show
@@ -21,7 +55,33 @@ def yesno(prompt, default = None):

    Returns:
      `True` if the answer was "yes", `False` if "no"
-"""
+
+    Examples:
+        >>> yesno("A number:", 20)
+        Traceback (most recent call last):
+        ...
+        ValueError: yesno(): default must be a boolean or None
+        >>> saved_stdin = sys.stdin
+        >>> try:
+        ...     sys.stdin = io.StringIO(u"x\nyes\nno\n\n")
+        ...     yesno("is it good 1")
+        ...     yesno("is it good 2", True)
+        ...     yesno("is it good 3", False)
+        ... finally:
+        ...     sys.stdin = saved_stdin
+         [?] is it good 1 [yes/no] Please answer yes or no
+         [?] is it good 1 [yes/no] True
+         [?] is it good 2 [Yes/no] False
+         [?] is it good 3 [yes/No] False
+
+    Tests:
+        >>> p = testpwnproc("print(yesno('is it ok??'))")
+        >>> b"is it ok" in p.recvuntil("??")
+        True
+        >>> p.sendline(b"x\nny")
+        >>> b"True" in p.recvall()
+        True
+    """

    if default is not None and not isinstance(default, bool):
        raise ValueError('yesno(): default must be a boolean or None')
@@ -30,9 +90,9 @@ def yesno(prompt, default = None):
        term.output(' [?] %s [' % prompt)
        yesfocus, yes = term.text.bold('Yes'), 'yes'
        nofocus, no = term.text.bold('No'), 'no'
-        hy = term.output(yesfocus if default == True else yes)
+        hy = term.output(yesfocus if default is True else yes)
        term.output('/')
-        hn = term.output(nofocus if default == False else no)
+        hn = term.output(nofocus if default is False else no)
        term.output(']\n')
        cur = default
        while True:
@@ -50,12 +110,12 @@ def yesno(prompt, default = None):
                    return cur
    else:
        prompt = ' [?] %s [%s/%s] ' % (prompt,
-                                       'Yes' if default == True else 'yes',
-                                       'No' if default == False else 'no',
+                                       'Yes' if default is True else 'yes',
+                                       'No' if default is False else 'no',
                                       )
        while True:
            opt = raw_input(prompt).lower()
-            if opt == '' and default != None:
+            if not opt and default is not None:
                return default
            elif opt in ('y','yes'):
                return True
@@ -64,7 +124,7 @@ def yesno(prompt, default = None):
            print('Please answer yes or no')

 def options(prompt, opts, default = None):
-    """Presents the user with a prompt (typically in the
+    r"""Presents the user with a prompt (typically in the
    form of a question) and a number of options.

    Arguments:
@@ -74,7 +134,44 @@ def options(prompt, opts, default = None):

    Returns:
      The users choice in the form of an integer.
-"""
+
+    Examples:
+        >>> options("Select a color", ("red", "green", "blue"), "green")
+        Traceback (most recent call last):
+        ...
+        ValueError: options(): default must be a number or None
+
+    Tests:
+        >>> p = testpwnproc("print(options('select a color', ('red', 'green', 'blue')))")
+        >>> p.sendline(b"\33[C\33[A\33[A\33[B\33[1;5A\33[1;5B 0310")
+        >>> _ = p.recvall()
+        >>> saved_stdin = sys.stdin
+        >>> try:
+        ...     sys.stdin = io.StringIO(u"\n4\n\n3\n")
+        ...     with context.local(log_level="INFO"):
+        ...         options("select a color A", ("red", "green", "blue"), 0)
+        ...         options("select a color B", ("red", "green", "blue"))
+        ... finally:
+        ...     sys.stdin = saved_stdin
+         [?] select a color A
+               1) red
+               2) green
+               3) blue
+             Choice [1] 0
+         [?] select a color B
+               1) red
+               2) green
+               3) blue
+             Choice  [?] select a color B
+               1) red
+               2) green
+               3) blue
+             Choice  [?] select a color B
+               1) red
+               2) green
+               3) blue
+             Choice 2
+    """

    if default is not None and not isinstance(default, six.integer_types):
        raise ValueError('options(): default must be a number or None')
@@ -93,7 +190,6 @@ def options(prompt, opts, default = None):
            term.output(opt + '\n', indent = len(num) + len(space))
            hs.append(h)
        ds = ''
-        prev = 0
        while True:
            prev = cur
            was_digit = False
@@ -115,16 +211,17 @@ def options(prompt, opts, default = None):
            elif k in ('<enter>', '<right>'):
                if cur is not None:
                    return cur
-            elif k in tuple('1234567890'):
+            elif k in tuple(string.digits):
                was_digit = True
                d = str(k)
                n = int(ds + d)
-                if n > 0 and n <= len(opts):
+                if 0 < n <= len(opts):
                    ds += d
+                    cur = n - 1
                elif d != '0':
                    ds = d
-                n = int(ds)
-                cur = n - 1
+                    n = int(ds)
+                    cur = n - 1

            if prev != cur:
                if prev is not None:
@@ -135,6 +232,8 @@ def options(prompt, opts, default = None):
                    hs[cur].update(arrow)
    else:
        linefmt =       '       %' + str(len(str(len(opts)))) + 'd) %s'
+        if default is not None:
+            default += 1
        while True:
            print(' [?] ' + prompt)
            for i, opt in enumerate(opts):
@@ -147,12 +246,39 @@ def options(prompt, opts, default = None):
            except (ValueError, TypeError):
                continue
            if x >= 1 and x <= len(opts):
-                return x
+                return x - 1

-def pause(n = None):
-    """Waits for either user input or a specific number of seconds."""
+def pause(n=None):
+    r"""Waits for either user input or a specific number of seconds.

-    if n == None:
+    Examples:
+        >>> with context.local(log_level="INFO"):
+        ...     pause(1)
+        [x] Waiting
+        [x] Waiting: 1...
+        [+] Waiting: Done
+        >>> pause("whatever")
+        Traceback (most recent call last):
+        ...
+        ValueError: pause(): n must be a number or None
+
+    Tests:
+        >>> saved_stdin = sys.stdin
+        >>> try:
+        ...     sys.stdin = io.StringIO(u"\n")
+        ...     with context.local(log_level="INFO"):
+        ...         pause()
+        ... finally:
+        ...     sys.stdin = saved_stdin
+        [*] Paused (press enter to continue)
+        >>> p = testpwnproc("pause()")
+        >>> b"Paused" in p.recvuntil(b"press any")
+        True
+        >>> p.send(b"x")
+        >>> _ = p.recvall()
+    """
+
+    if n is None:
        if term.term_mode:
            log.info('Paused (press any to continue)')
            term.getkey()
@@ -169,7 +295,7 @@ def pause(n = None):
        raise ValueError('pause(): n must be a number or None')

 def more(text):
-    """more(text)
+    r"""more(text)

    Shows text like the command line tool ``more``.

@@ -180,6 +306,14 @@ def more(text):

    Returns:
      :const:`None`
+
+    Tests:
+        >>> more("text")
+        text
+        >>> p = testpwnproc("more('text\\n' * (term.height + 2))")
+        >>> p.send(b"x")
+        >>> b"text" in p.recvall()
+        True
    """
    if term.term_mode:
        lines = text.split('\n')
@@ -6,7 +6,7 @@ import string

 from pwnlib.context import context, LocalNoarchContext
 from pwnlib.log import getLogger
-from pwnlib.util import packing
+from pwnlib.util import packing, iters

 log = getLogger(__name__)

@@ -129,19 +129,10 @@ def cyclic(length = None, alphabet = None, n = None):
        log.error("Can't create a pattern length=%i with len(alphabet)==%i and n==%i" \
                  % (length, len(alphabet), n))

-    out = []
-    for ndx, c in enumerate(de_bruijn(alphabet, n)):
-        if length != None and ndx >= length:
-            break
-        else:
-            out.append(c)
+    generator = de_bruijn(alphabet, n)
+    out = iters.take(length, generator)

-    if isinstance(alphabet, six.text_type):
-        return ''.join(out)
-    elif isinstance(alphabet, bytes):
-        return bytes(bytearray(out))
-    else:
-        return out
+    return _join_sequence(out, alphabet)

@LocalNoarchContext
 def cyclic_find(subseq, alphabet = None, n = None):
@@ -218,6 +209,7 @@ def cyclic_find(subseq, alphabet = None, n = None):

    if isinstance(subseq, six.integer_types):
        subseq = packing.pack(subseq, bytes=n)
+    subseq = context._encode(subseq)

    if len(subseq) != n:
        log.warn_once("cyclic_find() expects %i-byte subsequences by default, you gave %r\n"\
@@ -228,6 +220,7 @@ def cyclic_find(subseq, alphabet = None, n = None):

    if alphabet is None:
        alphabet = context.cyclic_alphabet
+    alphabet = context._encode(alphabet)

    if any(c not in alphabet for c in subseq):
        return -1
@@ -287,21 +280,15 @@ def cyclic_metasploit(length = None, sets = None):
        20280
    """
    sets = sets or [ string.ascii_uppercase.encode(), string.ascii_lowercase.encode(), string.digits.encode() ]
-    out = bytearray()

-    for ndx, c in enumerate(metasploit_pattern(sets)):
-        if length != None and ndx >= length:
-            break
-        else:
-            out.append(c)
-
-    out = bytes(out)
+    generator = metasploit_pattern(sets)
+    out = iters.take(length, generator)

    if length != None and len(out) < length:
        log.error("Can't create a pattern of length %i with sets of lengths %s. Maximum pattern length is %i." \
                  % (length, list(map(len, sets)), len(out)))

-    return out
+    return _join_sequence(out, sets[0])

 def cyclic_metasploit_find(subseq, sets = None):
    """cyclic_metasploit_find(subseq, sets = [ string.ascii_uppercase, string.ascii_lowercase, string.digits ]) -> int
@@ -344,3 +331,124 @@ def _gen_find(subseq, generator):
        if saved == subseq:
            return pos
    return -1
+
+def _join_sequence(seq, alphabet):
+    if isinstance(alphabet, six.text_type):
+        return ''.join(seq)
+    elif isinstance(alphabet, bytes):
+        return bytes(bytearray(seq))
+    else:
+        return seq
+
+class cyclic_gen(object):
+    """
+    Creates a stateful cyclic generator which can generate sequential chunks of de Bruijn sequences.
+
+    >>> g = cyclic_gen() # Create a generator
+    >>> g.get(4) # Get a chunk of length 4
+    b'aaaa'
+    >>> g.get(4) # Get a chunk of length 4
+    b'baaa'
+    >>> g.get(8) # Get a chunk of length 8
+    b'caaadaaa'
+    >>> g.get(4) # Get a chunk of length 4
+    b'eaaa'
+    >>> g.find(b'caaa') # Position 8, which is in chunk 2 at index 0
+    (8, 2, 0)
+    >>> g.find(b'aaaa') # Position 0, which is in chunk 0 at index 0
+    (0, 0, 0)
+    >>> g.find(b'baaa') # Position 4, which is in chunk 1 at index 0
+    (4, 1, 0)
+    >>> g.find(b'aaad') # Position 9, which is in chunk 2 at index 1
+    (9, 2, 1)
+    >>> g.find(b'aada') # Position 10, which is in chunk 2 at index 2
+    (10, 2, 2)
+    >>> g.get() # Get the rest of the sequence
+    b'faaagaaahaaaiaaajaaa...yyxzyzxzzyxzzzyyyyzyyzzyzyzzzz'
+    >>> g.find(b'racz') # Position 7760, which is in chunk 4 at index 7740
+    (7760, 4, 7740)
+    >>> g.get(12) # Generator is exhausted
+    Traceback (most recent call last):
+      ...
+    StopIteration
+
+    >>> g = cyclic_gen(string.ascii_uppercase, n=8) # Custom alphabet and item size
+    >>> g.get(12) # Get a chunk of length 12
+    'AAAAAAAABAAA'
+    >>> g.get(18) # Get a chunk of length 18
+    'AAAACAAAAAAADAAAAA'
+    >>> g.find('CAAAAAAA') # Position 16, which is in chunk 1 at index 4
+    (16, 1, 4)
+    """
+
+    def __init__(self, alphabet = None, n = None):
+        if n is None:
+            n = context.cyclic_size
+
+        if alphabet is None:
+            alphabet = context.cyclic_alphabet
+
+        self._generator = de_bruijn(alphabet, n)
+        self._alphabet = alphabet
+        self._total_length = 0
+        self._n = n
+        self._chunks = []
+
+    def get(self, length = None):
+        """
+        Get the next de Bruijn sequence from this generator.
+
+        >>> g = cyclic_gen()
+        >>> g.get(4) # Get a chunk of length 4
+        b'aaaa'
+        >>> g.get(4) # Get a chunk of length 4
+        b'baaa'
+        >>> g.get(8) # Get a chunk of length 8
+        b'caaadaaa'
+        >>> g.get(4) # Get a chunk of length 4
+        b'eaaa'
+        >>> g.get() # Get the rest of the sequence
+        b'faaagaaahaaaiaaajaaa...yyxzyzxzzyxzzzyyyyzyyzzyzyzzzz'
+        >>> g.get(12) # Generator is exhausted
+        Traceback (most recent call last):
+          ...
+        StopIteration
+        """
+        
+        if length != None:
+            self._chunks.append(length)
+            self._total_length += length
+            if len(self._alphabet) ** self._n < self._total_length:
+                log.error("Can't create a pattern length=%i with len(alphabet)==%i and n==%i" \
+                    % (self._total_length, len(self._alphabet), self._n))
+        else:
+            self._chunks.append(float("inf"))
+
+        out = [next(self._generator) for _ in range(length)] if length != None else [next(self._generator)] + list(self._generator)
+        return _join_sequence(out, self._alphabet)
+
+    def find(self, subseq):
+        """
+        Find a chunk and subindex from all the generates de Bruijn sequences.
+
+        >>> g = cyclic_gen()
+        >>> g.get(4)
+        b'aaaa'
+        >>> g.get(4)
+        b'baaa'
+        >>> g.get(8)
+        b'caaadaaa'
+        >>> g.get(4)
+        b'eaaa'
+        >>> g.find(b'caaa') # Position 8, which is in chunk 2 at index 0
+        (8, 2, 0)
+        """
+
+        global_index = cyclic_find(subseq, self._alphabet, self._n)
+        remaining_index = global_index
+        for chunk_idx in range(len(self._chunks)):
+            chunk = self._chunks[chunk_idx]
+            if remaining_index < chunk:
+                return (global_index, chunk_idx, remaining_index)
+            remaining_index -= chunk
+        return -1
@@ -226,4 +226,4 @@ def sockaddr(host, port, network = 'ipv4'):
        sockaddr += p32(0xffffffff) # Save three bytes 'push -1' vs 'push 0'
        sockaddr += host
        length    = len(sockaddr) + 4 # Save five bytes 'push 0'
-    return (sockaddr, length, address_family)
+    return (sockaddr, length, getattr(address_family, "name", address_family))
@@ -325,10 +325,11 @@ def state(pid):
    """
    return status(pid)['State']

-def wait_for_debugger(pid):
-    """wait_for_debugger(pid) -> None
+def wait_for_debugger(pid, debugger_pid=None):
+    """wait_for_debugger(pid, debugger_pid=None) -> None

    Sleeps until the process with PID `pid` is being traced.
+    If debugger_pid is set and debugger exits, raises an error.

    Arguments:
        pid (int): PID of the process.
@@ -337,6 +338,16 @@ def wait_for_debugger(pid):
        None
    """
    with log.waitfor('Waiting for debugger') as l:
-        while tracer(pid) is None:
-            time.sleep(0.01)
+        if debugger_pid:
+            debugger = psutil.Process(debugger_pid)
+            while tracer(pid) is None:
+                try:
+                    debugger.wait(0.01)
+                except psutil.TimeoutExpired:
+                    pass
+                else:
+                    l.error("debugger exited! (maybe check /proc/sys/kernel/yama/ptrace_scope)")
+        else:
+            while tracer(pid) is None:
+                time.sleep(0.01)
        l.success()
@@ -13,7 +13,7 @@ from pwnlib.util.misc import size
 log = getLogger(__name__)

 def wget(url, save=None, timeout=5, **kwargs):
-    """wget(url, save=None, timeout=5) -> str
+    r"""wget(url, save=None, timeout=5) -> str

    Downloads a file via HTTP/HTTPS.

@@ -28,7 +28,7 @@ def wget(url, save=None, timeout=5, **kwargs):
      >>> url    = 'https://httpbin.org/robots.txt'
      >>> result = wget(url, timeout=60)
      >>> result
-      b'User-agent: *\\nDisallow: /deny\\n'
+      b'User-agent: *\nDisallow: /deny\n'
      >>> result2 = wget(url, True, timeout=60)
      >>> result == open('robots.txt', 'rb').read()
      True
@@ -1 +1 @@
-__version__ = '4.0.0'
+__version__ = '4.1.0'
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python
 from __future__ import print_function

 import glob
@@ -53,7 +53,6 @@ install_requires     = ['paramiko>=1.15.2',
                        'pyserial>=2.7',
                        'requests>=2.0',
                        'pip>=6.0.8',
-                        'tox>=1.8.1',
                        'pygments>=2.0',
                        'pysocks',
                        'python-dateutil',
@@ -61,7 +60,8 @@ install_requires     = ['paramiko>=1.15.2',
                        'psutil>=3.3.0',
                        'intervaltree>=3.0',
                        'sortedcontainers',
-                        'unicorn']
+                        'unicorn>=1.0.2rc1', # see unicorn-engine/unicorn#1100, unicorn-engine/unicorn#1170
+]

 # Check that the user has installed the Python development headers
 PythonH = os.path.join(get_python_inc(), 'Python.h')
@@ -82,7 +82,7 @@ setup(
    name                 = 'pwntools',
    python_requires      = '>=2.7',
    packages             = find_packages(),
-    version              = '4.0.0',
+    version              = '4.1.0',
    data_files           = [('',
                             glob.glob('*.md') + glob.glob('*.txt')),
                            ],
@@ -1,5 +0,0 @@
-[tox]
-envlist = py27
-[testenv]
-deps=-r{toxinidir}/requirements.txt
-commands=make PWNLIB_NOTERM=1 -C docs doctest
@@ -64,6 +64,13 @@ setup_travis()
    rm -rf usr/share
 }

+setup_gdbserver()
+{
+    # https://docs.improbable.io/reference/14.3/shared/debug-cloud-workers#common-issues
+    wget http://archive.ubuntu.com/ubuntu/pool/main/g/gdb/gdbserver_8.3-0ubuntu1_amd64.deb
+    sudo apt-get install ./gdbserver_8.3-0ubuntu1_amd64.deb
+}
+
 setup_linux()
 {
    sudo apt-get install -y software-properties-common openssh-server libncurses5-dev libncursesw5-dev openjdk-8-jre-headless
@@ -119,60 +126,52 @@ setup_android_emulator()
    if (uname | grep -i Darwin &>/dev/null); then
        brew install android-sdk android-ndk
    else
-        if [ ! -f android-sdk/android ]; then
+        if [ ! -f android-sdk/tools/bin/sdkmanager ]; then
            # Install the SDK, which gives us the 'android' and 'emulator' commands
-            wget -nv https://dl.google.com/android/android-sdk_r24.4.1-linux.tgz
-            tar xf android-sdk_r24.4.1-linux.tgz
-            rm  -f android-sdk_r24.4.1-linux.tgz
+            wget -nv -O sdk-tools-linux.zip https://dl.google.com/android/repository/sdk-tools-linux-4333796.zip
+            unzip -q sdk-tools-linux.zip
+            rm    -f sdk-tools-linux.zip

            # Travis caching causes this to exist already
            rm -rf android-sdk

-            mv android-sdk-linux android-sdk
-            file android-sdk/tools/android
+            mkdir android-sdk
+            mv tools android-sdk/
+            file android-sdk/tools/bin/sdk-manager
        fi

        export PATH="$PWD/android-sdk/tools:$PATH"
+        export PATH="$PWD/android-sdk/tools/bin:$PATH"
        export PATH="$PWD/android-sdk/platform-tools:$PATH"
-        which android
-
-        # Install the NDK, which is required for adb.compile()
-        NDK_VERSION=android-ndk-r12b
-        if [ ! -f android-ndk/ndk-build ]; then
-            wget -nv https://dl.google.com/android/repository/$NDK_VERSION-linux-x86_64.zip
-            unzip -q android-ndk-*.zip
-            rm -f    android-ndk-*.zip
-
-            # Travis caching causes this to exist already
-            rm -rf android-ndk
-
-            mv     $NDK_VERSION android-ndk
-        fi
-
-        export NDK=$PWD/android-ndk
-        export PATH=$NDK:$PATH
+        export ANDROID_SDK_ROOT="$PWD/android-sdk"
+        export ANDROID_HOME="$PWD/android-sdk"
+        which sdkmanager
    fi

    # Grab prerequisites
-    echo y | android update sdk --no-ui --all --filter platform-tools,extra-android-support
-    echo y | android update sdk --no-ui --all --filter android-21
-
    # Valid ABIs:
    # - armeabi-v7a
    # - arm64-v8a
    # - x86
    # - x86_64
-    ABI='armeabi-v7a'
+    ANDROID_ABI='armeabi-v7a'
+    ANDROIDV=android-24
+    yes | sdkmanager --install platform-tools 'extras;android;m2repository' emulator ndk-bundle \
+          "platforms;$ANDROIDV" "system-images;$ANDROIDV;default;$ANDROID_ABI" >/dev/null
+    yes | sdkmanager --licenses

-    # Grab the emulator image
-    echo y | android update sdk --no-ui --all --filter sys-img-$ABI-android-21
+    # enable NDK for adb.compile()
+    for d in "$PWD/android-sdk/ndk-bundle/"*/; do
+        export PATH="$PATH:$d"
+    done

    # Create our emulator Android Virtual Device (AVD)
-    echo no | android --silent create avd --name android-$ABI   --target android-21 --force --snapshot --abi $ABI
+    # --snapshot flag is deprecated, see bitrise-steplib/steps-create-android-emulator#18
+    echo no | avdmanager --silent create avd --name android-$ANDROID_ABI --force --package "system-images;$ANDROIDV;default;$ANDROID_ABI"

    # In the future, it would be nice to be able to use snapshots.
    # However, I haven't gotten them to work nicely.
-    emulator -avd android-$ABI -no-window -no-boot-anim -no-skin -no-audio -no-window -no-snapshot &
+    android-sdk/emulator/emulator -avd android-$ANDROID_ABI -no-window -no-boot-anim -read-only -no-audio -no-window -no-snapshot &
    adb wait-for-device
    adb shell id
    adb shell getprop
@@ -187,6 +186,7 @@ setup_osx()

 if [[ "$USER" == "travis" ]]; then
 #   setup_travis
+    setup_gdbserver
    setup_android_emulator
 elif [[ "$USER" == "shippable" ]]; then
    sudo apt-get update
Author	SHA1	Message	Date
Arusekk	dd1f776c6c	Release 4.1.0	2020-05-08 13:54:09 +02:00
Arusekk	a8e594d7a3	Fix bytes in flag.py	2020-04-07 00:59:22 +02:00
Arusekk	1b6e663f93	Re-add recommends for cross-gcc	2020-04-07 00:41:17 +02:00
Arusekk	4fe423c1b4	Hope this will pass CI	2020-04-07 00:27:34 +02:00
Arusekk	8673f53aab	Solve build problems	2020-04-07 00:26:55 +02:00
Arusekk	3875a8ff18	Stabilize CI flow	2020-04-07 00:26:29 +02:00
Arusekk	97f5742f5c	More reliable crash on thumb	2020-04-06 22:35:37 +02:00
Arusekk	5b8acf0893	Try to make ADB work	2020-03-30 11:18:44 +02:00
Arusekk	9bb47493ba	Stable tests, some bootup permission issue	2020-03-29 00:45:10 +01:00
Arusekk	b952728fad	Ensure ADB quite working	2020-03-29 00:22:59 +01:00
Arusekk	d8b69dde24	GDB enhancements	2020-03-28 12:21:30 +01:00
Arusekk	e5f17630dc	Enhance debug messages if yama is on Also disable yama on CI	2020-03-27 12:39:49 +01:00
Arusekk	b7c985ad42	Make sure ADB is in root mode (unordered doctests)	2020-03-27 11:20:34 +01:00
Arusekk	34ef7dce64	Allow slower doctests	2020-03-26 10:29:16 +01:00
Arusekk	c9909eea0d	Fix ADB again	2020-03-26 10:10:24 +01:00
Arusekk	1d92ee3583	Prevent adb module from being introspected wrongly	2020-03-26 09:32:32 +01:00
Arusekk	8eb242df5a	Actually fix the problems	2020-03-26 09:18:14 +01:00
Arusekk	4b2037028e	Please please please work as expected	2020-03-25 21:49:20 +01:00
Arusekk	b6bc3629e9	More debugging	2020-03-25 17:39:55 +01:00
Arusekk	1d7d5ecd7f	More debug info	2020-03-22 21:37:00 +01:00
Arusekk	0a558391d9	Try to debug what is wrong	2020-03-22 20:00:05 +01:00
Arusekk	e33a3fd517	Whatever	2020-03-22 00:57:16 +01:00
Arusekk	ca0b08f500	Changes in AVD handling	2020-03-22 00:11:30 +01:00
Arusekk	358581a26f	Too complicated	2020-03-21 23:36:10 +01:00
Arusekk	60461af312	Enhance UI tests diagnostics	2020-03-21 23:24:35 +01:00
Arusekk	37492fd579	Try again and again	2020-03-21 22:42:25 +01:00
Arusekk	e4a0703178	Fix ssh user in gdb	2020-03-21 21:50:19 +01:00
Arusekk	a1e74b3d57	Merge branch 'stable' into beta	2020-03-21 21:20:37 +01:00
ArusekkandChristian Clauss	055d8b87ae	GitHub actions (#1460 ) * Try GitHub actions instead of Travis CI. This is just a proof-of-concept until we can have it the clean way it should be done. More commits will likely follow. * Travis CI: Use flake8 to detect syntax errors (#1385) * Travis CI: Use flake8 to detect syntax errors Repeat #1305 on the Python 3 branch. Also, remove the hardcoding of default values for dist and Py2. https://flake8.pycqa.org/en/latest/user/error-codes.html On the flake8 test selection, this PR does _not_ focus on "_style violations_" (the majority of flake8 error codes that [__psf/black__](https://github.com/psf/black) can autocorrect). Instead these tests are focus on runtime safety and correctness: * E9 tests are about Python syntax errors usually raised because flake8 can not build an Abstract Syntax Tree (AST). Often these issues are a sign of unused code or code that has not been ported to Python 3. These would be compile-time errors in a compiled language but in a dynamic language like Python they result in the script halting/crashing on the user. * F63 tests are usually about the confusion between identity and equality in Python. Use ==/!= to compare str, bytes, and int literals is the classic case. These are areas where __a == b__ is True but __a is b__ is False (or vice versa). Python >= 3.8 will raise SyntaxWarnings on these instances. * F7 tests logic errors and syntax errors in type hints * F82 tests are almost always _undefined names_ which are usually a sign of a typo, missing imports, or code that has not been ported to Python 3. These also would be compile-time errors in a compiled language but in Python a __NameError__ is raised which will halt/crash the script on the user. * Fail fast, exclude generated source files * --select=E9,F63,F7 * 2to3 -f print -wn examples * No more <> in python 3. * Use ==/!= to compare str, bytes, and int literals (#1386) Identity is not the same thing as equality in Python so use ==/!= to compare str, bytes, and int literals. In Python >= 3.8, these instances will raise _SyntaxWarnings_ so it is best to fix them now. https://docs.python.org/3.8/whatsnew/3.8.html#porting-to-python-3-8 $ python ```python >>> raw = 'ra' >>> raw += 'w' >>> raw == 'raw' True >>> raw is 'raw' False ``` * Use print() function in both Python 2 and Python 3 (#1387) $ __pip install black future isort__ $ __futurize -f libfuturize.fixes.fix_print_with_import -w android.py__ $ __isort android.py__ $ __black android.py__ * Add cache and building artifacts * Enable caching * Actually enable caching for all of it * Mistake * Try different ABI on CI * Try different ABI + version * Try it harder * Update SDK version * Try building using newer SDK * Accept licenses * Ugh android emulator documentation is unfriendly * Environment variables for AVD * Add QEMU-system for android virtual devices * Use the correct emulator * New android image * Was actually too new image * Update ssh credentials to reflect GHA setup * Fix spelling * Try finding endless loops * Actually use the timeout * Reduce possible hang surface in doctests * Try SSH immediately when set up * SSH strict host key checking disable for localhost * Try stackoverflow.com research * Try to fix failing doctests * Add required gcc for cross-arch tests * Ensure gcc not failing in fmtstr * Fix issues with shellcraft * Try different libs * Try again, with verbosity * Why do these conflict??? * Fix issues related to PIE binaries * Add bash-static * Verbose 'could not set transport: any' * Add AT_SECURE in auxv to prevent SEGV on libc bug * Killing ADB server actually kills it * Android image has /data/user/0 -> /data/data * Set coverage mode from dev * Try new actions * Fix coveralls complaining * A little time.sleep would not hurt or would it Co-authored-by: Christian Clauss <[email protected]>	2020-03-21 21:10:45 +01:00
Arusekk	d82bab2849	Merge branch 'stable' into beta	2020-01-22 22:23:57 +01:00
Arusekk	904872edc8	Release 4.0.1	2020-01-22 21:57:27 +01:00
Zach RiggleandArusekk	7b072dc7f7	Fix gadget caching for ROP objects with multiple ELFs (#1115 ) * Fix test breakage caused by Sphinx deprecation Closes #1112 * Fix gadget caching for ROP objects with multiple ELFs Fixes Gallopsled/pwntools#1114 * Update requirements, fix travis failing Co-authored-by: Arusekk <[email protected]>	2020-01-22 21:49:10 +01:00
Arusekk	d8219b8826	Merge branch 'stable' into beta	2020-01-18 20:49:12 +01:00
CptGibbon	eee1503252	Correct self.buffer.add() to self.buffer.unget() in recvline_pred() (#1412 )	2020-01-18 20:46:22 +01:00
Arusekk	c3784268a1	Release 4.1.0beta0	2020-01-09 19:00:58 +01:00
Félix Martel (lixtelnis)andArusekk	6d340bf2ac	Remove `AddressFamily.` prefix from address_family (#1316 ) * Remove `AddressFamily.` prefix from address_family * Fix python 2 regression Co-authored-by: Arusekk <[email protected]>	2020-01-07 15:31:21 +01:00
Calle Svensson	2124f3607a	Use bytes instead of string in serialtube class (#1402 )	2020-01-07 15:29:18 +01:00
Bet4andArusekk	f727b66475	Fix issues related with debugging (#1323 ) * Fix issues related with debugging * Update pwnlib/tubes/process.py Co-Authored-By: Arusekk <[email protected]> * Enable doctest for find_module_addresses * Experiment about adding doctests to gdb.py * Add doctests for most functions in gdb.py * Remove 'file' statement in gdb script * Encode cyclic alphabet * Fix travis build Co-authored-by: Arusekk <[email protected]>	2019-12-29 13:16:11 +01:00
Arusekk	c334cebade	Android here we go again	2019-12-28 13:50:49 +01:00
Arusekk	010c89c3b3	Fix android builds regression with #1385	2019-12-28 01:46:46 +01:00
Arusekk	82dce28722	Introduce tests for pwnlib.ui (#1398 ) * Introduce tests for pwnlib.ui This architecture can be changed later, but I'm quite proud of how it's done currently. * Try to debug CI * Ugly hack to get results back * Let's just keep it half-broken	2019-12-27 19:27:18 +01:00
Arusekk	5fdeb21de1	Enhance unlock_bootloader with better status messages (#1001 )	2019-12-27 18:45:07 +01:00
Arusekk	8a4288f895	Enhance CI caching (#1392 ) Currently, python 3 installed libraries and sphinx doctree cache are not retained across builds. This PR addresses this issue, hopefully resulting in a major speedup.	2019-12-27 16:10:10 +01:00
Arusekk	ecc160a36c	Merge branch 'beta' into dev	2019-12-13 23:41:07 +01:00
Itamar	df59b10eb3	transform `armv7l` arch to `arm` (#1388 )	2019-12-13 13:54:16 +01:00
mephi42	7b9bcf8d7b	Fix process.libs (#1391 ) Currently fails with "name maps is not defined".	2019-12-13 13:53:54 +01:00
Krzysztof Korościk	59ec1b1a1b	remove tox dependency (#1389 )	2019-12-12 14:36:26 +01:00
Christian Clauss	41702e89e6	Travis CI: Use flake8 to detect syntax errors (#1385 ) * Travis CI: Use flake8 to detect syntax errors Repeat #1305 on the Python 3 branch. Also, remove the hardcoding of default values for dist and Py2. https://flake8.pycqa.org/en/latest/user/error-codes.html On the flake8 test selection, this PR does _not_ focus on "_style violations_" (the majority of flake8 error codes that [__psf/black__](https://github.com/psf/black) can autocorrect). Instead these tests are focus on runtime safety and correctness: * E9 tests are about Python syntax errors usually raised because flake8 can not build an Abstract Syntax Tree (AST). Often these issues are a sign of unused code or code that has not been ported to Python 3. These would be compile-time errors in a compiled language but in a dynamic language like Python they result in the script halting/crashing on the user. * F63 tests are usually about the confusion between identity and equality in Python. Use ==/!= to compare str, bytes, and int literals is the classic case. These are areas where __a == b__ is True but __a is b__ is False (or vice versa). Python >= 3.8 will raise SyntaxWarnings on these instances. * F7 tests logic errors and syntax errors in type hints * F82 tests are almost always _undefined names_ which are usually a sign of a typo, missing imports, or code that has not been ported to Python 3. These also would be compile-time errors in a compiled language but in Python a __NameError__ is raised which will halt/crash the script on the user. * Fail fast, exclude generated source files * --select=E9,F63,F7 * 2to3 -f print -wn examples * No more <> in python 3.	2019-12-09 21:58:15 +01:00
Gulshan Singh	f6fe196780	Launch QEMU with sysroot if specified (#1241 ) * Launch QEMU with sysroot if specified * Update gdb.py * Append args, do not replace	2019-12-09 18:04:35 +01:00
Bet4	6667a239be	Use the last possible address as PLT address (#1146 )	2019-12-09 17:54:25 +01:00
Vignesh S Rao (a.k.a sherl0ck)	59c78105e1	Adding support for FileStructure exploitation (#1218 ) * Adding support for FileStructure exploitation * Actually run the tests + basic py3 fixes also rename the module to lower case to keep consistency * Python 3 working	2019-12-09 17:48:38 +01:00
Josh FandArusekk	b78fd6e3c8	Allow VMA 0 in make_elf (#1361 ) * Allow VMA 0 in make_elf * Update none check Co-Authored-By: Arusekk <[email protected]>	2019-12-09 17:33:11 +01:00
Arusekk	ab67da5d8e	Merge branch 'dev3' into dev	2019-12-09 17:30:00 +01:00
Christian Clauss	0491972d1c	Use print() function in both Python 2 and Python 3 (#1387 ) $ __pip install black future isort__ $ __futurize -f libfuturize.fixes.fix_print_with_import -w android.py__ $ __isort android.py__ $ __black android.py__	2019-12-08 19:43:18 +01:00
Christian Clauss	98bdcfd9f0	Use ==/!= to compare str, bytes, and int literals (#1386 ) Identity is not the same thing as equality in Python so use ==/!= to compare str, bytes, and int literals. In Python >= 3.8, these instances will raise _SyntaxWarnings_ so it is best to fix them now. https://docs.python.org/3.8/whatsnew/3.8.html#porting-to-python-3-8 $ python ```python >>> raw = 'ra' >>> raw += 'w' >>> raw == 'raw' True >>> raw is 'raw' False ```	2019-12-08 17:26:33 +01:00
Andreas Oberritter	dbf0549213	Make DynELF().libc work on current Debian 10 (amd64) (#1340 ) * libcdb: Add another offset for amd64 Tested with libc6_2.28-10_amd64.deb from Debian 10. * dynelf: Improve status message when looking for build id	2019-12-08 15:27:33 +01:00
Kyle Zeng	11e9d814c9	in python3, input builtin returns str not bytes (#1383 )	2019-12-08 15:24:18 +01:00
Arusekk	aaa3cb48b9	Fix readline under py2	2019-12-08 11:28:51 +01:00
Riccardo Landolfo	9476e2e5b0	use bytes variable instead of arch (#1384 )	2019-12-08 11:26:45 +01:00
ArusekkandAdi	99e9bcbf30	Special handling for gdb LD_* environment (#1371 ) Closes #1307 Co-authored-by: Adi <[email protected]>	2019-12-06 12:24:16 +01:00
Arusekk	adaba022e6	Pretty docstrings and libcdb fix	2019-12-06 11:30:55 +01:00
Arusekk	37aeb060c6	Merge branch 'beta' into dev	2019-12-06 11:28:41 +01:00
Kyle Zeng	faab6ea3dd	should not use first argument if exe is specified (#1377 ) * should not use first argument if exe is specified * use absolute path for execuable	2019-12-05 00:37:30 +01:00
Arusekk	5bba036c54	Merge branch 'dev3' into dev	2019-11-11 00:02:38 +01:00
Calle Svensson	11a70f2e9e	Added cyclic_gen class (#1359 ) * Added cyclic_gen class * Fix review comments * Add description and doctests * Added missing quotes * Improved tests	2019-11-10 23:54:13 +01:00
Arusekk	55dfe9b5e4	Begin working on 4.1.0	2019-11-05 19:53:20 +01:00
Zach Riggle	70de143278	Enhance unlock_bootloader with better status messages - Detect bootloaders that are already unklocked - Detect bootloader unlock success - Detect bootloaders that cannot be unlocked - Detect devices which have not enabled bootloader unlocking	2017-07-10 15:54:22 -05:00