Release 3.7.0

Importing stat module to fix some errors on lines 361 to 364. The errors are caused when using the parameter setuid=True.

CHANGELOG.md

@@ -9,9 +9,12 @@ The table below shows which release corresponds to each branch, and what date th
 
 | Version          | Branch   | Release Date           |
 | ---------------- | -------- | ---------------------- |
-| [3.7.0](#350)    | `dev`    | Jun 17, 2017 (planned)
-| [3.6.0](#350)    | `beta`   | May 6, 2017 (planned)
-| [3.5.1](#351)    | `stable` | Apr 15, 2017
+| [3.9.0](#390)    | `dev`    | Sep 8, 2017 (planned)
+| [3.8.0](#380)    | `beta`   | Jul 28, 2017 (planned)
+| [3.7.0](#370)    | `stable` | Jun 19, 2017
+| [3.6.1](#361)    |          | May 12, 2017
+| [3.6.0](#360)    |          | May 8, 2017
+| [3.5.1](#351)    |          | Apr 15, 2017
 | [3.5.0](#350)    |          | Mar 26, 2017
 | [3.4.1](#341)    |          | Feb 17, 2017
 | [3.4.0](#340)    |          | Feb 13, 2017
@@ -31,14 +34,42 @@ The table below shows which release corresponds to each branch, and what date th
 | [3.0.0](#300)    |          | Aug 20, 2016
 | [2.2.0](#220)    |          | Jan 5, 2015
 
+## 3.9.0
+
+To be released on Sep 8, 2017.
+
+## 3.8.0
+
+To be released on Jul 28, 2017.
+
+- [#981][981] Fixed RELRO detection logic
+- [#986][986] Enhancements to DynELF for controlling usage of LibcDB
+
+[981]: https://github.com/Gallopsled/pwntools/issues/981
+[986]: https://github.com/Gallopsled/pwntools/issues/986
+
 ## 3.7.0
 
-To be on Jun 17, 2017.
+- [#933][933] DynELF works better with different base addresses
+- [#952][952] A few small fixes were made to `pwn template`, and the CRC database was updated.
+- [5c72d62c][5c72d62c] Updated the CRC database
+
+[933]: https://github.com/Gallopsled/pwntools/issues/933
+[952]: https://github.com/Gallopsled/pwntools/issues/952
+[5c72d62c]: https://github.com/Gallopsled/pwntools/commit/5c72d62c
+
+## 3.6.1
+
+- [#979][979]+[1a4a1e1][1a4a1e1] Fixed [#974][974], a bug related to the terminal handling and numlock.
+- [#980][980] Fixed the `pwn template` command.
+
+[974]: https://github.com/Gallopsled/pwntools/issues/974
+[979]: https://github.com/Gallopsled/pwntools/pull/979
+[980]: https://github.com/Gallopsled/pwntools/pull/980
+[1a4a1e1]: https://github.com/Gallopsled/pwntools/commit/1a4a1e1
 
 ## 3.6.0
 
-To be released on May 6, 2017.
-
 - [#895][895] Added a Dockerfile to simplify testing setup and allow testing on OSX
 - [#897][897] Fixed some incorrect AArch64 syscals
 - [#893][893] Added the `pwnlib.config` module
@@ -51,6 +82,18 @@ To be released on May 6, 2017.
     + More information is available in the documentation [here](http://docs.pwntools.com/en/dev/commandline.html#pwn-debug)
 - [#919][919] Added a `pwn template` command-line utility to simplify the process of bootstrapping a new exploit.
     + More information is available in the documentation [here](http://docs.pwntools.com/en/dev/commandline.html#pwn-template).
+- [#948][948] Fix unnecessary warning for Core files
+- [#954][954] Fix list processing in `~/.pwn.conf`
+- [#967][967] Respect `TERM_PROGRAM` for `run_in_new_terminal`
+- [#970][970] Fix overly-aggressive corefile caching
+
+[947]: https://github.com/Gallopsled/pwntools/pull/947
+[948]: https://github.com/Gallopsled/pwntools/pull/948
+[954]: https://github.com/Gallopsled/pwntools/pull/954
+[960]: https://github.com/Gallopsled/pwntools/pull/960
+[967]: https://github.com/Gallopsled/pwntools/pull/967
+[968]: https://github.com/Gallopsled/pwntools/pull/968
+[970]: https://github.com/Gallopsled/pwntools/pull/970
 
 [895]: https://github.com/Gallopsled/pwntools/pull/895
 [897]: https://github.com/Gallopsled/pwntools/pull/897

MANIFEST.in

@@ -2,6 +2,6 @@ graft examples
 graft extra
 include *.md *.txt *.sh *.yml MANIFEST.in
 recursive-include docs   *.rst *.png Makefile *.py *.txt
-recursive-include pwnlib *.py *.asm *.rst *.md *.txt *.sh __doc__
+recursive-include pwnlib *.py *.asm *.rst *.md *.txt *.sh __doc__ *.mako
 recursive-include pwn    *.py *.asm *.rst *.md *.txt *.sh
 recursive-exclude *.pyc

README.md

@@ -2,7 +2,7 @@
 ![pwntools logo](https://github.com/Gallopsled/pwntools/blob/stable/docs/source/logo.png?raw=true)
 
 [![Docs](https://readthedocs.org/projects/pwntools/badge/?version=stable)](https://docs.pwntools.com/)
-[![PyPI](https://img.shields.io/badge/pypi-v3.5.1-green.svg?style=flat)](https://pypi.python.org/pypi/pwntools/)
+[![PyPI](https://img.shields.io/badge/pypi-v3.7.0-green.svg?style=flat)](https://pypi.python.org/pypi/pwntools/)
 [![Travis](https://travis-ci.org/Gallopsled/pwntools.svg)](https://travis-ci.org/Gallopsled/pwntools)
 [![Coveralls](https://img.shields.io/coveralls/Gallopsled/pwntools/dev.svg)](https://coveralls.io/github/Gallopsled/pwntools?branch=dev)
 [![Twitter](https://img.shields.io/badge/twitter-pwntools-4099FF.svg?style=flat)](https://twitter.com/pwntools)

TESTING.md

@@ -4,7 +4,6 @@ Pwntools makes extensive use of unit tests and integration tests to ensure every
 
 ## Test Suite
 
-
 To run the test suite, it is best to use Ubuntu 12.04 or 14.04, and run the following commands.  **Be aware** that this will add a user to the machine, and create a public key for SSH login!
 
 ```sh
@@ -14,6 +13,12 @@ pip install --upgrade --editable .
 PWNLIB_NOTERM=1 make -C docs doctest
 ```
 
+## Testing in Docker
+
+A `Dockerfile` has been provided which has a clean testing environment with Ubuntu Xenial.  It is very similar to the online Travis CI testing environment, but uses a more modern version of Ubuntu.
+
+See `travis/docker/README.md` for more information.
+
 ## New Tests
 
 To add a new test to an existing module, just add an inline doctest.  If the test needs access to an external module, add the import statement to the `testsetup` block in the corresponding file in `docs/source/<module>.rst`.

docs/source/conf.py

@@ -357,5 +357,46 @@ if build_dash:
 
 # -- Customization to Sphinx autodoc generation --------------------------------------------
 import sphinx.ext.autodoc
+from sphinx.util.inspect import safe_getmembers, safe_getattr
 
-sphinx.ext.autodoc
+# Test hidden members (e.g. def _foo(...))
+def dont_skip_any_doctests(app, what, name, obj, skip, options):
+    return False
+
+# Test non-exported members
+class ModuleDocumenter(sphinx.ext.autodoc.ModuleDocumenter):
+    def get_object_members(self, want_all):
+        if want_all:
+            # if not hasattr(self.object, '__all__'):
+            #     for implicit module members, check __module__ to avoid
+            #     documenting imported objects
+                return True, safe_getmembers(self.object)
+            # else:
+            #     memberlist = self.object.__all__
+            #     # Sometimes __all__ is broken...
+            #     if not isinstance(memberlist, (list, tuple)) or not \
+            #        all(isinstance(entry, string_types) for entry in memberlist):
+            #         self.directive.warn(
+            #             '__all__ should be a list of strings, not %r '
+            #             '(in module %s) -- ignoring __all__' %
+            #             (memberlist, self.fullname))
+            #         # fall back to all members
+            #         return True, safe_getmembers(self.object)
+        else:
+            memberlist = self.options.members or []
+        ret = []
+        for mname in memberlist:
+            try:
+                ret.append((mname, safe_getattr(self.object, mname)))
+            except AttributeError:
+                self.directive.warn(
+                    'missing attribute mentioned in :members: or __all__: '
+                    'module %s, attribute %s' % (
+                        safe_getattr(self.object, '__name__', '???'), mname))
+        return False, ret
+
+if 'doctest' in sys.argv:
+    def setup(app):
+        app.connect('autodoc-skip-member', dont_skip_any_doctests)
+
+    sphinx.ext.autodoc.ModuleDocumenter = ModuleDocumenter

docs/source/config.rst

@@ -0,0 +1,4 @@
+:mod:`pwnlib.config` --- Pwntools Configuration File
+====================================================
+
+.. automodule:: pwnlib.config

docs/source/context.rst

@@ -27,11 +27,7 @@ The recommended method is to use ``context.binary``  to automagically set all of
     context.binary = './challenge-binary'
 
 Module Members
-=====================================================
+----------------------------------------------------
 
-.. autodata:: pwnlib.context.context
-
-.. autoclass:: pwnlib.context.ContextType
-    :members:
-
-.. autoclass:: pwnlib.context.Thread
+.. automodule:: pwnlib.context
+   :members:

docs/source/elf/elf.rst

@@ -1,6 +1,7 @@
 .. testsetup:: *
 
    from pwn import *
+   from glob import glob
 
 :mod:`pwnlib.elf.elf` --- ELF Files
 ===========================================================

docs/source/index.rst

@@ -48,6 +48,7 @@ Each of the ``pwntools`` modules is documented here.
    atexception
    atexit
    constants
+   config
    context
    dynelf
    encoders

docs/source/intro.rst

@@ -72,7 +72,7 @@ a ``process`` tube.
 
 ::
 
-    >>> shell = ssh('bandit0', 'bandit.labs.overthewire.org', password='bandit0')
+    >>> shell = ssh('bandit0', 'bandit.labs.overthewire.org', password='bandit0', port=2220)
     >>> shell['whoami']
     'bandit0'
     >>> shell.download_file('/etc/motd')

docs/source/memleak.rst

@@ -1,11 +1,9 @@
 .. testsetup:: *
 
-   from pwnlib.memleak import *
+    from pwn import *
 
 :mod:`pwnlib.memleak` --- Helper class for leaking memory
 =========================================================
 
-.. module:: pwnlib.memleak
-
-.. autoclass:: MemLeak
-   :members:
+.. automodule:: pwnlib.memleak
+    :members:

docs/source/util/safeeval.rst

@@ -1,6 +1,7 @@
 .. testsetup:: *
 
    from pwnlib.util.safeeval import *
+   _get_opcodes = pwnlib.util.safeeval._get_opcodes
 
 
 :mod:`pwnlib.util.safeeval` --- Safe evaluation of python code

examples/ssh.py

@@ -4,7 +4,7 @@ Example showing how to use the ssh class.
 
 from pwn import *
 
-shell = ssh(host='bandit.labs.overthewire.org',user='bandit0',password='bandit0')
+shell = ssh(host='bandit.labs.overthewire.org',user='bandit0',password='bandit0', port=2220)
 
 # Show basic command syntax
 log.info("username: %s" % shell.whoami())

pwn/__init__.py

@@ -5,6 +5,7 @@ from pwn.toplevel import *
 
 pwnlib.args.initialize()
 pwnlib.log.install_default_handler()
+pwnlib.config.initialize()
 
 log = pwnlib.log.getLogger('pwnlib.exploit')
 args = pwnlib.args.args

pwnlib/__init__.py

@@ -14,6 +14,7 @@ __all__ = [
     'commandline',
     'constants',
     'context',
+    'data',
     'dynelf',
     'encoders',
     'elf',

pwnlib/adb/adb.py

@@ -346,7 +346,7 @@ class AdbDevice(Device):
         of this device.
 
         >>> property = 'ro.build.fingerprint'
-        >>> device = adb.current_device()
+        >>> device = adb.wait_for_device()
         >>> adb.getprop(property) == device.getprop(property)
         True
         """

pwnlib/args.py

@@ -51,8 +51,12 @@ import sys
 from pwnlib import term
 from pwnlib.context import context
 
+class PwnlibArgs(collections.defaultdict):
+    def __getattr__(self, attr):
+        return self[attr]
+
+args = PwnlibArgs(str)
 term_mode  = True
-args       = collections.defaultdict(str)
 env_prefix = 'PWNLIB_'
 free_form  = True
 

pwnlib/asm.py

@@ -146,8 +146,9 @@ def which_binutils(util):
     Examples:
 
         >>> import platform
+        >>> which_binutils = pwnlib.asm.which_binutils
         >>> which_binutils('as', arch=platform.machine())
-        '.../bin/as'
+        '.../bin/...as'
         >>> which_binutils('as', arch='arm') #doctest: +ELLIPSIS
         '.../bin/arm-...-as'
         >>> which_binutils('as', arch='powerpc') #doctest: +ELLIPSIS
@@ -783,7 +784,7 @@ def disasm(data, vma = 0, byte = True, offset = True, instructions = True):
 
         res = _run(objcopy + [step1, step2])
 
-        output0 = subprocess.check_output(objdump + [step2])
+        output0 = _run(objdump + [step2])
         output1 = output0.split('<.text>:\n')
 
         if len(output1) != 2:

pwnlib/commandline/__init__.py

@@ -5,7 +5,9 @@ __all__ = [
     'common',
     'constgrep',
     'cyclic',
+    'debug',
     'disasm',
+    'disablenx',
     'elfdiff',
     'elfpatch',
     'errno',

pwnlib/commandline/asm.py

@@ -96,6 +96,10 @@ parser.add_argument(
 def main(args):
     tty    = args.output.isatty()
 
+    if args.infile.isatty() and not args.lines:
+        parser.print_usage()
+        sys.exit(1)
+
     data   = '\n'.join(args.lines) or args.infile.read()
     output = asm(data.replace(';', '\n'))
     fmt    = args.format or ('hex' if tty else 'raw')
@@ -125,4 +129,4 @@ def main(args):
         args.output.write('\n')
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/checksec.py

@@ -37,4 +37,4 @@ def main(args):
         e = ELF(f.name)
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/constgrep.py

@@ -139,4 +139,4 @@ def main(args):
                 print '(%s) == %s' % (' | '.join(k for v, k in good), args.constant)
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/cyclic.py

@@ -95,4 +95,4 @@ def main(args):
             sys.stdout.write('\n')
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/debug.py

@@ -0,0 +1,69 @@
+#!/usr/bin/env python2
+from __future__ import absolute_import
+
+import argparse
+import sys
+
+from pwn import *
+from pwnlib.commandline import common
+
+parser = common.parser_commands.add_parser(
+    'debug',
+    help = 'Debug a binary in GDB'
+)
+parser.add_argument(
+    '-x', metavar='GDBSCRIPT',
+    type=file,
+    help='Execute GDB commands from this file.'
+)
+parser.add_argument(
+    '--pid',
+    type=int,
+    help="PID to attach to"
+)
+parser.add_argument(
+    '-c', '--context',
+    metavar = 'context',
+    action = 'append',
+    type   = common.context_arg,
+    choices = common.choices,
+    help = 'The os/architecture/endianness/bits the shellcode will run in (default: linux/i386), choose from: %s' % common.choices,
+)
+parser.add_argument(
+    '--exec', type=file, dest='executable',
+    help='File to debug'
+)
+parser.add_argument(
+    '--process', metavar='PROCESS_NAME',
+    help='Name of the process to attach to (e.g. "bash")'
+)
+
+def main(args):
+    gdbscript = ''
+    if args.x:
+        gdbscript = args.x.read()
+
+    if context.os == 'android':
+        context.device = adb.wait_for_device()
+
+    if args.executable:
+        context.binary = ELF(args.executable.name)
+        target = context.binary.path
+    elif args.pid:
+        target = int(args.pid)
+    elif args.process:
+        if context.os == 'android':
+            target = adb.pidof(process)
+        else:
+            target = pidof(process)
+    else:
+        parser.print_usage()
+        return 1
+
+    if args.pid or args.process:
+        gdb.attach(target, gdbscript=gdbscript)
+    else:
+        gdb.debug(target, gdbscript=gdbscript).interactive()
+
+if __name__ == '__main__':
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/disablenx.py

@@ -0,0 +1,25 @@
+#!/usr/bin/env python2
+from __future__ import absolute_import
+
+from pwn import *
+from pwnlib.commandline import common
+
+parser = common.parser_commands.add_parser(
+    'disablenx',
+    help = 'Disable NX for an ELF binary'
+)
+parser.add_argument(
+    'elf',
+    nargs='+',
+    type=file,
+    help='Files to check'
+)
+
+def main(args):
+    for f in args.elf:
+        e = ELF(f.name)
+        e.disable_nx()
+        ELF(e.path)
+
+if __name__ == '__main__':
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/disasm.py

@@ -92,4 +92,4 @@ def main(args):
     print disasm(dat, vma=safeeval.const(args.address))
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/elfdiff.py

@@ -58,4 +58,4 @@ def main(a):
     print diff(x, y)
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/elfpatch.py

@@ -35,4 +35,4 @@ def main(a):
     sys.stdout.write(elf.get_data())
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/errno.py

@@ -44,4 +44,4 @@ def main(args):
   print os.strerror(value)
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/hex.py

@@ -21,4 +21,4 @@ def main(args):
         print ' '.join(args.data).encode('hex')
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/main.py

@@ -7,7 +7,9 @@ from pwnlib.commandline import checksec
 from pwnlib.commandline import common
 from pwnlib.commandline import constgrep
 from pwnlib.commandline import cyclic
+from pwnlib.commandline import debug
 from pwnlib.commandline import disasm
+from pwnlib.commandline import disablenx
 from pwnlib.commandline import elfdiff
 from pwnlib.commandline import elfpatch
 from pwnlib.commandline import errno
@@ -16,6 +18,7 @@ from pwnlib.commandline import phd
 from pwnlib.commandline import pwnstrip
 from pwnlib.commandline import scramble
 from pwnlib.commandline import shellcraft
+from pwnlib.commandline import template
 from pwnlib.commandline import unhex
 from pwnlib.commandline import update
 from pwnlib.commandline.common import parser
@@ -26,7 +29,9 @@ commands = {
     'checksec': checksec.main,
     'constgrep': constgrep.main,
     'cyclic': cyclic.main,
+    'debug': debug.main,
     'disasm': disasm.main,
+    'disablenx': disablenx.main,
     'elfdiff': elfdiff.main,
     'elfpatch': elfpatch.main,
     'errno': errno.main,
@@ -35,6 +40,7 @@ commands = {
     'pwnstrip': pwnstrip.main,
     'scramble': scramble.main,
     'shellcraft': shellcraft.main,
+    'template': template.main,
     'unhex': unhex.main,
     'update': update.main,
 }

pwnlib/commandline/phd.py

@@ -101,4 +101,4 @@ def main(args):
         pass
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/pwnstrip.py

@@ -51,4 +51,4 @@ def main(args):
     args.output.write(result)
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/scramble.py

@@ -105,4 +105,4 @@ def main(args):
 
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/shellcraft.py

@@ -342,4 +342,4 @@ def main(args):
     args.out.write(code)
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/template.py

@@ -0,0 +1,64 @@
+#!/usr/bin/env python2
+from __future__ import absolute_import
+
+import re
+
+from pwn import *
+from pwnlib.commandline import common
+
+from mako.lookup import TemplateLookup
+
+parser = common.parser_commands.add_parser(
+    'template',
+    help = 'Generate an exploit template'
+)
+
+parser.add_argument('exe', nargs='?', help='Target binary')
+parser.add_argument('--host', help='Remote host / SSH server')
+parser.add_argument('--port', help='Remote port / SSH port', type=int)
+parser.add_argument('--user', help='SSH Username')
+parser.add_argument('--pass', help='SSH Password', dest='password')
+parser.add_argument('--path', help='Remote path of file on SSH server')
+
+def main(args):
+    cache = None
+
+    if cache:
+        cache = os.path.join(context.cache_dir, 'mako')
+
+    lookup = TemplateLookup(
+        directories      = [os.path.join(pwnlib.data.path, 'templates')],
+        module_directory = cache
+    )
+
+    # For the SSH scenario, check that the binary is at the
+    # same path on the remote host.
+    if args.user:
+        if not (args.path or args.exe):
+            log.error("Must specify --path or a exe")
+
+        s = ssh(args.user, args.host, args.port or 22, args.password or None)
+        s.download(args.path or args.exe)
+
+        if not args.exe:
+            args.exe = os.path.basename(args.path)
+
+    template = lookup.get_template('pwnup.mako')
+    output = template.render(args.exe,
+                             args.host,
+                             args.port,
+                             args.user,
+                             args.password,
+                             args.path)
+
+    # Fix Mako formatting bs
+    output = re.sub('\n\n\n', '\n\n', output)
+
+    print output
+
+    if not sys.stdout.isatty():
+        try: os.fchmod(sys.stdout.fileno(), 0700)
+        except OSError: pass
+
+if __name__ == '__main__':
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/unhex.py

@@ -27,4 +27,4 @@ def main(args):
         sys.stderr.write(str(e) + '\n')
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/commandline/update.py

@@ -28,4 +28,4 @@ def main(a):
         subprocess.check_call(result, shell=False)
 
 if __name__ == '__main__':
-    pwnlib.common.main(__file__)
+    pwnlib.commandline.common.main(__file__)

pwnlib/config.py

@@ -0,0 +1,60 @@
+# -*- coding: utf-8 -*-
+"""Allows per-user and per-host configuration of Pwntools settings.
+
+The list of configurable options includes all of the logging symbols
+and colors, as well as all of the default values on the global context
+object.
+
+The configuration file is read from ``~/.pwn.conf`` and ``/etc/pwn.conf``.
+
+The configuration file is only read in ``from pwn import *`` mode, and not
+when used in library mode (``import pwnlib``).  To read the configuration
+file in library mode, invoke :func:`.config.initialize`.
+
+The ``context`` section supports complex types, at least as far as is
+supported by ``pwnlib.util.safeeval.expr``.
+
+::
+
+    [log]
+    success.symbol=😎
+    error.symbol=☠
+    info.color=blue
+
+    [context]
+    adb_port=4141
+    randomize=1
+    timeout=60
+    terminal=['x-terminal-emulator', '-e']
+"""
+from __future__ import absolute_import
+
+import ConfigParser
+import os
+
+registered_configs = {}
+
+def register_config(section, function):
+    """Registers a configuration section.
+
+    Arguments:
+        section(str): Named configuration section
+        function(callable): Function invoked with a dictionary of
+            ``{option: value}`` for the entries in the section.
+    """
+    registered_configs[section] = function
+
+def initialize():
+    """Read the configuration files."""
+    from pwnlib.log import getLogger
+    log = getLogger(__name__)
+
+    c = ConfigParser.ConfigParser()
+    c.read(['/etc/pwn.conf', os.path.expanduser('~/.pwn.conf')])
+
+    for section in c.sections():
+        if section not in registered_configs:
+            log.warn("Unknown configuration section %r" % section)
+            continue
+        settings = dict(c.items(section))
+        registered_configs[section](settings)

pwnlib/constants/__init__.py

@@ -39,6 +39,16 @@ Example:
     ...     print int(constants.SYS_stat)
     4
 
+    >>> with context.local(arch = 'i386', os = 'linux'):
+    ...    print constants.SYS_execve + constants.PROT_WRITE
+    13
+    >>> with context.local(arch = 'amd64', os = 'linux'):
+    ...    print constants.SYS_execve + constants.PROT_WRITE
+    61
+    >>> with context.local(arch = 'amd64', os = 'linux'):
+    ...    print constants.SYS_execve + constants.PROT_WRITE
+    61
+
 """
 from __future__ import absolute_import
 
@@ -56,19 +66,6 @@ class ConstantsModule(ModuleType):
     ModuleType specialization in order to automatically
     route queries down to the correct module based on the
     current context arch / os.
-
-        >>> with context.local(arch = 'i386', os = 'linux'):
-        ...    print constants.SYS_execve + constants.PROT_WRITE
-        13
-        >>> with context.local(arch = 'amd64', os = 'linux'):
-        ...    print constants.SYS_execve + constants.PROT_WRITE
-        61
-        >>> with context.local(arch = 'amd64', os = 'linux'):
-        ...    print constants.SYS_execve + constants.PROT_WRITE
-        61
-        >>> False
-        True
-
     """
     Constant = Constant
 

pwnlib/constants/linux/aarch64.py

@@ -1,5 +1,4 @@
 from pwnlib.constants.constant import Constant
-
 __NR_io_setup = Constant('__NR_io_setup',0)
 __NR_io_destroy = Constant('__NR_io_destroy',1)
 __NR_io_submit = Constant('__NR_io_submit',2)
@@ -259,63 +258,6 @@ __NR_process_vm_readv = Constant('__NR_process_vm_readv',270)
 __NR_process_vm_writev = Constant('__NR_process_vm_writev',271)
 __NR_kcmp = Constant('__NR_kcmp',272)
 __NR_finit_module = Constant('__NR_finit_module',273)
-__NR_open = Constant('__NR_open',1024)
-__NR_link = Constant('__NR_link',1025)
-__NR_unlink = Constant('__NR_unlink',1026)
-__NR_mknod = Constant('__NR_mknod',1027)
-__NR_chmod = Constant('__NR_chmod',1028)
-__NR_chown = Constant('__NR_chown',1029)
-__NR_mkdir = Constant('__NR_mkdir',1030)
-__NR_rmdir = Constant('__NR_rmdir',1031)
-__NR_lchown = Constant('__NR_lchown',1032)
-__NR_access = Constant('__NR_access',1033)
-__NR_rename = Constant('__NR_rename',1034)
-__NR_readlink = Constant('__NR_readlink',1035)
-__NR_symlink = Constant('__NR_symlink',1036)
-__NR_utimes = Constant('__NR_utimes',1037)
-__NR_stat = Constant('__NR_stat',1038)
-__NR_lstat = Constant('__NR_lstat',1039)
-__NR_pipe = Constant('__NR_pipe',1040)
-__NR_dup2 = Constant('__NR_dup2',1041)
-__NR_epoll_create = Constant('__NR_epoll_create',1042)
-__NR_inotify_init = Constant('__NR_inotify_init',1043)
-__NR_eventfd = Constant('__NR_eventfd',1044)
-__NR_signalfd = Constant('__NR_signalfd',1045)
-__NR_sendfile64 = Constant('__NR_sendfile64',1046)
-__NR_ftruncate64 = Constant('__NR_ftruncate64',1047)
-__NR_truncate64 = Constant('__NR_truncate64',1048)
-__NR_stat64 = Constant('__NR_stat64',1049)
-__NR_lstat64 = Constant('__NR_lstat64',1050)
-__NR_fstat64 = Constant('__NR_fstat64',1051)
-__NR_fcntl64 = Constant('__NR_fcntl64',1052)
-__NR_newfstatat = Constant('__NR_newfstatat',1054)
-__NR_fstatfs64 = Constant('__NR_fstatfs64',1055)
-__NR_statfs64 = Constant('__NR_statfs64',1056)
-__NR_lseek64 = Constant('__NR_lseek64',1057)
-__NR_mmap64 = Constant('__NR_mmap64',1058)
-__NR_alarm = Constant('__NR_alarm',1059)
-__NR_getpgrp = Constant('__NR_getpgrp',1060)
-__NR_pause = Constant('__NR_pause',1061)
-__NR_time = Constant('__NR_time',1062)
-__NR_utime = Constant('__NR_utime',1063)
-__NR_creat = Constant('__NR_creat',1064)
-__NR_getdents = Constant('__NR_getdents',1065)
-__NR_futimesat = Constant('__NR_futimesat',1066)
-__NR_select = Constant('__NR_select',1067)
-__NR_poll = Constant('__NR_poll',1068)
-__NR_epoll_wait = Constant('__NR_epoll_wait',1069)
-__NR_ustat = Constant('__NR_ustat',1070)
-__NR_vfork = Constant('__NR_vfork',1071)
-__NR_oldwait4 = Constant('__NR_oldwait4',1072)
-__NR_recv = Constant('__NR_recv',1073)
-__NR_send = Constant('__NR_send',1074)
-__NR_bdflush = Constant('__NR_bdflush',1075)
-__NR_umount = Constant('__NR_umount',1076)
-__NR_uselib = Constant('__NR_uselib',1077)
-__NR__sysctl = Constant('__NR__sysctl',1078)
-__NR_fork = Constant('__NR_fork',1079)
-__NR_syscalls = Constant('__NR_syscalls',(1079+1))
-__NR_sigreturn = Constant('__NR_sigreturn',1999)
 MAP_32BIT = Constant('MAP_32BIT',0x40)
 INADDR_ANY = Constant('INADDR_ANY',0)
 INADDR_BROADCAST = Constant('INADDR_BROADCAST',0xffffffff)
@@ -944,19 +886,14 @@ CC_N_BIT = Constant('CC_N_BIT',(1 << 31))
 PCMASK = Constant('PCMASK',0)
 SYS_accept = Constant('SYS_accept',202)
 SYS_accept4 = Constant('SYS_accept4',242)
-SYS_access = Constant('SYS_access',1033)
 SYS_acct = Constant('SYS_acct',89)
 SYS_add_key = Constant('SYS_add_key',217)
 SYS_adjtimex = Constant('SYS_adjtimex',171)
-SYS_alarm = Constant('SYS_alarm',1059)
-SYS_bdflush = Constant('SYS_bdflush',1075)
 SYS_bind = Constant('SYS_bind',200)
 SYS_brk = Constant('SYS_brk',214)
 SYS_capget = Constant('SYS_capget',90)
 SYS_capset = Constant('SYS_capset',91)
 SYS_chdir = Constant('SYS_chdir',49)
-SYS_chmod = Constant('SYS_chmod',1028)
-SYS_chown = Constant('SYS_chown',1029)
 SYS_chroot = Constant('SYS_chroot',51)
 SYS_clock_getres = Constant('SYS_clock_getres',114)
 SYS_clock_gettime = Constant('SYS_clock_gettime',113)
@@ -965,17 +902,12 @@ SYS_clock_settime = Constant('SYS_clock_settime',112)
 SYS_clone = Constant('SYS_clone',220)
 SYS_close = Constant('SYS_close',57)
 SYS_connect = Constant('SYS_connect',203)
-SYS_creat = Constant('SYS_creat',1064)
 SYS_delete_module = Constant('SYS_delete_module',106)
 SYS_dup = Constant('SYS_dup',23)
-SYS_dup2 = Constant('SYS_dup2',1041)
 SYS_dup3 = Constant('SYS_dup3',24)
-SYS_epoll_create = Constant('SYS_epoll_create',1042)
 SYS_epoll_create1 = Constant('SYS_epoll_create1',20)
 SYS_epoll_ctl = Constant('SYS_epoll_ctl',21)
 SYS_epoll_pwait = Constant('SYS_epoll_pwait',22)
-SYS_epoll_wait = Constant('SYS_epoll_wait',1069)
-SYS_eventfd = Constant('SYS_eventfd',1044)
 SYS_eventfd2 = Constant('SYS_eventfd2',19)
 SYS_execve = Constant('SYS_execve',221)
 SYS_exit = Constant('SYS_exit',93)
@@ -991,27 +923,20 @@ SYS_fchmodat = Constant('SYS_fchmodat',53)
 SYS_fchown = Constant('SYS_fchown',55)
 SYS_fchownat = Constant('SYS_fchownat',54)
 SYS_fcntl = Constant('SYS_fcntl',25)
-SYS_fcntl64 = Constant('SYS_fcntl64',1052)
 SYS_fdatasync = Constant('SYS_fdatasync',83)
 SYS_fgetxattr = Constant('SYS_fgetxattr',10)
 SYS_flistxattr = Constant('SYS_flistxattr',13)
 SYS_flock = Constant('SYS_flock',32)
-SYS_fork = Constant('SYS_fork',1079)
 SYS_fremovexattr = Constant('SYS_fremovexattr',16)
 SYS_fsetxattr = Constant('SYS_fsetxattr',7)
 SYS_fstat = Constant('SYS_fstat',80)
-SYS_fstat64 = Constant('SYS_fstat64',1051)
 SYS_fstatat64 = Constant('SYS_fstatat64',79)
 SYS_fstatfs = Constant('SYS_fstatfs',44)
-SYS_fstatfs64 = Constant('SYS_fstatfs64',1055)
 SYS_fsync = Constant('SYS_fsync',82)
 SYS_ftruncate = Constant('SYS_ftruncate',46)
-SYS_ftruncate64 = Constant('SYS_ftruncate64',1047)
 SYS_futex = Constant('SYS_futex',98)
-SYS_futimesat = Constant('SYS_futimesat',1066)
 SYS_getcpu = Constant('SYS_getcpu',168)
 SYS_getcwd = Constant('SYS_getcwd',17)
-SYS_getdents = Constant('SYS_getdents',1065)
 SYS_getdents64 = Constant('SYS_getdents64',61)
 SYS_getegid = Constant('SYS_getegid',177)
 SYS_geteuid = Constant('SYS_geteuid',175)
@@ -1021,7 +946,6 @@ SYS_getitimer = Constant('SYS_getitimer',102)
 SYS_get_mempolicy = Constant('SYS_get_mempolicy',236)
 SYS_getpeername = Constant('SYS_getpeername',205)
 SYS_getpgid = Constant('SYS_getpgid',155)
-SYS_getpgrp = Constant('SYS_getpgrp',1060)
 SYS_getpid = Constant('SYS_getpid',172)
 SYS_getppid = Constant('SYS_getppid',173)
 SYS_getpriority = Constant('SYS_getpriority',141)
@@ -1039,7 +963,6 @@ SYS_getuid = Constant('SYS_getuid',174)
 SYS_getxattr = Constant('SYS_getxattr',8)
 SYS_init_module = Constant('SYS_init_module',105)
 SYS_inotify_add_watch = Constant('SYS_inotify_add_watch',27)
-SYS_inotify_init = Constant('SYS_inotify_init',1043)
 SYS_inotify_init1 = Constant('SYS_inotify_init1',26)
 SYS_inotify_rm_watch = Constant('SYS_inotify_rm_watch',28)
 SYS_io_cancel = Constant('SYS_io_cancel',3)
@@ -1053,9 +976,7 @@ SYS_io_submit = Constant('SYS_io_submit',2)
 SYS_kexec_load = Constant('SYS_kexec_load',104)
 SYS_keyctl = Constant('SYS_keyctl',219)
 SYS_kill = Constant('SYS_kill',129)
-SYS_lchown = Constant('SYS_lchown',1032)
 SYS_lgetxattr = Constant('SYS_lgetxattr',9)
-SYS_link = Constant('SYS_link',1025)
 SYS_linkat = Constant('SYS_linkat',37)
 SYS_listen = Constant('SYS_listen',201)
 SYS_listxattr = Constant('SYS_listxattr',11)
@@ -1064,15 +985,11 @@ SYS_lookup_dcookie = Constant('SYS_lookup_dcookie',18)
 SYS_lremovexattr = Constant('SYS_lremovexattr',15)
 SYS_lseek = Constant('SYS_lseek',62)
 SYS_lsetxattr = Constant('SYS_lsetxattr',6)
-SYS_lstat = Constant('SYS_lstat',1039)
-SYS_lstat64 = Constant('SYS_lstat64',1050)
 SYS_madvise = Constant('SYS_madvise',233)
 SYS_mbind = Constant('SYS_mbind',235)
 SYS_migrate_pages = Constant('SYS_migrate_pages',238)
 SYS_mincore = Constant('SYS_mincore',232)
-SYS_mkdir = Constant('SYS_mkdir',1030)
 SYS_mkdirat = Constant('SYS_mkdirat',34)
-SYS_mknod = Constant('SYS_mknod',1027)
 SYS_mknodat = Constant('SYS_mknodat',33)
 SYS_mlock = Constant('SYS_mlock',228)
 SYS_mlockall = Constant('SYS_mlockall',230)
@@ -1096,17 +1013,12 @@ SYS_munlock = Constant('SYS_munlock',229)
 SYS_munlockall = Constant('SYS_munlockall',231)
 SYS_munmap = Constant('SYS_munmap',215)
 SYS_nanosleep = Constant('SYS_nanosleep',101)
-SYS_newfstatat = Constant('SYS_newfstatat',1054)
 SYS_nfsservctl = Constant('SYS_nfsservctl',42)
-SYS_open = Constant('SYS_open',1024)
 SYS_openat = Constant('SYS_openat',56)
-SYS_pause = Constant('SYS_pause',1061)
 SYS_perf_event_open = Constant('SYS_perf_event_open',241)
 SYS_personality = Constant('SYS_personality',92)
-SYS_pipe = Constant('SYS_pipe',1040)
 SYS_pipe2 = Constant('SYS_pipe2',59)
 SYS_pivot_root = Constant('SYS_pivot_root',41)
-SYS_poll = Constant('SYS_poll',1068)
 SYS_ppoll = Constant('SYS_ppoll',73)
 SYS_prctl = Constant('SYS_prctl',167)
 SYS_pread64 = Constant('SYS_pread64',67)
@@ -1119,21 +1031,17 @@ SYS_pwritev = Constant('SYS_pwritev',70)
 SYS_quotactl = Constant('SYS_quotactl',60)
 SYS_read = Constant('SYS_read',63)
 SYS_readahead = Constant('SYS_readahead',213)
-SYS_readlink = Constant('SYS_readlink',1035)
 SYS_readlinkat = Constant('SYS_readlinkat',78)
 SYS_readv = Constant('SYS_readv',65)
 SYS_reboot = Constant('SYS_reboot',142)
-SYS_recv = Constant('SYS_recv',1073)
 SYS_recvfrom = Constant('SYS_recvfrom',207)
 SYS_recvmmsg = Constant('SYS_recvmmsg',243)
 SYS_recvmsg = Constant('SYS_recvmsg',212)
 SYS_remap_file_pages = Constant('SYS_remap_file_pages',234)
 SYS_removexattr = Constant('SYS_removexattr',14)
-SYS_rename = Constant('SYS_rename',1034)
 SYS_renameat = Constant('SYS_renameat',38)
 SYS_request_key = Constant('SYS_request_key',218)
 SYS_restart_syscall = Constant('SYS_restart_syscall',128)
-SYS_rmdir = Constant('SYS_rmdir',1031)
 SYS_rt_sigaction = Constant('SYS_rt_sigaction',134)
 SYS_rt_sigpending = Constant('SYS_rt_sigpending',136)
 SYS_rt_sigprocmask = Constant('SYS_rt_sigprocmask',135)
@@ -1152,14 +1060,11 @@ SYS_sched_setaffinity = Constant('SYS_sched_setaffinity',122)
 SYS_sched_setparam = Constant('SYS_sched_setparam',118)
 SYS_sched_setscheduler = Constant('SYS_sched_setscheduler',119)
 SYS_sched_yield = Constant('SYS_sched_yield',124)
-SYS_select = Constant('SYS_select',1067)
 SYS_semctl = Constant('SYS_semctl',191)
 SYS_semget = Constant('SYS_semget',190)
 SYS_semop = Constant('SYS_semop',193)
 SYS_semtimedop = Constant('SYS_semtimedop',192)
-SYS_send = Constant('SYS_send',1074)
 SYS_sendfile = Constant('SYS_sendfile',71)
-SYS_sendfile64 = Constant('SYS_sendfile64',1046)
 SYS_sendmsg = Constant('SYS_sendmsg',211)
 SYS_sendto = Constant('SYS_sendto',206)
 SYS_setdomainname = Constant('SYS_setdomainname',162)
@@ -1190,28 +1095,20 @@ SYS_shmdt = Constant('SYS_shmdt',197)
 SYS_shmget = Constant('SYS_shmget',194)
 SYS_shutdown = Constant('SYS_shutdown',210)
 SYS_sigaltstack = Constant('SYS_sigaltstack',132)
-SYS_signalfd = Constant('SYS_signalfd',1045)
 SYS_signalfd4 = Constant('SYS_signalfd4',74)
-SYS_sigreturn = Constant('SYS_sigreturn',1999)
 SYS_socket = Constant('SYS_socket',198)
 SYS_socketpair = Constant('SYS_socketpair',199)
 SYS_splice = Constant('SYS_splice',76)
-SYS_stat = Constant('SYS_stat',1038)
-SYS_stat64 = Constant('SYS_stat64',1049)
 SYS_statfs = Constant('SYS_statfs',43)
-SYS_statfs64 = Constant('SYS_statfs64',1056)
 SYS_swapoff = Constant('SYS_swapoff',225)
 SYS_swapon = Constant('SYS_swapon',224)
-SYS_symlink = Constant('SYS_symlink',1036)
 SYS_symlinkat = Constant('SYS_symlinkat',36)
 SYS_sync = Constant('SYS_sync',81)
 SYS_sync_file_range2 = Constant('SYS_sync_file_range2',84)
-SYS__sysctl = Constant('SYS__sysctl',1078)
 SYS_sysinfo = Constant('SYS_sysinfo',179)
 SYS_syslog = Constant('SYS_syslog',116)
 SYS_tee = Constant('SYS_tee',77)
 SYS_tgkill = Constant('SYS_tgkill',131)
-SYS_time = Constant('SYS_time',1062)
 SYS_timer_create = Constant('SYS_timer_create',107)
 SYS_timer_delete = Constant('SYS_timer_delete',111)
 SYS_timerfd_create = Constant('SYS_timerfd_create',85)
@@ -1223,20 +1120,12 @@ SYS_timer_settime = Constant('SYS_timer_settime',110)
 SYS_times = Constant('SYS_times',153)
 SYS_tkill = Constant('SYS_tkill',130)
 SYS_truncate = Constant('SYS_truncate',45)
-SYS_truncate64 = Constant('SYS_truncate64',1048)
 SYS_umask = Constant('SYS_umask',166)
-SYS_umount = Constant('SYS_umount',1076)
 SYS_umount2 = Constant('SYS_umount2',39)
 SYS_uname = Constant('SYS_uname',160)
-SYS_unlink = Constant('SYS_unlink',1026)
 SYS_unlinkat = Constant('SYS_unlinkat',35)
 SYS_unshare = Constant('SYS_unshare',97)
-SYS_uselib = Constant('SYS_uselib',1077)
-SYS_ustat = Constant('SYS_ustat',1070)
-SYS_utime = Constant('SYS_utime',1063)
 SYS_utimensat = Constant('SYS_utimensat',88)
-SYS_utimes = Constant('SYS_utimes',1037)
-SYS_vfork = Constant('SYS_vfork',1071)
 SYS_vhangup = Constant('SYS_vhangup',58)
 SYS_vmsplice = Constant('SYS_vmsplice',75)
 SYS_wait4 = Constant('SYS_wait4',260)

pwnlib/context/__init__.py

@@ -21,9 +21,12 @@ import time
 
 import socks
 
+from pwnlib.config import register_config
 from pwnlib.device import Device
 from pwnlib.timeout import Timeout
 
+__all__ = ['context', 'ContextType', 'Thread']
+
 _original_socket = socket.socket
 
 class _devnull(object):
@@ -246,9 +249,10 @@ def _longest(d):
     as it ensures the most complete match will be found.
 
     >>> data = {'a': 1, 'bb': 2, 'ccc': 3}
-    >>> _longest(data) == data
+    >>> pwnlib.context._longest(data) == data
     True
-    >>> for i in _longest(data): print i
+    >>> for i in pwnlib.context._longest(data):
+    ...     print i
     ccc
     bb
     a
@@ -347,7 +351,7 @@ class ContextType(object):
         'os': 'linux',
         'proxy': None,
         'signed': False,
-        'terminal': None,
+        'terminal': tuple(),
         'timeout': Timeout.maximum,
     }
 
@@ -1326,6 +1330,7 @@ def LocalContext(function):
 
     Example:
 
+        >>> context.clear()
         >>> @LocalContext
         ... def printArch():
         ...     print(context.arch)
@@ -1343,3 +1348,20 @@ def LocalContext(function):
         with context.local(**{k:kw.pop(k) for k,v in kw.items() if isinstance(getattr(ContextType, k, None), property)}):
             return function(*a, **kw)
     return setter
+
+# Read configuration options from the context section
+def update_context_defaults(section):
+    # Circular imports FTW!
+    from pwnlib.util import safeeval
+    from pwnlib.log import getLogger
+    log = getLogger(__name__)
+    for key, value in section.items():
+        if key not in ContextType.defaults:
+            log.warn("Unknown configuration option %r in section %r" % (key, 'context'))
+            continue
+        if isinstance(ContextType.defaults[key], (str, unicode, tuple)):
+            value = safeeval.expr(value)
+
+        ContextType.defaults[key] = value
+
+register_config('context', update_context_defaults)

pwnlib/data/__init__.py

@@ -0,0 +1,12 @@
+from __future__ import absolute_import
+
+try:
+    # These files are not distributed with Pwntools, but
+    # are in the source tree and used for testing.
+    from pwnlib.data import elf
+except ImportError:
+    pass
+
+import os
+path = os.path.dirname(__file__)
+

pwnlib/data/crcsums.txt

@@ -4,75 +4,102 @@
 # At the time of download, that page was available under the GPLv3 license, which can found here:
 # https://www.gnu.org/licenses/gpl.html
 #
-crc.cat-bits.3 width=3  poly=0x3  init=0x7  refin=true  refout=true  xorout=0x0  check=0x6  name="CRC-3/ROHC"
-crc.cat-bits.4 width=4  poly=0x3  init=0x0  refin=true  refout=true  xorout=0x0  check=0x7  name="CRC-4/ITU"
-crc.cat-bits.5 width=5  poly=0x09  init=0x09  refin=false  refout=false  xorout=0x00  check=0x00  name="CRC-5/EPC"
-crc.cat.crc-5-itu width=5  poly=0x15  init=0x00  refin=true  refout=true  xorout=0x00  check=0x07  name="CRC-5/ITU"
-crc.cat.crc-5-usb width=5  poly=0x05  init=0x1f  refin=true  refout=true  xorout=0x1f  check=0x19  name="CRC-5/USB"
-crc.cat-bits.6 width=6  poly=0x27  init=0x3f  refin=false  refout=false  xorout=0x00  check=0x0d  name="CRC-6/CDMA2000-A"
-crc.cat.crc-6-cdma2000-b width=6  poly=0x07  init=0x3f  refin=false  refout=false  xorout=0x00  check=0x3b  name="CRC-6/CDMA2000-B"
-crc.cat.crc-6-darc width=6  poly=0x19  init=0x00  refin=true  refout=true  xorout=0x00  check=0x26  name="CRC-6/DARC"
-crc.cat.crc-6-itu width=6  poly=0x03  init=0x00  refin=true  refout=true  xorout=0x00  check=0x06  name="CRC-6/ITU"
-crc.cat-bits.7 width=7  poly=0x09  init=0x00  refin=false  refout=false  xorout=0x00  check=0x75  name="CRC-7"
-crc.cat.crc-7-rohc width=7  poly=0x4f  init=0x7f  refin=true  refout=true  xorout=0x00  check=0x53  name="CRC-7/ROHC"
-crc.cat-bits.8 width=8  poly=0x07  init=0x00  refin=false  refout=false  xorout=0x00  check=0xf4  name="CRC-8"
-crc.cat.crc-8-cdma2000 width=8  poly=0x9b  init=0xff  refin=false  refout=false  xorout=0x00  check=0xda  name="CRC-8/CDMA2000"
-crc.cat.crc-8-darc width=8  poly=0x39  init=0x00  refin=true  refout=true  xorout=0x00  check=0x15  name="CRC-8/DARC"
-crc.cat.crc-8-dvb-s2 width=8  poly=0xd5  init=0x00  refin=false  refout=false  xorout=0x00  check=0xbc  name="CRC-8/DVB-S2"
-crc.cat.crc-8-ebu width=8  poly=0x1d  init=0xff  refin=true  refout=true  xorout=0x00  check=0x97  name="CRC-8/EBU"
-crc.cat.crc-8-i-code width=8  poly=0x1d  init=0xfd  refin=false  refout=false  xorout=0x00  check=0x7e  name="CRC-8/I-CODE"
-crc.cat.crc-8-itu width=8  poly=0x07  init=0x00  refin=false  refout=false  xorout=0x55  check=0xa1  name="CRC-8/ITU"
-crc.cat.crc-8-maxim width=8  poly=0x31  init=0x00  refin=true  refout=true  xorout=0x00  check=0xa1  name="CRC-8/MAXIM"
-crc.cat.crc-8-rohc width=8  poly=0x07  init=0xff  refin=true  refout=true  xorout=0x00  check=0xd0  name="CRC-8/ROHC"
-crc.cat.crc-8-wdcma width=8  poly=0x9b  init=0x00  refin=true  refout=true  xorout=0x00  check=0x25  name="CRC-8/WCDMA"
-crc.cat-bits.10 width=10  poly=0x233  init=0x000  refin=false  refout=false  xorout=0x000  check=0x199  name="CRC-10"
-crc.cat.crc-10-cdma2000 width=10  poly=0x3d9  init=0x3ff  refin=false  refout=false  xorout=0x000  check=0x233  name="CRC-10/CDMA2000"
-crc.cat-bits.11 width=11  poly=0x385  init=0x01a  refin=false  refout=false  xorout=0x000  check=0x5a3  name="CRC-11"
-crc.cat-bits.12 width=12  poly=0x80f  init=0x000  refin=false  refout=true  xorout=0x000  check=0xdaf  name="CRC-12/3GPP"
-crc.cat.crc-12-cdma2000 width=12  poly=0xf13  init=0xfff  refin=false  refout=false  xorout=0x000  check=0xd4d  name="CRC-12/CDMA2000"
-crc.cat.crc-12-dect width=12  poly=0x80f  init=0x000  refin=false  refout=false  xorout=0x000  check=0xf5b  name="CRC-12/DECT"
-crc.cat-bits.13 width=13  poly=0x1cf5  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0x04fa  name="CRC-13/BBC"
-crc.cat-bits.14 width=14  poly=0x0805  init=0x0000  refin=true  refout=true  xorout=0x0000  check=0x082d  name="CRC-14/DARC"
-crc.cat-bits.15 width=15  poly=0x4599  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0x059e  name="CRC-15"
-crc.cat.crc-15-mpt1327 width=15  poly=0x6815  init=0x0000  refin=false  refout=false  xorout=0x0001  check=0x2566  name="CRC-15/MPT1327"
-crc.cat-bits.16 width=16  poly=0x8005  init=0x0000  refin=true  refout=true  xorout=0x0000  check=0xbb3d  name="ARC"
-crc.cat.crc-16-aug-ccitt width=16  poly=0x1021  init=0x1d0f  refin=false  refout=false  xorout=0x0000  check=0xe5cc  name="CRC-16/AUG-CCITT"
-crc.cat.crc-16-buypass width=16  poly=0x8005  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0xfee8  name="CRC-16/BUYPASS"
-crc.cat.crc-16-ccitt-false width=16  poly=0x1021  init=0xffff  refin=false  refout=false  xorout=0x0000  check=0x29b1  name="CRC-16/CCITT-FALSE"
-crc.cat.crc-16-cdma2000 width=16  poly=0xc867  init=0xffff  refin=false  refout=false  xorout=0x0000  check=0x4c06  name="CRC-16/CDMA2000"
-crc.cat.crc-16-dds-110 width=16  poly=0x8005  init=0x800d  refin=false  refout=false  xorout=0x0000  check=0x9ecf  name="CRC-16/DDS-110"
-crc.cat.crc-16-dect-r width=16  poly=0x0589  init=0x0000  refin=false  refout=false  xorout=0x0001  check=0x007e  name="CRC-16/DECT-R"
-crc.cat.crc-16-dect-x width=16  poly=0x0589  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0x007f  name="CRC-16/DECT-X"
-crc.cat.crc-16-dnp width=16  poly=0x3d65  init=0x0000  refin=true  refout=true  xorout=0xffff  check=0xea82  name="CRC-16/DNP"
-crc.cat.crc-16-en-13757 width=16  poly=0x3d65  init=0x0000  refin=false  refout=false  xorout=0xffff  check=0xc2b7  name="CRC-16/EN-13757"
-crc.cat.crc-16-genibus width=16  poly=0x1021  init=0xffff  refin=false  refout=false  xorout=0xffff  check=0xd64e  name="CRC-16/GENIBUS"
-crc.cat.crc-16-maxim width=16  poly=0x8005  init=0x0000  refin=true  refout=true  xorout=0xffff  check=0x44c2  name="CRC-16/MAXIM"
-crc.cat.crc-16-mcrf4xx width=16  poly=0x1021  init=0xffff  refin=true  refout=true  xorout=0x0000  check=0x6f91  name="CRC-16/MCRF4XX"
-crc.cat.crc-16-riello width=16  poly=0x1021  init=0xb2aa  refin=true  refout=true  xorout=0x0000  check=0x63d0  name="CRC-16/RIELLO"
-crc.cat.crc-16-t10-dif width=16  poly=0x8bb7  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0xd0db  name="CRC-16/T10-DIF"
-crc.cat.crc-16-teledisk width=16  poly=0xa097  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0x0fb3  name="CRC-16/TELEDISK"
-crc.cat.crc-16-tms37157 width=16  poly=0x1021  init=0x89ec  refin=true  refout=true  xorout=0x0000  check=0x26b1  name="CRC-16/TMS37157"
-crc.cat.crc-16-usb width=16  poly=0x8005  init=0xffff  refin=true  refout=true  xorout=0xffff  check=0xb4c8  name="CRC-16/USB"
-crc.cat.crc-a width=16  poly=0x1021  init=0xc6c6  refin=true  refout=true  xorout=0x0000  check=0xbf05  name="CRC-A"
-crc.cat.kermit width=16  poly=0x1021  init=0x0000  refin=true  refout=true  xorout=0x0000  check=0x2189  name="KERMIT"
-crc.cat.modbus width=16  poly=0x8005  init=0xffff  refin=true  refout=true  xorout=0x0000  check=0x4b37  name="MODBUS"
-crc.cat.x-25 width=16  poly=0x1021  init=0xffff  refin=true  refout=true  xorout=0xffff  check=0x906e  name="X-25"
-crc.cat.xmodem width=16  poly=0x1021  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0x31c3  name="XMODEM"
-crc.cat-bits.24 width=24  poly=0x864cfb  init=0xb704ce  refin=false  refout=false  xorout=0x000000  check=0x21cf02  name="CRC-24"
-crc.cat.crc-24-flexray-a width=24  poly=0x5d6dcb  init=0xfedcba  refin=false  refout=false  xorout=0x000000  check=0x7979bd  name="CRC-24/FLEXRAY-A"
-crc.cat.crc-24-flexray-b width=24  poly=0x5d6dcb  init=0xabcdef  refin=false  refout=false  xorout=0x000000  check=0x1f23b8  name="CRC-24/FLEXRAY-B"
-crc.cat-bits.31 width=31  poly=0x04c11db7  init=0x7fffffff  refin=false  refout=false  xorout=0x7fffffff  check=0x0ce9e46c  name="CRC-31/PHILIPS"
-crc.cat-bits.32 width=32  poly=0x04c11db7  init=0xffffffff  refin=true  refout=true  xorout=0xffffffff  check=0xcbf43926  name="CRC-32"
-crc.cat.crc-32-bzip2 width=32  poly=0x04c11db7  init=0xffffffff  refin=false  refout=false  xorout=0xffffffff  check=0xfc891918  name="CRC-32/BZIP2"
-crc.cat.crc-32c width=32  poly=0x1edc6f41  init=0xffffffff  refin=true  refout=true  xorout=0xffffffff  check=0xe3069283  name="CRC-32C"
-crc.cat.crc-32d width=32  poly=0xa833982b  init=0xffffffff  refin=true  refout=true  xorout=0xffffffff  check=0x87315576  name="CRC-32D"
-crc.cat.crc-32-mpeg-2 width=32  poly=0x04c11db7  init=0xffffffff  refin=false  refout=false  xorout=0x00000000  check=0x0376e6e7  name="CRC-32/MPEG-2"
-crc.cat.crc-32-posix width=32  poly=0x04c11db7  init=0x00000000  refin=false  refout=false  xorout=0xffffffff  check=0x765e7680  name="CRC-32/POSIX"
-crc.cat.crc-32q width=32  poly=0x814141ab  init=0x00000000  refin=false  refout=false  xorout=0x00000000  check=0x3010bf7f  name="CRC-32Q"
-crc.cat.jamcrc width=32  poly=0x04c11db7  init=0xffffffff  refin=true  refout=true  xorout=0x00000000  check=0x340bc6d9  name="JAMCRC"
-crc.cat.xfer width=32  poly=0x000000af  init=0x00000000  refin=false  refout=false  xorout=0x00000000  check=0xbd0be338  name="XFER"
-crc.cat-bits.40 width=40  poly=0x0004820009  init=0x0000000000  refin=false  refout=false  xorout=0xffffffffff  check=0xd4164fc646  name="CRC-40/GSM"
-crc.cat-bits.64 width=64  poly=0x42f0e1eba9ea3693  init=0x0000000000000000  refin=false  refout=false  xorout=0x0000000000000000  check=0x6c40df5f0b497347  name="CRC-64"
-crc.cat.crc-64-we width=64  poly=0x42f0e1eba9ea3693  init=0xffffffffffffffff  refin=false  refout=false  xorout=0xffffffffffffffff  check=0x62ec59e3f1a4f00a  name="CRC-64/WE"
-crc.cat.crc-64-xz width=64  poly=0x42f0e1eba9ea3693  init=0xffffffffffffffff  refin=true  refout=true  xorout=0xffffffffffffffff  check=0x995dc9bbdf1939fa  name="CRC-64/XZ"
-crc.cat-bits.82 width=82  poly=0x0308c0111011401440411  init=0x000000000000000000000  refin=true  refout=true  xorout=0x000000000000000000000  check=0x09ea83f625023801fd612  name="CRC-82/DARC"
+crc.cat.crc-3-rohc width=3  poly=0x3  init=0x7  refin=true  refout=true  xorout=0x0  check=0x6  residue=0x0  name="CRC-3/ROHC"
+crc.cat-bits.3 width=3  poly=0x3  init=0x0  refin=false  refout=false  xorout=0x7  check=0x4  residue=0x2  name="CRC-3/GSM"
+crc.cat-bits.4 width=4  poly=0x3  init=0xf  refin=false  refout=false  xorout=0xf  check=0xb  residue=0x2  name="CRC-4/INTERLAKEN"
+crc.cat.crc-4-itu width=4  poly=0x3  init=0x0  refin=true  refout=true  xorout=0x0  check=0x7  residue=0x0  name="CRC-4/ITU"
+crc.cat-bits.5 width=5  poly=0x09  init=0x09  refin=false  refout=false  xorout=0x00  check=0x00  residue=0x00  name="CRC-5/EPC"
+crc.cat.crc-5-itu width=5  poly=0x15  init=0x00  refin=true  refout=true  xorout=0x00  check=0x07  residue=0x00  name="CRC-5/ITU"
+crc.cat.crc-5-usb width=5  poly=0x05  init=0x1f  refin=true  refout=true  xorout=0x1f  check=0x19  residue=0x06  name="CRC-5/USB"
+crc.cat-bits.6 width=6  poly=0x27  init=0x3f  refin=false  refout=false  xorout=0x00  check=0x0d  residue=0x00  name="CRC-6/CDMA2000-A"
+crc.cat.crc-6-cdma2000-b width=6  poly=0x07  init=0x3f  refin=false  refout=false  xorout=0x00  check=0x3b  residue=0x00  name="CRC-6/CDMA2000-B"
+crc.cat.crc-6-darc width=6  poly=0x19  init=0x00  refin=true  refout=true  xorout=0x00  check=0x26  residue=0x00  name="CRC-6/DARC"
+crc.cat.crc-6-gsm width=6  poly=0x2f  init=0x00  refin=false  refout=false  xorout=0x3f  check=0x13  residue=0x3a  name="CRC-6/GSM"
+crc.cat.crc-6-itu width=6  poly=0x03  init=0x00  refin=true  refout=true  xorout=0x00  check=0x06  residue=0x00  name="CRC-6/ITU"
+crc.cat-bits.7 width=7  poly=0x09  init=0x00  refin=false  refout=false  xorout=0x00  check=0x75  residue=0x00  name="CRC-7"
+crc.cat.crc-7-rohc width=7  poly=0x4f  init=0x7f  refin=true  refout=true  xorout=0x00  check=0x53  residue=0x00  name="CRC-7/ROHC"
+crc.cat.crc-7-umts width=7  poly=0x45  init=0x00  refin=false  refout=false  xorout=0x00  check=0x61  residue=0x00  name="CRC-7/UMTS"
+crc.cat-bits.8 width=8  poly=0x07  init=0x00  refin=false  refout=false  xorout=0x00  check=0xf4  residue=0x00  name="CRC-8"
+crc.cat.crc-8-autosar width=8  poly=0x2f  init=0xff  refin=false  refout=false  xorout=0xff  check=0xdf  residue=0x42  name="CRC-8/AUTOSAR"
+crc.cat.crc-8-cdma2000 width=8  poly=0x9b  init=0xff  refin=false  refout=false  xorout=0x00  check=0xda  residue=0x00  name="CRC-8/CDMA2000"
+crc.cat.crc-8-darc width=8  poly=0x39  init=0x00  refin=true  refout=true  xorout=0x00  check=0x15  residue=0x00  name="CRC-8/DARC"
+crc.cat.crc-8-dvb-s2 width=8  poly=0xd5  init=0x00  refin=false  refout=false  xorout=0x00  check=0xbc  residue=0x00  name="CRC-8/DVB-S2"
+crc.cat.crc-8-ebu width=8  poly=0x1d  init=0xff  refin=true  refout=true  xorout=0x00  check=0x97  residue=0x00  name="CRC-8/EBU"
+crc.cat.crc-8-gsm-a width=8  poly=0x1d  init=0x00  refin=false  refout=false  xorout=0x00  check=0x37  residue=0x00  name="CRC-8/GSM-A"
+crc.cat.crc-8-gsm-b width=8  poly=0x49  init=0x00  refin=false  refout=false  xorout=0xff  check=0x94  residue=0x53  name="CRC-8/GSM-B"
+crc.cat.crc-8-i-code width=8  poly=0x1d  init=0xfd  refin=false  refout=false  xorout=0x00  check=0x7e  residue=0x00  name="CRC-8/I-CODE"
+crc.cat.crc-8-itu width=8  poly=0x07  init=0x00  refin=false  refout=false  xorout=0x55  check=0xa1  residue=0xac  name="CRC-8/ITU"
+crc.cat.crc-8-lte width=8  poly=0x9b  init=0x00  refin=false  refout=false  xorout=0x00  check=0xea  residue=0x00  name="CRC-8/LTE"
+crc.cat.crc-8-maxim width=8  poly=0x31  init=0x00  refin=true  refout=true  xorout=0x00  check=0xa1  residue=0x00  name="CRC-8/MAXIM"
+crc.cat.crc-8-opensafety width=8  poly=0x2f  init=0x00  refin=false  refout=false  xorout=0x00  check=0x3e  residue=0x00  name="CRC-8/OPENSAFETY"
+crc.cat.crc-8-rohc width=8  poly=0x07  init=0xff  refin=true  refout=true  xorout=0x00  check=0xd0  residue=0x00  name="CRC-8/ROHC"
+crc.cat.crc-8-sae-j1850 width=8  poly=0x1d  init=0xff  refin=false  refout=false  xorout=0xff  check=0x4b  residue=0xc4  name="CRC-8/SAE-J1850"
+crc.cat.crc-8-wdcma width=8  poly=0x9b  init=0x00  refin=true  refout=true  xorout=0x00  check=0x25  residue=0x00  name="CRC-8/WCDMA"
+crc.cat-bits.10 width=10  poly=0x233  init=0x000  refin=false  refout=false  xorout=0x000  check=0x199  residue=0x000  name="CRC-10"
+crc.cat.crc-10-cdma2000 width=10  poly=0x3d9  init=0x3ff  refin=false  refout=false  xorout=0x000  check=0x233  residue=0x000  name="CRC-10/CDMA2000"
+crc.cat.crc-10-gsm width=10  poly=0x175  init=0x000  refin=false  refout=false  xorout=0x3ff  check=0x12a  residue=0x0c6  name="CRC-10/GSM"
+crc.cat-bits.11 width=11  poly=0x385  init=0x01a  refin=false  refout=false  xorout=0x000  check=0x5a3  residue=0x000  name="CRC-11"
+crc.cat.crc-11-umts width=11  poly=0x307  init=0x000  refin=false  refout=false  xorout=0x000  check=0x061  residue=0x000  name="CRC-11/UMTS"
+crc.cat-bits.12 width=12  poly=0xf13  init=0xfff  refin=false  refout=false  xorout=0x000  check=0xd4d  residue=0x000  name="CRC-12/CDMA2000"
+crc.cat.crc-12-dect width=12  poly=0x80f  init=0x000  refin=false  refout=false  xorout=0x000  check=0xf5b  residue=0x000  name="CRC-12/DECT"
+crc.cat.crc-12-gsm width=12  poly=0xd31  init=0x000  refin=false  refout=false  xorout=0xfff  check=0xb34  residue=0x178  name="CRC-12/GSM"
+crc.cat.crc-12-umts width=12  poly=0x80f  init=0x000  refin=false  refout=true  xorout=0x000  check=0xdaf  residue=0x000  name="CRC-12/UMTS"
+crc.cat-bits.13 width=13  poly=0x1cf5  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0x04fa  residue=0x0000  name="CRC-13/BBC"
+crc.cat-bits.14 width=14  poly=0x0805  init=0x0000  refin=true  refout=true  xorout=0x0000  check=0x082d  residue=0x0000  name="CRC-14/DARC"
+crc.cat.crc-14-gsm width=14  poly=0x202d  init=0x0000  refin=false  refout=false  xorout=0x3fff  check=0x30ae  residue=0x031e  name="CRC-14/GSM"
+crc.cat-bits.15 width=15  poly=0x4599  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0x059e  residue=0x0000  name="CRC-15"
+crc.cat.crc-15-mpt1327 width=15  poly=0x6815  init=0x0000  refin=false  refout=false  xorout=0x0001  check=0x2566  residue=0x6815  name="CRC-15/MPT1327"
+crc.cat-bits.16 width=16  poly=0x8005  init=0x0000  refin=true  refout=true  xorout=0x0000  check=0xbb3d  residue=0x0000  name="ARC"
+crc.cat.crc-16-aug-ccitt width=16  poly=0x1021  init=0x1d0f  refin=false  refout=false  xorout=0x0000  check=0xe5cc  residue=0x0000  name="CRC-16/AUG-CCITT"
+crc.cat.crc-16-buypass width=16  poly=0x8005  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0xfee8  residue=0x0000  name="CRC-16/BUYPASS"
+crc.cat.crc-16-ccitt-false width=16  poly=0x1021  init=0xffff  refin=false  refout=false  xorout=0x0000  check=0x29b1  residue=0x0000  name="CRC-16/CCITT-FALSE"
+crc.cat.crc-16-cdma2000 width=16  poly=0xc867  init=0xffff  refin=false  refout=false  xorout=0x0000  check=0x4c06  residue=0x0000  name="CRC-16/CDMA2000"
+crc.cat.crc-16-cms width=16  poly=0x8005  init=0xffff  refin=false  refout=false  xorout=0x0000  check=0xaee7  residue=0x0000  name="CRC-16/CMS"
+crc.cat.crc-16-dds-110 width=16  poly=0x8005  init=0x800d  refin=false  refout=false  xorout=0x0000  check=0x9ecf  residue=0x0000  name="CRC-16/DDS-110"
+crc.cat.crc-16-dect-r width=16  poly=0x0589  init=0x0000  refin=false  refout=false  xorout=0x0001  check=0x007e  residue=0x0589  name="CRC-16/DECT-R"
+crc.cat.crc-16-dect-x width=16  poly=0x0589  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0x007f  residue=0x0000  name="CRC-16/DECT-X"
+crc.cat.crc-16-dnp width=16  poly=0x3d65  init=0x0000  refin=true  refout=true  xorout=0xffff  check=0xea82  residue=0x66c5  name="CRC-16/DNP"
+crc.cat.crc-16-en-13757 width=16  poly=0x3d65  init=0x0000  refin=false  refout=false  xorout=0xffff  check=0xc2b7  residue=0xa366  name="CRC-16/EN-13757"
+crc.cat.crc-16-genibus width=16  poly=0x1021  init=0xffff  refin=false  refout=false  xorout=0xffff  check=0xd64e  residue=0x1d0f  name="CRC-16/GENIBUS"
+crc.cat.crc-16-gsm width=16  poly=0x1021  init=0x0000  refin=false  refout=false  xorout=0xffff  check=0xce3c  residue=0x1d0f  name="CRC-16/GSM"
+crc.cat.crc-16-lj1200 width=16  poly=0x6f63  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0xbdf4  residue=0x0000  name="CRC-16/LJ1200"
+crc.cat.crc-16-maxim width=16  poly=0x8005  init=0x0000  refin=true  refout=true  xorout=0xffff  check=0x44c2  residue=0xb001  name="CRC-16/MAXIM"
+crc.cat.crc-16-mcrf4xx width=16  poly=0x1021  init=0xffff  refin=true  refout=true  xorout=0x0000  check=0x6f91  residue=0x0000  name="CRC-16/MCRF4XX"
+crc.cat.crc-16-opensafety-a width=16  poly=0x5935  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0x5d38  residue=0x0000  name="CRC-16/OPENSAFETY-A"
+crc.cat.crc-16-opensafety-a width=16  poly=0x755b  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0x20fe  residue=0x0000  name="CRC-16/OPENSAFETY-B"
+crc.cat.crc-16-profibus width=16  poly=0x1dcf  init=0xffff  refin=false  refout=false  xorout=0xffff  check=0xa819  residue=0xe394  name="CRC-16/PROFIBUS"
+crc.cat.crc-16-riello width=16  poly=0x1021  init=0xb2aa  refin=true  refout=true  xorout=0x0000  check=0x63d0  residue=0x0000  name="CRC-16/RIELLO"
+crc.cat.crc-16-t10-dif width=16  poly=0x8bb7  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0xd0db  residue=0x0000  name="CRC-16/T10-DIF"
+crc.cat.crc-16-teledisk width=16  poly=0xa097  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0x0fb3  residue=0x0000  name="CRC-16/TELEDISK"
+crc.cat.crc-16-tms37157 width=16  poly=0x1021  init=0x89ec  refin=true  refout=true  xorout=0x0000  check=0x26b1  residue=0x0000  name="CRC-16/TMS37157"
+crc.cat.crc-16-usb width=16  poly=0x8005  init=0xffff  refin=true  refout=true  xorout=0xffff  check=0xb4c8  residue=0xb001  name="CRC-16/USB"
+crc.cat.crc-a width=16  poly=0x1021  init=0xc6c6  refin=true  refout=true  xorout=0x0000  check=0xbf05  residue=0x0000  name="CRC-A"
+crc.cat.kermit width=16  poly=0x1021  init=0x0000  refin=true  refout=true  xorout=0x0000  check=0x2189  residue=0x0000  name="KERMIT"
+crc.cat.modbus width=16  poly=0x8005  init=0xffff  refin=true  refout=true  xorout=0x0000  check=0x4b37  residue=0x0000  name="MODBUS"
+crc.cat.x-25 width=16  poly=0x1021  init=0xffff  refin=true  refout=true  xorout=0xffff  check=0x906e  residue=0xf0b8  name="X-25"
+crc.cat.xmodem width=16  poly=0x1021  init=0x0000  refin=false  refout=false  xorout=0x0000  check=0x31c3  residue=0x0000  name="XMODEM"
+crc.cat-bits.24 width=24  poly=0x864cfb  init=0xb704ce  refin=false  refout=false  xorout=0x000000  check=0x21cf02  residue=0x000000  name="CRC-24"
+crc.cat.crc-24-ble width=24  poly=0x00065b  init=0x555555  refin=true  refout=true  xorout=0x000000  check=0xc25a56  residue=0x000000  name="CRC-24/BLE"
+crc.cat.crc-24-flexray-a width=24  poly=0x5d6dcb  init=0xfedcba  refin=false  refout=false  xorout=0x000000  check=0x7979bd  residue=0x000000  name="CRC-24/FLEXRAY-A"
+crc.cat.crc-24-flexray-b width=24  poly=0x5d6dcb  init=0xabcdef  refin=false  refout=false  xorout=0x000000  check=0x1f23b8  residue=0x000000  name="CRC-24/FLEXRAY-B"
+crc.cat.crc-24-interlaken width=24  poly=0x328b63  init=0xffffff  refin=false  refout=false  xorout=0xffffff  check=0xb4f3e6  residue=0x144e63  name="CRC-24/INTERLAKEN"
+crc.cat.crc-24-lte-a width=24  poly=0x864cfb  init=0x000000  refin=false  refout=false  xorout=0x000000  check=0xcde703  residue=0x000000  name="CRC-24/LTE-A"
+crc.cat.crc-24-lte-b width=24  poly=0x800063  init=0x000000  refin=false  refout=false  xorout=0x000000  check=0x23ef52  residue=0x000000  name="CRC-24/LTE-B"
+crc.cat-bits.30 width=30  poly=0x2030b9c7  init=0x3fffffff  refin=false  refout=false  xorout=0x3fffffff  check=0x04c34abf  residue=0x34efa55a  name="CRC-30/CDMA"
+crc.cat-bits.31 width=31  poly=0x04c11db7  init=0x7fffffff  refin=false  refout=false  xorout=0x7fffffff  check=0x0ce9e46c  residue=0x4eaf26f1  name="CRC-31/PHILIPS"
+crc.cat-bits.32 width=32  poly=0x04c11db7  init=0xffffffff  refin=true  refout=true  xorout=0xffffffff  check=0xcbf43926  residue=0xdebb20e3  name="CRC-32"
+crc.cat.crc-32-autosar width=32  poly=0xf4acfb13  init=0xffffffff  refin=true  refout=true  xorout=0xffffffff  check=0x1697d06a  residue=0x904cddbf  name="CRC-32/AUTOSAR"
+crc.cat.crc-32-bzip2 width=32  poly=0x04c11db7  init=0xffffffff  refin=false  refout=false  xorout=0xffffffff  check=0xfc891918  residue=0xc704dd7b  name="CRC-32/BZIP2"
+crc.cat.crc-32c width=32  poly=0x1edc6f41  init=0xffffffff  refin=true  refout=true  xorout=0xffffffff  check=0xe3069283  residue=0xb798b438  name="CRC-32C"
+crc.cat.crc-32d width=32  poly=0xa833982b  init=0xffffffff  refin=true  refout=true  xorout=0xffffffff  check=0x87315576  residue=0x45270551  name="CRC-32D"
+crc.cat.crc-32-mpeg-2 width=32  poly=0x04c11db7  init=0xffffffff  refin=false  refout=false  xorout=0x00000000  check=0x0376e6e7  residue=0x00000000  name="CRC-32/MPEG-2"
+crc.cat.crc-32-posix width=32  poly=0x04c11db7  init=0x00000000  refin=false  refout=false  xorout=0xffffffff  check=0x765e7680  residue=0xc704dd7b  name="CRC-32/POSIX"
+crc.cat.crc-32q width=32  poly=0x814141ab  init=0x00000000  refin=false  refout=false  xorout=0x00000000  check=0x3010bf7f  residue=0x00000000  name="CRC-32Q"
+crc.cat.jamcrc width=32  poly=0x04c11db7  init=0xffffffff  refin=true  refout=true  xorout=0x00000000  check=0x340bc6d9  residue=0x00000000  name="JAMCRC"
+crc.cat.xfer width=32  poly=0x000000af  init=0x00000000  refin=false  refout=false  xorout=0x00000000  check=0xbd0be338  residue=0x00000000  name="XFER"
+crc.cat-bits.40 width=40  poly=0x0004820009  init=0x0000000000  refin=false  refout=false  xorout=0xffffffffff  check=0xd4164fc646  residue=0xc4ff8071ff  name="CRC-40/GSM"
+crc.cat-bits.64 width=64  poly=0x42f0e1eba9ea3693  init=0x0000000000000000  refin=false  refout=false  xorout=0x0000000000000000  check=0x6c40df5f0b497347  residue=0x0000000000000000  name="CRC-64"
+crc.cat.crc-64-go-iso width=64  poly=0x000000000000001b  init=0xffffffffffffffff  refin=true  refout=true  xorout=0xffffffffffffffff  check=0xb90956c775a41001  residue=0x5300000000000000  name="CRC-64/GO-ISO"
+crc.cat.crc-64-we width=64  poly=0x42f0e1eba9ea3693  init=0xffffffffffffffff  refin=false  refout=false  xorout=0xffffffffffffffff  check=0x62ec59e3f1a4f00a  residue=0xfcacbebd5931a992  name="CRC-64/WE"
+crc.cat.crc-64-xz width=64  poly=0x42f0e1eba9ea3693  init=0xffffffffffffffff  refin=true  refout=true  xorout=0xffffffffffffffff  check=0x995dc9bbdf1939fa  residue=0x49958c9abd7d353f  name="CRC-64/XZ"
+crc.cat-bits.82 width=82  poly=0x0308c0111011401440411  init=0x000000000000000000000  refin=true  refout=true  xorout=0x000000000000000000000  check=0x09ea83f625023801fd612  residue=0x000000000000000000000  name="CRC-82/DARC"

pwnlib/data/elf/__init__.py

@@ -0,0 +1,5 @@
+import os
+path = os.path.dirname(__file__)
+
+def get(x):
+    return os.path.join(path, x)

pwnlib/data/includes/generator/linux/diet/aarch64/syscalls.h

@@ -262,62 +262,3 @@
 #define __NR_process_vm_writev 271
 #define __NR_kcmp 272
 #define __NR_finit_module 273
-#define __NR_open 1024
-#define __NR_link 1025
-#define __NR_unlink 1026
-#define __NR_mknod 1027
-#define __NR_chmod 1028
-#define __NR_chown 1029
-#define __NR_mkdir 1030
-#define __NR_rmdir 1031
-#define __NR_lchown 1032
-#define __NR_access 1033
-#define __NR_rename 1034
-#define __NR_readlink 1035
-#define __NR_symlink 1036
-#define __NR_utimes 1037
-#define __NR_stat 1038
-#define __NR_lstat 1039
-#define __NR_pipe 1040
-#define __NR_dup2 1041
-#define __NR_epoll_create 1042
-#define __NR_inotify_init 1043
-#define __NR_eventfd 1044
-#define __NR_signalfd 1045
-#define __NR_sendfile64 1046
-#define __NR_ftruncate64 1047
-#define __NR_truncate64 1048
-#define __NR_stat64 1049
-#define __NR_lstat64 1050
-#define __NR_fstat64 1051
-#define __NR_fcntl64 1052
-/* #define __NR_fadvise64 1053 */
-#define __NR_newfstatat 1054
-#define __NR_fstatfs64 1055
-#define __NR_statfs64 1056
-#define __NR_lseek64 1057
-#define __NR_mmap64 1058
-#define __NR_alarm 1059
-#define __NR_getpgrp 1060
-#define __NR_pause 1061
-#define __NR_time 1062
-#define __NR_utime 1063
-#define __NR_creat 1064
-#define __NR_getdents 1065
-#define __NR_futimesat 1066
-#define __NR_select 1067
-#define __NR_poll 1068
-#define __NR_epoll_wait 1069
-#define __NR_ustat 1070
-#define __NR_vfork 1071
-#define __NR_oldwait4 1072
-#define __NR_recv 1073
-#define __NR_send 1074
-#define __NR_bdflush 1075
-#define __NR_umount 1076
-#define __NR_uselib 1077
-#define __NR__sysctl 1078
-#define __NR_fork 1079
-#define __NR_syscalls (__NR_fork+1)
-
-#define __NR_sigreturn 1999

pwnlib/data/includes/linux/aarch64.h

@@ -257,63 +257,6 @@
 #define __NR_process_vm_writev 271
 #define __NR_kcmp 272
 #define __NR_finit_module 273
-#define __NR_open 1024
-#define __NR_link 1025
-#define __NR_unlink 1026
-#define __NR_mknod 1027
-#define __NR_chmod 1028
-#define __NR_chown 1029
-#define __NR_mkdir 1030
-#define __NR_rmdir 1031
-#define __NR_lchown 1032
-#define __NR_access 1033
-#define __NR_rename 1034
-#define __NR_readlink 1035
-#define __NR_symlink 1036
-#define __NR_utimes 1037
-#define __NR_stat 1038
-#define __NR_lstat 1039
-#define __NR_pipe 1040
-#define __NR_dup2 1041
-#define __NR_epoll_create 1042
-#define __NR_inotify_init 1043
-#define __NR_eventfd 1044
-#define __NR_signalfd 1045
-#define __NR_sendfile64 1046
-#define __NR_ftruncate64 1047
-#define __NR_truncate64 1048
-#define __NR_stat64 1049
-#define __NR_lstat64 1050
-#define __NR_fstat64 1051
-#define __NR_fcntl64 1052
-#define __NR_newfstatat 1054
-#define __NR_fstatfs64 1055
-#define __NR_statfs64 1056
-#define __NR_lseek64 1057
-#define __NR_mmap64 1058
-#define __NR_alarm 1059
-#define __NR_getpgrp 1060
-#define __NR_pause 1061
-#define __NR_time 1062
-#define __NR_utime 1063
-#define __NR_creat 1064
-#define __NR_getdents 1065
-#define __NR_futimesat 1066
-#define __NR_select 1067
-#define __NR_poll 1068
-#define __NR_epoll_wait 1069
-#define __NR_ustat 1070
-#define __NR_vfork 1071
-#define __NR_oldwait4 1072
-#define __NR_recv 1073
-#define __NR_send 1074
-#define __NR_bdflush 1075
-#define __NR_umount 1076
-#define __NR_uselib 1077
-#define __NR__sysctl 1078
-#define __NR_fork 1079
-#define __NR_syscalls (1079+1)
-#define __NR_sigreturn 1999
 #define MAP_32BIT 0x40
 #define INADDR_ANY 0
 #define INADDR_BROADCAST 0xffffffff
@@ -942,19 +885,14 @@
 #define PCMASK 0
 #define SYS_accept 202
 #define SYS_accept4 242
-#define SYS_access 1033
 #define SYS_acct 89
 #define SYS_add_key 217
 #define SYS_adjtimex 171
-#define SYS_alarm 1059
-#define SYS_bdflush 1075
 #define SYS_bind 200
 #define SYS_brk 214
 #define SYS_capget 90
 #define SYS_capset 91
 #define SYS_chdir 49
-#define SYS_chmod 1028
-#define SYS_chown 1029
 #define SYS_chroot 51
 #define SYS_clock_getres 114
 #define SYS_clock_gettime 113
@@ -963,17 +901,12 @@
 #define SYS_clone 220
 #define SYS_close 57
 #define SYS_connect 203
-#define SYS_creat 1064
 #define SYS_delete_module 106
 #define SYS_dup 23
-#define SYS_dup2 1041
 #define SYS_dup3 24
-#define SYS_epoll_create 1042
 #define SYS_epoll_create1 20
 #define SYS_epoll_ctl 21
 #define SYS_epoll_pwait 22
-#define SYS_epoll_wait 1069
-#define SYS_eventfd 1044
 #define SYS_eventfd2 19
 #define SYS_execve 221
 #define SYS_exit 93
@@ -989,27 +922,20 @@
 #define SYS_fchown 55
 #define SYS_fchownat 54
 #define SYS_fcntl 25
-#define SYS_fcntl64 1052
 #define SYS_fdatasync 83
 #define SYS_fgetxattr 10
 #define SYS_flistxattr 13
 #define SYS_flock 32
-#define SYS_fork 1079
 #define SYS_fremovexattr 16
 #define SYS_fsetxattr 7
 #define SYS_fstat 80
-#define SYS_fstat64 1051
 #define SYS_fstatat64 79
 #define SYS_fstatfs 44
-#define SYS_fstatfs64 1055
 #define SYS_fsync 82
 #define SYS_ftruncate 46
-#define SYS_ftruncate64 1047
 #define SYS_futex 98
-#define SYS_futimesat 1066
 #define SYS_getcpu 168
 #define SYS_getcwd 17
-#define SYS_getdents 1065
 #define SYS_getdents64 61
 #define SYS_getegid 177
 #define SYS_geteuid 175
@@ -1019,7 +945,6 @@
 #define SYS_get_mempolicy 236
 #define SYS_getpeername 205
 #define SYS_getpgid 155
-#define SYS_getpgrp 1060
 #define SYS_getpid 172
 #define SYS_getppid 173
 #define SYS_getpriority 141
@@ -1037,7 +962,6 @@
 #define SYS_getxattr 8
 #define SYS_init_module 105
 #define SYS_inotify_add_watch 27
-#define SYS_inotify_init 1043
 #define SYS_inotify_init1 26
 #define SYS_inotify_rm_watch 28
 #define SYS_io_cancel 3
@@ -1051,9 +975,7 @@
 #define SYS_kexec_load 104
 #define SYS_keyctl 219
 #define SYS_kill 129
-#define SYS_lchown 1032
 #define SYS_lgetxattr 9
-#define SYS_link 1025
 #define SYS_linkat 37
 #define SYS_listen 201
 #define SYS_listxattr 11
@@ -1062,15 +984,11 @@
 #define SYS_lremovexattr 15
 #define SYS_lseek 62
 #define SYS_lsetxattr 6
-#define SYS_lstat 1039
-#define SYS_lstat64 1050
 #define SYS_madvise 233
 #define SYS_mbind 235
 #define SYS_migrate_pages 238
 #define SYS_mincore 232
-#define SYS_mkdir 1030
 #define SYS_mkdirat 34
-#define SYS_mknod 1027
 #define SYS_mknodat 33
 #define SYS_mlock 228
 #define SYS_mlockall 230
@@ -1094,17 +1012,12 @@
 #define SYS_munlockall 231
 #define SYS_munmap 215
 #define SYS_nanosleep 101
-#define SYS_newfstatat 1054
 #define SYS_nfsservctl 42
-#define SYS_open 1024
 #define SYS_openat 56
-#define SYS_pause 1061
 #define SYS_perf_event_open 241
 #define SYS_personality 92
-#define SYS_pipe 1040
 #define SYS_pipe2 59
 #define SYS_pivot_root 41
-#define SYS_poll 1068
 #define SYS_ppoll 73
 #define SYS_prctl 167
 #define SYS_pread64 67
@@ -1117,21 +1030,17 @@
 #define SYS_quotactl 60
 #define SYS_read 63
 #define SYS_readahead 213
-#define SYS_readlink 1035
 #define SYS_readlinkat 78
 #define SYS_readv 65
 #define SYS_reboot 142
-#define SYS_recv 1073
 #define SYS_recvfrom 207
 #define SYS_recvmmsg 243
 #define SYS_recvmsg 212
 #define SYS_remap_file_pages 234
 #define SYS_removexattr 14
-#define SYS_rename 1034
 #define SYS_renameat 38
 #define SYS_request_key 218
 #define SYS_restart_syscall 128
-#define SYS_rmdir 1031
 #define SYS_rt_sigaction 134
 #define SYS_rt_sigpending 136
 #define SYS_rt_sigprocmask 135
@@ -1150,14 +1059,11 @@
 #define SYS_sched_setparam 118
 #define SYS_sched_setscheduler 119
 #define SYS_sched_yield 124
-#define SYS_select 1067
 #define SYS_semctl 191
 #define SYS_semget 190
 #define SYS_semop 193
 #define SYS_semtimedop 192
-#define SYS_send 1074
 #define SYS_sendfile 71
-#define SYS_sendfile64 1046
 #define SYS_sendmsg 211
 #define SYS_sendto 206
 #define SYS_setdomainname 162
@@ -1188,28 +1094,20 @@
 #define SYS_shmget 194
 #define SYS_shutdown 210
 #define SYS_sigaltstack 132
-#define SYS_signalfd 1045
 #define SYS_signalfd4 74
-#define SYS_sigreturn 1999
 #define SYS_socket 198
 #define SYS_socketpair 199
 #define SYS_splice 76
-#define SYS_stat 1038
-#define SYS_stat64 1049
 #define SYS_statfs 43
-#define SYS_statfs64 1056
 #define SYS_swapoff 225
 #define SYS_swapon 224
-#define SYS_symlink 1036
 #define SYS_symlinkat 36
 #define SYS_sync 81
 #define SYS_sync_file_range2 84
-#define SYS__sysctl 1078
 #define SYS_sysinfo 179
 #define SYS_syslog 116
 #define SYS_tee 77
 #define SYS_tgkill 131
-#define SYS_time 1062
 #define SYS_timer_create 107
 #define SYS_timer_delete 111
 #define SYS_timerfd_create 85
@@ -1221,20 +1119,12 @@
 #define SYS_times 153
 #define SYS_tkill 130
 #define SYS_truncate 45
-#define SYS_truncate64 1048
 #define SYS_umask 166
-#define SYS_umount 1076
 #define SYS_umount2 39
 #define SYS_uname 160
-#define SYS_unlink 1026
 #define SYS_unlinkat 35
 #define SYS_unshare 97
-#define SYS_uselib 1077
-#define SYS_ustat 1070
-#define SYS_utime 1063
 #define SYS_utimensat 88
-#define SYS_utimes 1037
-#define SYS_vfork 1071
 #define SYS_vhangup 58
 #define SYS_vmsplice 75
 #define SYS_wait4 260

pwnlib/data/templates/pwnup.mako

@@ -0,0 +1,139 @@
+<%page args="binary, host=None, port=None, user=None, password=None, remote_path=None"/>\
+<%
+from pwnlib.context import context as ctx
+from pwnlib.elf.elf import ELF
+from elftools.common.exceptions import ELFError
+
+import os
+try:
+    if binary:
+        ctx.binary = ELF(binary, checksec=False)
+except ELFError:
+    pass
+
+if not binary:
+    binary = './path/to/binary'
+
+exe = os.path.basename(binary)
+
+ssh = user or password
+if ssh and not port:
+    port = 22
+elif host and not port:
+    port = 4141
+
+remote_path = remote_path or exe
+password = password or 'secret1234'
+binary_repr = repr(binary)
+%>\
+#!/usr/bin/env python2
+# -*- coding: utf-8 -*-
+from pwn import *
+
+# Set up pwntools for the correct architecture
+%if ctx.binary:
+exe = context.binary = ELF(${binary_repr})
+<% binary_repr = 'exe.path' %>
+%else:
+context.update(arch='i386')
+exe = ${binary_repr}
+<% binary_repr = 'exe' %>
+%endif
+
+# Many built-in settings can be controlled on the command-line and show up
+# in "args".  For example, to dump all data sent/received, and disable ASLR
+# for all created processes...
+# ./exploit.py DEBUG NOASLR
+%if host or port or user:
+# ./exploit.py GDB HOST=example.com PORT=4141
+%endif
+%if host:
+host = args.HOST or ${repr(host)}
+%endif
+%if port:
+port = int(args.PORT or ${port})
+%endif
+%if user:
+user = args.USER or ${repr(user)}
+password = args.PASSWORD or ${repr(password)}
+%endif
+%if ssh:
+remote_path = ${repr(remote_path)}
+%endif
+
+%if exe or remote_path:
+# Specify your GDB script here for debugging
+# GDB will be launched if the exploit is run via e.g.
+# ./exploit.py GDB
+gdbscript = '''
+%if ctx.binary:
+  %if 'main' in ctx.binary.symbols:
+break *0x{exe.symbols.main:x}
+  %else:
+break *0x{exe.entry:x}
+  %endif
+%endif
+continue
+'''.format(**locals())
+%endif
+
+%if ssh:
+# Connect to the remote SSH server
+shell = None
+if not args.LOCAL:
+    shell = ssh(user, host, port, password)
+    shell.set_working_directory(symlink=True)
+%endif
+
+%if host:
+# Execute the target binary locally
+def local(argv=[], *a, **kw):
+    if args.GDB:
+        return gdb.debug([exe.path] + argv, gdbscript=gdbscript, *a, **kw)
+    else:
+        return process([exe.path] + argv, *a, **kw)
+
+%if ssh:
+# Execute the target binary on the remote host
+%else:
+# Connect to the process on the remote host
+%endif
+def remote(argv=[], *a, **kw):
+  %if ssh:
+    if args.GDB:
+        return gdb.debug([remote_path] + argv, gdbscript=gdbscript, ssh=shell, *a, **kw)
+    else:
+        return shell.process([remote_path] + argv, *a, **kw)
+  %else:
+    io = connect(host, port)
+    if args.GDB:
+        gdb.attach(io, gdbscript=gdbscript)
+    return io
+  %endif
+%endif
+
+%if host:
+start = local if args.LOCAL else remote
+%else:
+def start(argv=[], *a, **kw):
+    if args.GDB:
+        return gdb.debug([${binary_repr}] + argv, gdbscript=gdbscript, *a, **kw)
+    else:
+        return process([${binary_repr}] + argv, *a, **kw)
+%endif
+
+#===========================================================
+#                    EXPLOIT GOES HERE
+#===========================================================
+io = start()
+
+# shellcode = asm(shellcraft.sh())
+# payload = fit({
+#     32: 0xdeadbeef,
+#     'iaaa': [1, 2, 'Hello', 3]
+# }, length=128)
+# io.send(payload)
+# flag = io.recv(...)
+# log.success(flag)
+
+io.interactive()

pwnlib/data/useragents/useragents.txt

@@ -64,6 +64,7 @@ Activeworlds
 Ad Muncher v4.xx.x
 Ad Muncher v4x Build xxxxx
 Adaxas Spider (http://www.adaxas.net/)
+Adobe Application Manager 2.0
 AdsBot-Google ( http://www.google.com/adsbot.html)
 Advanced Browser (http://www.avantbrowser.com)
 Agent-SharewarePlazaFileCheckBot/2.0+(+http://www.SharewarePlaza.com)
@@ -93,6 +94,7 @@ AmigaVoyager/2.95 (compatible; MC680x0; AmigaOS; SV1)
 AmigaVoyager/3.2 (AmigaOS/MC680x0)
 AmigaVoyager/3.4.4 (MorphOS/PPC native)
 Amoi 8512/R21.0 NF-Browser/3.3
+AndroidDownloadManager/5.1 (Linux; U; Android 5.1; Z820 Build/LMY47D)
 AnnoMille spider 0.1 alpha - http://www.annomille.it
 Anonymized by ProxyOS: http://www.megaproxy.com
 Anonymizer/1.1
@@ -704,6 +706,7 @@ FLATARTS_FAVICO
 FSurf15a 01
 FTB-Bot http://www.findthebest.co.uk/
 FaEdit/2.0.x
+Facebot
 Factbot 1.09 (see http://www.factbites.com/webmasters.php)
 FairAd Client
 Fast Crawler Gold Edition
@@ -1442,6 +1445,7 @@ MnogoSearch/3.2.xx
 Mnogosearch-3.1.21
 Mo College 1.9
 Mobile Safari 1.1.3 (iPhone U; CPU like Mac OS X; en)
+MobileSafari/600.1.4 CFNetwork/711.1.12 Darwin/14.0.0
 MojeekBot/0.2 (archi; http://www.mojeek.com/bot.html)
 MojeekBot/0.x (archi; http://www.mojeek.com/bot.html)
 Mojoo Robot (http://www.mojoo.com/)
@@ -8135,6 +8139,7 @@ Mozilla/5.0 (Android 2.2.2; Linux; Opera Mobi/ADR-1103311355; U; en; rv:1.9.1.6)
 Mozilla/5.0 (Android 2.2; Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4
 Mozilla/5.0 (Android 2.2; zh-cn; HTC Desire)/GoBrowser
 Mozilla/5.0 (Android 4.2; rv:19.0) Gecko/20121129 Firefox/19.0
+Mozilla/5.0 (Android 6.0.1; Mobile; rv:48.0) Gecko/48.0 Firefox/48.0
 Mozilla/5.0 (Android) Gecko Firefox Fennec/4.0
 Mozilla/5.0 (Android) Gecko/20110318 Firefox/4.0 Fennec/4.0
 Mozilla/5.0 (Android; Linux arm71; rv:2.1.1) Gecko/20110415 Firefox/4.0.2pre Fennec/4.0.1
@@ -8226,10 +8231,26 @@ Mozilla/5.0 (Linux i686; U; en; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 1
 Mozilla/5.0 (Linux i686; de-AT; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3
 Mozilla/5.0 (Linux) Gecko Iceweasel (Debian) Mnenhy
 Mozilla/5.0 (Linux) Gecko/20061122 Firefox/2.0
+Mozilla/5.0 (Linux; Android 4.0.4; BNTV400 Build/IMM76L) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.111 Safari/537.36
 Mozilla/5.0 (Linux; Android 4.1.2; SHV-E250S Build/JZO54K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.82 Mobile Safari/537.36
+Mozilla/5.0 (Linux; Android 4.3; SPH-L710 Build/JSS15J) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.99 Mobile Safari/537.36
+Mozilla/5.0 (Linux; Android 4.4.2; LG-V410 Build/KOT49I.V41010d) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.103 Safari/537.36
+Mozilla/5.0 (Linux; Android 4.4.2; LGMS323 Build/KOT49I.MS32310b) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.103 Mobile Safari/537.36
+Mozilla/5.0 (Linux; Android 4.4.2; SAMSUNG-SM-G900A Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.94 Mobile Safari/537.36
 Mozilla/5.0 (Linux; Android 4.4.2; SAMSUNG-SM-T537A Build/KOT49H) AppleWebKit/537.36 (KHTML like Gecko) Chrome/35.0.1916.141 Safari/537.36
+Mozilla/5.0 (Linux; Android 4.4.2; SM-T230NU Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.81 Safari/537.36
 Mozilla/5.0 (Linux; Android 4.4.4; Nexus 7 Build/KTU84P) AppleWebKit/537.36 (KHTML like Gecko) Chrome/36.0.1985.135 Safari/537.36
+Mozilla/5.0 (Linux; Android 4.4.4; XT1032 Build/KXB21.14-L1.61) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.94 Mobile Safari/537.36
 Mozilla/5.0 (Linux; Android 4.4; Nexus 5 Build/BuildID) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36
+Mozilla/5.0 (Linux; Android 5.0.1; SCH-R970 Build/LRX22C) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.84 Mobile Safari/537.36
+Mozilla/5.0 (Linux; Android 5.0.2; SAMSUNG SM-T530NU Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/3.2 Chrome/38.0.2125.102 Safari/537.36
+Mozilla/5.0 (Linux; Android 5.1.1; Nexus 7 Build/LMY47V) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.78 Safari/537.36 OPR/30.0.1856.93524
+Mozilla/5.0 (Linux; Android 5.1; C6740N Build/LMY47O) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.111 Mobile Safari/537.36
+Mozilla/5.0 (Linux; Android 6.0.1; SM-G900H Build/MMB29K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.98 Mobile Safari/537.36
+Mozilla/5.0 (Linux; Android 6.0; HTC One M9 Build/MRA58K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.98 Mobile Safari/537.36
+Mozilla/5.0 (Linux; Android 6.0; LG-D850 Build/MRA58K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.97 Mobile Safari/537.36
+Mozilla/5.0 (Linux; Android 6.0; Nexus 5X Build/MDB08L) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.124 Mobile Safari/537.36
+Mozilla/5.0 (Linux; Android 7.0; Nexus 9 Build/NRD90R) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.124 Safari/537.36
 Mozilla/5.0 (Linux; U) Opera 6.02  [en]
 Mozilla/5.0 (Linux; U; Android 0.5; en-us) AppleWebKit/522  (KHTML, like Gecko) Safari/419.3
 Mozilla/5.0 (Linux; U; Android 1.0; en-us; dream) AppleWebKit/525.10  (KHTML, like Gecko) Version/3.0.4 Mobile Safari/523.12.2
@@ -8323,9 +8344,11 @@ Mozilla/5.0 (Linux; U; Android 4.0.3; de-ch; HTC Sensation Build/IML74K) AppleWe
 Mozilla/5.0 (Linux; U; Android 4.0.3; de-de; Galaxy S II Build/GRJ22) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
 Mozilla/5.0 (Linux; U; Android 4.0.3; en-us; KFTT Build/IML74K) AppleWebKit/535.19 (KHTML, like Gecko) Silk/2.1 Mobile Safari/535.19 Silk-Accelerated=true
 Mozilla/5.0 (Linux; U; Android 4.0.3; ko-kr; LG-L160L Build/IML74K) AppleWebkit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
+Mozilla/5.0 (Linux; U; Android 4.1.2; en-us; LG-P870/P87020d Build/JZO54K) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
 Mozilla/5.0 (Linux; U; Android 4.1; en-us; sdk Build/MR1) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.1 Safari/534.30
 Mozilla/5.0 (Linux; U; Android 4.2; en-us; sdk Build/MR1) AppleWebKit/535.19 (KHTML, like Gecko) Version/4.2 Safari/535.19
 Mozilla/5.0 (Linux; U; Android 4.3; en-us; sdk Build/MR1) AppleWebKit/536.23 (KHTML, like Gecko) Version/4.3 Mobile Safari/536.23
+Mozilla/5.0 (Linux; U; Android 4.4.2; en-us; GT-P5210 Build/KOT49H) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Safari/534.30
 Mozilla/5.0 (Linux; U; Debian Linux; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12
 Mozilla/5.0 (Linux; U; Debian Linux; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9
 Mozilla/5.0 (Linux; U; Intel Linux; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
@@ -8339,10 +8362,13 @@ Mozilla/5.0 (Linux; U; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.
 Mozilla/5.0 (Linux; U;Linux X86_64; en-US; rv:1.9.1) Gecko/20090624 Firefox/3.5
 Mozilla/5.0 (Linux; webOS/2.2.4; U; en-US) AppleWebKit/534.6 (KHTML, like Gecko) webOSBrowser/221.56 Safari/534.6 Pre/3.0
 Mozilla/5.0 (MSIE 7.0; Macintosh; U; SunOS; X11; gu; SV1; InfoPath.2; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648)
+Mozilla/5.0 (MSIE 9.0; Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14931
 Mozilla/5.0 (Macintosh; ; Intel Mac OS X; fr; rv:1.8.1.1) Gecko/20061204 Opera
 Mozilla/5.0 (Macintosh; AMD Mac OS X 10_8_2) AppleWebKit/535.22 (KHTML, like Gecko) Chrome/18.6.872
 Mozilla/5.0 (Macintosh; I; Intel Mac OS X 11_7_9; de-LI; rv:1.9b4) Gecko/2012010317 Firefox/10.0a4
 Mozilla/5.0 (Macintosh; I; PPC Mac OS X Mach-O; en-US; rv:1.9a1) Gecko/20061204 Firefox/3.0a1
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:40.0) Gecko/20100101 Firefox/40.0
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 SeaMonkey/2.7.1
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:5.0) Gecko/20100101 Firefox/5.0
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:15.0) Gecko/20120909 Firefox/15.0.1 SeaMonkey/2.12.1
@@ -8373,11 +8399,23 @@ Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Firefox/24.
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:33.0) Gecko/20100101 Firefox/33.0 SeaMonkey/2.30
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:35.0) Gecko/20100101 Firefox/35.0
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32a2
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:47.0) Gecko/20100101 Firefox/47.0
 Mozilla/5.0 (Macintosh; Intel Mac OS X 1083) AppleWebKit/537.36 (KHTML like Gecko) Chrome/28.0.1469.0 Safari/537.36
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10; rv:33.0) Gecko/20100101 Firefox/33.0
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.1 Safari/537.36
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.1.25 (KHTML, like Gecko)
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36 OPR/28.0.1750.51
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.82 Safari/537.36 OPR/29.0.1795.41
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.105 Safari/537.36 Vivaldi/1.0.162.9
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2859.0 Safari/537.36
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/600.8.9 (KHTML, like Gecko) Maxthon/4.5.2
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/600.8.9 (KHTML, like Gecko) Version/8.0.8 Safari/600.8.9
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.56 (KHTML, like Gecko) Version/9.0 Safari/601.1.56
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_5_8) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.68 Safari/534.24
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_5_8) AppleWebKit/534.31 (KHTML, like Gecko) Chrome/13.0.748.0 Safari/534.31
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_5_8) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/14.0.801.0 Safari/535.1
@@ -9947,6 +9985,7 @@ Mozilla/5.0 (Maemo; Linux armv7l; rv:6.0a1) Gecko/20110522 Firefox/6.0a1 Fennec/
 Mozilla/5.0 (Maemo; Linux armv7l; rv:6.0a1) Gecko/20110526 Firefox/6.0a1 Fennec/6.0a1
 Mozilla/5.0 (MeeGo; NokiaN9) AppleWebKit/534.13 (KHTML, like Gecko) NokiaBrowser/8.5.0 Mobile Safari/534.13
 Mozilla/5.0 (MeeGo; NokiaN950-00/00) AppleWebKit/534.13 (KHTML, like Gecko) NokiaBrowser/8.5.0 Mobile Safari/534.13
+Mozilla/5.0 (Mobile; Windows Phone 8.1; Android 4.0; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 929) like iPhone OS 7_0_3 Mac OS X AppleWebKit/537 (KHTML, like Gecko) Mobile Safari/537
 Mozilla/5.0 (Mozilla/5.0 (iPhone; U; CPU iPhone OS 2_0_1 like Mac OS X; fr-fr) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1 Mobile/5G77 Safari/525.20
 Mozilla/5.0 (OS/2; U; OS/2; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Arora/0.11.0 Safari/533.3
 Mozilla/5.0 (OS/2; U; OS/2; en-US) AppleWebKit/533.3 (KHTML, like Gecko) QupZilla/1.3.1 Safari/533.3
@@ -9960,6 +9999,12 @@ Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:1.8.1.3pre) Gecko/20070307 SeaMonkey/1
 Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:1.9a1) Gecko/20051119 MultiZilla/1.8.1.0s SeaMonkey/1.5a
 Mozilla/5.0 (OS/2; Warp 4.5; rv:10.0.12) Gecko/20100101 Firefox/10.0.12
 Mozilla/5.0 (OS/2; Warp 4.5; rv:10.0.12) Gecko/20130108 Firefox/10.0.12 SeaMonkey/2.7.2
+Mozilla/5.0 (OS/2; Warp 4.5; rv:24.0) Gecko/20100101 Firefox/24.0
+Mozilla/5.0 (OS/2; Warp 4.5; rv:24.0) Gecko/20100101 Firefox/24.0 SeaMonkey/2.21
+Mozilla/5.0 (OS/2; Warp 4.5; rv:31.0) Gecko/20100101 Firefox/31.0
+Mozilla/5.0 (OS/2; Warp 4.5; rv:31.0) Gecko/20100101 Firefox/31.0 SeaMonkey/2.28
+Mozilla/5.0 (OS/2; Warp 4.5; rv:38.0) Gecko/20100101 Firefox/38.0
+Mozilla/5.0 (OS/2; Warp 4.5; rv:38.0) Gecko/20100101 Firefox/38.0 SeaMonkey/2.35
 Mozilla/5.0 (PLAYSTATION 3; 1.0)
 Mozilla/5.0 (PLAYSTATION 3; 1.00)
 Mozilla/5.0 (PLAYSTATION 3; 1.10)
@@ -10007,6 +10052,7 @@ Mozilla/5.0 (U; en-US; rv:1.8.1.10) Gecko/20071126  Firefox/2.0.0.10
 Mozilla/5.0 (U; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6
 Mozilla/5.0 (U;) AppleWebKit/532.0 (KHTML, like Gecko) Iron/3.0.197.0 Safari/532.0
 Mozilla/5.0 (Unknown; U; UNIX BSD/SYSV system; C -) AppleWebKit/527  (KHTML, like Gecko, Safari/419.3) Arora/0.10.2
+Mozilla/5.0 (Unknown; UNIX BSD/SYSV system) AppleWebKit/538.1 (KHTML, like Gecko) QupZilla/1.7.0 Safari/538.1
 Mozilla/5.0 (Version: xxxx Type:xx)
 Mozilla/5.0 (Windows  U  Windows NT 5.1  en-US) AppleWebKit/534.12 (KHTML, like Gecko) Chrome/9.0.583.0 Safari/534.12
 Mozilla/5.0 (Windows 2000; U) Opera 6.01  [de]
@@ -10025,6 +10071,19 @@ Mozilla/5.0 (Windows 98; U; en; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.6
 Mozilla/5.0 (Windows ME; U) Opera 6.05  [de]
 Mozilla/5.0 (Windows ME; U; en) Opera 8.51
 Mozilla/5.0 (Windows ME; U; en; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.63
+Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240
+Mozilla/5.0 (Windows NT 10.0; ARM; Lumia 950 Dual SIM) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393
+Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36
+Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 UBrowser/5.6.13705.206 Safari/537.36
+Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36 OPR/36.0.2130.46
+Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36
+Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.90 Safari/537.36 Vivaldi/1.4.589.11
+Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko
+Mozilla/5.0 (Windows NT 10.0; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0
+Mozilla/5.0 (Windows NT 10.0; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0
+Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Safari/537.36 Edge/12.0
+Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2869.0 Safari/537.36
+Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:52.0) Gecko/20100101 Firefox/52.0
 Mozilla/5.0 (Windows NT 4.0; U) Opera 6.05  [en]
 Mozilla/5.0 (Windows NT 4.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2049.0 Safari/537.36
 Mozilla/5.0 (Windows NT 5.0) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1150.1  Iron/20.0.1150.1 Safari/536.11
@@ -10206,6 +10265,7 @@ Mozilla/5.0 (Windows NT 5.1; rv:26.0) Gecko/20100101 Firefox/26.0 SeaMonkey/2.23
 Mozilla/5.0 (Windows NT 5.1; rv:28.0) Gecko/20100101 Firefox/28.0 SeaMonkey/2.25 Lightning/3.0b1
 Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Firefox/31.0
 Mozilla/5.0 (Windows NT 5.1; rv:33.0) Gecko/20100101 Firefox/33.0 SeaMonkey/2.3
+Mozilla/5.0 (Windows NT 5.1; rv:38.0) Gecko/20100101 Firefox/38.0 SeaMonkey/2.35
 Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0
 Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20110624 Thunderbird/5.0
 Mozilla/5.0 (Windows NT 5.1; rv:6.0) Gecko/20100101 Firefox/6.0 FirePHP/0.6
@@ -10284,6 +10344,7 @@ Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.9
 Mozilla/5.0 (Windows NT 6.0) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1150.1  Iron/20.0.1150.1 Safari/536.11
 Mozilla/5.0 (Windows NT 6.0) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.36 Safari/536.5
 Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1200.0 Iron/21.0.1200.0  Safari/537.1
+Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36
 Mozilla/5.0 (Windows NT 6.0) yi; AppleWebKit/345667.12221 (KHTML, like Gecko) Chrome/23.0.1271.26 Safari/453667.1221
 Mozilla/5.0 (Windows NT 6.0; U; en; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 10.00
 Mozilla/5.0 (Windows NT 6.0; U; en; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.51
@@ -10306,6 +10367,7 @@ Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome
 Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Iron/16.0.950.0 Chrome/16.0.950.0 Safari/535.7
 Mozilla/5.0 (Windows NT 6.0; WOW64; rv:12.2) Gecko/20120605 Firefox/12.2 PaleMoon/12.2
 Mozilla/5.0 (Windows NT 6.0; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0
+Mozilla/5.0 (Windows NT 6.0; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0
 Mozilla/5.0 (Windows NT 6.0; rv:11.0) Gecko/20120319 Firefox/11.0 PaleMoon/11.0
 Mozilla/5.0 (Windows NT 6.0; rv:12.0) Gecko/20120424 Firefox/12.0 PaleMoon/12.0
 Mozilla/5.0 (Windows NT 6.0; rv:12.3) Gecko/20120717 Firefox/12.3 PaleMoon/12.3
@@ -10317,6 +10379,7 @@ Mozilla/5.0 (Windows NT 6.0; rv:17.0) Gecko/20130620 Thunderbird/17.0.7
 Mozilla/5.0 (Windows NT 6.0; rv:19.0) Gecko/20100101 Firefox/19.0 SeaMonkey/2.16.1
 Mozilla/5.0 (Windows NT 6.0; rv:2.0) Gecko/20100101 Firefox/4.0 Opera 12.14
 Mozilla/5.0 (Windows NT 6.0; rv:2.1.1) Gecko/20110415 Firefox/4.0.2pre Fennec/4.0.1
+Mozilla/5.0 (Windows NT 6.0; rv:36.0) Gecko/20100101 Firefox/36.0 SeaMonkey/2.33.1
 Mozilla/5.0 (Windows NT 6.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.694.0 Safari/534.24
 Mozilla/5.0 (Windows NT 6.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.3 Safari/534.24
 Mozilla/5.0 (Windows NT 6.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.68 Safari/534.24
@@ -10425,7 +10488,10 @@ Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrom
 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36 OPR/19.0.1326.56
 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.154 Safari/537.36 OPR/20.0.1387.91
 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.67 Safari/537.36
+Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.89 Vivaldi/1.0.94.2 Safari/537.36
 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36
+Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.155 Safari/537.36 OPR/31.0.1889.174
+Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Maxthon/4.4.6.1000 Chrome/30.0.1599.101 Safari/537.36
 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1250.0 Iron/22.0.2150.0 Safari/537.4
 Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.1) Gecko/20100101 Firefox/10.0.1
@@ -10563,6 +10629,8 @@ Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.13 (KHTML, like Gecko) Chrome/24.0.
 Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1464.0 Safari/537.36
 Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1467.0 Safari/537.36
 Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4
+Mozilla/5.0 (Windows NT 6.2; ARM; Trident/7.0; Touch; rv:11.0; WPDesktop; NOKIA; Lumia 635) like Gecko
+Mozilla/5.0 (Windows NT 6.2; ARM; Trident/7.0; Touch; rv:11.0; WPDesktop; NOKIA; Lumia 920) like Geckoo
 Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/535.11 (KHTML, like Gecko)  Iron/17.0.1000.0 Chrome/17.0.1000.0 Safari/535.11
 Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.65 Safari/535.11
 Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.66 Safari/535.11
@@ -10579,6 +10647,7 @@ Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrom
 Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1500.55 Safari/537.36
 Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.2 Safari/537.36
 Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.17 Safari/537.36
+Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.76 Safari/537.36 OPR/28.0.1750.40
 Mozilla/5.0 (Windows NT 6.2; WOW64; rv:1.8.0.7) Gecko/20110321 MultiZilla/4.33.2.6a SeaMonkey/8.6.55
 Mozilla/5.0 (Windows NT 6.2; WOW64; rv:12.2) Gecko/20120605 Firefox/12.2 PaleMoon/12.2
 Mozilla/5.0 (Windows NT 6.2; WOW64; rv:15.0) Gecko/20120910144328 Firefox/15.0.2
@@ -10591,6 +10660,7 @@ Mozilla/5.0 (Windows NT 6.2; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0 SeaMonk
 Mozilla/5.0 (Windows NT 6.2; WOW64; rv:23.0) Gecko/20100101 Thunderbird/23.0 Lightning/2.5b1
 Mozilla/5.0 (Windows NT 6.2; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.0
 Mozilla/5.0 (Windows NT 6.2; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0
+Mozilla/5.0 (Windows NT 6.2; WOW64; rv:39.0) Gecko/20100101 Firefox/39.0
 Mozilla/5.0 (Windows NT 6.2; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0
 Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36
 Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:12.2.1) Gecko/20120616 Firefox/12.2.1-x64 PaleMoon/12.2.1-x64
@@ -10622,6 +10692,10 @@ Mozilla/5.0 (Windows NT 6.4; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrom
 Mozilla/5.0 (Windows NT 7.1) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.112 Safari/534.30
 Mozilla/5.0 (Windows NT) AppleWebKit/534.20 (KHTML, like Gecko) Chrome/11.0.672.2 Safari/534.20
 Mozilla/5.0 (Windows NT; U; en) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1 Iris/1.1.7 Safari/525.20
+Mozilla/5.0 (Windows Phone 10.0; Android 4.2.1; DEVICE INFO) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Mobile Safari/537.36 Edge/12.0
+Mozilla/5.0 (Windows Phone 8.1; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 530) like Gecko
+Mozilla/5.0 (Windows Phone 8.1; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 630) like Gecko
+Mozilla/5.0 (Windows Phone 8.1; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 920) like Gecko
 Mozilla/5.0 (Windows XP; U) Opera 6.05  [de]
 Mozilla/5.0 (Windows XP; U) Opera 6.06  [en]
 Mozilla/5.0 (Windows x86; rv:19.0) Gecko/20100101 Firefox/19.0
@@ -15130,10 +15204,14 @@ Mozilla/5.0 (X11; CrOS i686 2268.111.0) AppleWebKit/536.11 (KHTML, like Gecko) C
 Mozilla/5.0 (X11; CrOS i686 3912.101.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36
 Mozilla/5.0 (X11; CrOS i686 4319.74.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.57 Safari/537.36
 Mozilla/5.0 (X11; CrOS x86_64 5841.83.0) AppleWebKit/537.36 (KHTML like Gecko) Chrome/36.0.1985.138 Safari/537.36
+Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36
+Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0
 Mozilla/5.0 (X11; FreeBSD 7.1-RELEASE i386; U; en; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.63
 Mozilla/5.0 (X11; FreeBSD amd64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.65 Safari/535.11
+Mozilla/5.0 (X11; FreeBSD amd64) AppleWebKit/535.22+ (KHTML, like Gecko) Chromium/17.0.963.56 Chrome/17.0.963.56 Safari/535.22+ Epiphany/2.30.6
 Mozilla/5.0 (X11; FreeBSD amd64) AppleWebKit/536.5 (KHTML like Gecko) Chrome/19.0.1084.56 Safari/1EA69
 Mozilla/5.0 (X11; FreeBSD amd64) AppleWebKit/536.5 (KHTML like Gecko) Chrome/19.0.1084.56 Safari/536.5
+Mozilla/5.0 (X11; FreeBSD amd64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36
 Mozilla/5.0 (X11; FreeBSD amd64) AppleWebKit/537.4 (KHTML like Gecko) Chrome/22.0.1229.79 Safari/537.4
 Mozilla/5.0 (X11; FreeBSD amd64; rv:27.0) Gecko/20100101 Firefox/27.0 SeaMonkey/2.24
 Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20100101 Firefox/5.0
@@ -15160,6 +15238,7 @@ Mozilla/5.0 (X11; Linux armv7l; rv:2.0b4pre) Gecko/20100812 Firefox/4.0b4pre Fen
 Mozilla/5.0 (X11; Linux armv7l; rv:2.0b4pre) Gecko/20100818 Firefox/4.0b4pre Fennec/2.0a1pre
 Mozilla/5.0 (X11; Linux i386; U) Opera 7.60  [en-GB]
 Mozilla/5.0 (X11; Linux i586; rv:31.0) Gecko/20100101 Firefox/31.0
+Mozilla/5.0 (X11; Linux i686 (x86_64)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.130 Safari/537.36
 Mozilla/5.0 (X11; Linux i686 on x86_64; rv:10.0.6) Gecko/20100101 Firefox/10.0.6 Iceweasel/10.0.6
 Mozilla/5.0 (X11; Linux i686 on x86_64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
 Mozilla/5.0 (X11; Linux i686 on x86_64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1 Fennec/2.0.1
@@ -15203,6 +15282,9 @@ Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.
 Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.22 (KHTML like Gecko) Ubuntu Chromium/25.0.1364.160 Chrome/25.0.1364.160 Safari/537.22
 Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1478.0 Safari/537.36
 Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2166.2 Safari/537.36
+Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.101 Safari/537.36 OPR/40.0.2308.62
+Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/51.0.2704.79 Chrome/51.0.2704.79 Safari/537.36
+Mozilla/5.0 (X11; Linux i686) AppleWebKit/538.1 (KHTML, like Gecko) QupZilla/1.8.6 Safari/538.1
 Mozilla/5.0 (X11; Linux i686; Debian Testing; rv:7.0.1) Gecko/20100101 Firefox/7.0.1 Iceweasel/7.0.1
 Mozilla/5.0 (X11; Linux i686; U) Opera 7.54  [en]
 Mozilla/5.0 (X11; Linux i686; U) Opera 7.54 [en]
@@ -15301,6 +15383,10 @@ Mozilla/5.0 (X11; Linux i686; rv:28.0) Gecko/20100101 Firefox/28.0
 Mozilla/5.0 (X11; Linux i686; rv:28.0) Gecko/20100101 Firefox/28.0 SeaMonkey/2.25
 Mozilla/5.0 (X11; Linux i686; rv:31.0) Gecko/20100101 Firefox/31.0 SeaMonkey/2.28a1
 Mozilla/5.0 (X11; Linux i686; rv:32.0) Gecko/20100101 Firefox/32.0
+Mozilla/5.0 (X11; Linux i686; rv:40.0) Gecko/20100101 Firefox/40.0
+Mozilla/5.0 (X11; Linux i686; rv:43.0) Gecko/20100101 Firefox/43.0
+Mozilla/5.0 (X11; Linux i686; rv:46.0) Gecko/20100101 Firefox/46.0
+Mozilla/5.0 (X11; Linux i686; rv:49.0) Gecko/20100101 Firefox/49.0
 Mozilla/5.0 (X11; Linux i686; rv:5.0) Gecko/20100101 Firefox/5.0
 Mozilla/5.0 (X11; Linux i686; rv:5.0) Gecko/20100101 Firefox/5.0 Iceweasel/5.0
 Mozilla/5.0 (X11; Linux i686; rv:6.0) Gecko/20100101 Firefox/6.0
@@ -15353,12 +15439,20 @@ Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20
 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.9 Safari/536.5
 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.5 (KHTML, like Gecko) Iron/19.0.1100.0 Chrome/19.0.1100.0 Safari/536.5
 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1200.0 Iron/21.0.1200.0  Safari/537.1
+Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.21 (KHTML, like Gecko) konqueror/4.14.10 Safari/537.21
+Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.32 (KHTML, like Gecko) Chromium/25.0.1349.2 Chrome/25.0.1349.2 Safari/537.32 Epiphany/3.8.2
 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/36.0.1985.125 Safari/537.36
 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.166 Safari/537.36 OPR/20.0.1396.73172
 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.517 Safari/537.36
 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.137 Safari/4E423F
 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36
+Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.93 Safari/537.36
+Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36 OPR/32.0.1948.25
+Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36 Vivaldi/1.0.344.37
+Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2876.0 Safari/537.36
+Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/33.0.1750.152 Chrome/33.0.1750.152 Safari/537.36
 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.4 (KHTML like Gecko) Chrome/22.0.1229.56 Safari/537.4
+Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/538.1 (KHTML, like Gecko) QupZilla/1.9.0 Safari/538.1
 Mozilla/5.0 (X11; Linux x86_64) Gecko Firefox/5.0
 Mozilla/5.0 (X11; Linux x86_64; U; de; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 10.62
 Mozilla/5.0 (X11; Linux x86_64; U; en; rv:1.8.1) Gecko/20061208 Firefox/2.0.0
@@ -15417,6 +15511,8 @@ Mozilla/5.0 (X11; Linux x86_64; rv:2.2a1pre) Gecko/20110324 Firefox/4.2a1pre
 Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.0 Lightning/2.6b3
 Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 Lightning/2.6.4
 Mozilla/5.0 (X11; Linux x86_64; rv:28.0) Gecko/20100101  Firefox/28.0
+Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Firefox/38.0
+Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.2.1
 Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20100101 Firefox/5.0 FirePHP/0.5
 Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20100101 Firefox/5.0 Firefox/5.0
 Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20100101 Firefox/5.0 Iceweasel/5.0
@@ -15430,9 +15526,12 @@ Mozilla/5.0 (X11; Linux) KHTML/4.9.1 (like Gecko) Konqueror/4.9
 Mozilla/5.0 (X11; Linux; rv:2.0.1) Gecko/20100101 Firefox/4.0.1 Midori/0.4
 Mozilla/5.0 (X11; Mageia; Linux x86_64; rv:10.0.9) Gecko/20100101 Firefox/10.0.9
 Mozilla/5.0 (X11; NetBSD amd64; rv:16.0) Gecko/20121102 Firefox/16.0
+Mozilla/5.0 (X11; NetBSD amd64; rv:30.0) Gecko/20100101 Firefox/30.0
+Mozilla/5.0 (X11; NetBSD x86; en-us) AppleWebKit/666.6+ (KHTML, like Gecko) Chromium/20.0.0000.00 Chrome/20.0.0000.00 Safari/666.6+
 Mozilla/5.0 (X11; NetBSD) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36
 Mozilla/5.0 (X11; OpenBSD amd64; rv:24.0) Gecko/20100101 Firefox/24.0 SeaMonkey/2.21
 Mozilla/5.0 (X11; OpenBSD amd64; rv:28.0) Gecko/20100101 Firefox/28.0
+Mozilla/5.0 (X11; OpenBSD amd64; rv:30.0) Gecko/20100101 Firefox/30.0
 Mozilla/5.0 (X11; OpenBSD i386) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36
 Mozilla/5.0 (X11; U; AIX 0043031B4C00; en-US; rv:1.2.1) Gecko/20021209
 Mozilla/5.0 (X11; U; AIX 0048013C4C00; en-US; rv:1.0.1) Gecko/20021009 Netscape/7.0
@@ -15570,6 +15669,7 @@ Mozilla/5.0 (X11; U; Linux armv6l; en-US; rv:1.9.2a1pre) Gecko/20081222 Fennec/1
 Mozilla/5.0 (X11; U; Linux armv6l; rv 1.8.1.5pre) Gecko/20070619 Minimo/0.020
 Mozilla/5.0 (X11; U; Linux armv6l; rv: 1.8.1.5pre) Gecko/20070619 Minimo/0.020
 Mozilla/5.0 (X11; U; Linux armv6l; sv-se) AppleWebKit/528.5+ (KHTML, like Gecko, Safari/528.5+) midori
+Mozilla/5.0 (X11; U; Linux armv7l like Android; en-us) AppleWebKit/531.2+ (KHTML, like Gecko) Version/5.0 Safari/533.2+ Kindle/3.0+
 Mozilla/5.0 (X11; U; Linux armv7l; en-GB; rv:1.9.2.3pre) Gecko/20100723 Firefox/3.6.11
 Mozilla/5.0 (X11; U; Linux armv7l; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.204 Safari/534.16
 Mozilla/5.0 (X11; U; Linux armv7l; en-US; rv:1.9.0.1) Gecko/2009010915 Minefield/3.0.1
@@ -17815,6 +17915,7 @@ Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.7 (KHTML, like Gecko)
 Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Epiphany/2.30.6 Safari/534.7
 Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Iron/7.0.520.0 Chrome/7.0.520.0 Safari/534.7
 Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.20 Safari/535.1
+Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.114 Safari/537.36 Puffin/4.5.0IT
 Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/540.0 (KHTML, like Gecko) Ubuntu/10.10 Chrome/8.1.0.0 Safari/540.0
 Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/540.0 (KHTML, like Gecko) Ubuntu/10.10 Chrome/9.1.0.0 Safari/540.0
 Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/540.0 (KHTML,like Gecko) Chrome/9.1.0.0 Safari/540.0
@@ -18109,6 +18210,7 @@ Mozilla/5.0 (X11; U; Linux x86_64; en-us) AppleWebKit/528.5+ (KHTML, like Gecko,
 Mozilla/5.0 (X11; U; Linux x86_64; en-us) AppleWebKit/531.2+ (KHTML, like Gecko) Version/5.0 Safari/531.2+
 Mozilla/5.0 (X11; U; Linux x86_64; en-us) AppleWebKit/532+ (KHTML, like Gecko) Safari/419.3 Midori/0.1.8
 Mozilla/5.0 (X11; U; Linux x86_64; en-us) AppleWebKit/534.35 (KHTML, like Gecko) Chrome/11.0.696.65 Safari/534.35 Puffin/2.9174AT
+Mozilla/5.0 (X11; U; Linux x86_64; en-us) AppleWebKit/537.36 (KHTML, like Gecko)  Chrome/30.0.1599.114 Safari/537.36 Puffin/4.8.0.2965AT
 Mozilla/5.0 (X11; U; Linux x86_64; en; rv:1.8.1.13) Gecko/20080322 Epiphany/2.22 Firefox/2.0.0.4
 Mozilla/5.0 (X11; U; Linux x86_64; en; rv:1.8.1.13) Gecko/20080326 (Debian-1.8.1.13-1) Epiphany/2.20
 Mozilla/5.0 (X11; U; Linux x86_64; en; rv:1.8.1.14) Gecko/20080616 Fedora/2.20.3-4.fc8 Epiphany/2.20 Firefox/2.0.0.14
@@ -18460,6 +18562,7 @@ Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) Gecko/20100101 Firefox/21.0
 Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) Gecko/20130331 Firefox/21.0
 Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0
 Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:35.0) Gecko/20100101 Firefox/35.0
+Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0
 Mozilla/5.0 (X11; debian; Linux x86_64; rv:15.0) Gecko/20100101 Iceweasel/15.0
 Mozilla/5.0 (X11;U; Linux i686; en-GB; rv:1.9.1) Gecko/20090624 Ubuntu/9.04 (jaunty) Firefox/3.5
 Mozilla/5.0 (X11;U;Linux i686;en-US;rv:1.8.1) Gecko/2006101022 Firefox/2.0
@@ -19248,12 +19351,16 @@ Mozilla/5.0 (compatible;archive.org_bot/heritrix-1.9.0-200608171144 +http://pand
 Mozilla/5.0 (en-US;rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2
 Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Web Preview) Version/3.1 Safari/525.13
 Mozilla/5.0 (hp-tablet; Linux; hpwOS/3.0.2; U; de-DE) AppleWebKit/534.6 (KHTML, like Gecko) wOSBrowser/234.40.1 Safari/534.6 TouchPad/1.0
+Mozilla/5.0 (iPad; CPU OS 10_0 like Mac OS X) AppleWebKit/601.1 (KHTML, like Gecko) CriOS/49.0.2623.109 Mobile/14A5335b Safari/601.1.46
 Mozilla/5.0 (iPad; CPU OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176
 Mozilla/5.0 (iPad; CPU OS 5_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko ) Version/5.1 Mobile/9B176 Safari/7534.48.3
 Mozilla/5.0 (iPad; CPU OS 5_1_1 like Mac OS X; ru-ru) AppleWebKit/534.46.0 (KHTML, like Gecko) CriOS/21.0.1180.80 Mobile/9B206 Safari/7534.48.3
 Mozilla/5.0 (iPad; CPU OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5355d Safari/8536.25
 Mozilla/5.0 (iPad; CPU OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) CriOS/30.0.1599.12 Mobile/11A465 Safari/8536.25 (3B92C18B-D9DE-4CB7-A02A-22FD2AF17C8F)
+Mozilla/5.0 (iPad; CPU OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53
 Mozilla/5.0 (iPad; CPU OS 8_0_2 like Mac OS X) AppleWebKit/600.1.4 (KHTML like Gecko) Mobile/12A405 Version/7.0 Safari/9537.53
+Mozilla/5.0 (iPad; CPU OS 8_4_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12H321 Safari/600.1.4
+Mozilla/5.0 (iPad; CPU OS 9_3_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13F69 Safari/601.1
 Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; de-de) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7W367a Safari/531.21.10
 Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B334b Safari/531.21.10
 Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B334b Safari/531.21.1021.10gin_lib.cc
@@ -19281,7 +19388,13 @@ Mozilla/5.0 (iPhone Simulator; U; CPU iPhone OS 2_2_1 like Mac OS X; en-us) Appl
 Mozilla/5.0 (iPhone Simulator; U; CPU iPhone OS 3_0 like Mac OS X; en-us) AppleWebKit/528.11 (KHTML, like Gecko) Version/3.1.1 Mobile/7A238j Safari/525.20
 Mozilla/5.0 (iPhone Simulator; U; CPU iPhone OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7D11 Safari/531.21.10
 Mozilla/5.0 (iPhone; CPU  iPhone OS 3_0 like Mac OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko) Version/4.0 Mobile/7A341 Safari/528.16
+Mozilla/5.0 (iPhone; CPU iPhone OS 10_0 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) GSA/18.0.130791545 Mobile/14A5345a Safari/600.1.4
+Mozilla/5.0 (iPhone; CPU iPhone OS 10_0 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Mobile/14A346 Safari/602.1
+Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25
 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53
+Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12F70 Safari/600.1.4
+Mozilla/5.0 (iPhone; CPU iPhone OS 8_4_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) GSA/8.0.57838 Mobile/12H321 Safari/600.1.4
+Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13C75 Safari/601.1
 Mozilla/5.0 (iPhone; U; CPU OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B334b Safari/531.21.10
 Mozilla/5.0 (iPhone; U; CPU comme Mac OS X; en) AppleWebKit/420 + (KHTML, comme Gecko) Version/3.0 Mobile/1A543 Safari/419.3
 Mozilla/5.0 (iPhone; U; CPU iPhone OS 1_2_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5
@@ -19371,6 +19484,7 @@ Mozilla/5.0 (iPhone; U; fr; CPU iPhone OS 4_2_1 like Mac OS X; fr) AppleWebKit/5
 Mozilla/5.0 (iPhone; U; ru; CPU iPhone OS 4_2_1 like Mac OS X; fr) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8C148a Safari/6533.18.5
 Mozilla/5.0 (iPhone; U; ru; CPU iPhone OS 4_2_1 like Mac OS X; ru) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8C148a Safari/6533.18.5
 Mozilla/5.0 (iPod touch; CPU iPhone OS 7_1 like Mac OS X) AppleWebKit/537.51.2 (KHTML like Gecko) Version/7.0 Mobile/11D167 Safari/123E71C
+Mozilla/5.0 (iPod; CPU iPhone OS 8_4 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) CriOS/44.0.2403.67 Mobile/12H143 Safari/600.1.4
 Mozilla/5.0 (iPod; U; CPU iPhone OS 2_1_1 like Mac OS X; ja-jp) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1 Mobile/5F138 Safari/525.20
 Mozilla/5.0 (iPod; U; CPU iPhone OS 2_2 like Mac OS X; de-de) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1 Mobile/5G77 Safari/525.20
 Mozilla/5.0 (iPod; U; CPU iPhone OS 2_2 like Mac OS X; de-de) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1 Mobile/5G77a Safari/525.20
@@ -20525,6 +20639,7 @@ Opera/9.80 (J2ME/MIDP; Opera Mini/6.24093/26.1305; U; en) Presto/2.8.119 Version
 Opera/9.80 (J2ME/MIDP; Opera Mini/6.24288/25.729; U; en) Presto/2.5.25 Version/10.54
 Opera/9.80 (J2ME/MIDP; Opera Mini/6.5.26955/27.1407; U; en) Presto/2.8.119 Version/11.10
 Opera/9.80 (J2ME/MIDP; Opera Mini/7.1.32052/35.5706; U; id) Presto/2.8.119 Version/11.10
+Opera/9.80 (J2ME/MIDP; Opera Mini/8.0.35626/37.8918; U; en) Presto/2.12.423 Version/12.16
 Opera/9.80 (J2ME/MIDP; Opera Mini/9 (Compatible; MSIE:9.0; iPhone; BlackBerry9700; AppleWebKit/24.746; U; en) Presto/2.5.25 Version/10.54
 Opera/9.80 (J2ME/MIDP; Opera Mini/9.80 (J2ME/22.478; U; en) Presto/2.5.25 Version/10.54
 Opera/9.80 (J2ME/MIDP; Opera Mini/9.80 (J2ME/23.377; U; en) Presto/2.5.25 Version/10.54
@@ -20872,6 +20987,7 @@ Robot/www.pj-search.com
 Robot: NutchCrawler- Owner: [email protected]
 [email protected]
 Robozilla/1.0
+Roku/DVP-4.1 (024.01E01250A)
 Rome Client (http://tinyurl.com/64t5n) Ver: 0.9
 Rotondo/3.1 libwww/5.3.1
 RssBandit/1.5.0.10 (.NET CLR 1.1.4322.2407; WinNT 5.1.2600.0; http://www.rssbandit.org) (.NET CLR 1.1.4322.2407; WinNT 5.1.2600.0; )
@@ -21408,6 +21524,7 @@ Wget/1.10.2
 Wget/1.10.2 (Red Hat modified)
 Wget/1.11.1 (Red Hat modified)
 Wget/1.11.4 (Red Hat modified)
+Wget/1.12 (freebsd8.1)
 Wget/1.15 (darwin13.0.2)
 Wget/1.5.3
 Wget/1.6
@@ -21685,6 +21802,8 @@ exactseek-pagereaper-2.63 ([email protected])
 exactseek.com
 exooba crawler/exooba crawler (crawler for exooba.com; http://www.exooba.com/; info at exooba dot com)
 exooba/exooba crawler (exooba; exooba)
+facebookexternalhit/1.1
+facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
 facebookscraper/1.0( http://www.facebook.com/sharescraper_help.php)
 factbot : http://www.factbites.com/robots
 fastbot crawler beta 2.0 (+http://www.fastbot.de)

pwnlib/dynelf.py

@@ -157,6 +157,7 @@ class DynELF(object):
             elf(str,ELF):  Path to the ELF file on disk, or a loaded :class:`pwnlib.elf.ELF`.
         '''
         self._elfclass = None
+        self._elftype  = None
         self._link_map = None
         self._waitfor  = None
         self._bases    = {}
@@ -204,6 +205,22 @@ class DynELF(object):
                               constants.ELFCLASS64: 64}[elfclass]
         return self._elfclass
 
+    @property
+    def elftype(self):
+        """e_type from the elf header. In practice the value will almost always
+        be 'EXEC' or 'DYN'. If the value is architecture-specific (between
+        ET_LOPROC and ET_HIPROC) or invalid, KeyError is raised.
+        """
+        if not self._elftype:
+            Ehdr  = {32: elf.Elf32_Ehdr, 64: elf.Elf64_Ehdr}[self.elfclass]
+            elftype = self.leak.field(self.libbase, Ehdr.e_type)
+            self._elftype = {constants.ET_NONE: 'NONE',
+                             constants.ET_REL: 'REL',
+                             constants.ET_EXEC: 'EXEC',
+                             constants.ET_DYN: 'DYN',
+                             constants.ET_CORE: 'CORE'}[elftype]
+        return self._elftype
+
     @property
     def link_map(self):
         """Pointer to the runtime link_map object"""
@@ -356,9 +373,7 @@ class DynELF(object):
         dynamic = leak.field(phead, Phdr.p_vaddr)
         self.status("PT_DYNAMIC @ %#x" % dynamic)
 
-        #Sometimes this is an offset instead of an address
-        if 0 < dynamic < 0x400000:
-            dynamic += base
+        dynamic = self._make_absolute_ptr(dynamic)
 
         return dynamic
 
@@ -392,9 +407,7 @@ class DynELF(object):
         self.status("Found %s at %#x" % (name, dynamic))
         ptr = leak.field(dynamic, Dyn.d_ptr)
 
-        # Sometimes this is an offset rather than an actual pointer.
-        if 0 < ptr < 0x400000:
-            ptr += self.libbase
+        ptr = self._make_absolute_ptr(ptr)
 
         return ptr
 
@@ -438,8 +451,7 @@ class DynELF(object):
             w.failure("Could not find DT_PLTGOT or DT_DEBUG")
             return None
 
-        if 0 < result < 0x400000:
-            result += self.libbase
+        result = self._make_absolute_ptr(result)
 
         w.success('%#x' % result)
         return result
@@ -672,11 +684,9 @@ class DynELF(object):
         if not all([strtab, symtab, hshtab]):
             self.failure("Could not find all tables")
 
-        # glibc pointers are relocated but uclibc are not
-        if 0 < strtab < 0x400000:
-            strtab  += self.libbase
-            symtab  += self.libbase
-            hshtab  += self.libbase
+        strtab = self._make_absolute_ptr(strtab)
+        symtab = self._make_absolute_ptr(symtab)
+        hshtab = self._make_absolute_ptr(hshtab)
 
         #
         # Perform the hash lookup
@@ -843,6 +853,29 @@ class DynELF(object):
                 self.status("Magic did not match")
                 pass
 
+    def _make_absolute_ptr(self, ptr_or_offset):
+        """For shared libraries (or PIE executables), many ELF fields may
+        contain offsets rather than actual pointers. If the ELF type is 'DYN',
+        the argument may be an offset. It will not necessarily be an offset,
+        because the run-time linker may have fixed it up to be a real pointer
+        already. In this case an educated guess is made, and the ELF base
+        address is added to the value if it is determined to be an offset.
+        """
+        if_ptr = ptr_or_offset
+        if_offset = ptr_or_offset + self.libbase
+
+        # if the ELF type is not DYN, the value is a pointer
+
+        if self.elftype != 'DYN':
+            return if_ptr
+
+        # if the ELF type may be DYN, guess
+
+        if 0 < ptr_or_offset < self.libbase:
+            return if_offset
+        else:
+            return if_ptr
+
     def stack(self):
         """Finds a pointer to the stack via __environ, which is an exported
         symbol in libc, which points to the environment block.

pwnlib/elf/__init__.py

@@ -10,5 +10,6 @@ from pwnlib.elf.corefile import Core
 from pwnlib.elf.datatypes import *
 from pwnlib.elf.elf import ELF
 from pwnlib.elf.elf import load
+from pwnlib.elf import plt
 
 __all__ = ['load', 'ELF', 'Core'] + sorted(filter(lambda x: not x.startswith('_'), datatypes.__dict__.keys()))

pwnlib/elf/config.py

@@ -127,6 +127,7 @@ kernel_configuration = [
     Enabled('SECURITY_DMESG_RESTRICT'),
     Enabled('SECURITY_NETWORK'),
     Enabled('SECURITY_SELINUX'),
+    Enabled('ARCH_HAS_ELF_RANDOMIZE'),
     EnabledIfPresent('MSM_KERNEL_PROTECT'),
 
 # Not really security relevant, but come on guys
@@ -182,6 +183,8 @@ kernel_configuration = [
     Enabled('CPU_SW_DOMAIN_PAN', requires=['ARM'], minver=4.3),
     Enabled('CONFIG_ARM64_PAN', requires=['ARM64'], minver=4.3),
     Disabled('OABI_COMPAT'),
+    Disabled('CP_ACCESS', requires=['ARM']),
+    Disabled('CP_ACCESS64', requires=['ARM64']),
 
 # Only relevant of MODULES are enabled
     Enabled('MODULE_SIG', requires=['MODULES']),

pwnlib/elf/corefile.py

@@ -102,9 +102,9 @@ prstatus_types = {
     'aarch64': elf_prstatus_aarch64
 }
 
-prspinfo_types = {
-    32: elf_prspinfo_32,
-    64: elf_prspinfo_64,
+prpsinfo_types = {
+    32: elf_prpsinfo_32,
+    64: elf_prpsinfo_64,
 }
 
 siginfo_types = {
@@ -437,8 +437,8 @@ class Corefile(ELF):
         #: The NT_PRSTATUS object.
         self.prstatus = None
 
-        #: The NT_PRSPINFO object
-        self.prspinfo = None
+        #: The NT_PRPSINFO object
+        self.prpsinfo = None
 
         #: The NT_SIGINFO object
         self.siginfo = None
@@ -484,7 +484,7 @@ class Corefile(ELF):
             log.warn_once("%s does not use a supported corefile architecture, registers are unavailable" % self.file.name)
 
         prstatus_type = prstatus_types.get(self.arch, None)
-        prspinfo_type = prspinfo_types.get(self.bits, None)
+        prpsinfo_type = prpsinfo_types.get(self.bits, None)
         siginfo_type = siginfo_types.get(self.bits, None)
 
         with log.waitfor("Parsing corefile...") as w:
@@ -505,10 +505,10 @@ class Corefile(ELF):
                     # Try to find NT_PRPSINFO
                     # Note that pyelftools currently mis-identifies the enum name
                     # as 'NT_GNU_BUILD_ID'
-                    if note.n_descsz == ctypes.sizeof(prspinfo_type) and \
+                    if note.n_descsz == ctypes.sizeof(prpsinfo_type) and \
                        note.n_type == 'NT_GNU_BUILD_ID':
-                        self.NT_PRSPINFO = note
-                        self.prspinfo = prspinfo_type.from_buffer_copy(note.n_desc)
+                        self.NT_PRPSINFO = note
+                        self.prpsinfo = prpsinfo_type.from_buffer_copy(note.n_desc)
 
                     # Try to find NT_SIGINFO so we can see the fault
                     if note.n_type == 0x53494749:
@@ -943,6 +943,10 @@ class Corefile(ELF):
 
         return super(Corefile, self).__getattribute__(attribute)
 
+    # Override routines which don't make sense for Corefiles
+    def _populate_got(*a): pass
+    def _populate_plt(*a): pass
+
 class Core(Corefile):
     """Alias for :class:`.Corefile`"""
 

pwnlib/elf/datatypes.py

@@ -595,7 +595,7 @@ class Elf64_auxv_t(ctypes.Structure):
     _fields_ = [('a_type', ctypes.c_uint64),
                 ('a_val', ctypes.c_uint64),]
 
-def generate_prspinfo(long):
+def generate_prpsinfo(long):
     return [
         ('pr_state', byte),
         ('pr_sname', char),
@@ -612,11 +612,11 @@ def generate_prspinfo(long):
         ('pr_psargs', char * 80)
     ]
 
-class elf_prspinfo_32(ctypes.Structure):
-    _fields_ = generate_prspinfo(Elf32_Addr)
+class elf_prpsinfo_32(ctypes.Structure):
+    _fields_ = generate_prpsinfo(Elf32_Addr)
 
-class elf_prspinfo_64(ctypes.Structure):
-    _fields_ = generate_prspinfo(Elf64_Addr)
+class elf_prpsinfo_64(ctypes.Structure):
+    _fields_ = generate_prpsinfo(Elf64_Addr)
 
 def generate_siginfo(int_t, long_t):
     class siginfo_t(ctypes.Structure):

pwnlib/elf/elf.py

@@ -51,6 +51,7 @@ from elftools.elf.constants import P_FLAGS
 from elftools.elf.constants import SHN_INDICES
 from elftools.elf.descriptions import describe_e_type
 from elftools.elf.elffile import ELFFile
+from elftools.elf.enums import ENUM_P_TYPE
 from elftools.elf.gnuversions import GNUVerDefSection
 from elftools.elf.relocation import RelocationSection
 from elftools.elf.sections import SymbolTableSection
@@ -63,13 +64,14 @@ from pwnlib.context import LocalContext
 from pwnlib.context import context
 from pwnlib.elf.config import kernel_configuration
 from pwnlib.elf.config import parse_kconfig
+from pwnlib.elf.plt import emulate_plt_instructions
 from pwnlib.log import getLogger
 from pwnlib.qemu import get_qemu_arch
 from pwnlib.term import text
 from pwnlib.tubes.process import process
 from pwnlib.util import misc
 from pwnlib.util import packing
-from pwnlib.util import sh_string
+from pwnlib.util.sh_string import sh_string
 
 log = getLogger(__name__)
 
@@ -122,6 +124,8 @@ class dotdict(dict):
     Is a real :class:`dict` object, but also serves up keys as attributes
     when reading attributes.
 
+    Supports recursive instantiation for keys which contain dots.
+
     Example:
 
         >>> x = pwnlib.elf.elf.dotdict()
@@ -130,9 +134,22 @@ class dotdict(dict):
         >>> x['foo'] = 3
         >>> x.foo
         3
+        >>> x['bar.baz'] = 4
+        >>> x.bar.baz
+        4
     """
     def __getattr__(self, name):
-        return self[name]
+        if name in self:
+            return self[name]
+
+        name_dot = name + '.'
+        name_len = len(name_dot)
+        subkeys = {k[name_len:]: self[k] for k in self if k.startswith(name_dot)}
+
+        if subkeys:
+            return dotdict(subkeys)
+
+        return getattr(super(dotdict, self), name)
 
 class ELF(ELFFile):
     """Encapsulates information about an ELF file.
@@ -169,7 +186,7 @@ class ELF(ELFFile):
     _fill_gaps = True
 
 
-    def __init__(self, path):
+    def __init__(self, path, checksec=True):
         # elftools uses the backing file for all reads and writes
         # in order to permit writing without being able to write to disk,
         # mmap() the file.
@@ -231,16 +248,20 @@ class ELF(ELFFile):
         self.bytes = self.bits / 8
 
         if self.arch == 'mips':
-            if self.header['e_flags'] & E_FLAGS.EF_MIPS_ARCH_64 \
-            or self.header['e_flags'] & E_FLAGS.EF_MIPS_ARCH_64R2:
+            mask = lambda a, b: a & b == b
+            flags = self.header['e_flags']
+
+            if mask(flags, E_FLAGS.EF_MIPS_ARCH_32) \
+            or mask(flags, E_FLAGS.EF_MIPS_ARCH_32R2):
+                pass
+            elif mask(flags, E_FLAGS.EF_MIPS_ARCH_64) \
+            or mask(flags, E_FLAGS.EF_MIPS_ARCH_64R2):
                 self.arch = 'mips64'
                 self.bits = 64
 
         self._address = 0
         if self.elftype != 'DYN':
-            for seg in self.segments:
-                if seg.header.p_type != 'PT_LOAD':
-                    continue
+            for seg in self.iter_segments_by_type('PT_LOAD'):
                 addr = seg.header.p_vaddr
                 if addr == 0:
                     continue
@@ -268,13 +289,41 @@ class ELF(ELFFile):
             if config:
                 self.config = parse_kconfig(config)
 
-        self._populate_got_plt()
-        self._populate_symbols()
+        #: ``True`` if the ELF is a statically linked executable
+        self.statically_linked = bool(self.elftype == 'EXEC' and self.load_addr)
+
+        #: ``True`` if the ELF is an executable
+        self.executable = bool(self.elftype == 'EXEC')
+
+        for seg in self.iter_segments_by_type('PT_INTERP'):
+            self.executable = True
+            self.statically_linked = False
+
+        #: ``True`` if the ELF is a shared library
+        self.library = not self.executable and self.elftype == 'DYN'
+
+        try:
+            self._populate_symbols()
+        except Exception as e:
+            log.warn("Could not populate symbols: %s", e)
+
+        try:
+            self._populate_got()
+        except Exception as e:
+            log.warn("Could not populate GOT: %s", e)
+
+        try:
+            self._populate_plt()
+        except Exception as e:
+            log.warn("Could not populate PLT: %s", e)
+
+        self._populate_synthetic_symbols()
         self._populate_libraries()
         self._populate_functions()
         self._populate_kernel_version()
 
-        self._describe()
+        if checksec:
+            self._describe()
 
     @staticmethod
     @LocalContext
@@ -401,6 +450,15 @@ class ELF(ELFFile):
         """
         return list(self.iter_segments())
 
+    def iter_segments_by_type(self, t):
+        """
+        Yields:
+            Segments matching the specified type.
+        """
+        for seg in self.iter_segments():
+            if t == seg.header.p_type or t in str(seg.header.p_type):
+                yield seg
+
     @property
     def sections(self):
         """
@@ -551,7 +609,7 @@ class ELF(ELFFile):
             return
 
         try:
-            cmd = sh_string.sh_command_with('ulimit -s unlimited; LD_TRACE_LOADED_OBJECTS=1 LD_WARN=1 LD_BIND_NOW=1 %s 2>/dev/null', self.path)
+            cmd = 'ulimit -s unlimited; LD_TRACE_LOADED_OBJECTS=1 LD_WARN=1 LD_BIND_NOW=1 %s 2>/dev/null' % sh_string(self.path)
 
             data = subprocess.check_output(cmd, shell = True, stderr = subprocess.STDOUT)
             libs = misc.parse_ldd_output(data)
@@ -598,15 +656,11 @@ class ELF(ELFFile):
     def _populate_symbols(self):
         """
         >>> bash = ELF(which('bash'))
-        >>> bash.symbols['_start'] == bash.header.e_entry
+        >>> bash.symbols['_start'] == bash.entry
         True
         """
-        # By default, have 'symbols' include everything in the PLT.
-        #
-        # This way, elf.symbols['write'] will be a valid address to call
-        # for write().
-        self.symbols.update(self.plt)
 
+        # Populate all of the "normal" symbols from the symbol tables
         for section in self.sections:
             if not isinstance(section, SymbolTableSection):
                 continue
@@ -617,90 +671,162 @@ class ELF(ELFFile):
                     continue
                 self.symbols[symbol.name] = value
 
-        # Add 'plt.foo' and 'got.foo' to the symbols for entries
-        self.symbols.update({'plt.%s' % sym: addr for sym, addr in self.plt.items()})
-        self.symbols.update({'got.%s' % sym: addr for sym, addr in self.got.items()})
+    def _populate_synthetic_symbols(self):
+        """Adds symbols from the GOT and PLT to the symbols dictionary.
 
-    def _populate_got_plt(self):
-        """Loads the GOT and the PLT symbols and addresses.
+        Does not overwrite any existing symbols, and prefers PLT symbols.
 
-        The following doctest checks the valitidy of the addresses.
-        This assumes that each GOT entry points to its PLT entry,
-        usually +6 bytes but could be anywhere within 0-16 bytes.
+        Synthetic plt.xxx and got.xxx symbols are added for each PLT and
+        GOT entry, respectively.
 
-        >>> from pwnlib.util.packing import unpack
-        >>> bash = ELF(which('bash'))
-        >>> def validate_got_plt(sym):
-        ...     got      = bash.got[sym]
-        ...     plt      = bash.plt[sym]
-        ...     got_addr = unpack(bash.read(got, bash.bytes), bash.bits)
-        ...     return got_addr in range(plt,plt+0x10)
-        ...
-        >>> all(map(validate_got_plt, bash.got.keys()))
-        True
+        Example:bash.
+
+            >>> bash = ELF(which('bash'))
+            >>> bash.symbols.wcscmp == bash.plt.wcscmp
+            True
+            >>> bash.symbols.wcscmp == bash.symbols.plt.wcscmp
+            True
+            >>> bash.symbols.stdin  == bash.got.stdin
+            True
+            >>> bash.symbols.stdin  == bash.symbols.got.stdin
+            True
         """
-        plt = self.get_section_by_name('.plt')
-        got = self.get_section_by_name('.got')
+        for symbol, address in self.plt.items():
+            self.symbols.setdefault(symbol, address)
+            self.symbols['plt.' + symbol] = address
 
-        if not plt:
+        for symbol, address in self.got.items():
+            self.symbols.setdefault(symbol, address)
+            self.symbols['got.' + symbol] = address
+
+    def _populate_got(self):
+        """Loads the symbols for all relocations"""
+        # Statically linked implies no relocations, since there is no linker
+        # Could always be self-relocating like Android's linker *shrug*
+        if self.statically_linked:
             return
 
-        # Find the relocation section for PLT
-        try:
-            rel_plt = next(s for s in self.sections if
-                            s.header.sh_info == self.sections.index(plt) and
-                            isinstance(s, RelocationSection))
-        except StopIteration:
-            # Evidently whatever android-ndk uses to build binaries zeroed out sh_info for rel.plt
-            rel_plt = self.get_section_by_name('.rel.plt') or self.get_section_by_name('.rela.plt')
+        for section in self.iter_sections():
+            # We are only interested in relocations
+            if not isinstance(section, RelocationSection):
+                continue
 
-        if not rel_plt:
-            log.warning("Couldn't find relocations against PLT to get symbols")
-            return
+            # Only get relocations which link to another section (for symbols)
+            if section.header.sh_link == SHN_INDICES.SHN_UNDEF:
+                continue
 
-        if rel_plt.header.sh_link != SHN_INDICES.SHN_UNDEF:
-            # Find the symbols for the relocation section
-            sym_rel_plt = self.sections[rel_plt.header.sh_link]
+            symbols = self.get_section(section.header.sh_link)
 
-            # Populate the GOT
-            for rel in rel_plt.iter_relocations():
+            for rel in section.iter_relocations():
                 sym_idx  = rel.entry.r_info_sym
-                symbol   = sym_rel_plt.get_symbol(sym_idx)
-                name     = symbol.name
 
-                self.got[name] = rel.entry.r_offset
+                if not sym_idx:
+                    continue
 
-        # Depending on the architecture, the beginning of the .plt will differ
-        # in size, and each entry in the .plt will also differ in size.
-        offset     = None
-        multiplier = None
+                symbol = symbols.get_symbol(sym_idx)
 
-        # Map architecture: offset, multiplier
-        header_size, entry_size = {
-            'i386':   (0x10, 0x10),
-            'amd64': (0x10, 0x10),
-            'arm':   (0x14, 0xC),
-            'aarch64': (0x20, 0x20),
-        }.get(self.arch, (0,0))
+                if symbol and symbol.name:
+                    self.got[symbol.name] = rel.entry.r_offset
 
-        address = plt.header.sh_addr + header_size
+        if self.arch == 'mips':
+            try:
+                self._populate_mips_got()
+            except Exception as e:
+                log.warn("Could not populate MIPS GOT: %s", e)
 
-        # Based on the ordering of the GOT symbols, populate the PLT
-        for i,(addr,name) in enumerate(sorted((addr,name) for name, addr in self.got.items())):
-            self.plt[name] = address
+        if not self.got:
+            log.warn("Did not find any GOT entries")
 
-            # Some PLT entries in ARM binaries have a thumb-mode stub that looks like:
-            #
-            # 00008304 <__gmon_start__@plt>:
-            #     8304:   4778        bx  pc
-            #     8306:   46c0        nop         ; (mov r8, r8)
-            #     8308:   e28fc600    add ip, pc, #0, 12
-            #     830c:   e28cca08    add ip, ip, #8, 20  ; 0x8000
-            #     8310:   e5bcf228    ldr pc, [ip, #552]! ; 0x228
-            if self.arch in ('arm', 'thumb') and self.u16(address) == 0x4778:
-                address += 4
+    def _populate_mips_got(self):
+        self._mips_got = {}
+        strings = self.get_section(self.header.e_shstrndx)
 
-            address += entry_size
+        ELF_MIPS_GNU_GOT1_MASK = 0x80000000
+
+        if self.bits == 64:
+            ELF_MIPS_GNU_GOT1_MASK <<= 32
+
+        # Beginning of the GOT
+        got = self.dynamic_value_by_tag('DT_PLTGOT') or 0
+
+        # Find the beginning of the GOT pointers
+        got1_mask = (self.unpack(got) & ELF_MIPS_GNU_GOT1_MASK)
+        i = 2 if got1_mask else 1
+        self._mips_skip = i
+
+        # We don't care about local GOT entries, skip them
+        local_gotno = self.dynamic_value_by_tag('DT_MIPS_LOCAL_GOTNO')
+        got += local_gotno * context.bytes
+
+        # Iterate over the dynamic symbol table
+        dynsym = self.get_section_by_name('.dynsym')
+        symbol_iter = dynsym.iter_symbols()
+
+        # 'gotsym' is the index of the first GOT symbol
+        gotsym = self.dynamic_value_by_tag('DT_MIPS_GOTSYM')
+        for i in range(gotsym):
+            symbol_iter.next()
+
+        # 'symtabno' is the total number of symbols
+        symtabno = self.dynamic_value_by_tag('DT_MIPS_SYMTABNO')
+
+        for i in range(symtabno - gotsym):
+            symbol = symbol_iter.next()
+            self._mips_got[i + gotsym] = got
+            self.got[symbol.name] = got
+            got += self.bytes
+
+    def _populate_plt(self):
+        """Loads the PLT symbols
+
+        >>> path = pwnlib.data.elf.path
+        >>> for test in glob(os.path.join(path, 'test-*')):
+        ...     test = ELF(test)
+        ...     assert '__stack_chk_fail' in test.got, test
+        ...     if test.arch != 'ppc':
+        ...         assert '__stack_chk_fail' in test.plt, test
+        """
+        if self.statically_linked:
+            log.debug("%r is statically linked, skipping GOT/PLT symbols" % self.path)
+            return
+
+        if not self.got:
+            log.debug("%r doesn't have any GOT symbols, skipping PLT" % self.path)
+            return
+
+        # This element holds an address associated with the procedure linkage table
+        # and/or the global offset table.
+        #
+        # Zach's note: This corresponds to the ".got.plt" section, in a PIE non-RELRO binary.
+        #              This corresponds to the ".got" section, in a PIE full-RELRO binary.
+        #              In particular, this is where EBX points when it points into the GOT.
+        dt_pltgot = self.dynamic_value_by_tag('DT_PLTGOT') or 0
+
+        # There are two PLTs we may need to search
+        plt = self.get_section_by_name('.plt')          # <-- Functions only
+        plt_got = self.get_section_by_name('.plt.got')  # <-- Functions used as data
+        plt_mips = self.get_section_by_name('.MIPS.stubs')
+
+        # Invert the GOT symbols we already have, so we can look up by address
+        inv_symbols = {v:k for k,v in self.got.items()}
+        inv_symbols.update({v:k for k,v in self.symbols.items()})
+
+        with context.local(arch=self.arch, bits=self.bits, endian=self.endian):
+            for section in (plt, plt_got, plt_mips):
+                if not section:
+                    continue
+
+                res = emulate_plt_instructions(self,
+                                                dt_pltgot,
+                                                section.header.sh_addr,
+                                                section.data(),
+                                                inv_symbols)
+
+                for address, target in reversed(sorted(res.items())):
+                    self.plt[inv_symbols[target]] = address
+
+        for a,n in sorted({v:k for k,v in self.plt.items()}.items()):
+            log.debug('PLT %#x %s', a, n)
 
     def _populate_kernel_version(self):
         if 'linux_banner' not in self.symbols:
@@ -1069,7 +1195,8 @@ class ELF(ELFFile):
         if self.arch == 'arm' and address & 1:
             arch = 'thumb'
             address -= 1
-        return disasm(self.read(address, n_bytes), vma=address, arch=arch)
+
+        return disasm(self.read(address, n_bytes), vma=address, arch=arch, endian=self.endian)
 
     def asm(self, address, assembly):
         """asm(address, assembly)
@@ -1118,6 +1245,18 @@ class ELF(ELFFile):
 
         return dt
 
+    def dynamic_value_by_tag(self, tag):
+        """dynamic_value_by_tag(tag) -> int
+
+        Retrieve the value from a dynamic tag a la ``DT_XXX``.
+
+        If the tag is missing, returns ``None``.
+        """
+        tag = self.dynamic_by_tag(tag)
+
+        if tag:
+            return tag.entry.d_val
+
     def dynamic_string(self, offset):
         """dynamic_string(offset) -> bytes
 
@@ -1142,35 +1281,168 @@ class ELF(ELFFile):
         return string.rstrip('\x00')
 
 
+
     @property
     def relro(self):
-        """:class:`bool`: Whether the current binary uses RELRO protections."""
+        """:class:`bool`: Whether the current binary uses RELRO protections.
+
+        This requires both presence of the dynamic tag ``DT_BIND_NOW``, and
+        a ``GNU_RELRO`` program header.
+
+        The `ELF Specification`_ describes how the linker should resolve
+        symbols immediately, as soon as a binary is loaded.  This can be
+        emulated with the ``LD_BIND_NOW=1`` environment variable.
+
+            ``DT_BIND_NOW``
+
+            If present in a shared object or executable, this entry instructs
+            the dynamic linker to process all relocations for the object
+            containing this entry before transferring control to the program.
+            The presence of this entry takes precedence over a directive to use
+            lazy binding for this object when specified through the environment
+            or via ``dlopen(BA_LIB)``.
+
+            (`page 81`_)
+
+        Separately, an extension to the GNU linker allows a binary to specify
+        a PT_GNU_RELRO_ program header, which describes the *region of memory
+        which is to be made read-only after relocations are complete.*
+
+        Finally, a new-ish extension which doesn't seem to have a canonical
+        source of documentation is DF_BIND_NOW_, which has supposedly superceded
+        ``DT_BIND_NOW``.
+
+            ``DF_BIND_NOW``
+
+            If set in a shared object or executable, this flag instructs the
+            dynamic linker to process all relocations for the object containing
+            this entry before transferring control to the program. The presence
+            of this entry takes precedence over a directive to use lazy binding
+            for this object when specified through the environment or via
+            ``dlopen(BA_LIB)``.
+
+        .. _ELF Specification: https://refspecs.linuxbase.org/elf/elf.pdf
+        .. _page 81: https://refspecs.linuxbase.org/elf/elf.pdf#page=81
+        .. _DT_BIND_NOW: https://refspecs.linuxbase.org/elf/elf.pdf#page=81
+        .. _PT_GNU_RELRO: https://refspecs.linuxbase.org/LSB_3.1.1/LSB-Core-generic/LSB-Core-generic.html#PROGHEADER
+        .. _DF_BIND_NOW: http://refspecs.linuxbase.org/elf/gabi4+/ch5.dynamic.html#df_bind_now
+
+        """
         if self.dynamic_by_tag('DT_BIND_NOW'):
             return "Full"
 
         if any('GNU_RELRO' in str(s.header.p_type) for s in self.segments):
             return "Partial"
+
         return None
 
     @property
     def nx(self):
-        """:class:`bool`: Whether the current binary uses NX protections."""
-        if not any('GNU_STACK' in str(seg.header.p_type) for seg in self.segments):
-            return False
+        """:class:`bool`: Whether the current binary uses NX protections.
 
-        # Can't call self.executable_segments because of dependency loop.
-        exec_seg = [s for s in self.segments if s.header.p_flags & P_FLAGS.PF_X]
-        return not any('GNU_STACK' in str(seg.header.p_type) for seg in exec_seg)
+        Specifically, we are checking for ``READ_IMPLIES_EXEC`` being set
+        by the kernel, as a result of honoring ``PT_GNU_STACK`` in the kernel.
+
+        The **Linux kernel** directly honors ``PT_GNU_STACK`` to `mark the
+        stack as executable.`__
+
+        .. __: https://github.com/torvalds/linux/blob/v4.9/fs/binfmt_elf.c#L784-L789
+
+        .. code-block:: c
+
+            case PT_GNU_STACK:
+                if (elf_ppnt->p_flags & PF_X)
+                    executable_stack = EXSTACK_ENABLE_X;
+                else
+                    executable_stack = EXSTACK_DISABLE_X;
+                break;
+
+        Additionally, it then sets ``read_implies_exec``, so that `all readable pages
+        are executable`__.
+
+        .. __: https://github.com/torvalds/linux/blob/v4.9/fs/binfmt_elf.c#L849-L850
+
+        .. code-block:: c
+
+            if (elf_read_implies_exec(loc->elf_ex, executable_stack))
+                current->personality |= READ_IMPLIES_EXEC;
+        """
+        if not self.executable:
+            return True
+
+        for seg in self.iter_segments_by_type('GNU_STACK'):
+            return not bool(seg.header.p_flags & P_FLAGS.PF_X)
+
+        # If you NULL out the PT_GNU_STACK section via ELF.disable_nx(),
+        # everything is executable.
+        return False
 
     @property
     def execstack(self):
-        """:class:`bool`: Whether the current binary uses an executable stack."""
-        return not self.nx
+        """:class:`bool`: Whether the current binary uses an executable stack.
+
+        This is based on the presence of a program header PT_GNU_STACK_
+        being present, and its setting.
+
+            ``PT_GNU_STACK``
+
+            The p_flags member specifies the permissions on the segment
+            containing the stack and is used to indicate wether the stack
+            should be executable. The absense of this header indicates
+            that the stack will be executable.
+
+        In particular, if the header is missing the stack is executable.
+        If the header is present, it may **explicitly** mark that the stack is
+        executable.
+
+        This is only somewhat accurate.  When using the GNU Linker, it usees
+        DEFAULT_STACK_PERMS_ to decide whether a lack of ``PT_GNU_STACK``
+        should mark the stack as executable:
+
+        .. code-block:: c
+
+            /* On most platforms presume that PT_GNU_STACK is absent and the stack is
+             * executable.  Other platforms default to a nonexecutable stack and don't
+             * need PT_GNU_STACK to do so.  */
+            uint_fast16_t stack_flags = DEFAULT_STACK_PERMS;
+
+        By searching the source for ``DEFAULT_STACK_PERMS``, we can see which
+        architectures have which settings.
+
+        ::
+
+            $ git grep '#define DEFAULT_STACK_PERMS' | grep -v PF_X
+            sysdeps/aarch64/stackinfo.h:31:#define DEFAULT_STACK_PERMS (PF_R|PF_W)
+            sysdeps/nios2/stackinfo.h:31:#define DEFAULT_STACK_PERMS (PF_R|PF_W)
+            sysdeps/tile/stackinfo.h:31:#define DEFAULT_STACK_PERMS (PF_R|PF_W)
+
+        .. _PT_GNU_STACK: https://refspecs.linuxbase.org/LSB_3.0.0/LSB-PDA/LSB-PDA/progheader.html
+        .. _DEFAULT_STACK_PERMS: https://github.com/bminor/glibc/blob/glibc-2.25/elf/dl-load.c#L1036-L1038
+        """
+        # Dynamic objects do not have the ability to change the executable state of the stack.
+        if not self.executable:
+            return False
+
+        # If NX is completely off for the process, the stack is executable.
+        if not self.nx:
+            return True
+
+        # If the ``PT_GNU_STACK`` program header is missing, then use the
+        # default rules.  Only AArch64 gets a non-executable stack by default.
+        for _ in self.iter_segments_by_type('GNU_STACK'):
+            break
+        else:
+            return self.arch != 'aarch64'
+
+        return False
 
     @property
     def canary(self):
         """:class:`bool`: Whether the current binary uses stack canaries."""
-        return '__stack_chk_fail' in self.symbols
+
+        # Sometimes there is no function for __stack_chk_fail,
+        # but there is an entry in the GOT
+        return '__stack_chk_fail' in (set(self.symbols) | set(self.got))
 
     @property
     def packed(self):
@@ -1240,17 +1512,18 @@ class ELF(ELFFile):
             "PIE:".ljust(10) + {
                 True: green("PIE enabled"),
                 False: red("No PIE (%#x)" % self.address)
-            }[self.pie]
+            }[self.pie],
         ])
 
+        # Execstack may be a thing, even with NX enabled, because of glibc
+        if self.execstack and self.nx:
+            res.append("Stack:".ljust(10) + red("Executable"))
 
         # Are there any RWX areas in the binary?
         #
         # This will occur if NX is disabled and *any* area is
         # RW, or can expressly occur.
-        rwx = self.rwx_segments
-
-        if self.nx and rwx:
+        if self.rwx_segments or (not self.nx and self.writable_segments):
             res += [ "RWX:".ljust(10) + red("Has RWX segments") ]
 
         if self.rpath:
@@ -1331,45 +1604,59 @@ class ELF(ELFFile):
         Undefined Behavior Sanitizer (``UBSAN``)."""
         return any(s.startswith('__ubsan_') for s in self.symbols)
 
+    def _update_args(self, kw):
+        kw.setdefault('arch', self.arch)
+        kw.setdefault('bits', self.bits)
+        kw.setdefault('endian', self.endian)
 
     def p64(self,  address, data, *a, **kw):
         """Writes a 64-bit integer ``data`` to the specified ``address``"""
+        self._update_args(kw)
         return self.write(address, packing.p64(data, *a, **kw))
 
     def p32(self,  address, data, *a, **kw):
         """Writes a 32-bit integer ``data`` to the specified ``address``"""
+        self._update_args(kw)
         return self.write(address, packing.p32(data, *a, **kw))
 
     def p16(self,  address, data, *a, **kw):
         """Writes a 16-bit integer ``data`` to the specified ``address``"""
+        self._update_args(kw)
         return self.write(address, packing.p16(data, *a, **kw))
 
     def p8(self,   address, data, *a, **kw):
         """Writes a 8-bit integer ``data`` to the specified ``address``"""
+        self._update_args(kw)
         return self.write(address, packing.p8(data, *a, **kw))
 
     def pack(self, address, data, *a, **kw):
         """Writes a packed integer ``data`` to the specified ``address``"""
+        self._update_args(kw)
         return self.write(address, packing.pack(data, *a, **kw))
 
     def u64(self,    address, *a, **kw):
         """Unpacks an integer from the specified ``address``."""
+        self._update_args(kw)
         return packing.u64(self.read(address, 8), *a, **kw)
 
     def u32(self,    address, *a, **kw):
         """Unpacks an integer from the specified ``address``."""
+        self._update_args(kw)
         return packing.u32(self.read(address, 4), *a, **kw)
 
     def u16(self,    address, *a, **kw):
         """Unpacks an integer from the specified ``address``."""
+        self._update_args(kw)
         return packing.u16(self.read(address, 2), *a, **kw)
 
     def u8(self,     address, *a, **kw):
         """Unpacks an integer from the specified ``address``."""
+        self._update_args(kw)
         return packing.u8(self.read(address, 1), *a, **kw)
 
     def unpack(self, address, *a, **kw):
         """Unpacks an integer from the specified ``address``."""
+        self._update_args(kw)
         return packing.unpack(self.read(address, context.bytes), *a, **kw)
 
     def string(self, address):
@@ -1400,3 +1687,31 @@ class ELF(ELFFile):
 
     def parse_kconfig(self, data):
         self.config.update(parse_kconfig(data))
+
+    def disable_nx(self):
+        """Disables NX for the ELF.
+
+        Zeroes out the ``PT_GNU_STACK`` program header ``p_type`` field.
+        """
+        PT_GNU_STACK = packing.p32(ENUM_P_TYPE['PT_GNU_STACK'])
+
+        if not self.executable:
+            log.error("Can only make stack executable with executables")
+
+        for i, segment in enumerate(self.iter_segments()):
+            if not segment.header.p_type:
+                continue
+            if 'GNU_STACK' not in segment.header.p_type:
+                continue
+
+            phoff = self.header.e_phoff
+            phentsize = self.header.e_phentsize
+            offset = phoff + phentsize * i
+
+            if self.mmap[offset:offset+4] == PT_GNU_STACK:
+                self.mmap[offset:offset+4] = '\x00' * 4
+                self.save()
+                return
+
+        log.error("Could not find PT_GNU_STACK, stack should already be executable")
+

pwnlib/elf/plt.py

@@ -0,0 +1,180 @@
+"""Emulates instructions in the PLT to locate symbols more accurately.
+"""
+import logging
+
+from pwnlib.args import args
+from pwnlib.log import getLogger
+from pwnlib.util import fiddling
+from pwnlib.util import packing
+
+log = getLogger(__name__)
+
+DEBUG = args.PLT_DEBUG or 0
+
+if not DEBUG:
+    log.setLevel(logging.DEBUG + 1)
+
+def emulate_plt_instructions(elf, got, address, data, targets):
+    """Emulates instructions in ``data``
+
+    Arguments:
+        elf(ELF): ELF that we are emulating
+        got(int): Address of the GOT, as expected in e.g. EBX
+        address(int): Address of ``data`` for emulation
+        data(str): Array of bytes to emulate
+        targets(list): List of target addresses
+
+    Returns:
+        :class:`dict`: Map of ``{address: target}`` for each address which
+            reaches one of the selected targets.
+    """
+    rv = {}
+
+    # Unicorn doesn't support big-endian for everything yet.
+    if elf.endian == 'big' and elf.arch == 'mips':
+        data = packing.unpack_many(data, bits=32, endian='little')
+        data = packing.flat(data, bits=32, endian='big')
+
+    # Brute force addresses, assume that PLT entry points are at 4-byte aligned
+    # Do not emulate more than a handful of instructions.
+    for i, pc in enumerate(range(address, address + len(data), 4)):
+        if log.isEnabledFor(logging.DEBUG):
+            log.debug('%s %#x', fiddling.enhex(data[i*4:(i+1) * 4]), pc)
+            log.debug(elf.disasm(pc, 4))
+        target = emulate_plt_instructions_inner(elf, got, pc, data[i*4:], targets)
+
+        if target in targets:
+            log.debug("%#x -> %#x", pc, target)
+            rv[pc] = target
+
+    return rv
+
+def emulate_plt_instructions_inner(elf, got, pc, data, targets):
+    # Deferred import to not affect load time
+    import unicorn as U
+
+    # Instantiate the emulator with the correct arguments for the current
+    # architecutre.
+    arch = {
+        'aarch64': U.UC_ARCH_ARM64,
+        'amd64': U.UC_ARCH_X86,
+        'arm': U.UC_ARCH_ARM,
+        'i386': U.UC_ARCH_X86,
+        'mips': U.UC_ARCH_MIPS,
+        # 'powerpc': U.UC_ARCH_PPC, <-- Not actually supported
+        'thumb': U.UC_ARCH_ARM,
+    }.get(elf.arch, None)
+
+    if arch is None:
+        log.warn("Could not emulate PLT instructions for %r" % elf)
+        return {}
+
+    emulation_bits = elf.bits
+
+    # x32 uses 64-bit instructions, just restricts itself to a 32-bit
+    # address space.
+    if elf.arch == 'amd64' and elf.bits == 32:
+        emulation_bits = 64
+
+    mode = {
+        32: U.UC_MODE_32,
+        64: U.UC_MODE_64
+    }.get(emulation_bits)
+
+    if elf.arch in ('arm', 'aarch64'):
+        mode = U.UC_MODE_ARM
+
+    uc = U.Uc(arch, mode)
+
+    # Map the page of memory, and fill it with the contents
+    start = pc & (~0xfff)
+    stop  = (pc + len(data) + 0xfff) & (~0xfff)
+
+    if not (0 <= start <= stop <= (1 << elf.bits)):
+        return None
+
+    uc.mem_map(start, stop-start)
+    uc.mem_write(pc, data)
+    assert uc.mem_read(pc, len(data)) == data
+
+    # MIPS is unique in that it relies entirely on _DYNAMIC, at the beginning
+    # of the GOT.  Each PLT stub loads an address stored here.
+    # Because of this, we have to support loading memory from this location.
+    #
+    # https://www.cr0.org/paper/mips.elf.external.resolution.txt
+    magic_addr = 0xdbdbdbdb
+
+    if elf.arch == 'mips':
+        # Map the GOT so that MIPS can access it
+        p_magic = packing.p32(magic_addr)
+        start = got & (~0xfff)
+        try:
+            uc.mem_map(start, start+0x1000)
+        except Exception:
+            # Ignore double-mapping
+            pass
+
+        uc.mem_write(got, p_magic)
+
+        # Separately, Unicorn is apparently unable to hook unmapped memory
+        # accesses on MIPS.  So we also have to map the page that contains
+        # the magic address.
+        start = magic_addr & (~0xfff)
+        try:
+            uc.mem_map(start, start+0x1000)
+        except Exception:
+            # Ignore double-mapping
+            pass
+        trap = packing.p32(0x34000000, endian=elf.endian)
+        uc.mem_write(magic_addr, trap)
+
+    # Hook invalid addresses and any accesses out of the specified address range
+    stopped_addr = []
+
+    def hook_mem(uc, access, address, size, value, user_data):
+        # Special case to allow MIPS to dereference the _DYNAMIC pointer
+        # in the GOT.
+        if elf.arch == 'mips' and address == got:
+            return True
+
+        user_data.append(address)
+        uc.emu_stop()
+        return False
+
+    uc.hook_add(U.UC_HOOK_MEM_READ, hook_mem, stopped_addr)
+    uc.hook_add(U.UC_HOOK_MEM_UNMAPPED, hook_mem, stopped_addr)
+
+    # callback for tracing instructions
+    # def hook_code(uc, address, size, user_data):
+    #     print(">>> Tracing instruction at 0x%x, instruction size = 0x%x, data=%r" %(address, size, uc.mem_read(address, size)))
+    # uc.hook_add(U.UC_HOOK_CODE, hook_code)
+
+    # For Intel, set the value of EBX
+    if elf.arch == 'i386':
+        uc.reg_write(U.x86_const.UC_X86_REG_EBX, got)
+
+    # Special case for MIPS, which is the most silly architecture
+    # https://sourceware.org/ml/binutils/2004-11/msg00116.html
+    if elf.arch == 'mips' and elf.bits == 32:
+        OFFSET_GP_GOT = 0x7ff0
+        uc.reg_write(U.mips_const.UC_MIPS_REG_GP, got + 0x7ff0)
+
+    try:
+        uc.emu_start(pc, until=-1, count=5)
+    except U.UcError as error:
+        UC_ERR = (k for k,v in \
+                    U.unicorn_const.__dict__.items()
+                    if error.errno == v and k.startswith('UC_ERR_')).next()
+        log.debug("%#x: %s (%s)", pc, error, UC_ERR)
+
+    if elf.arch == 'mips':
+        pc = uc.reg_read(U.mips_const.UC_MIPS_REG_PC)
+        if pc+1 == magic_addr:
+            t8 = uc.reg_read(U.mips_const.UC_MIPS_REG_T8)
+            stopped_addr.append(elf._mips_got.get(t8, 0))
+
+    retval = 0
+    if stopped_addr:
+        retval = stopped_addr.pop()
+
+    return retval

pwnlib/gdb.py

@@ -524,7 +524,7 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
         .. code-block:: python
 
             # Connect to the SSH server
-            shell = ssh('bandit0', 'bandit.labs.overthewire.org', password='bandit0')
+            shell = ssh('bandit0', 'bandit.labs.overthewire.org', password='bandit0', port=2220)
 
             # Start a process on the server
             cat = shell.process(['cat'])
@@ -596,7 +596,7 @@ def attach(target, gdbscript = None, exe = None, need_ptrace_scope = True, gdb_a
             cmd = ['sshpass', '-p', shell.password] + cmd
         if shell.keyfile:
             cmd += ['-i', shell.keyfile]
-        cmd += ['gdb %r %s -x "%s"' % (target.executable,
+        cmd += ['gdb -q %r %s -x "%s"' % (target.executable,
                                        target.pid,
                                        tmpfile)]
 
@@ -763,7 +763,7 @@ def find_module_addresses(binary, ssh=None, ulimit=False):
     Example:
 
     >>> with context.local(log_level=9999): # doctest: +SKIP
-    ...     shell = ssh(host='bandit.labs.overthewire.org',user='bandit0',password='bandit0')
+    ...     shell = ssh(host='bandit.labs.overthewire.org',user='bandit0',password='bandit0', port=2220)
     ...     bash_libs = gdb.find_module_addresses('/bin/bash', shell)
     >>> os.path.basename(bash_libs[0].path) # doctest: +SKIP
     'libc.so.6'
@@ -790,7 +790,7 @@ def find_module_addresses(binary, ssh=None, ulimit=False):
     # Get the addresses from GDB
     #
     libs = {}
-    cmd  = "gdb --args %s" % (binary)
+    cmd  = "gdb -q --args %s" % (binary)
     expr = re.compile(r'(0x\S+)[^/]+(.*)')
 
     if ulimit:

pwnlib/log.py

@@ -103,6 +103,7 @@ import threading
 import time
 
 from pwnlib import term
+from pwnlib.config import register_config
 from pwnlib.context import Thread
 from pwnlib.context import context
 from pwnlib.exception import PwnlibException
@@ -132,26 +133,26 @@ _msgtype_prefixes = {
     }
 
 
-# permit setting logging colors from a configuration file
-config = os.path.expanduser('~/.pwn.conf')
-if os.path.exists(config):
-    c = ConfigParser.ConfigParser()
-    c.read([config])
-
-    for section in c.sections():
-        if section not in _msgtype_prefixes:
+def read_log_config(settings):
+    log = getLogger(__name__)
+    for key, value in settings.items():
+        if '.' not in key:
+            log.warn("Invalid configuration option %r in section %r" % (key, 'log'))
             continue
 
-        for key, value in c.items(section):
-            if key == 'color':
-                try:
-                    _msgtype_prefixes[section][0] = getattr(text, value)
-                except AttributeError:
-                    pass
+        msgtype, key = key.split('.', 1)
 
-            elif key == 'symbol':
-                _msgtype_prefixes[section][1] = value
+        if key == 'color':
+            current = _msgtype_prefixes[msgtype][0]
+            _msgtype_prefixes[msgtype][0] = getattr(text, value, current)
 
+        elif key == 'symbol':
+            _msgtype_prefixes[msgtype][1] = value
+
+        else:
+            log.warn("Unknown configuration option %r in section %r" % (key, 'log'))
+
+register_config('log', read_log_config)
 
 # the text decoration to use for spinners.  the spinners themselves can be found
 # in the `pwnlib.term.spinners` module

pwnlib/memleak.py

@@ -11,6 +11,8 @@ from pwnlib.util.packing import unpack
 
 log = getLogger(__name__)
 
+__all__ = ['MemLeak', 'RelativeMemLeak']
+
 class MemLeak(object):
     """MemLeak is a caching and heuristic tool for exploiting memory leaks.
 

pwnlib/rop/rop.py

@@ -475,6 +475,16 @@ class ROP(object):
     def generatePadding(self, offset, count):
         """
         Generates padding to be inserted into the ROP stack.
+
+        >>> rop = ROP([])
+        >>> val = rop.generatePadding(5,15)
+        >>> cyclic_find(val[:4])
+        5
+        >>> len(val)
+        15
+        >>> rop.generatePadding(0,0)
+        ''
+
         """
         if count:
             return cyclic.cyclic(offset + count)[-count:]
@@ -786,6 +796,18 @@ class ROP(object):
 
         Arguments:
             data(int/str): The raw value to put onto the rop chain.
+
+        >>> rop = ROP([])
+        >>> rop.raw('AAAAAAAA')
+        >>> rop.raw('BBBBBBBB')
+        >>> rop.raw('CCCCCCCC')
+        >>> print rop.dump()
+        0x0000:           'AAAA' 'AAAAAAAA'
+        0x0004:           'AAAA'
+        0x0008:           'BBBB' 'BBBBBBBB'
+        0x000c:           'BBBB'
+        0x0010:           'CCCC' 'CCCCCCCC'
+        0x0014:           'CCCC'
         """
         if self.migrated:
             log.error('Cannot append to a migrated chain')

pwnlib/rop/srop.py

@@ -15,7 +15,7 @@ i386 Example:
 
     Let's just print a message out using SROP.
 
-    >>> message = "Hello, World"
+    >>> message = "Hello, World\\n"
 
     First, we'll create our example binary.
     It just reads some data onto the stack, and invokes
@@ -47,11 +47,10 @@ i386 Example:
 
     >>> p = process(binary.path)
     >>> p.send(str(frame))
-    >>> p.recvn(len(message)) == message
-    True
-    >>> p.wait_for_close()
-    >>> p.poll() == 0
-    True
+    >>> p.recvline()
+    'Hello, World\n'
+    >>> p.poll(block=True)
+    0
 
 amd64 Example:
 
@@ -73,11 +72,10 @@ amd64 Example:
     >>> frame.rip = binary.symbols['syscall']
     >>> p = process(binary.path)
     >>> p.send(str(frame))
-    >>> p.recvn(len(message)) == message
-    True
-    >>> p.wait_for_close()
-    >>> p.poll() == 0
-    True
+    >>> p.recvline()
+    'Hello, World\n'
+    >>> p.poll(block=True)
+    0
 
 arm Example:
 
@@ -99,11 +97,11 @@ arm Example:
     >>> frame.pc = binary.symbols['syscall']
     >>> p = process(binary.path)
     >>> p.send(str(frame))
-    >>> p.recvn(len(message)) == message
-    True
+    >>> p.recvline()
+    'Hello, World\n'
     >>> p.wait_for_close()
-    >>> p.poll() == 0
-    True
+    >>> p.poll(block=True)
+    0
 
 Mips Example:
 
@@ -125,11 +123,10 @@ Mips Example:
     >>> frame.pc = binary.symbols['syscall']
     >>> p = process(binary.path)
     >>> p.send(str(frame))
-    >>> p.recvn(len(message)) == message
-    True
-    >>> p.wait_for_close()
-    >>> p.poll() == 0
-    True
+    >>> p.recvline()
+    'Hello, World\n'
+    >>> p.poll(block=True)
+    0
 
 Mipsel Example:
 
@@ -151,11 +148,10 @@ Mipsel Example:
     >>> frame.pc = binary.symbols['syscall']
     >>> p = process(binary.path)
     >>> p.send(str(frame))
-    >>> p.recvn(len(message)) == message
-    True
-    >>> p.wait_for_close()
-    >>> p.poll() == 0
-    True
+    >>> p.recvline()
+    'Hello, World\n'
+    >>> p.poll(block=True)
+    0
 
 """
 from __future__ import absolute_import

pwnlib/shellcraft/templates/aarch64/breakpoint.asm

@@ -0,0 +1,10 @@
+<% import pwnlib.shellcraft as S %>
+<%docstring>
+Inserts a debugger breakpoint (raises SIGTRAP).
+
+Example:
+
+    >>> run_assembly(shellcraft.breakpoint()).poll(True)
+    -5
+</%docstring>
+    brk #0

pwnlib/shellcraft/templates/aarch64/crash.asm

@@ -0,0 +1,11 @@
+<% import pwnlib.shellcraft as S %>
+<%docstring>
+Crashes the process.
+
+Example:
+
+    >>> run_assembly(shellcraft.crash()).poll(True)
+    -11
+</%docstring>
+    ${S.mov('x30', 0)}
+    ret

pwnlib/shellcraft/templates/aarch64/infloop.asm

@@ -1,5 +1,15 @@
 <% from pwnlib.shellcraft import common %>
-<%docstring>An infinite loop.</%docstring>
+<%docstring>
+An infinite loop.
+
+Example:
+
+    >>> io = run_assembly(shellcraft.infloop())
+    >>> io.recvall(timeout=1)
+    ''
+    >>> io.close()
+
+</%docstring>
 <% infloop = common.label("infloop") %>
 ${infloop}:
     b ${infloop}