* Fix corefile environment variables and speed up ELF.string()
* Even more doctests
* Fix doctest arch
* No longer split on = since it isn't necessary
* Fix arch again
* Delete intermediate corefiles
* Do not flag for extended testing when the commit SHA is what matches
* Add support for finding QEMU's LD_PREFIX
* Refactor pwnlib.qemu and add official documentation
* Add more documentation, support for sysroot
* More docs, binfmt info
* Fix variable name overload
* Fix missed refactor
* Check for empty/null env
* Add qemu.rst to index
* Add cyclic default args to context
Specifically, this adds context.cyclic_alphabet and context.cyclic_size
This helps cut down on writing "n=8" all the time for users who use 8-byte
subsequences on 64-bit architectures.
Exposing the default alphabet means that users can avoid "bad bytes" in
cyclic patterns which are generated by pwntools internally (e.g. ROP padding)
Some pwnlib.util.packing docstrings were changed to clarify that word_size is
in bits.
The doctests in pwnlib.util.cyclic were updated and extended to have more
and better doctests, as well as better exaples.
* Add an example showing an alphabet with NUL bytes
This is important because old code would use pack(..., "all") when an integer
value was provided, instead of using the correct width.
* Overhaul ROP.setRegisters to support more complex cases needed by AMD64
Additionally, have a separate cache for ROPs composed of multiple ELF files
* Better presentation for stack padding slots
* Remove unnecessary stack adjustment on amd64
For amd64, it is possible that len(stackArguments) == 0.
* Add a doctest for amd64 register dependency stuff
* More i386 adjust optimizations
* Use nextGadgetAddr as the adjust value
* Always use padding for the last "ret" adjustment
* Better dump display
* Update doctests for new ROP dump() enhancements
* Add badchars support for ROP gadgets
* More doctest fixes, also ensure ROP.search does not return gadgets ending in e.g. int 0x80
* Prevent Sphinx from dumping all level=INFO logging output to stdout.
See Gallopsled/pwntools#1036 for more information.
* Filter out Paramiko logs too, apply filter to all root handlers during testing
* Remove unused import
(cherry picked from commit 54c47b3daa)
Technically this introduces issues with e.g. 'unhex ffdf | python foo.py' if
foo.py uses .interactive(). However, that was already broken for other reasons.
Fixes#1004
(cherry picked from commit ebbe917713)
* Slightly enhance template and advertise for the feature itself in scripts
Add single-dash to the "very good" characters in shell escaping, to avoid
the need to single-quote things like --host and --port.
* Add --quiet option for pwn template, move gdbscript down a bit
* Slightly better docs for 'pwn template'
* Add checksec information directly to 'pwn template' output
* Prevent Sphinx from dumping all level=INFO logging output to stdout.
See Gallopsled/pwntools#1036 for more information.
* Filter out Paramiko logs too, apply filter to all root handlers during testing
* Remove unused import
* DynELF: trust ELF base address
* Bug fixes for DynELF:
* Fixed lookup when not using libcdb
* Fixed _find_linkmap returning 0 instead of None when linkmap doesn't
exist
* Fixed _lookup_build_id crashing if linkmap cannot be found
* Misc cleanup
* Changes based on feedback
Technically this introduces issues with e.g. 'unhex ffdf | python foo.py' if
foo.py uses .interactive(). However, that was already broken for other reasons.
Fixes#1004
* Force use of Ubuntu 12 builds on Travis CI until we can migrate to 14
* Fix Travis Dockerfile
* Add bash command logging to Travis scripts, since Travis removed its own logging
* curl should follow redirects when grabbing Ubuntu packages
* Update Travis cache directories to newer version of Python, as seen in Travis logs
* Use ::1 instead of ip6-localhost since Travis 12.04 changes broke it
* fix typo in comment
* new config option: context.gdbinit
This allows setting the default gdbinit for GDB, which is helpful when you want
to enable some GDB extension only when using pwntools and not by default.
* Fix Full RELRO detection in ELFs.
This add supports for the newer DF_BIND_NOW and DF_1_NOW flags.
These are used instead of DT_BIND_NOW when linking with gold.
Also fixed the order of checks, now it reports "No RELRO" when
BIND_NOW is present but GNU_RELRO is not (in which case the bindings
are done at start but they will not be marked as read-only).
* Add x86 test binaries for relro check.
The check for when certain ELF fields contain offsets instead of pointers can be
made more precise by checking for the type field in the ELF header instead of
comparing the value to some hard-coded range.
* Clean up socket-class and socket-type code in sock/remote/listen
* Slight change to docs
* Add doctests for listen and attribute documentation
* Fix bug in listen.close()
* Fix regression on ipv6 sockets
* Simplify documentation
* Fix doctest
* Do not use UC_MODE_BIG_ENDIAN, its support is poor
* Ghetto-swap endianness for emulation of PLT instructions
* Reduce instruction alignment so that all ARM PLT entries are found
* Add tests for ELF PLT loading
* Fix canary detection on PowerPC
* Fix x32 emulation
* Fix two failing doctests
* Add AArch64 to supported big-endian architectures
* Add MIPS GOT parsing code, fix unpack macros to use ELF settings
* Why, MIPS, are you so complicated?
* Catch weird edge cases
* I don't know what's real anymore
* Fix synthetic symbol generation and add tests
* Remove canary test
* Guard against errors in MIPS GOT
* Fix doctests
* Do not fail with gdb.debug() over ssh when given an absolute path
FixesGallopsled/pwntools#885
* Prevent failing on stable since this directory doesn't exist
* Doctest all members
* Fix doctest failures in asm and remove canary in context
* Fix failing doctest
* Use automodule for pwnlib.context
* Fix failing tests in context module
* Use automodule for pwnlib.memleak
* Fix checksec nx, execstack, relro reporting
Also add a disable_nx function
* Add disablenx command-line utility
Also fix bug where invoking command-line utilities directly by module name
(python -m pwnlib.commandline.foo) would fail
This adds functionality for user configuration files at ~/.pwn.conf
and /etc/pwn.conf. Previously this was only used by the pwnlib.log
module, and was entirely undocumented.
This is now documented, and offers an easy mechanism for other parts
of the code to have extension points.
* Remove unavailable syscalls from AArch64
The Linux kernel has apparently stopped supporting these
* Rewrite shellcraft.aarch64.linux.cat, add open which uses openat
FixesGallopsled/pwntools#896
* Fix doctests to use openat
* Add templates from generator
* Add abi.stack to all ABIs
* Enable shellcraft.arch.os.foobar.template
* Add pwnlib.registers.current()
* Add syscalls symlinks
* Add syscall generator
* Enable shellcraft.foo() shortcut for shellcraft.arch.os.syscalls.foo()
* Make templates more verbose
* Remove old templates
* Add makefile
* Delete old templates, make shellcraft --list work
This also speeds up boot time by like 0.05 seconds
* Remove unnecessary execve wrappers
* Check for common arch as well
* Don't explode on string arguments that are register names
* Catch bad argument names
* Remove accidental import
* Add aarch64 ABIs since we enabled aarch64 tests
* Fix mmap vs mmap2 syscall
* Make shellcraft command-line work for syscalls
* Remove unnecessary wrapper templates
* Set all default values to zero
* Add example of automatic determination of types
* Add missing register architectures
* Add PowerPC ABIs
* More syscall examples
* Handle mmap vs mmap2
* Add missing import, use full path
* Just use the string name
* Fix rt_sigreturn on amd64
* __NR_ --> SYS_ because of old expectations
* rstrip to remove blankline
* Fix Thumb doctests which uses unpredictable jump labels
* Import additional syscalls by updating the function source.
Support variable-argument syscalls.
Support spilling to the stack.
* Simplified constants.eval logic means no extra parens
* Simplify vararg representation; fix oepn() argument name explicitly
* Fix syscall examples
* Fix reordering of arguments, since we have to support stack spill
* Fix argument name in doctest
* Make Codacy happy
* Restore connect template accidentally removed in 9fceb12b
* Scrape all architectures for syscall names
* Enable void* and void** to be pushed
* Make Codacy happy
* Add __doc__ to sdist
* Follow shellcraft template symlinks
* Detect syscall shellcodes in shellcraft based on path
(cherry picked from commit 94462845fa)
Adds a large amount of functionality around Corefiles, to support smarter exploitation
Adds a ".corefile" property to both local and remote (ssh) processes. See the
documentation for the corefile module for extensive examples and tests.
Additionally, context.delete_corefiles and context.rename_corefiles have been added,
to control the behavior of the ".corefile" property.
* Add aarch64 documentation, which enables doctests
* Fix failing doctest
* Attempt to figure out whats going on with Travis
* Use the Xenial version of qemu-static since the Zesty version is broken for AArch64
- Show the PID when a process is created
- Make process.cwd into a property, which fetches the real CWD from procs
- Record the UID/GID and SUID/SGID of the executed file
- Add a few doctests
- Make _stop_noticed a timestamp
- Break out ssh_channel into subclass ssh_process
- Make ssh.process() directly instantiate ssh_process
- Add ssh_process().ELF
- Add ssh_process().corefile
- Record the UID and GID of the executed binary
- Add the created PID to the ssh_process message
* Fix chunked reading of ELF files
This introduces a new dependency on 'intervaltree', which is used
to automatically select segments which are relevant for a given
virtual-address range.
This also introduces a new ELF.memory attribute, which is the tree
of memory ranges, which point at the segment objects that describe
the memory range.
These are recalculated when ELF.address is adjusted, and we have
tests for that.
This also optimizes some of the other routines to avoid the wrapped
file-seeking, and to just use the mmap.
* Rewrite install scripts to use more modern versions of binutils
* Load segments in file-order, chop any overlaps
* Add pwnlib.protocols module
Migrate pwnlib.adb.protocol into that directory, and add compatibility shims
* Add documentation links, and run doctests
* Make doctests run
* pwnlib.protocols.adb.Client.transport should work with no device
We cannot determine the length of the file if passed e.g. a tube or a pipe or PTY or character device (e.g. stdin or /dev/null).
Supercedes: 9c2f0f4e
(cherry picked from commit 14f717b7b4)
This code wasnt updated when the code was changed to operate on file descriptors.
Additionally, there was a functional regression, that the first line would be skipped.
Fixes: Gallopsled/pwntools#836
Caused by: Gallopsled/pwntools#695 (d65991d)
Looks like this:
[x] Connecting to example.pwnme on port 22
[+] Connecting to example.pwnme on port 22: Done
[*] [email protected]:
Distro Ubuntu 14.04
OS: linux
Arch: amd64
Version: 3.11.0
ASLR: Enabled
Note: Susceptible to ASLR ulimit trick (CVE-2016-3672)
FixesGallopsled/pwntools#753
* Defer attempts to start ADB server until we actually fail to connect
* Upgrade to a warning message; fix .version() multiple calls with _autoclose
* Add doctest which tests the kill/restart behavior
* Add PR_SET_PTRACER to initialization, and add doctests for process.leak/corefile
* Add PR_SET_PTRACER to SSH processes
* Do not warn about YAMA if we dont need to
* Fix doctest to use raw strings
* Fix typo in doctest
* Apparently GDB is not installed on Travis by default
* Add warning message to process("foo") vs process("./foo")
* Add gdb.corefile; make run_in_new_terminal return a PID
* Intentionally break doctest because I dont believe that it worked
* Fix codacy issues
* Fix gdb.attach return value
* Add missing import
* Fix typo, use waitpid
* Refactor some code into gdb.version / gdb.binary
* Permit GDB with multipart versions
* Fix repr() on Corefile
* Silence run-in-new-terminal message for gdb.corefile
* Fix version number
* Remove Yama warning message entirely, it should no longer be necessary
* Version detection is hard. Regular expressions to the rescue!
* Remove process.kill/stop/resume, those will be a different PR
* Fix doctests
* The GDB used by Travis must be so old!
* Apparently Travis is entirely borked. Use a less robust test.
* Enable leaking memory relatively with MemLeak
FixesGallopsled/pwntools#758
* Add RelativeMemLeak alias for MemLeak(..., relative=True)
* Make codacy happy
If pandoc is not installed locally, pwntools' setup.py
tells pypandoc to download one.
Unfortunately, it fails on non-amd64 arches because
pypandocs provides precompiled binaries only for amd64.
Since pypandoc is not hard dependency for pwntools,
this error can be skipped without major harm.
In fact, pwntools' setup.py already handles ImportError
gracefully. The problem is, arch mismatch causes
a completely different exception.
* Remove support for installation without psutil.
OpenBSD was the edge case we cared about, it is supported for over a year (giampaolo/psutil#615)
* Bump psutil version to when they introduced OpenBSD support
* Fix ADB wait_for_device with ANDROID_SERIAL set, and fix performance
* Fix QEMU usage for native binaries
* Fix adb.write and adb.listdir operating on the wrong type of object
* Fix failing unit test with new behavior in adb.protocol.Client.list
* Fix adb server startup issue
* Rework deferred initialization to work around class-scope superclass attributes, and add test for AdbDevice
* Add adb.{unlink,mkdir,makedirs,isdir,exists}
* Change unlink() to throw an exception rather than return a string
* Fix doctest and code for isdir
* Remove unused variable
* Fix adb.partitions listing
* Add adb.walk and adb.find functions to ensure adb.partitions works on devices without a find binary
* Add adb.readlink to simplify logic for reading symlinks
Support custom buffer-read sizes for `tube` objects via their internal `Buffer` object. Default buffer size is 4096. Buffer size can be set per-object at instantiation-time (`tube(buffer_fill_size=...)`), by attribute (`tube.buffer_fill_size=...`), or in a context (`context.buffer_size=...`).
Previously, if getaddrinfo or socket.socket threw an exception,
_connect would return without ever calling h.success or h.failure. As a
result, progress indicators would continue to update indefinitely.
Fix this by using a with statement instead.
(Also remove an unused variable and fix a typo.)
* adding output format for python hex-notation
* cleaned up implementation
* changed format identifier to {a} --escaped
* {a] was taken by asm, changed format identifier to {d} --escaped
* adding {e}lf to help
* Extract copy-pasted callback routine, add better errors and doctests for adb.{push,pull}
* Add protocol submodule to documentation and doctests
* Add and fix doctests in adb.protocol
* Use pprint to sort dicts
* Re-implement sh_string to use embedded literals, add lots of tests
* Add docs for sh_string so that unit tests get picked up
* Mark the docstrings as raw
* Rewrite doctests to not "print" since this breaks with invisible characters (issue with doctest itself)
* Add test which uses 16KB of random non-NUL data
An error message will be printed like this when a package is found:
>>> context.arch='powerpc'
>>> pwnlib.asm.print_binutils_instructions('objdump', context)
[ERROR] Could not find 'objdump' installed for ContextType(arch = 'powerpc', bits = 32, endian = 'big')
Try installing binutils for this architecture:
$ sudo apt-get install binutils-powerpc-linux-gnu
* Add Android emulator to Travis
* More robust handling of ADB device status, and lack of devices
* Add platform-tools to PATH for adb, fastboot, etc.
* Add basic tests for some ADB code
* Add with_device annotation to auto-detect a device
* Skip Android emulator installation if we dont need it
* Do not install emulator in all cases (e.g. OSX)
Adds an install-time option to disable legacy commands, and only use 'pwn':
$ pip install --install-option='--only-use-pwn-command'
Documentation is updated to show 'pwn' entry point and sub-commands.
Fixes#404Fixes#660
Adds an install-time option to disable legacy commands, and only use 'pwn':
$ pip install --install-option='--only-use-pwn-command'
Documentation is updated to show 'pwn' entry point and sub-commands.
Fixes#404Fixes#660
* Add MIPS documentation, which should also trigger doctests
* Fix broken and missing doctests for MIPS pushstr / syscall
* Add optimization for mips.mov to fix doctest
* Exit cleanly so QEMU doesn't complain
In two instances, the file was not removed because of an incorrect lambda.
In two further instances, use of shell concatenation via run_in_new_terminal
were changed into GDB 'shell' commands.
This fixesGallopsled/pwntools#629
The clear is necessary because vars(context) will only retrieve the set keys. Any other context bits set later will not be correctly set to default values.
This bug happened because we no longer disable input processing in the terminal.
In particular `\r` is now translated to `\n`. The fix is to interpret both `\r`
and `\n` as <enter> in `term/key.py`.
(cherry picked from commit 383f245082)
When installing pwntools, setup.py required a very new version of
requests -- even though we never used any new features.
This is a problem, since the required version is much never than what is
in e.g. ubuntu's package repo. While this by itself is not much of a
problem, it is a problem if one installs python-pip from the distro's
repo as well, since python-pip will try to use the newest version of the
requests library, which caused pip to break.
The solution: Do not force such a new version of python, instead live
with a much older version.
(cherry picked from commit f91299986e)
Code says context.bits should be > 0, but error message says >= 0.
Since 0 bit architecture doesn't make sense at all, make them all be > 0.
(cherry picked from commit def9400d48)
Changes:
- `pwn/init` initialization code simplified, and some was moved to `pwnlib/args`
- `scramble` script added to `pwnlib.commandline`; support for encoders added to relevant scripts
- Added shellcode encoders for several architectures
- Added CGC architecture shellcode and constants
- Added Shellcraft templates for every syscall
- Added `xor_key` helper for generating a 4-byte XOR key for a data stream
- Added `getdents` for use with the relevant shellcraft scripts
- Add ELF loaders for all architectures
- Lots of MIPS and AArch64 shellcode
Add information about various ARM coprocessors that might be present.
Correct typo
Add in comments
Change how magic numbers and offsets are stored
Use namedtuple
Renaming variables
srop.py doc modified
ARM sp check added in
Cleanup
- rop.py does not import _registers_* anymore; instead imports get_registers from srop.py
- rop.py now has a self.arch that is set to context.arch inside __init__
- (_registers_32, _registers_64) => (_registers_i386, _registers_amd64)
- get_registers and get_regpos_mapping added in
- _initialize_values now handles all archs
- get_stackpointer_index changed to get_spindex; made cleaner
- get_frame made cleaner
So now we can populate the accumulator register with the sigreturn syscall number
rop.py now finds "int 0x80" instructions.
Adding in more to srop skeleton; integrating with finding int 0x80 instruction
The following changes were made :-
- s/find/index correction for list
- required imports added in
- None as argument removed
Now generates srop frames whenever function cannot be resolved
Add in code that pivots only if the gadget is not a "pop"
dump is now modified to print out the register information from srop
Uses constants.linux.i386 for syscall numbers now
Added in some exception handling code
Rely on constants to get us the right syscall number based on context.arch
The following changes were made :-
- getattr when getting syscall_number now returns None by default
- ropgadget now looks for syscall and sysenter
- added in getattr("syscall") and getattr("sysenter")
rop.search now returns a namedtuple
The following changes were made :-
1. self.int80, self.syscall and self.sysenter now return namedtuples.
2. We have functions self.raw_address and self.raw_data to specify
how the raw values that are added in, need to be interpreted so that
the chain can be built accordingly.
Added in a function _get_syscall_inst that returns the address of a syscall instruction
Checking if travis builds it properly/
The following changes were made :-
- Change the body of _get_syscall_inst, make it cleaner
- Add in a self._syscall_instruction that caches the address of
a syscall instruction that was found
Removed raw_address and raw_data; modified srop implementation to use raw()
Cleanup getattr
Now the stack pointer is modified so that we can continue constructing the ROP chain if rop.base is specified
No need of enumerate; removed
doctest for rop-srop integration added in
Dropping the doctest ifmain statements
Correct underline in rst
Add testsetup to srop.rst
Now uses packing.pack instead of struct.pack
Added in context.arch information to srop.rst and srop.py doctest
Added "context.clear()" to the doctest
Cleaned up _reg_pos_mapping up a bit
Corrected reference link
Replace "print repr()" with "unpack_many"
Added in import to srop.rst
In this commit, I only fix 'i386/linux/acceptloop_ipv4.asm' and
'i386/linux/mprotect_all.asm'.
BUG=None
TEST=confirmed the behavior is still the same under
gdb and strace.
When reporting an issue, be sure that you are running the latest released version of pwntools (`pip install --upgrade pwntools`).
Please verify that your issue occurs on 64-bit Ubuntu 14.04. You can use the Dockerfile on `docker.io` for quick testing.
```
$ docker pull pwntools/pwntools:stable
$ docker run -it pwntools/pwntools:stable
```
If possible, provide a proof-of-concept which demonstrates the problem. Include any binaries or scripts necessary to reproduce the issue, and please include the full debug output via setting the environment variable `PWNLIB_DEBUG=1`.
Thanks for contributing to Pwntools! Take a moment to look at [`CONTRIBUTING.md`][contributing] to make sure you're familiar with Pwntools development.
Please provide a high-level explanation of what this pull request is for.
## Testing
Pull Requests that introduce new code should try to add doctests for that code. See [`TESTING.md`][testing] for more information.
## Target Branch
Depending on what the PR is for, it needs to target a different branch.
You can always [change the branch][change] after you create the PR if it's against the wrong branch.
| [3.13.0](#3130) | `dev` | Mar 31, 2018 (planned)
| [3.12.0](#3120) | `beta` | Feb 17, 2018 (planned)
| [3.11.0](#3110) | `stable` | Jan 3, 2018
| [3.10.0](#3100) | | Oct 25, 2017
| [3.9.2](#392) | | Oct 5, 2017
| [3.9.1](#391) | | Sep 28, 2017
| [3.9.0](#390) | | Sep 11, 2017
| [3.8.0](#380) | | Jul 29, 2017
| [3.7.1](#371) | | Jul 14, 2017
| [3.7.0](#370) | | Jun 19, 2017
| [3.6.1](#361) | | May 12, 2017
| [3.6.0](#360) | | May 8, 2017
| [3.5.1](#351) | | Apr 15, 2017
| [3.5.0](#350) | | Mar 26, 2017
| [3.4.1](#341) | | Feb 17, 2017
| [3.4.0](#340) | | Feb 13, 2017
| [3.3.4](#334) | | Jan 12, 2016
| [3.3.3](#333) | | Jan 10, 2016
| [3.3.2](#332) | | Jan 10, 2016
| [3.3.1](#331) | | Jan 10, 2016
| [3.3.0](#330) | | Dec 24, 2016
| [3.2.1](#321) | | Dec 24, 2016
| [3.2.0](#320) | | Nov 12, 2016
| [3.1.1](#311) | | Oct 23, 2016
| [3.1.0](#310) | | Oct 2, 2016
| [3.0.4](#304) | | Sept 19, 2016
| [3.0.3](#303) | | Sept 18, 2016
| [3.0.2](#302) | | Sept 6, 2016
| [3.0.1](#301) | | Aug 20, 2016
| [3.0.0](#300) | | Aug 20, 2016
| [2.2.0](#220) | | Jan 5, 2015
## 3.12.0
To be released on Feb 17, 2018.
- [#1083][1083] Better error messages for `gdb` when `LD_PRELOAD` is incorrect
- [#1085][1085] Add support for extracting Android `BOOTLDR!` images
- [#1075][1075] Add support for detecting GNU Screen for `run_in_new_terminal`
- [#1074][1074] Add support for running `pwntools-gdb` wrapper script instead of `gdb`
- [#1067][1067] Add `pwnlib.tubes.server` module, which adds a reusable `server` listener
- [#1063][1063] Add support for labels in `fit()`, allowing dynamic contents to be injected. (This feature is really cool, check out the pull request!)
+ Much better support for 64-bit Intel (amd64) ROP
+ ROP gadget selection is optimized to favor multi-pops instead of multiple single-pop gadgets
+ Added support for blacklisting byte values in ROP gadget addresses
- [#1049][1049] Enhancements to `cyclic`
+ `context` now has two additional attributes, `cyclic_alphabet` and `cyclic_length`, which correspond to the arguments `alphabet` and `n` to `cyclic()` and `cyclic_find()` and related routines.
+ The motivation for this change is to allow setting the `alphabet` globally, so that any padding / patterns generated internally to pwntools can be controlled. The specific motivation is blacklisting values in ROP padding.
- [#1052][1052] Enhancements for detecting `QEMU_LD_PREFIX` used by QEMU user-mode emulation for sysroots
- [#1035][1035] Minor documentation changes
- [#1032][1032] Enhancements to `pwn template`
- [#1031][1031] More accurate `Coredump.fault_addr` on amd64
- [#1084][1084] Fix broken tests due to `ftp.debian.org` going down
- [300f8e0][300f8e0] Slightly speed up processing of large ELF files
- [#861][861] Adds support for extracting `IKCONFIG` configs from Linux kernel images, and extends `checksec` to report on any insecure configurations discovered
- [#871][871] Moves all of the basic syscall templates to `shellcraft/common` and exposes them via symlinks. Closed [#685][685]
+ Should not have any visible effects from any documented APIs
+ `shellcraft.arch.os.syscall_function()` still works the same
+ We now have the ability to differentiate between the `connect` syscall, and a TCP `connect` helper
- [#887][887] `sh_string` now returns a quoted empty string `''` rather than just an empty string
- [#839][839] Exposes a huge amount of functionality via corefiles which was not previously availble. See the [docs][corefile_docs] for examples.
+ `process().corefile` will automatically instantiate a Corefile for the process
+ QEMU-emulated processes are supported
+ Native processes are supported, including extraction of coredumps from `apport` crash logs
+ Native processes can be dumped *while running*, in a manner similar to `GDB`'s `gcore` script
- [#875][857] Added [documentation][aarch64] (and tests) for AArch64 shellcode
- [#882][882] The `ROP` class now respects `context.bytes` instead of using the hard-coded value of `4` (fixed [#879][879])
- [#869][869] Added several fields to the `process` class (`uid`, `gid`, `suid`, `sgid`) which are recorded at execution time, based on the file permissions
- [#868][868] Changed the way that `ssh.process()` works internally, and it now returns a more specialized class, `ssh_process`.
+ Added `ssh_process.corefile` for fetching remote corefiles
+ Added `ssh_process.ELF` for getting an ELF of the remote executable
+ The `uid`, `gid`, and `suid`, and `sgid` which are recorded at execution time, based on the file permissions
- [#865][865] Fixes `ELF.read` to support contiguous memory reads across non-contiguous file-backed segments
- [#862][862] Adds a `symlink=` argument to `ssh.set_working_directory`, which will automatically symlink all of the files in the "old" working directory into the "new" working directory
- [#695][695] Fixed a performance regression in `phd`.
- [452605e][452605e] Fixed [#629][629] related to correct removal of temporary files.
- [ea94ee4][ea94ee4] Disallows semi-colons in for the `run_in_terminal` function, since it did not work properly in all cases.
- [6376d07][6376d07] Added the mips shellcode `pushstr_array`.
- [#700][700] Added missing MIPS shellcode documentation to readthedocs, and enabled unit tests
- [#701][701] Command line tools refactored to have a common `pwn` entry point.
+ Added an option to *not* install the traditional `asm`, `disasm`, `checksec`, etc scripts
+ All existing tools can be accessed from the `pwn` command (e.g. `pwn asm nop`).
- [#704][704] The `process` object has a new, optional argument `alarm` for setting a `SIGALRM` timeout for processes.
- [#705][705] Added the Android Emulator to the test suite and Travis CI.
+ Android Emulator is now required for the full test suite
+ Android Emulator tests are skipped if no Android-related changes are detected
- [#711][711] `DynELF` has a new attribute, `heap`, which leaks the current `brk` address (heap base). This is useful for finding heap allocations with dlmalloc-derived allocators like those used by Glibc.
- [#717][717] `sh_string` was rewritten to emit more compact and compatible strings
+ This was achieved by embedding single-quoted non-printable literals
+ Much more testing was added
+ Emitted strings are no longer copy-paste compatible, but work fine with e.g. `tubes` module and the default `subprocess` module
- [#709][709] The `adb` module now directly talks to the `adb` server process via a new module, `adb.protocol`
+ Removes the need to shell out to `adb`
+ Avoids version-compatibility issues with `adb` server vs. client
- [#703][703] Added new methods to `adb`
+ `install` - Installs an APK
+ `uninstall` - Uninstalls a package
+ `packages` - Lists installed packages
- [4893819][4893819] Modified `shellcraft.sh` on all platforms to provide `argv[0]` and set `argc==1`
+ This is needed for systems which have Busybox or other minimal shell for `/bin/sh` which does not behave well with `argc==0` or `argv[0]==NULL`.
- [1e414af][1e414af] Added `connect()` alias for `remote()`
+ For example, `io=connect('google.com', 80)`
+ This also works with `tcp(...)` and `udp(...)` aliases
- [869ec42][869ec42] Added `ssh.read()` and `ssh.write()` aliases
- [2af55c9][2af55c9] `AdbDevice` objects exposed via e.g. `adb.devices()` now offer scoped access to all `adb` module properties
+ It is now possible to e.g. `map(lambda d: d.process(['id']).recvall(), adb.devices())`
A number of smaller bugfixes and documentation tweaks.
## 3.0.4
- Fixed a bug that made 3.0.3 uninstallable (Issue: #751, PR: #752)
## 3.0.3
- Fixed some performance and usability problems with the update system (Issues:
#723, #724, #736. PRs: #729, #738, #747).
- Fixed a bug related to internals in pyelftools (PRs: #730, #746).
- Fixed an issue with travis (Issue: #741, PRs: #743, #744, #745).
## 3.0.2
- Cherry-pick #695, as this was a regression-fix.
- Added a fix for the update checker, as it would suggest prereleases as updates to stable releases.
- Various documentation fixes.
## 3.0.1
A small bugfix release. There were a lot of references to the `master`-branch, however after 3.0.0 we use the names `stable`, `beta` and `dev` for our branches.
## 3.0.0
This was a large release (1305 commits since 2.2.0) with a lot of bugfixes and changes. The Binjitsu project, a fork of Pwntools, was merged back into Pwntools. As such, its features are now available here.
As always, the best source of information on specific features is the comprehensive docs at https://pwntools.readthedocs.org.
This list of changes is non-complete, but covers all of the significant changes which were appropriately documented.
#### Android
Android support via a new `adb` module, `context.device`, `context.adb_host`, and `context.adb_port`.
#### Assembly and Shellcode
- Assembly module enhancements for making ELF modules from assembly or pre-assembled shellcode. See `asm.make_elf` and `asm.make_elf_from_assembly`.
-`asm` and `shellcraft` command-line tools support flags for the new shellcode encoders
-`asm` and `shellcraft` command-line tools support `--debug` flag for automatically launching GDB on the result
- Added MIPS, PowerPC, and AArch64 support to the `shellcraft` module
- Added Cyber Grand Challenge (CGC) support to the `shellcraft` module
- Added syscall wrappers for every Linux syscall for all supported architectures to the `shellcraft` module
+ e.g. `shellcraft.<arch>.gettimeofday`
- (e.g. `shellcraft.i386.linux.`)
- Added in-memory ELF loaders for most supported architectures
+ Only supports statically-linked binaries
+ `shellcraft.<arch>.linux.loader`
#### Context Module
- Added `context.aslr` which controls ASLR on launched processes. This works with both `process()` and `ssh.process()`, and can be specified per-process with the `aslr=` keyword argument.
- Added `context.binary` which automatically sets all `context` variables from an ELF file.
- Added `context.device`, `context.adb`, `context.adb_port`, and `context.adb_host` for connecting to Android devices.
- Added `context.kernel` setting for SigReturn-Oriented-Programming (SROP).
- Added `context.log_file` setting for sending logs to a file. This can be set with the `LOG_FILE` magic command-line option.
- Added `context.noptrace` setting for disabling actions which require `ptrace` support. This is useful for turning all `gdb.debug` and `gdb.attach` options into no-ops, and can be set via the `NOPTRACE` magic command-line option.
- Added `context.proxy` which hooks all connections and sends them to a SOCKS4/SOCKS5. This can be set via the `PROXY` magic command-line option.
- Added `context.randomize` to control randomization of settings like XOR keys and register ordering (default off).
- Added `context.terminal` for setting how to launch commands in a new terminal.
#### DynELF and MemLeak Module
- Added a `DynELF().libc` property which attempt to find the remote libc and download the ELF from LibcDB.
- Added a `DynELF().stack` property which leaks the `__environ` pointer from libc, making it easy to leak stack addresses.
- Added `MemLeak.String` and `MemLeak.NoNewlines` and other related helpers for handling special leakers which cannot e.g. handle newlines in the leaked addresses and which leak a C string (e.g. auto-append a `'\x00'`).
- Enhancements for leaking speed via `MemLeak.compare` to avoid leaking an entire field if we can tell from a partial leak that it does not match what we are searching for.
#### Encoders Module
- Added a `pwnlib.encoders` module for assembled-shellcode encoders/decoders
- Includes position-independent basic XOR encoders
- Includes position-independent delta encoders
- Includes non-position-independent alphanumeric encoders for Intel
- Includes position-independent alphanumeric encoders for ARM/Thumb
#### ELF Module
- Added a `Core` object which can parse core-files, in order to extract / search for memory contents, and extract register states (e.g. `Core('./corefile').eax`).
#### Format Strings
- Added a basic `fmtstr` module for assisting with Format String exploitation
#### GDB Module
- Added support for debugging Android devices when `context.os=='android'`
- Added helpers for debugging shellcode snippets with `gdb.debug_assembly()` and `gdb.debug_shellcode()`
#### ROP Module
- Added support for SigReturn via `pwnlib.rop.srop`
+ Occurs automatically when syscalls are invoked and a function cannot be found
+ SigReturn frames can be constructed manually with `SigreturnFrame()` objects
- Added functional doctests for ROP and SROP
#### Tubes Process Module
-`process()` has many new options, check out the documentation
+ `aslr` controls ASLR
+ `setuid` can disable the effect of setuid, allowing core dumps (useful for extracting crash state via the new `Core()` object)
+ TTY echo and control characters can be enabled via `raw` argument
-`stdout` and `stderr` are now PTYs by default
+ `stdin` can be set to a PTY also via setting `stdin=process.PTY`
#### Tubes SSH Module
- Massive enhancements all over
-`ssh` objects now have a `ssh.process()` method which avoids the need to handle shell expansion via the old `ssh.run()` method
- Files are downloaded via SFTP if available
- New `download` and `upload` methods auto-detect whether the target is a file or directory and acts accordingly
- Added `listen()` method alias for `listen_remote()`
- Added `remote()` method alias for `connect_remote()`
#### Utilities
- Added `fit()` method to combine the functionality of `flat()` with the functionality of `cyclic()`
- Added `negative()` method to negate the value of an integer via two's complement, with respect to the current integer size (`context.bytes`).
- Added `xor_key()` method to generate an XOR key which avoids undesirable bytes over a given input.
- Added a multi-threaded `bruteforce()` implementation, `mbruteforce()`.
- Added `dealarm_shell()` helper to remove the effects of `alarm()` after you've popped a shell.
## 2.2.0
This was a large release with a lot of bugfixes and changes. Only the most significant
In general, we like to keep things documented. You should add documentation to any new functionality, and update it for any changed functionality. Our docstrings use the [Google Style Python Docstrings](https://sphinxcontrib-napoleon.readthedocs.org/en/latest/example_google.html#example-google).
@@ -21,18 +19,17 @@ Finally, it is probably a good idea to run the test suite locally before doing
the pull-request to make sure everything works, however this is not a
requirement.
Once you are ready to do a pull-request, you should figure out if your changes
constitutes a new feature or a bugfix in stable or beta. If it is a bugfix in
stable or beta, you should do the pull-request against the branch in question,
and otherwise your pull-request should be against the dev branch.
Once you do the pull-request Travis CI will run the test-suite on it. Once it
passes one of the core developers will look at your pull request, possibly
comment on it and then hopefully merge it into master.
comment on it and then hopefully merge it into the branch in question.
## Test Suite
## Automated Testing
To run the test suite, you should be running Ubuntu 12.04 or 14.04, and run the following commands. **Be aware** that this will add a user to your machine, and create a public key for SSH login!
Pull requests against Pwntools require at a minimum that no tests have been broken, and ideally each pull request will include new tests to ensure that all of the functionality works as intended.
```sh
bash .travis_install.sh
bash .travis_ssh_setup.sh
cd docs
pip install -r requirements.txt
make doctest
```
You can find more information on testing in [TESTING.md](TESTING.md).
This is the CTF framework used by Gallopsled in every CTF.
Pwntools is a CTF framework and exploit development library. Written in Python, it is designed for rapid prototyping and development, and intended to make exploit writing as simple as possible.
```python
frompwnimport*
@@ -20,32 +20,30 @@ r.send(asm(shellcraft.sh()))
r.interactive()
```
However we have made command-line frontends for some of the functionality
in `pwnlib`. These are:
# Try It Now!
*`asm`/`disasm`: Small wrapper for various assemblers.
*`constgrep`: Tool for finding constants defined in header files.
*`cyclic`: De Bruijn sequence generator and lookup tool.
*`hex`/`unhex`: Command line tools for doing common hexing/unhexing operations.
*`shellcraft`: Frontend to our shellcode.
*`phd`: Replacement for `hexdump` with colors.
You can now do a live demo of Pwntools, [right in your browser](https://demo.pwntools.com).
# Documentation
Our documentation is available at [pwntools.readthedocs.org](https://pwntools.readthedocs.org/en/latest/)
Our documentation is available at [docs.pwntools.com](https://docs.pwntools.com/)
To get you started, we've provided some example solutions for past CTF challenges in our [write-ups repository](https://github.com/Gallopsled/pwntools-write-ups).
# Installation
pwntools is best supported on Ubuntu 12.04 and 14.04, but most functionality should work on any Posix-like distribution (Debian, Arch, FreeBSD, OSX, etc.).
Pwntools is best supported on 64-bit Ubuntu LTE releases (12.04, 14.04, and 16.04). Most functionality should work on any Posix-like distribution (Debian, Arch, FreeBSD, OSX, etc.). Python 2.7 is required.
Most of the functionality of pwntools is self-contained and Python-only. You should be able to get running quickly with
However, some of the features (ROP generation and assembling/disassembling foreign architectures) require non-Python dependencies. For more information, see the [complete installation instructions here](https://pwntools.readthedocs.org/en/latest/install.html).
However, some of the features (assembling/disassembling foreign architectures) require non-Python dependencies. For more information, see the [complete installation instructions here](https://docs.pwntools.com/en/stable/install.html).
# Contribution
@@ -53,7 +51,7 @@ However, some of the features (ROP generation and assembling/disassembling forei
See [CONTRIBUTING.md](CONTRIBUTING.md)
# Contact
If you have any questions not worthy of a [bug report](https://github.com/Gallopsled/pwntools/issues), feel free to join us
If you have any questions not worthy of a [bug report](https://github.com/Gallopsled/pwntools/issues), feel free to ping us
at [`#pwntools` on Freenode](irc://irc.freenode.net/pwntools) and ask away.
Click [here](https://kiwiirc.com/client/irc.freenode.net/pwntools) to connect.
There is also a [mailing list](https://groups.google.com/forum/#!forum/pwntools-users) for higher latency discussion.
Pwntools makes extensive use of unit tests and integration tests to ensure everything is in working order, and no regressions occur.
## Test Suite
To run the test suite, it is best to use Ubuntu 12.04 or 14.04, and run the following commands. **Be aware** that this will add a user to the machine, and create a public key for SSH login!
```sh
bash travis/install.sh
bash travis/ssh_setup.sh
pip install --upgrade --editable .
PWNLIB_NOTERM=1 make -C docs doctest
```
## Testing in Docker
A `Dockerfile` has been provided which has a clean testing environment with Ubuntu Xenial. It is very similar to the online Travis CI testing environment, but uses a more modern version of Ubuntu.
See `travis/docker/README.md` for more information.
## New Tests
To add a new test to an existing module, just add an inline doctest. If the test needs access to an external module, add the import statement to the `testsetup` block in the corresponding file in `docs/source/<module>.rst`.
To add an entirely new module, create a new `module.rst` and add it to the list in `index.rst`. The best way to see if your tests are actually being run is to add an intentionally-failing test like:
```py
>>>assertFalse
```
## Example Test Module
The module `pwnlib.testexample` exists to demonstrate how everything is tied together. The only files which were modified to add this example module and run its tests automatically are:
-`pwnlib/testexample.py`
-`docs/source/testexample.rst`
-`docs/source/index.rst`
## Shellcode and ROP
These are both less easy to test, as they require actually executing code, or loading real binaries. In order to make the process simpler, the `runner` library was created to wrap common tasks. For an example of testing shellcode with these helpers, see [exit.asm](pwnlib/shellcraft/templates/i386/linux/exit.asm).
Additionally, for loading ELF files on-the-fly, the helpers `ELF.from_bytes` and `ELF.from_assembly` are available.
Many settings in ``pwntools`` are controlled via the global variable :data:`.context`, such as the selected target operating system, architecture, and bit-width.
..autoclass:: pwnlib.context.ContextType
:members:
In general, exploits will start with something like:
..autoclass::pwnlib.context.Thread
..code-block::python
frompwnimport*
context.arch='amd64'
Which sets up everything in the exploit for exploiting a 64-bit Intel binary.
The recommended method is to use ``context.binary`` to automagically set all of the appropriate values.
@@ -7,12 +7,13 @@ this replacement module also defines :func:`unregister`.
This module also fixes a the issue that exceptions raised by an exit handler is
printed twice when the standard :mod:`atexit` is used.
"""
from__future__importabsolute_import
importsys
importthreading
importtraceback
from.contextimportcontext
frompwnlib.contextimportcontext
__all__=['register','unregister']
@@ -83,7 +84,7 @@ def _run_handlers():
func(*args,**kwargs)
exceptSystemExit:
pass
except:
exceptException:
# extract the current exception and rewind the traceback to where it
# originated
typ,val,tb=sys.exc_info()
Loaded 100 of 943 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Mathias Svensson
