* [elf.py] convert linux string banner into an actual str
``self.string(address)`` actually return a bytes object containing the raw bytes, not a string in a pythonesque sense !
Since quite some time on Python3, ``re.search`` only works on proper str so we need to convert it before feeding it to ``re.search``
``.decode()`` works both on Python2 and Python3
* Update kernel ELF handling
Co-authored-by: Arusekk <[email protected]>
* Implement property getter/setter for context.newline, and make tubes honor it
The default newline can now be set context-wide, or per-tube.
* [changelog] Update for 1541
* [docs] Use r-prefix for embedded newlines
Co-authored-by: Arusekk <[email protected]>
* use non-greedy operator to handle corner cases where the program output contains END in it
* END can appear in the string leak
Co-authored-by: Arusekk <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* Keep ROP cache in user's home like regular cache
I don't have statistics numbers on which would be more likely to be writable, an user's home directory or the OS's temp directory, but seems to me it'd make sense to try and write all caches to the same location.
Optimally this would be easily configurable to allow adjusting to different set-ups, obviously.
* Use cache_dir instead
Co-authored-by: Arusekk <[email protected]>
* fix --multibr issue
* modify changelong
* In which I learn how to count
* put doctest for ROP().syscall in unfortunate place
* fix context malapropism
* fix ROP gadget loading when binary contains a gadget with add rsp, <register>
* add documentation regarding the fact that loading gadget "add $sp, <reg>" will fail
All three instructions: `int 0x80`, `sysenter`, and `syscall` require
different stack frames and registers to pass arguments, but
`ROP._srop_call` uses them interchangeably.
This fix limits syscall instruction lists to gadgets that work.
Co-authored-by: Arusekk <[email protected]>
* Change lookup algorithm of `adb.which`.
This is done because the current algorithm does not work when the binary (e.g. `su`) is installed in the last element of $PATH.
Example:
```shell
$ echo $PATH
/sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin
$ echo $PATH | while read -r -d: directory; do
> echo "$directory/su";
> done
/sbin/su
/vendor/bin/su
/system/sbin/su
/system/bin/su
$ (IFS=:
> for directory in $PATH; do
> echo "$directory/su";
> done
> )
/sbin/su
/vendor/bin/su
/system/sbin/su
/system/bin/su
/system/xbin/su
```
* Update CHANGELOG
Turns out that python 2 does not raise correct socket.timeout, but
SSLError instead. With no errno. Just a message that might be
localized in some circumstances.
**THIS IS NOT THE ONLY TIME PYTHON 2 IS CAUSING PROBLEMS.**
Time to stop using it, dear exploit developers. It is dead now.
But here you are: the simple fix.
Closes#1356
* [docker] Upgrade Docker images of Pwntools to use Python3
Also installs qemu-user-static and various binutils packages for building (binutils)
and running (qemu) cross-arch binaries.
Fixes#1500
* Update CHANGELOG
* Switch to Python3 for stable/beta/dev Docker images
* [changelog] Add missing links
* [shellcraft] Avoid recursive walk of all templates for command line
Fixes#1650
* Avoid double-fetch
* Add changelog
* [pylint] Only warn for additions
* tubes/remote: Support Server Name Indication and enable by default
This allows customizing the server_name via sni="custom" or sending the
name specified in the host parameter via sni=True.
We switch to the SSLContext.wrap_socket API because ssl.wrap_socket is
deprecated and does not support SNI.
Unfortunately, SSLContext.wrap_socket dropped some of the parameters
that are supported by ssl.wrap_socket. These parameters have been moved
to the SSLContext API and can now be used by passing a custom SSLContext
to the ssl_context parameter of `remote`.
* update CHANGELOG.md
* Enhance gdb error message when unable to launch
A common issue is when the binary is not executable,
or the binary is simply missing.
* Use six.ensure_str
* [tests] Change test slightly so that failing output will be emitted
* Disable flaky UI test on Github Actions
* [ui] Try a sleep rather than disabling tests
* Add a bfdname for msp430 to make asm and disasm work
* Add a bfdname for s390 to make asm and disasm work for all architectures
* Add tests for new bfdnames
* ci: Add binutils for newly-tested architectures
Co-authored-by: Arusekk <[email protected]>
* [packing] Support negative indices for flat() and fit()
Fixes#1531
* [changelog] Add Changelog for PR
* [packing] Only take negative path if there are any
min([]) is not valid Python
* [packing] Move out_negative to a higher scope
* [stable] Fix incorrectly-merged code
* [aarch64] Use byte-strings instead of strings
* [aarch64] Have different tests for str vs bytes
* [aarch64] Convert all items to binary before processing
* [aarch64] Fix doctest results
* [aarch64] Stopgap to fix Py3 CI tests
* [aarch64] Remove failing test, will address in a future release
* [aarch64] Fix aarch64.pushstr and pushstr_array
* pushstr
* We were pushing the correct values, and each 16-byte span was correct, but the 16-byte segments were ordered wrong
* pushstr_array:
* Was entirely broken. This commit has a non-optmimized version but it works
Closes#1549
* [packing] Better alternative to #1580
pack() was failing for dictionaries that are not ordered
Closes#1582Closes#1579
* [docs] Add better documentation for flat()
* [changelog] Add 4.1.4
* [release] Bump version to 4.1.4
Co-authored-by: Arusekk <[email protected]>
* [docker] Get Travis CI dockerfile working more
* [docker] Do not upgrade all packages, only update if required
* [docker] Move requirements installation higher up before the copy
* Fix several sphinx warnings
* Several code style fixes
Mostly grep -Er '[=!]= (None|True|False)'
* Compile-time str concat
also reduced redundant characters at line ends
* Fix python2 toplevel regression
* No equality comparison with singletons again
* Clarification: lambda: [] is just list
* Reintroduce reverted fragment
* sock-based GDB connections didn't set the PID, so it didn't work
* Now uses a real Timeout object
* Added special case handling for socat, since we were attaching to the wrong process
* ssh-based doctests improved
* Works locally on everything I tested
* One test is ignored, since it doesnt work on Travis CI
* Added libc6-dbg to Github Actions
* run_in_new_terminal supports unicode strings now
* This fixes SSH-based debugging
* wait_for_debugger now has a timeout
* Fixes the case where the debugger never comes up, mostly for tests
If passed an empty dictionary defining the environment variables, the process invoked by gdb.debug() has environment variable 'PWD' set to the directory containing the executable. This behaviour is not observered when NOPTRACE is enabled: the program runs without any environment variables set. Similarly, if an empty environment variable dictionary is passed to process(), it appears to also run under a truly empty environment. Additionally, attempts to overwrite the "PWD" environment variable with a dictionary entry are not successful under gdb.debug().
This fix brings the gdb.debug() treatment of environment variables in line with and process() and gdb.debug() with NOPTRACE enabled.
It allows both empty environments, and overwriting of the 'PWD' environment variable.
Both Debian and Ubuntu provide a `binutils-avr` package, which contains
an assembler for AVR as `avr-as`. However, pwntools was looking for
something with `linux` in the name, such as `avr-linux-gnu-as`. Of course,
such a target triplet is nonsense, Linux does not run on AVRs
(apart from that one hack that actually emulated some other CPU which
therefore doesn't count).
* Disable corefile tests for Github Actions
* Disable the UI tests, as theyre broken
* Install gdbserver
* Rename step
* Remove SIGALARM to see if it fixes tests
* Set PWNLIB_NOTERM for command-line runs
`ELF.functions` wasn't updated after the binary base address was changed using `ELF.address` like `ELF.symbols` is. The documentation mentions `ELF.functions` getting updated as well, the update was missing though.
Closes#1512
Co-authored-by: Arusekk <[email protected]>
* Update Corefile warnings and replace asserts with normal checks
This means that most of the Corefile will still work, even if
we encounter issues while parsing data from the stack.
* Update pwnlib/elf/corefile.py
* Add missing return statement
Co-authored-by: Arusekk <[email protected]>
Detect if pwntools is running in a Windows Subsystem for Linux "VM" and set the terminal to open a new `cmd.exe` window on the host.
This replaces the need to add `context.terminal = ['cmd.exe', '/c', 'start', 'ubuntu', 'run']` to every script.
Debugging using `gdb.attach` under WSL with gdb in a new window out of the box!
This is a WONTFIX, but default gcc on ubuntu compiles .bss (and other RW
segments) so far away from .text that ret2dlresolve mathematically
just has to segfault unless .text and .rodata contain crafted structures
already.
Compiling and working fine on gentoo, though. Hope this proves useful.
Also closes#1448 by the way, it was really annoying.
Closes#1476.
* Enable text colors in Windows console
Consoles on Windows 10 support ANSI escape sequences including color codes.
Emulate unix curses `tigetstr` terminal capabilities by hardcoding the given escape sequences for used curses capabilities.
ANSI escape sequences have to be enabled using `kernel32.SetConsoleMode(hndl, ENABLE_VIRTUAL_TERMINAL_PROCESSING)` for the current console.
* Use object identity operator
* Import ABC from collections.abc for Python 3.
* Fix deprecation warnings due to invalid escape sequences.
* Use raw strings instead of escaping individually.
Previously, the rop._srop_call function would append the args without checking whether their value would work
directly in a register. It now uses AppendedArgument to determine whether the value is pushed to the end of
the ROP chain.
* Update proc.py
reversed([p.pid for p in processes]) only return a iter.
if use gdb.attach('some process name') will cause error like :
File "/home/tsj/.local/lib/python2.7/site-packages/pwnlib/gdb.py", line 657, in attach
pid = pids[0]
TypeError: 'listreverseiterator' object has no attribute '__getitem__'
In this case ,it treat 'pids' as a list,so here should return a list as well
* Apply suggestions from code review
Co-authored-by: Arusekk <[email protected]>
if the value returned from the leak function is a str, it converts it to a bytearray for pytohn3 compatibility. Assuming a string is returned is flaky and likely wrong.
removed the double quotes around the temp filename of the gdbscript as this broke the gdbscript functionality.
The doublequotes would be considered part of the filename and it would fail to read the file..
Tested this fix on Kali.
* The SSH tube now keeps the environment variables of processes it creates in order.
The way of achieving that is through passing the env to the script as a list of tuples,
since Python3's dictionaries are ordered by default, or you can use `OrderedDict`,
the result would be that the spawned process on the remote machine will keep the environment variables in order.
This is very important to exploits that are based on the stack's order and especially the environment variables.
For instance, a common challenge is comparing argc == 0, and afterwards accessing the argv array, as an out-of-bounds array.
The shellcode needs to be at a certain index consistently and cannot rely on the stack moving around.
* Relocated the code.
* Try GitHub actions instead of Travis CI.
This is just a proof-of-concept until we can
have it the clean way it should be done.
More commits will likely follow.
* Travis CI: Use flake8 to detect syntax errors (#1385)
* Travis CI: Use flake8 to detect syntax errors
Repeat #1305 on the Python 3 branch. Also, remove the hardcoding of default values for dist and Py2.
https://flake8.pycqa.org/en/latest/user/error-codes.html
On the flake8 test selection, this PR does _not_ focus on "_style violations_" (the majority of flake8 error codes that [__psf/black__](https://github.com/psf/black) can autocorrect). Instead these tests are focus on runtime safety and correctness:
* E9 tests are about Python syntax errors usually raised because flake8 can not build an Abstract Syntax Tree (AST). Often these issues are a sign of unused code or code that has not been ported to Python 3. These would be compile-time errors in a compiled language but in a dynamic language like Python they result in the script halting/crashing on the user.
* F63 tests are usually about the confusion between identity and equality in Python. Use ==/!= to compare str, bytes, and int literals is the classic case. These are areas where __a == b__ is True but __a is b__ is False (or vice versa). Python >= 3.8 will raise SyntaxWarnings on these instances.
* F7 tests logic errors and syntax errors in type hints
* F82 tests are almost always _undefined names_ which are usually a sign of a typo, missing imports, or code that has not been ported to Python 3. These also would be compile-time errors in a compiled language but in Python a __NameError__ is raised which will halt/crash the script on the user.
* Fail fast, exclude generated source files
* --select=E9,F63,F7
* 2to3 -f print -wn examples
* No more <> in python 3.
* Use ==/!= to compare str, bytes, and int literals (#1386)
Identity is not the same thing as equality in Python so use ==/!= to compare str, bytes, and int literals. In Python >= 3.8, these instances will raise _SyntaxWarnings_ so it is best to fix them now. https://docs.python.org/3.8/whatsnew/3.8.html#porting-to-python-3-8
$ python
```python
>>> raw = 'ra'
>>> raw += 'w'
>>> raw == 'raw'
True
>>> raw is 'raw'
False
```
* Use print() function in both Python 2 and Python 3 (#1387)
$ __pip install black future isort__
$ __futurize -f libfuturize.fixes.fix_print_with_import -w android.py__
$ __isort android.py__
$ __black android.py__
* Add cache and building artifacts
* Enable caching
* Actually enable caching for all of it
* Mistake
* Try different ABI on CI
* Try different ABI + version
* Try it harder
* Update SDK version
* Try building using newer SDK
* Accept licenses
* Ugh android emulator documentation is unfriendly
* Environment variables for AVD
* Add QEMU-system for android virtual devices
* Use the correct emulator
* New android image
* Was actually too new image
* Update ssh credentials to reflect GHA setup
* Fix spelling
* Try finding endless loops
* Actually use the timeout
* Reduce possible hang surface in doctests
* Try SSH immediately when set up
* SSH strict host key checking disable for localhost
* Try stackoverflow.com research
* Try to fix failing doctests
* Add required gcc for cross-arch tests
* Ensure gcc not failing in fmtstr
* Fix issues with shellcraft
* Try different libs
* Try again, with verbosity
* Why do these conflict???
* Fix issues related to PIE binaries
* Add bash-static
* Verbose 'could not set transport: any'
* Add AT_SECURE in auxv to prevent SEGV on libc bug
* Killing ADB server actually kills it
* Android image has /data/user/0 -> /data/data
* Set coverage mode from dev
* Try new actions
* Fix coveralls complaining
* A little time.sleep would not hurt or would it
Co-authored-by: Christian Clauss <[email protected]>
* Allow pwntools to load on Windows
Fixes loading of pwntools on Windows. `from pwn import *` works.
It allows to use some operating system agnostic features of pwntools out of the box.
This is inspired by #996.
Some features fail or even crash (local processes, gdb, assemble/disassemble)
and console colors are still not displayed correctly, but those can be addressed separately.
* Use `os.devnull` instead of manual selection
* Add heuristic to find return address into __libc_start_main
Find the return address from main into __libc_start_main in a libc shared binary. Mimic's niklasb's heuristic in libc-database.
* Add support for different `call` instructions
* Introduce tests for pwnlib.ui
This architecture can be changed later, but I'm quite proud
of how it's done currently.
* Try to debug CI
* Ugly hack to get results back
* Let's just keep it half-broken
Currently, python 3 installed libraries
and sphinx doctree cache are not retained across builds.
This PR addresses this issue, hopefully resulting in a major speedup.
* Travis CI: Use flake8 to detect syntax errors
Repeat #1305 on the Python 3 branch. Also, remove the hardcoding of default values for dist and Py2.
https://flake8.pycqa.org/en/latest/user/error-codes.html
On the flake8 test selection, this PR does _not_ focus on "_style violations_" (the majority of flake8 error codes that [__psf/black__](https://github.com/psf/black) can autocorrect). Instead these tests are focus on runtime safety and correctness:
* E9 tests are about Python syntax errors usually raised because flake8 can not build an Abstract Syntax Tree (AST). Often these issues are a sign of unused code or code that has not been ported to Python 3. These would be compile-time errors in a compiled language but in a dynamic language like Python they result in the script halting/crashing on the user.
* F63 tests are usually about the confusion between identity and equality in Python. Use ==/!= to compare str, bytes, and int literals is the classic case. These are areas where __a == b__ is True but __a is b__ is False (or vice versa). Python >= 3.8 will raise SyntaxWarnings on these instances.
* F7 tests logic errors and syntax errors in type hints
* F82 tests are almost always _undefined names_ which are usually a sign of a typo, missing imports, or code that has not been ported to Python 3. These also would be compile-time errors in a compiled language but in Python a __NameError__ is raised which will halt/crash the script on the user.
* Fail fast, exclude generated source files
* --select=E9,F63,F7
* 2to3 -f print -wn examples
* No more <> in python 3.
* Adding support for FileStructure exploitation
* Actually run the tests + basic py3 fixes
also rename the module to lower case to keep consistency
* Python 3 working
Identity is not the same thing as equality in Python so use ==/!= to compare str, bytes, and int literals. In Python >= 3.8, these instances will raise _SyntaxWarnings_ so it is best to fix them now. https://docs.python.org/3.8/whatsnew/3.8.html#porting-to-python-3-8
$ python
```python
>>> raw = 'ra'
>>> raw += 'w'
>>> raw == 'raw'
True
>>> raw is 'raw'
False
```
* feat(fmtstr): place pointers after format operations
This means that the format string can now also work in certain
cases even if the pointers contain null bytes.
That's especially important on 64 bit where most pointers contain null bytes.
* fix(fmtstr): more stuff
* feat(fmtstr): optimize
* perf(fmtstr): make sorting run in reasonable time (not perfect yet)
* perf(fmtstr): be faster
* style(fmtstr): more docs, some cleanup
* fix(fmtstr): small fixes + more docs
* style(fmtstr): cleanup small style issues
* fix(fmtstr): fix handling of szmax in merge_atoms_overlapping
* fix(fmtstr): fix Atom __getitem__ for out-of-bounds index
* refactor(fmtstr): docs and cleaner code
* fix(fmtstr): small test/doc fixes
* tests(fmtstr): add example as tests for other arches
* doc(fmtstr): more docstrings
* fix(fmtstr): fix cyclic_find warnings
* refactor(fmtstr): remove another nested function
* tests(fmtstr): fix doctest output for wrong arch
* style(fmtstr): remove unused imports
* tests(fmtstr): add binaries for tests
* Updates to requirements.txt.
Removed the < 1.8.0 requirement for sphinx. It appeared no longer
necessary.
Alphabetized the listing for easier searching.
Moved the requirements file under the docker folder. Docker does not
like relative paths to parent directories due to security restrictions.
* Updated Dockerfile to use Python3.
* Fixes to srop test errors.
Moving the imports to the beginning of the file resolved the issue.
The following errors were popping up during testing within Docker:
Document: rop/srop
------------------
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.recvline()
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.5/doctest.py", line 1321, in __run
compileflags, 1), test.globs)
File "<doctest default[18]>", line 1, in <module>
p.recvline()
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 467, in recvline
return self.recvuntil(self.newline, drop = not keepends, timeout = timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 310, in recvuntil
res = self.recv(timeout=self.timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 82, in recv
return self._recv(numb, timeout) or b''
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 160, in _recv
if not self.buffer and not self._fillbuffer(timeout):
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 131, in _fillbuffer
data = self.recv_raw(self.buffer.get_fill_size())
File "/home/travis/pwntools/pwnlib/tubes/process.py", line 707, in recv_raw
raise EOFError
EOFError
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.poll(block=True)
Expected:
0
Got:
-11
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.recvline()
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.5/doctest.py", line 1321, in __run
compileflags, 1), test.globs)
File "<doctest default[18]>", line 1, in <module>
p.recvline()
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 467, in recvline
return self.recvuntil(self.newline, drop = not keepends, timeout = timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 310, in recvuntil
res = self.recv(timeout=self.timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 82, in recv
return self._recv(numb, timeout) or b''
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 160, in _recv
if not self.buffer and not self._fillbuffer(timeout):
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 131, in _fillbuffer
data = self.recv_raw(self.buffer.get_fill_size())
File "/home/travis/pwntools/pwnlib/tubes/process.py", line 707, in recv_raw
raise EOFError
EOFError
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.poll(block=True)
Expected:
0
Got:
-11
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.recvline()
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.5/doctest.py", line 1321, in __run
compileflags, 1), test.globs)
File "<doctest default[18]>", line 1, in <module>
p.recvline()
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 467, in recvline
return self.recvuntil(self.newline, drop = not keepends, timeout = timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 310, in recvuntil
res = self.recv(timeout=self.timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 82, in recv
return self._recv(numb, timeout) or b''
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 160, in _recv
if not self.buffer and not self._fillbuffer(timeout):
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 131, in _fillbuffer
data = self.recv_raw(self.buffer.get_fill_size())
File "/home/travis/pwntools/pwnlib/tubes/process.py", line 707, in recv_raw
raise EOFError
EOFError
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.poll(block=True)
Expected:
0
Got:
-11
**********************************************************************
1 items had failures:
6 of 159 in default
159 tests in 1 items.
153 passed and 6 failed.
***Test Failed*** 6 failures.
* Fix for docstring warning in dynelf.py.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/dynelf.py:docstring of pwnlib.dynelf.DynELF._resolve_symbol_sysv:11: WARNING: Definition list ends without a blank line; unexpected unindent.
* Fix for docstring warnings in gdb.py.
The following warnings are resolved with this fix:
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_assembly:8: WARNING: Inline strong start-string without end-string.
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_assembly:21: WARNING: Block quote ends without a blank line; unexpected unindent.
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_shellcode:4: WARNING: Inline strong start-string without end-string.
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_shellcode:17: WARNING: Block quote ends without a blank line; unexpected unindent.
* Fix for adb/__init__.py.
The following error is resolved with this fix:
Document: protocols/adb
-----------------------
**********************************************************************
File "../../pwnlib/protocols/adb/__init__.py", line ?, in default
Failed example:
pprint(AdbClient().list('/data/user'))
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.5/doctest.py", line 1321, in __run
compileflags, 1), test.globs)
File "<doctest default[0]>", line 1, in <module>
pprint(AdbClient().list('/data/user'))
TypeError: 'module' object is not callable
**********************************************************************
1 items had failures:
1 of 10 in default
10 tests in 1 items.
9 passed and 1 failed.
***Test Failed*** 1 failures.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/protocols/adb/__init__.py:docstring of pwnlib.protocols.adb.AdbClient.stat:6: WARNING: Inline literal start-string without end-string.
* Fix for docstring warning in rop.py.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/rop/rop.py:docstring of pwnlib.rop.rop.DescriptiveStack.descriptions:3: WARNING: Inline literal start-string without end-string.
* Fix for docstring warning in ssh.py.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/tubes/ssh.py:docstring of pwnlib.tubes.ssh.ssh._init_remote_platform_info:6: WARNING: Block quote ends without a blank line; unexpected unindent.
* Doctest dictionary ordering problem in misc.py.
There was an ordering problem while running the doctest in misc.py.
Sorting the dictionary with pprint and updating the results fixed the
issue.
Document: util/misc
-------------------
**********************************************************************
File "../../pwnlib/util/misc.py", line ?, in default
Failed example:
sizes
Expected:
{'eax': 32, 'ax': 16, 'al': 8, 'ah': 8, 'ebx': 32, 'bx': 16, 'bl': 8, 'bh': 8, 'ecx': 32, 'cx': 16, 'cl': 8, 'ch': 8, 'edx': 32, 'dx': 16, 'dl': 8, 'dh': 8, 'edi': 32, 'di': 16, 'esi': 32, 'si': 16, 'ebp': 32, 'bp': 16, 'esp': 32, 'sp': 16}
Got:
{'si': 16, 'cx': 16, 'cl': 8, 'dl': 8, 'dh': 8, 'eax': 32, 'bx': 16, 'sp': 16, 'bl': 8, 'bh': 8, 'di': 16, 'esi': 32, 'edx': 32, 'dx': 16, 'ebx': 32, 'ebp': 32, 'ax': 16, 'ch': 8, 'al': 8, 'ah': 8, 'edi': 32, 'esp': 32, 'bp': 16, 'ecx': 32}
**********************************************************************
File "../../pwnlib/util/misc.py", line ?, in default
Failed example:
bigger
Expected:
{'eax': ['eax'], 'ax': ['eax', 'ax'], 'al': ['eax', 'ax', 'al'], 'ah': ['eax', 'ax', 'ah'], 'ebx': ['ebx'], 'bx': ['ebx', 'bx'], 'bl': ['ebx', 'bx', 'bl'], 'bh': ['ebx', 'bx', 'bh'], 'ecx': ['ecx'], 'cx': ['ecx', 'cx'], 'cl': ['ecx', 'cx', 'cl'], 'ch': ['ecx', 'cx', 'ch'], 'edx': ['edx'], 'dx': ['edx', 'dx'], 'dl': ['edx', 'dx', 'dl'], 'dh': ['edx', 'dx', 'dh'], 'edi': ['edi'], 'di': ['edi', 'di'], 'esi': ['esi'], 'si': ['esi', 'si'], 'ebp': ['ebp'], 'bp': ['ebp', 'bp'], 'esp': ['esp'], 'sp': ['esp', 'sp']}
Got:
{'si': ['esi', 'si'], 'cx': ['ecx', 'cx'], 'cl': ['ecx', 'cx', 'cl'], 'dl': ['edx', 'dx', 'dl'], 'dh': ['edx', 'dx', 'dh'], 'eax': ['eax'], 'bx': ['ebx', 'bx'], 'sp': ['esp', 'sp'], 'bl': ['ebx', 'bx', 'bl'], 'bh': ['ebx', 'bx', 'bh'], 'di': ['edi', 'di'], 'esi': ['esi'], 'edx': ['edx'], 'dx': ['edx', 'dx'], 'ebx': ['ebx'], 'ebp': ['ebp'], 'ax': ['eax', 'ax'], 'ch': ['ecx', 'cx', 'ch'], 'al': ['eax', 'ax', 'al'], 'ah': ['eax', 'ax', 'ah'], 'edi': ['edi'], 'esp': ['esp'], 'bp': ['ebp', 'bp'], 'ecx': ['ecx']}
**********************************************************************
File "../../pwnlib/util/misc.py", line ?, in default
Failed example:
smaller
Expected:
{'eax': ['ax', 'al', 'ah'], 'ax': ['al', 'ah'], 'al': [], 'ah': [], 'ebx': ['bx', 'bl', 'bh'], 'bx': ['bl', 'bh'], 'bl': [], 'bh': [], 'ecx': ['cx', 'cl', 'ch'], 'cx': ['cl', 'ch'], 'cl': [], 'ch': [], 'edx': ['dx', 'dl', 'dh'], 'dx': ['dl', 'dh'], 'dl': [], 'dh': [], 'edi': ['di'], 'di': [], 'esi': ['si'], 'si': [], 'ebp': ['bp'], 'bp': [], 'esp': ['sp'], 'sp': []}
Got:
{'si': [], 'cx': ['cl', 'ch'], 'cl': [], 'dl': [], 'dh': [], 'eax': ['ax', 'al', 'ah'], 'bx': ['bl', 'bh'], 'sp': [], 'bl': [], 'bh': [], 'di': [], 'esi': ['si'], 'edx': ['dx', 'dl', 'dh'], 'dx': ['dl', 'dh'], 'ebx': ['bx', 'bl', 'bh'], 'ebp': ['bp'], 'ax': ['al', 'ah'], 'ch': [], 'al': [], 'ah': [], 'edi': ['di'], 'esp': ['sp'], 'bp': [], 'ecx': ['cx', 'cl', 'ch']}
**********************************************************************
1 items had failures:
3 of 20 in default
20 tests in 1 items.
17 passed and 3 failed.
***Test Failed*** 3 failures.
* Fix for docstring warning in packing.py.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/util/packing.py:docstring of pwnlib.util.packing.flat:2: WARNING: Inline emphasis start-string without end-string.
* Removed commented out test in Dockerfile.
* Forgot to update location of requirements.txt.
Forgot to update the location of requirements.txt in .travis.yml.
* Made recommended changes from Arusekk.
pwnlib.shellcraft installs its custom import finder (LazyImporter) into
sys.meta_path.
According to PEP 302, finder is expected to have prototype
finder.find_module(fullname, path=None)
Note that the second argument is optional. pwnlib's LazyImporter has second
argument required, which leads to compatibility problems, surfacing
when trying to use pwnlib with other third-party modules:
>>> import pwnlib
>>> import faker
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/tmp/.venv/lib/python2.7/site-packages/faker/__init__.py", line 2, in <module>
from faker.factory import Factory # noqa F401
File "/tmp/.venv/lib/python2.7/site-packages/faker/factory.py", line 12, in <module>
from faker.config import AVAILABLE_LOCALES, DEFAULT_LOCALE, PROVIDERS
File "/tmp/.venv/lib/python2.7/site-packages/faker/config.py", line 15, in <module>
AVAILABLE_LOCALES = find_available_locales(PROVIDERS)
File "/tmp/.venv/lib/python2.7/site-packages/faker/utils/loading.py", line 45, in find_available_locales
provider_module = import_module(provider_path)
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/tmp/.venv/lib/python2.7/site-packages/faker/providers/internet/__init__.py", line 4, in <module>
from text_unidecode import unidecode
File "/tmp/.venv/lib/python2.7/site-packages/text_unidecode/__init__.py", line 6, in <module>
_replaces = pkgutil.get_data(__name__, 'data.bin').decode('utf8').split('\x00')
File "/usr/lib64/python2.7/pkgutil.py", line 576, in get_data
loader = get_loader(package)
File "/usr/lib64/python2.7/pkgutil.py", line 462, in get_loader
return find_loader(fullname)
File "/usr/lib64/python2.7/pkgutil.py", line 473, in find_loader
loader = importer.find_module(fullname)
TypeError: find_module() takes exactly 3 arguments (2 given)
* Removed old dependency for intervaltree and sortedcontainers
Also make the pwn command show usage if no arguments supplied
* Fix errno subcommand
The errno library moved out of os and into its own library
* Change template script to use system env python instead of only py2
* Require updated intervaltree in setup.py
* Update .travis.yml
* Update .travis.yml
* Tubes fixed
* Typo
* Typo + default value
* Fixes on and on
* Many many more
* Missing lines and tests
* Refactoring
* Missing line again
* Missing again
* Copypaste design patterns
Use ==/!= to compare str, bytes, and int literals: __fmt is not 'raw'__ --> __fmt != 'raw'__
$ __python__
```python
>>> raw = 'ra'
>>> raw += 'w'
>>> raw
'raw'
>>> raw == 'raw'
True
>>> raw is 'raw'
False
```
coredumpctl is a tool that can be used to retrieve and process
core dumps and metadata which were saved by systemd-coredump(8).
systemd-coredump works by setting a piped core pattern. For example:
$ cat /proc/sys/kernel/core_pattern
|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e
This commit detects `systemd-coredump` in the pattern and then uses
`coredumpctl`, which is the CLI utility for reading the coredumps.
`coredumpctl` is instructed to write the coredump to a file. The
coredump is specified using the process ID and the basename of the
executable.
This commit also updates some doc strings to fix them and make them
more useful.
This fixes#1177, #1178
Ubuntu 16.04 finally includes `binutils-mips-linux-gnu`,
which is required. So there is no need to download
packages manually anymore. This can help supporting
python 3.
* corefile: Update NT_XXX type names for pyelftools==0.25
* travis: Do not use Sphinx version 1.8.0, it fails with recursive errors
* corefile: Fix stack detection and Mapping.__contains__ logic
* corefile: Better detection of AT_EXECFN when NT_AUXV does not have it
* corefile: Avoid trying to parse corrupt stacks
* Fix doctests for new warning
* log: Fix warning_once caching
* corefile: also expose page_offset
This is useful to load the code from the original files.
* Undo constructor field reordering
* Fix ordering in Mapping instantiation
* Commas are important
* corefile: also put page_offset as last parameter
* Improve pwnup template
* Don't want to break at main again if its been hit already
* Always start the program
* Handle extra newline printed by gdb
* Remove moved recvline
* Remove controversial set sysroot
* Update gdb.py
* Update pwnup.mako
* fix wrong gadget selection in setRegisters() (#1118)
* Fix detection logic
* Update rop.py
* add test cases for filtering out the bad instructions in rop.py
* Replace division with floor division as of PEP-238
This will also get us closer to python3.
See https://www.python.org/dev/peps/pep-0238/
* Add from __future__ import division to source files
* Fix tests relying on broken autocontrib version (#1121)
* Removed tuple parameters unpacking due to PEP-3113
As a step towards python3 compatibility, removed tuple parameters
in :mod:`pwnlib.term.term`. They are now replaced with * unpacking.
See https://www.python.org/dev/peps/pep-3113/
Check for sha256 and sha256sum before trying openssl sha256 as "openssl sha256" doesn't return an error code in some old versions of openssl (OpenSSL 0.9.8g 19 Oct 2007 at least) that doesn't support sha256.
* Add sysroot= argument to gdb.debug, expose via 'pwn debug'
Also sets a sane default sysroot for Android debugging
* Add missing newline for "set deatch-on-fork off"
Introduced in e89a9b66ac
* Add support for extracting Android boot.img
* Fix incorrect variable in 'pwn debug'
* Ensure "file /path/to/file" is the first command
When one passes a `LD_PRELOAD` which cannot be preloaded the `process_created` string might end up looking like this:
```
"ERROR: ld.so: object '/bin/bash' from LD_PRELOAD cannot be preloaded (cannot dynamically load executable): ignored.\n"
```
When we end up in such situation, pwndbg shown unintuitive error:
```
$ python -c 'from pwn import *; gdb.debug("/bin/ls", env={"LD_PRELOAD": "/bin/bash"})'
[+] Starting local process '/usr/bin/gdbserver': pid 20691
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "pwnlib/context/__init__.py", line 1349, in setter
return function(*a, **kw)
File "pwnlib/gdb.py", line 397, in debug
port = _gdbserver_port(gdbserver, ssh)
File "pwnlib/gdb.py", line 234, in _gdbserver_port
gdbserver.pid = int(process_created.split()[-1], 0)
ValueError: invalid literal for int() with base 0: 'ignored.'
```
After this patch it will show:
```
$ python -c 'from pwn import *; gdb.debug("/bin/ls", env={"LD_PRELOAD": "/bin/bash"})'
[+] Starting local process '/usr/bin/gdbserver': pid 20863
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "pwnlib/context/__init__.py", line 1349, in setter
return function(*a, **kw)
File "pwnlib/gdb.py", line 402, in debug
port = _gdbserver_port(gdbserver, ssh)
File "pwnlib/gdb.py", line 236, in _gdbserver_port
'Failed to spawn process under gdbserver. gdbserver error message: %s' % process_created
ValueError: Failed to spawn process under gdbserver. gdbserver error message: ERROR: ld.so: object '/bin/bash' from LD_PRELOAD cannot be preloaded (cannot dynamically load executable): ignored.
[*] Stopped process '/bin/ls' (pid 20863)
```
As @bennofs noticed, our detection logic was incorrect:
"This is not a good test if I understand the issue correctly.
From my tests, it seems that fault_addr was only null before this patch if it
would've been >= 2**47.
This is because on 64 bit, that's where kernel memory is mapped so we get
"protection fault" instead of "segfault".
A protection fault does not have the same siginfo as a normal segfault:
si_code will be SI_KERNEL = 0x80 and there won't be a fault_addr
(see http://elixir.free-electrons.com/linux/v4.14-rc8/source/kernel/signal.c#L1052)"
* Handle debugging under Android better with 'pwn debug'
Previously, when using --process or --pid, we would terminate the gdbserver instance
when pwntools exited. Now we wait for the user to tell us to exit.
* Handle launching Android processes by filename
* Use the --multi option with gdbserver; disable detach-on-fork
* Intentionally break doctest to make sure its running
* Revert "Intentionally break doctest to make sure its running"
This reverts commit 00578ad177.
* Copy env in process._validate using .copy method to keep the order in case of OrderedDict
* Use supplied env object to preserve order in case of OrderedDict
* Fix corefile environment variables and speed up ELF.string()
* Even more doctests
* Fix doctest arch
* No longer split on = since it isn't necessary
* Fix arch again
* Delete intermediate corefiles
* Do not flag for extended testing when the commit SHA is what matches
* Add support for finding QEMU's LD_PREFIX
* Refactor pwnlib.qemu and add official documentation
* Add more documentation, support for sysroot
* More docs, binfmt info
* Fix variable name overload
* Fix missed refactor
* Check for empty/null env
* Add qemu.rst to index
* Add cyclic default args to context
Specifically, this adds context.cyclic_alphabet and context.cyclic_size
This helps cut down on writing "n=8" all the time for users who use 8-byte
subsequences on 64-bit architectures.
Exposing the default alphabet means that users can avoid "bad bytes" in
cyclic patterns which are generated by pwntools internally (e.g. ROP padding)
Some pwnlib.util.packing docstrings were changed to clarify that word_size is
in bits.
The doctests in pwnlib.util.cyclic were updated and extended to have more
and better doctests, as well as better exaples.
* Add an example showing an alphabet with NUL bytes
This is important because old code would use pack(..., "all") when an integer
value was provided, instead of using the correct width.
* Overhaul ROP.setRegisters to support more complex cases needed by AMD64
Additionally, have a separate cache for ROPs composed of multiple ELF files
* Better presentation for stack padding slots
* Remove unnecessary stack adjustment on amd64
For amd64, it is possible that len(stackArguments) == 0.
* Add a doctest for amd64 register dependency stuff
* More i386 adjust optimizations
* Use nextGadgetAddr as the adjust value
* Always use padding for the last "ret" adjustment
* Better dump display
* Update doctests for new ROP dump() enhancements
* Add badchars support for ROP gadgets
* More doctest fixes, also ensure ROP.search does not return gadgets ending in e.g. int 0x80
* Prevent Sphinx from dumping all level=INFO logging output to stdout.
See Gallopsled/pwntools#1036 for more information.
* Filter out Paramiko logs too, apply filter to all root handlers during testing
* Remove unused import
(cherry picked from commit 54c47b3daa)
Technically this introduces issues with e.g. 'unhex ffdf | python foo.py' if
foo.py uses .interactive(). However, that was already broken for other reasons.
Fixes#1004
(cherry picked from commit ebbe917713)
* Slightly enhance template and advertise for the feature itself in scripts
Add single-dash to the "very good" characters in shell escaping, to avoid
the need to single-quote things like --host and --port.
* Add --quiet option for pwn template, move gdbscript down a bit
* Slightly better docs for 'pwn template'
* Add checksec information directly to 'pwn template' output
* Prevent Sphinx from dumping all level=INFO logging output to stdout.
See Gallopsled/pwntools#1036 for more information.
* Filter out Paramiko logs too, apply filter to all root handlers during testing
* Remove unused import
* DynELF: trust ELF base address
* Bug fixes for DynELF:
* Fixed lookup when not using libcdb
* Fixed _find_linkmap returning 0 instead of None when linkmap doesn't
exist
* Fixed _lookup_build_id crashing if linkmap cannot be found
* Misc cleanup
* Changes based on feedback
Technically this introduces issues with e.g. 'unhex ffdf | python foo.py' if
foo.py uses .interactive(). However, that was already broken for other reasons.
Fixes#1004
* Force use of Ubuntu 12 builds on Travis CI until we can migrate to 14
* Fix Travis Dockerfile
* Add bash command logging to Travis scripts, since Travis removed its own logging
* curl should follow redirects when grabbing Ubuntu packages
* Update Travis cache directories to newer version of Python, as seen in Travis logs
* Use ::1 instead of ip6-localhost since Travis 12.04 changes broke it
* fix typo in comment
* new config option: context.gdbinit
This allows setting the default gdbinit for GDB, which is helpful when you want
to enable some GDB extension only when using pwntools and not by default.
- Detect bootloaders that are already unklocked
- Detect bootloader unlock success
- Detect bootloaders that cannot be unlocked
- Detect devices which have not enabled bootloader unlocking
* Fix Full RELRO detection in ELFs.
This add supports for the newer DF_BIND_NOW and DF_1_NOW flags.
These are used instead of DT_BIND_NOW when linking with gold.
Also fixed the order of checks, now it reports "No RELRO" when
BIND_NOW is present but GNU_RELRO is not (in which case the bindings
are done at start but they will not be marked as read-only).
* Add x86 test binaries for relro check.
The check for when certain ELF fields contain offsets instead of pointers can be
made more precise by checking for the type field in the ELF header instead of
comparing the value to some hard-coded range.
* Clean up socket-class and socket-type code in sock/remote/listen
* Slight change to docs
* Add doctests for listen and attribute documentation
* Fix bug in listen.close()
* Fix regression on ipv6 sockets
* Simplify documentation
* Fix doctest
* Do not use UC_MODE_BIG_ENDIAN, its support is poor
* Ghetto-swap endianness for emulation of PLT instructions
* Reduce instruction alignment so that all ARM PLT entries are found
* Add tests for ELF PLT loading
* Fix canary detection on PowerPC
* Fix x32 emulation
* Fix two failing doctests
* Add AArch64 to supported big-endian architectures
* Add MIPS GOT parsing code, fix unpack macros to use ELF settings
* Why, MIPS, are you so complicated?
* Catch weird edge cases
* I don't know what's real anymore
* Fix synthetic symbol generation and add tests
* Remove canary test
* Guard against errors in MIPS GOT
* Fix doctests
* Do not fail with gdb.debug() over ssh when given an absolute path
FixesGallopsled/pwntools#885
* Prevent failing on stable since this directory doesn't exist
* Doctest all members
* Fix doctest failures in asm and remove canary in context
* Fix failing doctest
* Use automodule for pwnlib.context
* Fix failing tests in context module
* Use automodule for pwnlib.memleak
* Fix checksec nx, execstack, relro reporting
Also add a disable_nx function
* Add disablenx command-line utility
Also fix bug where invoking command-line utilities directly by module name
(python -m pwnlib.commandline.foo) would fail
This adds functionality for user configuration files at ~/.pwn.conf
and /etc/pwn.conf. Previously this was only used by the pwnlib.log
module, and was entirely undocumented.
This is now documented, and offers an easy mechanism for other parts
of the code to have extension points.
* Remove unavailable syscalls from AArch64
The Linux kernel has apparently stopped supporting these
* Rewrite shellcraft.aarch64.linux.cat, add open which uses openat
FixesGallopsled/pwntools#896
* Fix doctests to use openat
* Add templates from generator
* Add abi.stack to all ABIs
* Enable shellcraft.arch.os.foobar.template
* Add pwnlib.registers.current()
* Add syscalls symlinks
* Add syscall generator
* Enable shellcraft.foo() shortcut for shellcraft.arch.os.syscalls.foo()
* Make templates more verbose
* Remove old templates
* Add makefile
* Delete old templates, make shellcraft --list work
This also speeds up boot time by like 0.05 seconds
* Remove unnecessary execve wrappers
* Check for common arch as well
* Don't explode on string arguments that are register names
* Catch bad argument names
* Remove accidental import
* Add aarch64 ABIs since we enabled aarch64 tests
* Fix mmap vs mmap2 syscall
* Make shellcraft command-line work for syscalls
* Remove unnecessary wrapper templates
* Set all default values to zero
* Add example of automatic determination of types
* Add missing register architectures
* Add PowerPC ABIs
* More syscall examples
* Handle mmap vs mmap2
* Add missing import, use full path
* Just use the string name
* Fix rt_sigreturn on amd64
* __NR_ --> SYS_ because of old expectations
* rstrip to remove blankline
* Fix Thumb doctests which uses unpredictable jump labels
* Import additional syscalls by updating the function source.
Support variable-argument syscalls.
Support spilling to the stack.
* Simplified constants.eval logic means no extra parens
* Simplify vararg representation; fix oepn() argument name explicitly
* Fix syscall examples
* Fix reordering of arguments, since we have to support stack spill
* Fix argument name in doctest
* Make Codacy happy
* Restore connect template accidentally removed in 9fceb12b
* Scrape all architectures for syscall names
* Enable void* and void** to be pushed
* Make Codacy happy
* Add __doc__ to sdist
* Follow shellcraft template symlinks
* Detect syscall shellcodes in shellcraft based on path
(cherry picked from commit 94462845fa)
Adds a large amount of functionality around Corefiles, to support smarter exploitation
Adds a ".corefile" property to both local and remote (ssh) processes. See the
documentation for the corefile module for extensive examples and tests.
Additionally, context.delete_corefiles and context.rename_corefiles have been added,
to control the behavior of the ".corefile" property.
* Add aarch64 documentation, which enables doctests
* Fix failing doctest
* Attempt to figure out whats going on with Travis
* Use the Xenial version of qemu-static since the Zesty version is broken for AArch64
- Show the PID when a process is created
- Make process.cwd into a property, which fetches the real CWD from procs
- Record the UID/GID and SUID/SGID of the executed file
- Add a few doctests
- Make _stop_noticed a timestamp
- Break out ssh_channel into subclass ssh_process
- Make ssh.process() directly instantiate ssh_process
- Add ssh_process().ELF
- Add ssh_process().corefile
- Record the UID and GID of the executed binary
- Add the created PID to the ssh_process message
* Fix chunked reading of ELF files
This introduces a new dependency on 'intervaltree', which is used
to automatically select segments which are relevant for a given
virtual-address range.
This also introduces a new ELF.memory attribute, which is the tree
of memory ranges, which point at the segment objects that describe
the memory range.
These are recalculated when ELF.address is adjusted, and we have
tests for that.
This also optimizes some of the other routines to avoid the wrapped
file-seeking, and to just use the mmap.
* Rewrite install scripts to use more modern versions of binutils
* Load segments in file-order, chop any overlaps
* Add pwnlib.protocols module
Migrate pwnlib.adb.protocol into that directory, and add compatibility shims
* Add documentation links, and run doctests
* Make doctests run
* pwnlib.protocols.adb.Client.transport should work with no device
We cannot determine the length of the file if passed e.g. a tube or a pipe or PTY or character device (e.g. stdin or /dev/null).
Supercedes: 9c2f0f4e
(cherry picked from commit 14f717b7b4)
This code wasnt updated when the code was changed to operate on file descriptors.
Additionally, there was a functional regression, that the first line would be skipped.
Fixes: Gallopsled/pwntools#836
Caused by: Gallopsled/pwntools#695 (d65991d)
Looks like this:
[x] Connecting to example.pwnme on port 22
[+] Connecting to example.pwnme on port 22: Done
[*] [email protected]:
Distro Ubuntu 14.04
OS: linux
Arch: amd64
Version: 3.11.0
ASLR: Enabled
Note: Susceptible to ASLR ulimit trick (CVE-2016-3672)
FixesGallopsled/pwntools#753
* Defer attempts to start ADB server until we actually fail to connect
* Upgrade to a warning message; fix .version() multiple calls with _autoclose
* Add doctest which tests the kill/restart behavior
* Add PR_SET_PTRACER to initialization, and add doctests for process.leak/corefile
* Add PR_SET_PTRACER to SSH processes
* Do not warn about YAMA if we dont need to
* Fix doctest to use raw strings
* Fix typo in doctest
* Apparently GDB is not installed on Travis by default
* Add warning message to process("foo") vs process("./foo")
* Add gdb.corefile; make run_in_new_terminal return a PID
* Intentionally break doctest because I dont believe that it worked
* Fix codacy issues
* Fix gdb.attach return value
* Add missing import
* Fix typo, use waitpid
* Refactor some code into gdb.version / gdb.binary
* Permit GDB with multipart versions
* Fix repr() on Corefile
* Silence run-in-new-terminal message for gdb.corefile
* Fix version number
* Remove Yama warning message entirely, it should no longer be necessary
* Version detection is hard. Regular expressions to the rescue!
* Remove process.kill/stop/resume, those will be a different PR
* Fix doctests
* The GDB used by Travis must be so old!
* Apparently Travis is entirely borked. Use a less robust test.
* Enable leaking memory relatively with MemLeak
FixesGallopsled/pwntools#758
* Add RelativeMemLeak alias for MemLeak(..., relative=True)
* Make codacy happy
If pandoc is not installed locally, pwntools' setup.py
tells pypandoc to download one.
Unfortunately, it fails on non-amd64 arches because
pypandocs provides precompiled binaries only for amd64.
Since pypandoc is not hard dependency for pwntools,
this error can be skipped without major harm.
In fact, pwntools' setup.py already handles ImportError
gracefully. The problem is, arch mismatch causes
a completely different exception.
* Remove support for installation without psutil.
OpenBSD was the edge case we cared about, it is supported for over a year (giampaolo/psutil#615)
* Bump psutil version to when they introduced OpenBSD support
* Fix ADB wait_for_device with ANDROID_SERIAL set, and fix performance
* Fix QEMU usage for native binaries
* Fix adb.write and adb.listdir operating on the wrong type of object
* Fix failing unit test with new behavior in adb.protocol.Client.list
* Fix adb server startup issue
* Rework deferred initialization to work around class-scope superclass attributes, and add test for AdbDevice
* Add adb.{unlink,mkdir,makedirs,isdir,exists}
* Change unlink() to throw an exception rather than return a string
* Fix doctest and code for isdir
* Remove unused variable
* Fix adb.partitions listing
* Add adb.walk and adb.find functions to ensure adb.partitions works on devices without a find binary
* Add adb.readlink to simplify logic for reading symlinks
Support custom buffer-read sizes for `tube` objects via their internal `Buffer` object. Default buffer size is 4096. Buffer size can be set per-object at instantiation-time (`tube(buffer_fill_size=...)`), by attribute (`tube.buffer_fill_size=...`), or in a context (`context.buffer_size=...`).
Previously, if getaddrinfo or socket.socket threw an exception,
_connect would return without ever calling h.success or h.failure. As a
result, progress indicators would continue to update indefinitely.
Fix this by using a with statement instead.
(Also remove an unused variable and fix a typo.)
* adding output format for python hex-notation
* cleaned up implementation
* changed format identifier to {a} --escaped
* {a] was taken by asm, changed format identifier to {d} --escaped
* adding {e}lf to help
When reporting an issue, be sure that you are running the latest released version of pwntools (`pip install --upgrade pwntools`).
Please verify that your issue occurs on 64-bit Ubuntu 14.04. You can use the Dockerfile on `docker.io` for quick testing.
```
$ docker pull pwntools/pwntools:stable
$ docker run -it pwntools/pwntools:stable
```
If possible, provide a proof-of-concept which demonstrates the problem. Include any binaries or scripts necessary to reproduce the issue, and please include the full debug output via setting the environment variable `PWNLIB_DEBUG=1`.
When reporting an issue, be sure that you are running the latest released version of pwntools (`pip install --upgrade pwntools`).
## Debug Output
Having the extra debug output really helps us, and might help you diagnose the problem yourself.
When submitting an issue that has output from Pwntools, make sure to run your script as shown below, to enable the extra debugging data.
```sh
$ python exploit.py DEBUG LOG_FILE=log.txt
```
You should see `[DEBUG]` statements that show what's happening behind the scenes:
```
[+] Starting local process '/bin/sh' argv=['sh'] : pid 16823
[DEBUG] Sent 0x7 bytes:
'whoami\n'
[DEBUG] Sent 0x5 bytes:
'exit\n'
[+] Receiving all data: Done (11B)
[DEBUG] Received 0xb bytes:
'crashheap\n'
[*] Process '/bin/sh' stopped with exit code 0 (pid 16823)
```
## Verify on Ubuntu
If possible, please verify that your issue occurs on 64-bit Ubuntu 18.04. We provide a Dockerfile based on Ubuntu 18.04 via `docker.io` to make this super simple, no VM required!
Thanks for using Pwntools! If you have a question, here are some tips that will make it easier for us to help you.
## Check the Documentation
Pwntools has REALLY extensive documentation. Please check there first! https://docs.pwntools.com
## Debug Output
If you're having an issue with an exploit, run your script with full debugging output, i.e. `python exploit.py DEBUG`. This may help you to see what's happening under the hood, and the debug output will help to diagnose any issues you're encountering.
- [#1242][1242] Use IntervalTree 2.xx, disallow use of 3.xx
- [#1243][1243] Fix a typo that caused an exception when executing a binary with `process()` which returns `-ENOEXEC` and the system does not have `qemu-user` binaries installed.
- [#1083][1083] Better error messages for `gdb` when `LD_PRELOAD` is incorrect
- [#1085][1085] Add support for extracting Android `BOOTLDR!` images
- [#1075][1075] Add support for detecting GNU Screen for `run_in_new_terminal`
- [#1074][1074] Add support for running `pwntools-gdb` wrapper script instead of `gdb`
- [#1068][1068] Work around very old OpenSSL versions which don't have sha256 support *AND* don't exit with an error code when trying to use it
- [#1067][1067] Add `pwnlib.tubes.server` module, which adds a reusable `server` listener
- [#1063][1063] Add support for labels in `fit()`, allowing dynamic contents to be injected. (This feature is really cool, check out the pull request!)
+ Much better support for 64-bit Intel (amd64) ROP
+ ROP gadget selection is optimized to favor multi-pops instead of multiple single-pop gadgets
+ Added support for blacklisting byte values in ROP gadget addresses
- [#1049][1049] Enhancements to `cyclic`
+ `context` now has two additional attributes, `cyclic_alphabet` and `cyclic_length`, which correspond to the arguments `alphabet` and `n` to `cyclic()` and `cyclic_find()` and related routines.
+ The motivation for this change is to allow setting the `alphabet` globally, so that any padding / patterns generated internally to pwntools can be controlled. The specific motivation is blacklisting values in ROP padding.
- [#1052][1052] Enhancements for detecting `QEMU_LD_PREFIX` used by QEMU user-mode emulation for sysroots
- [#1035][1035] Minor documentation changes
- [#1032][1032] Enhancements to `pwn template`
- [#1031][1031] More accurate `Coredump.fault_addr` on amd64
- [#1084][1084] Fix broken tests due to `ftp.debian.org` going down
- [300f8e0][300f8e0] Slightly speed up processing of large ELF files
- [#861][861] Adds support for extracting `IKCONFIG` configs from Linux kernel images, and extends `checksec` to report on any insecure configurations discovered
- [#871][871] Moves all of the basic syscall templates to `shellcraft/common` and exposes them via symlinks. Closed [#685][685]
+ Should not have any visible effects from any documented APIs
+ `shellcraft.arch.os.syscall_function()` still works the same
+ We now have the ability to differentiate between the `connect` syscall, and a TCP `connect` helper
- [#887][887] `sh_string` now returns a quoted empty string `''` rather than just an empty string
- [#839][839] Exposes a huge amount of functionality via corefiles which was not previously availble. See the [docs][corefile_docs] for examples.
+ `process().corefile` will automatically instantiate a Corefile for the process
+ QEMU-emulated processes are supported
+ Native processes are supported, including extraction of coredumps from `apport` crash logs
+ Native processes can be dumped *while running*, in a manner similar to `GDB`'s `gcore` script
- [#875][857] Added [documentation][aarch64] (and tests) for AArch64 shellcode
- [#882][882] The `ROP` class now respects `context.bytes` instead of using the hard-coded value of `4` (fixed [#879][879])
- [#869][869] Added several fields to the `process` class (`uid`, `gid`, `suid`, `sgid`) which are recorded at execution time, based on the file permissions
- [#868][868] Changed the way that `ssh.process()` works internally, and it now returns a more specialized class, `ssh_process`.
+ Added `ssh_process.corefile` for fetching remote corefiles
+ Added `ssh_process.ELF` for getting an ELF of the remote executable
+ The `uid`, `gid`, and `suid`, and `sgid` which are recorded at execution time, based on the file permissions
- [#865][865] Fixes `ELF.read` to support contiguous memory reads across non-contiguous file-backed segments
- [#862][862] Adds a `symlink=` argument to `ssh.set_working_directory`, which will automatically symlink all of the files in the "old" working directory into the "new" working directory
@@ -146,7 +795,7 @@ Android support via a new `adb` module, `context.device`, `context.adb_host`, an
- Added `context.log_file` setting for sending logs to a file. This can be set with the `LOG_FILE` magic command-line option.
- Added `context.noptrace` setting for disabling actions which require `ptrace` support. This is useful for turning all `gdb.debug` and `gdb.attach` options into no-ops, and can be set via the `NOPTRACE` magic command-line option.
- Added `context.proxy` which hooks all connections and sends them to a SOCKS4/SOCKS5. This can be set via the `PROXY` magic command-line option.
- Added `context.randomize` to control randommization of settings like XOR keys and register ordering (default off).
- Added `context.randomize` to control randomization of settings like XOR keys and register ordering (default off).
- Added `context.terminal` for setting how to launch commands in a new terminal.
#### DynELF and MemLeak Module
@@ -159,7 +808,7 @@ Android support via a new `adb` module, `context.device`, `context.adb_host`, an
#### Encoders Module
- Added a `pwnlib.encoders` module for assembled-shellcode encoders/decoders
- Includes position-indepentent basic XOR encoders
- Includes position-independent basic XOR encoders
- Includes position-independent delta encoders
- Includes non-position-independent alphanumeric encoders for Intel
- Includes position-independent alphanumeric encoders for ARM/Thumb
pwntools is a CTF framework and exploit development library. Written in Python, it is designed for rapid prototyping and development, and intended to make exploit writing as simple as possible.
Pwntools is a CTF framework and exploit development library. Written in Python, it is designed for rapid prototyping and development, and intended to make exploit writing as simple as possible.
```python
frompwnimport*
@@ -20,26 +21,25 @@ r.send(asm(shellcraft.sh()))
r.interactive()
```
# Try It Now!
You can now do a live demo of Pwntools, [right in your browser](https://demo.pwntools.com).
# Documentation
Our documentation is available at [docs.pwntools.com](https://docs.pwntools.com/)
A series of tutorials is also [available online](https://github.com/Gallopsled/pwntools-tutorial#readme)
To get you started, we've provided some example solutions for past CTF challenges in our [write-ups repository](https://github.com/Gallopsled/pwntools-write-ups).
# Installation
pwntools is best supported on 64-bit Ubuntu 12.04 and 14.04, but most functionality should work on any Posix-like distribution (Debian, Arch, FreeBSD, OSX, etc.). Python 2.7 is required.
Pwntools is best supported on 64-bit Ubuntu LTE releases (14.04, 16.04, 18.04, and 20.04). Most functionality should work on any Posix-like distribution (Debian, Arch, FreeBSD, OSX, etc.). Python >= 2.7 is required (Python 3 suggested as best).
Most of the functionality of pwntools is self-contained and Python-only. You should be able to get running quickly with
However, some of the features (assembling/disassembling foreign architectures) require non-Python dependencies. For more information, see the [complete installation instructions here](https://docs.pwntools.com/en/stable/install.html).
@@ -4,8 +4,7 @@ Pwntools makes extensive use of unit tests and integration tests to ensure every
## Test Suite
To run the test suite, it is beste to use Ubuntu 12.04 or 14.04, and run the following commands. **Be aware** that this will add a user to the machine, and create a public key for SSH login!
To run the test suite, it is best to use Ubuntu 12.04 or 14.04, and run the following commands. **Be aware** that this will add a user to the machine, and create a public key for SSH login!
A `Dockerfile` has been provided which has a clean testing environment with Ubuntu Xenial. It is very similar to the online Travis CI testing environment, but uses a more modern version of Ubuntu.
See `travis/docker/README.md` for more information.
## New Tests
To add a new test to an existing module, just add an inline doctest. If the test needs access to an external module, add the import statement to the `testsetup` block in the corresponding file in `docs/source/<module>.rst`.
Many settings in ``pwntools`` are controlled via the global variable :data:`.context`, such as the selected target operating system, architecture, and bit-width.
..autoclass:: pwnlib.context.ContextType
:members:
In general, exploits will start with something like:
..autoclass::pwnlib.context.Thread
..code-block::python
frompwnimport*
context.arch='amd64'
Which sets up everything in the exploit for exploiting a 64-bit Intel binary.
The recommended method is to use ``context.binary`` to automagically set all of the appropriate values.
Loaded 100 of 3853 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Arusekk