* [elf.py] convert linux string banner into an actual str
``self.string(address)`` actually return a bytes object containing the raw bytes, not a string in a pythonesque sense !
Since quite some time on Python3, ``re.search`` only works on proper str so we need to convert it before feeding it to ``re.search``
``.decode()`` works both on Python2 and Python3
* Update kernel ELF handling
Co-authored-by: Arusekk <[email protected]>
* Implement property getter/setter for context.newline, and make tubes honor it
The default newline can now be set context-wide, or per-tube.
* [changelog] Update for 1541
* [docs] Use r-prefix for embedded newlines
Co-authored-by: Arusekk <[email protected]>
* use non-greedy operator to handle corner cases where the program output contains END in it
* END can appear in the string leak
Co-authored-by: Arusekk <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* Keep ROP cache in user's home like regular cache
I don't have statistics numbers on which would be more likely to be writable, an user's home directory or the OS's temp directory, but seems to me it'd make sense to try and write all caches to the same location.
Optimally this would be easily configurable to allow adjusting to different set-ups, obviously.
* Use cache_dir instead
Co-authored-by: Arusekk <[email protected]>
* fix --multibr issue
* modify changelong
* In which I learn how to count
* put doctest for ROP().syscall in unfortunate place
* fix context malapropism
* fix ROP gadget loading when binary contains a gadget with add rsp, <register>
* add documentation regarding the fact that loading gadget "add $sp, <reg>" will fail
All three instructions: `int 0x80`, `sysenter`, and `syscall` require
different stack frames and registers to pass arguments, but
`ROP._srop_call` uses them interchangeably.
This fix limits syscall instruction lists to gadgets that work.
Co-authored-by: Arusekk <[email protected]>
* Change lookup algorithm of `adb.which`.
This is done because the current algorithm does not work when the binary (e.g. `su`) is installed in the last element of $PATH.
Example:
```shell
$ echo $PATH
/sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin
$ echo $PATH | while read -r -d: directory; do
> echo "$directory/su";
> done
/sbin/su
/vendor/bin/su
/system/sbin/su
/system/bin/su
$ (IFS=:
> for directory in $PATH; do
> echo "$directory/su";
> done
> )
/sbin/su
/vendor/bin/su
/system/sbin/su
/system/bin/su
/system/xbin/su
```
* Update CHANGELOG
Turns out that python 2 does not raise correct socket.timeout, but
SSLError instead. With no errno. Just a message that might be
localized in some circumstances.
**THIS IS NOT THE ONLY TIME PYTHON 2 IS CAUSING PROBLEMS.**
Time to stop using it, dear exploit developers. It is dead now.
But here you are: the simple fix.
Closes#1356
* [docker] Upgrade Docker images of Pwntools to use Python3
Also installs qemu-user-static and various binutils packages for building (binutils)
and running (qemu) cross-arch binaries.
Fixes#1500
* Update CHANGELOG
* Switch to Python3 for stable/beta/dev Docker images
* [changelog] Add missing links
* [shellcraft] Avoid recursive walk of all templates for command line
Fixes#1650
* Avoid double-fetch
* Add changelog
* [pylint] Only warn for additions
* tubes/remote: Support Server Name Indication and enable by default
This allows customizing the server_name via sni="custom" or sending the
name specified in the host parameter via sni=True.
We switch to the SSLContext.wrap_socket API because ssl.wrap_socket is
deprecated and does not support SNI.
Unfortunately, SSLContext.wrap_socket dropped some of the parameters
that are supported by ssl.wrap_socket. These parameters have been moved
to the SSLContext API and can now be used by passing a custom SSLContext
to the ssl_context parameter of `remote`.
* update CHANGELOG.md
* Enhance gdb error message when unable to launch
A common issue is when the binary is not executable,
or the binary is simply missing.
* Use six.ensure_str
* [tests] Change test slightly so that failing output will be emitted
* Disable flaky UI test on Github Actions
* [ui] Try a sleep rather than disabling tests
* Add a bfdname for msp430 to make asm and disasm work
* Add a bfdname for s390 to make asm and disasm work for all architectures
* Add tests for new bfdnames
* ci: Add binutils for newly-tested architectures
Co-authored-by: Arusekk <[email protected]>
* [packing] Support negative indices for flat() and fit()
Fixes#1531
* [changelog] Add Changelog for PR
* [packing] Only take negative path if there are any
min([]) is not valid Python
* [packing] Move out_negative to a higher scope
* [stable] Fix incorrectly-merged code
* [aarch64] Use byte-strings instead of strings
* [aarch64] Have different tests for str vs bytes
* [aarch64] Convert all items to binary before processing
* [aarch64] Fix doctest results
* [aarch64] Stopgap to fix Py3 CI tests
* [aarch64] Remove failing test, will address in a future release
* [aarch64] Fix aarch64.pushstr and pushstr_array
* pushstr
* We were pushing the correct values, and each 16-byte span was correct, but the 16-byte segments were ordered wrong
* pushstr_array:
* Was entirely broken. This commit has a non-optmimized version but it works
Closes#1549
* [packing] Better alternative to #1580
pack() was failing for dictionaries that are not ordered
Closes#1582Closes#1579
* [docs] Add better documentation for flat()
* [changelog] Add 4.1.4
* [release] Bump version to 4.1.4
Co-authored-by: Arusekk <[email protected]>
* [docker] Get Travis CI dockerfile working more
* [docker] Do not upgrade all packages, only update if required
* [docker] Move requirements installation higher up before the copy
* Fix several sphinx warnings
* Several code style fixes
Mostly grep -Er '[=!]= (None|True|False)'
* Compile-time str concat
also reduced redundant characters at line ends
* Fix python2 toplevel regression
* No equality comparison with singletons again
* Clarification: lambda: [] is just list
* Reintroduce reverted fragment
* sock-based GDB connections didn't set the PID, so it didn't work
* Now uses a real Timeout object
* Added special case handling for socat, since we were attaching to the wrong process
* ssh-based doctests improved
* Works locally on everything I tested
* One test is ignored, since it doesnt work on Travis CI
* Added libc6-dbg to Github Actions
* run_in_new_terminal supports unicode strings now
* This fixes SSH-based debugging
* wait_for_debugger now has a timeout
* Fixes the case where the debugger never comes up, mostly for tests
If passed an empty dictionary defining the environment variables, the process invoked by gdb.debug() has environment variable 'PWD' set to the directory containing the executable. This behaviour is not observered when NOPTRACE is enabled: the program runs without any environment variables set. Similarly, if an empty environment variable dictionary is passed to process(), it appears to also run under a truly empty environment. Additionally, attempts to overwrite the "PWD" environment variable with a dictionary entry are not successful under gdb.debug().
This fix brings the gdb.debug() treatment of environment variables in line with and process() and gdb.debug() with NOPTRACE enabled.
It allows both empty environments, and overwriting of the 'PWD' environment variable.
Both Debian and Ubuntu provide a `binutils-avr` package, which contains
an assembler for AVR as `avr-as`. However, pwntools was looking for
something with `linux` in the name, such as `avr-linux-gnu-as`. Of course,
such a target triplet is nonsense, Linux does not run on AVRs
(apart from that one hack that actually emulated some other CPU which
therefore doesn't count).
* Disable corefile tests for Github Actions
* Disable the UI tests, as theyre broken
* Install gdbserver
* Rename step
* Remove SIGALARM to see if it fixes tests
* Set PWNLIB_NOTERM for command-line runs
`ELF.functions` wasn't updated after the binary base address was changed using `ELF.address` like `ELF.symbols` is. The documentation mentions `ELF.functions` getting updated as well, the update was missing though.
Closes#1512
Co-authored-by: Arusekk <[email protected]>
* Update Corefile warnings and replace asserts with normal checks
This means that most of the Corefile will still work, even if
we encounter issues while parsing data from the stack.
* Update pwnlib/elf/corefile.py
* Add missing return statement
Co-authored-by: Arusekk <[email protected]>
Detect if pwntools is running in a Windows Subsystem for Linux "VM" and set the terminal to open a new `cmd.exe` window on the host.
This replaces the need to add `context.terminal = ['cmd.exe', '/c', 'start', 'ubuntu', 'run']` to every script.
Debugging using `gdb.attach` under WSL with gdb in a new window out of the box!
This is a WONTFIX, but default gcc on ubuntu compiles .bss (and other RW
segments) so far away from .text that ret2dlresolve mathematically
just has to segfault unless .text and .rodata contain crafted structures
already.
Compiling and working fine on gentoo, though. Hope this proves useful.
Also closes#1448 by the way, it was really annoying.
Closes#1476.
* Enable text colors in Windows console
Consoles on Windows 10 support ANSI escape sequences including color codes.
Emulate unix curses `tigetstr` terminal capabilities by hardcoding the given escape sequences for used curses capabilities.
ANSI escape sequences have to be enabled using `kernel32.SetConsoleMode(hndl, ENABLE_VIRTUAL_TERMINAL_PROCESSING)` for the current console.
* Use object identity operator
* Import ABC from collections.abc for Python 3.
* Fix deprecation warnings due to invalid escape sequences.
* Use raw strings instead of escaping individually.
Previously, the rop._srop_call function would append the args without checking whether their value would work
directly in a register. It now uses AppendedArgument to determine whether the value is pushed to the end of
the ROP chain.
* Update proc.py
reversed([p.pid for p in processes]) only return a iter.
if use gdb.attach('some process name') will cause error like :
File "/home/tsj/.local/lib/python2.7/site-packages/pwnlib/gdb.py", line 657, in attach
pid = pids[0]
TypeError: 'listreverseiterator' object has no attribute '__getitem__'
In this case ,it treat 'pids' as a list,so here should return a list as well
* Apply suggestions from code review
Co-authored-by: Arusekk <[email protected]>
if the value returned from the leak function is a str, it converts it to a bytearray for pytohn3 compatibility. Assuming a string is returned is flaky and likely wrong.
removed the double quotes around the temp filename of the gdbscript as this broke the gdbscript functionality.
The doublequotes would be considered part of the filename and it would fail to read the file..
Tested this fix on Kali.
* The SSH tube now keeps the environment variables of processes it creates in order.
The way of achieving that is through passing the env to the script as a list of tuples,
since Python3's dictionaries are ordered by default, or you can use `OrderedDict`,
the result would be that the spawned process on the remote machine will keep the environment variables in order.
This is very important to exploits that are based on the stack's order and especially the environment variables.
For instance, a common challenge is comparing argc == 0, and afterwards accessing the argv array, as an out-of-bounds array.
The shellcode needs to be at a certain index consistently and cannot rely on the stack moving around.
* Relocated the code.
* Try GitHub actions instead of Travis CI.
This is just a proof-of-concept until we can
have it the clean way it should be done.
More commits will likely follow.
* Travis CI: Use flake8 to detect syntax errors (#1385)
* Travis CI: Use flake8 to detect syntax errors
Repeat #1305 on the Python 3 branch. Also, remove the hardcoding of default values for dist and Py2.
https://flake8.pycqa.org/en/latest/user/error-codes.html
On the flake8 test selection, this PR does _not_ focus on "_style violations_" (the majority of flake8 error codes that [__psf/black__](https://github.com/psf/black) can autocorrect). Instead these tests are focus on runtime safety and correctness:
* E9 tests are about Python syntax errors usually raised because flake8 can not build an Abstract Syntax Tree (AST). Often these issues are a sign of unused code or code that has not been ported to Python 3. These would be compile-time errors in a compiled language but in a dynamic language like Python they result in the script halting/crashing on the user.
* F63 tests are usually about the confusion between identity and equality in Python. Use ==/!= to compare str, bytes, and int literals is the classic case. These are areas where __a == b__ is True but __a is b__ is False (or vice versa). Python >= 3.8 will raise SyntaxWarnings on these instances.
* F7 tests logic errors and syntax errors in type hints
* F82 tests are almost always _undefined names_ which are usually a sign of a typo, missing imports, or code that has not been ported to Python 3. These also would be compile-time errors in a compiled language but in Python a __NameError__ is raised which will halt/crash the script on the user.
* Fail fast, exclude generated source files
* --select=E9,F63,F7
* 2to3 -f print -wn examples
* No more <> in python 3.
* Use ==/!= to compare str, bytes, and int literals (#1386)
Identity is not the same thing as equality in Python so use ==/!= to compare str, bytes, and int literals. In Python >= 3.8, these instances will raise _SyntaxWarnings_ so it is best to fix them now. https://docs.python.org/3.8/whatsnew/3.8.html#porting-to-python-3-8
$ python
```python
>>> raw = 'ra'
>>> raw += 'w'
>>> raw == 'raw'
True
>>> raw is 'raw'
False
```
* Use print() function in both Python 2 and Python 3 (#1387)
$ __pip install black future isort__
$ __futurize -f libfuturize.fixes.fix_print_with_import -w android.py__
$ __isort android.py__
$ __black android.py__
* Add cache and building artifacts
* Enable caching
* Actually enable caching for all of it
* Mistake
* Try different ABI on CI
* Try different ABI + version
* Try it harder
* Update SDK version
* Try building using newer SDK
* Accept licenses
* Ugh android emulator documentation is unfriendly
* Environment variables for AVD
* Add QEMU-system for android virtual devices
* Use the correct emulator
* New android image
* Was actually too new image
* Update ssh credentials to reflect GHA setup
* Fix spelling
* Try finding endless loops
* Actually use the timeout
* Reduce possible hang surface in doctests
* Try SSH immediately when set up
* SSH strict host key checking disable for localhost
* Try stackoverflow.com research
* Try to fix failing doctests
* Add required gcc for cross-arch tests
* Ensure gcc not failing in fmtstr
* Fix issues with shellcraft
* Try different libs
* Try again, with verbosity
* Why do these conflict???
* Fix issues related to PIE binaries
* Add bash-static
* Verbose 'could not set transport: any'
* Add AT_SECURE in auxv to prevent SEGV on libc bug
* Killing ADB server actually kills it
* Android image has /data/user/0 -> /data/data
* Set coverage mode from dev
* Try new actions
* Fix coveralls complaining
* A little time.sleep would not hurt or would it
Co-authored-by: Christian Clauss <[email protected]>
* Allow pwntools to load on Windows
Fixes loading of pwntools on Windows. `from pwn import *` works.
It allows to use some operating system agnostic features of pwntools out of the box.
This is inspired by #996.
Some features fail or even crash (local processes, gdb, assemble/disassemble)
and console colors are still not displayed correctly, but those can be addressed separately.
* Use `os.devnull` instead of manual selection
* Add heuristic to find return address into __libc_start_main
Find the return address from main into __libc_start_main in a libc shared binary. Mimic's niklasb's heuristic in libc-database.
* Add support for different `call` instructions
* Introduce tests for pwnlib.ui
This architecture can be changed later, but I'm quite proud
of how it's done currently.
* Try to debug CI
* Ugly hack to get results back
* Let's just keep it half-broken
Currently, python 3 installed libraries
and sphinx doctree cache are not retained across builds.
This PR addresses this issue, hopefully resulting in a major speedup.
* Travis CI: Use flake8 to detect syntax errors
Repeat #1305 on the Python 3 branch. Also, remove the hardcoding of default values for dist and Py2.
https://flake8.pycqa.org/en/latest/user/error-codes.html
On the flake8 test selection, this PR does _not_ focus on "_style violations_" (the majority of flake8 error codes that [__psf/black__](https://github.com/psf/black) can autocorrect). Instead these tests are focus on runtime safety and correctness:
* E9 tests are about Python syntax errors usually raised because flake8 can not build an Abstract Syntax Tree (AST). Often these issues are a sign of unused code or code that has not been ported to Python 3. These would be compile-time errors in a compiled language but in a dynamic language like Python they result in the script halting/crashing on the user.
* F63 tests are usually about the confusion between identity and equality in Python. Use ==/!= to compare str, bytes, and int literals is the classic case. These are areas where __a == b__ is True but __a is b__ is False (or vice versa). Python >= 3.8 will raise SyntaxWarnings on these instances.
* F7 tests logic errors and syntax errors in type hints
* F82 tests are almost always _undefined names_ which are usually a sign of a typo, missing imports, or code that has not been ported to Python 3. These also would be compile-time errors in a compiled language but in Python a __NameError__ is raised which will halt/crash the script on the user.
* Fail fast, exclude generated source files
* --select=E9,F63,F7
* 2to3 -f print -wn examples
* No more <> in python 3.
* Adding support for FileStructure exploitation
* Actually run the tests + basic py3 fixes
also rename the module to lower case to keep consistency
* Python 3 working
Identity is not the same thing as equality in Python so use ==/!= to compare str, bytes, and int literals. In Python >= 3.8, these instances will raise _SyntaxWarnings_ so it is best to fix them now. https://docs.python.org/3.8/whatsnew/3.8.html#porting-to-python-3-8
$ python
```python
>>> raw = 'ra'
>>> raw += 'w'
>>> raw == 'raw'
True
>>> raw is 'raw'
False
```
* feat(fmtstr): place pointers after format operations
This means that the format string can now also work in certain
cases even if the pointers contain null bytes.
That's especially important on 64 bit where most pointers contain null bytes.
* fix(fmtstr): more stuff
* feat(fmtstr): optimize
* perf(fmtstr): make sorting run in reasonable time (not perfect yet)
* perf(fmtstr): be faster
* style(fmtstr): more docs, some cleanup
* fix(fmtstr): small fixes + more docs
* style(fmtstr): cleanup small style issues
* fix(fmtstr): fix handling of szmax in merge_atoms_overlapping
* fix(fmtstr): fix Atom __getitem__ for out-of-bounds index
* refactor(fmtstr): docs and cleaner code
* fix(fmtstr): small test/doc fixes
* tests(fmtstr): add example as tests for other arches
* doc(fmtstr): more docstrings
* fix(fmtstr): fix cyclic_find warnings
* refactor(fmtstr): remove another nested function
* tests(fmtstr): fix doctest output for wrong arch
* style(fmtstr): remove unused imports
* tests(fmtstr): add binaries for tests
* Updates to requirements.txt.
Removed the < 1.8.0 requirement for sphinx. It appeared no longer
necessary.
Alphabetized the listing for easier searching.
Moved the requirements file under the docker folder. Docker does not
like relative paths to parent directories due to security restrictions.
* Updated Dockerfile to use Python3.
* Fixes to srop test errors.
Moving the imports to the beginning of the file resolved the issue.
The following errors were popping up during testing within Docker:
Document: rop/srop
------------------
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.recvline()
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.5/doctest.py", line 1321, in __run
compileflags, 1), test.globs)
File "<doctest default[18]>", line 1, in <module>
p.recvline()
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 467, in recvline
return self.recvuntil(self.newline, drop = not keepends, timeout = timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 310, in recvuntil
res = self.recv(timeout=self.timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 82, in recv
return self._recv(numb, timeout) or b''
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 160, in _recv
if not self.buffer and not self._fillbuffer(timeout):
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 131, in _fillbuffer
data = self.recv_raw(self.buffer.get_fill_size())
File "/home/travis/pwntools/pwnlib/tubes/process.py", line 707, in recv_raw
raise EOFError
EOFError
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.poll(block=True)
Expected:
0
Got:
-11
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.recvline()
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.5/doctest.py", line 1321, in __run
compileflags, 1), test.globs)
File "<doctest default[18]>", line 1, in <module>
p.recvline()
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 467, in recvline
return self.recvuntil(self.newline, drop = not keepends, timeout = timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 310, in recvuntil
res = self.recv(timeout=self.timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 82, in recv
return self._recv(numb, timeout) or b''
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 160, in _recv
if not self.buffer and not self._fillbuffer(timeout):
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 131, in _fillbuffer
data = self.recv_raw(self.buffer.get_fill_size())
File "/home/travis/pwntools/pwnlib/tubes/process.py", line 707, in recv_raw
raise EOFError
EOFError
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.poll(block=True)
Expected:
0
Got:
-11
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.recvline()
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.5/doctest.py", line 1321, in __run
compileflags, 1), test.globs)
File "<doctest default[18]>", line 1, in <module>
p.recvline()
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 467, in recvline
return self.recvuntil(self.newline, drop = not keepends, timeout = timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 310, in recvuntil
res = self.recv(timeout=self.timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 82, in recv
return self._recv(numb, timeout) or b''
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 160, in _recv
if not self.buffer and not self._fillbuffer(timeout):
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 131, in _fillbuffer
data = self.recv_raw(self.buffer.get_fill_size())
File "/home/travis/pwntools/pwnlib/tubes/process.py", line 707, in recv_raw
raise EOFError
EOFError
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.poll(block=True)
Expected:
0
Got:
-11
**********************************************************************
1 items had failures:
6 of 159 in default
159 tests in 1 items.
153 passed and 6 failed.
***Test Failed*** 6 failures.
* Fix for docstring warning in dynelf.py.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/dynelf.py:docstring of pwnlib.dynelf.DynELF._resolve_symbol_sysv:11: WARNING: Definition list ends without a blank line; unexpected unindent.
* Fix for docstring warnings in gdb.py.
The following warnings are resolved with this fix:
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_assembly:8: WARNING: Inline strong start-string without end-string.
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_assembly:21: WARNING: Block quote ends without a blank line; unexpected unindent.
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_shellcode:4: WARNING: Inline strong start-string without end-string.
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_shellcode:17: WARNING: Block quote ends without a blank line; unexpected unindent.
* Fix for adb/__init__.py.
The following error is resolved with this fix:
Document: protocols/adb
-----------------------
**********************************************************************
File "../../pwnlib/protocols/adb/__init__.py", line ?, in default
Failed example:
pprint(AdbClient().list('/data/user'))
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.5/doctest.py", line 1321, in __run
compileflags, 1), test.globs)
File "<doctest default[0]>", line 1, in <module>
pprint(AdbClient().list('/data/user'))
TypeError: 'module' object is not callable
**********************************************************************
1 items had failures:
1 of 10 in default
10 tests in 1 items.
9 passed and 1 failed.
***Test Failed*** 1 failures.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/protocols/adb/__init__.py:docstring of pwnlib.protocols.adb.AdbClient.stat:6: WARNING: Inline literal start-string without end-string.
* Fix for docstring warning in rop.py.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/rop/rop.py:docstring of pwnlib.rop.rop.DescriptiveStack.descriptions:3: WARNING: Inline literal start-string without end-string.
* Fix for docstring warning in ssh.py.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/tubes/ssh.py:docstring of pwnlib.tubes.ssh.ssh._init_remote_platform_info:6: WARNING: Block quote ends without a blank line; unexpected unindent.
* Doctest dictionary ordering problem in misc.py.
There was an ordering problem while running the doctest in misc.py.
Sorting the dictionary with pprint and updating the results fixed the
issue.
Document: util/misc
-------------------
**********************************************************************
File "../../pwnlib/util/misc.py", line ?, in default
Failed example:
sizes
Expected:
{'eax': 32, 'ax': 16, 'al': 8, 'ah': 8, 'ebx': 32, 'bx': 16, 'bl': 8, 'bh': 8, 'ecx': 32, 'cx': 16, 'cl': 8, 'ch': 8, 'edx': 32, 'dx': 16, 'dl': 8, 'dh': 8, 'edi': 32, 'di': 16, 'esi': 32, 'si': 16, 'ebp': 32, 'bp': 16, 'esp': 32, 'sp': 16}
Got:
{'si': 16, 'cx': 16, 'cl': 8, 'dl': 8, 'dh': 8, 'eax': 32, 'bx': 16, 'sp': 16, 'bl': 8, 'bh': 8, 'di': 16, 'esi': 32, 'edx': 32, 'dx': 16, 'ebx': 32, 'ebp': 32, 'ax': 16, 'ch': 8, 'al': 8, 'ah': 8, 'edi': 32, 'esp': 32, 'bp': 16, 'ecx': 32}
**********************************************************************
File "../../pwnlib/util/misc.py", line ?, in default
Failed example:
bigger
Expected:
{'eax': ['eax'], 'ax': ['eax', 'ax'], 'al': ['eax', 'ax', 'al'], 'ah': ['eax', 'ax', 'ah'], 'ebx': ['ebx'], 'bx': ['ebx', 'bx'], 'bl': ['ebx', 'bx', 'bl'], 'bh': ['ebx', 'bx', 'bh'], 'ecx': ['ecx'], 'cx': ['ecx', 'cx'], 'cl': ['ecx', 'cx', 'cl'], 'ch': ['ecx', 'cx', 'ch'], 'edx': ['edx'], 'dx': ['edx', 'dx'], 'dl': ['edx', 'dx', 'dl'], 'dh': ['edx', 'dx', 'dh'], 'edi': ['edi'], 'di': ['edi', 'di'], 'esi': ['esi'], 'si': ['esi', 'si'], 'ebp': ['ebp'], 'bp': ['ebp', 'bp'], 'esp': ['esp'], 'sp': ['esp', 'sp']}
Got:
{'si': ['esi', 'si'], 'cx': ['ecx', 'cx'], 'cl': ['ecx', 'cx', 'cl'], 'dl': ['edx', 'dx', 'dl'], 'dh': ['edx', 'dx', 'dh'], 'eax': ['eax'], 'bx': ['ebx', 'bx'], 'sp': ['esp', 'sp'], 'bl': ['ebx', 'bx', 'bl'], 'bh': ['ebx', 'bx', 'bh'], 'di': ['edi', 'di'], 'esi': ['esi'], 'edx': ['edx'], 'dx': ['edx', 'dx'], 'ebx': ['ebx'], 'ebp': ['ebp'], 'ax': ['eax', 'ax'], 'ch': ['ecx', 'cx', 'ch'], 'al': ['eax', 'ax', 'al'], 'ah': ['eax', 'ax', 'ah'], 'edi': ['edi'], 'esp': ['esp'], 'bp': ['ebp', 'bp'], 'ecx': ['ecx']}
**********************************************************************
File "../../pwnlib/util/misc.py", line ?, in default
Failed example:
smaller
Expected:
{'eax': ['ax', 'al', 'ah'], 'ax': ['al', 'ah'], 'al': [], 'ah': [], 'ebx': ['bx', 'bl', 'bh'], 'bx': ['bl', 'bh'], 'bl': [], 'bh': [], 'ecx': ['cx', 'cl', 'ch'], 'cx': ['cl', 'ch'], 'cl': [], 'ch': [], 'edx': ['dx', 'dl', 'dh'], 'dx': ['dl', 'dh'], 'dl': [], 'dh': [], 'edi': ['di'], 'di': [], 'esi': ['si'], 'si': [], 'ebp': ['bp'], 'bp': [], 'esp': ['sp'], 'sp': []}
Got:
{'si': [], 'cx': ['cl', 'ch'], 'cl': [], 'dl': [], 'dh': [], 'eax': ['ax', 'al', 'ah'], 'bx': ['bl', 'bh'], 'sp': [], 'bl': [], 'bh': [], 'di': [], 'esi': ['si'], 'edx': ['dx', 'dl', 'dh'], 'dx': ['dl', 'dh'], 'ebx': ['bx', 'bl', 'bh'], 'ebp': ['bp'], 'ax': ['al', 'ah'], 'ch': [], 'al': [], 'ah': [], 'edi': ['di'], 'esp': ['sp'], 'bp': [], 'ecx': ['cx', 'cl', 'ch']}
**********************************************************************
1 items had failures:
3 of 20 in default
20 tests in 1 items.
17 passed and 3 failed.
***Test Failed*** 3 failures.
* Fix for docstring warning in packing.py.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/util/packing.py:docstring of pwnlib.util.packing.flat:2: WARNING: Inline emphasis start-string without end-string.
* Removed commented out test in Dockerfile.
* Forgot to update location of requirements.txt.
Forgot to update the location of requirements.txt in .travis.yml.
* Made recommended changes from Arusekk.
pwnlib.shellcraft installs its custom import finder (LazyImporter) into
sys.meta_path.
According to PEP 302, finder is expected to have prototype
finder.find_module(fullname, path=None)
Note that the second argument is optional. pwnlib's LazyImporter has second
argument required, which leads to compatibility problems, surfacing
when trying to use pwnlib with other third-party modules:
>>> import pwnlib
>>> import faker
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/tmp/.venv/lib/python2.7/site-packages/faker/__init__.py", line 2, in <module>
from faker.factory import Factory # noqa F401
File "/tmp/.venv/lib/python2.7/site-packages/faker/factory.py", line 12, in <module>
from faker.config import AVAILABLE_LOCALES, DEFAULT_LOCALE, PROVIDERS
File "/tmp/.venv/lib/python2.7/site-packages/faker/config.py", line 15, in <module>
AVAILABLE_LOCALES = find_available_locales(PROVIDERS)
File "/tmp/.venv/lib/python2.7/site-packages/faker/utils/loading.py", line 45, in find_available_locales
provider_module = import_module(provider_path)
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/tmp/.venv/lib/python2.7/site-packages/faker/providers/internet/__init__.py", line 4, in <module>
from text_unidecode import unidecode
File "/tmp/.venv/lib/python2.7/site-packages/text_unidecode/__init__.py", line 6, in <module>
_replaces = pkgutil.get_data(__name__, 'data.bin').decode('utf8').split('\x00')
File "/usr/lib64/python2.7/pkgutil.py", line 576, in get_data
loader = get_loader(package)
File "/usr/lib64/python2.7/pkgutil.py", line 462, in get_loader
return find_loader(fullname)
File "/usr/lib64/python2.7/pkgutil.py", line 473, in find_loader
loader = importer.find_module(fullname)
TypeError: find_module() takes exactly 3 arguments (2 given)
* Removed old dependency for intervaltree and sortedcontainers
Also make the pwn command show usage if no arguments supplied
* Fix errno subcommand
The errno library moved out of os and into its own library
* Change template script to use system env python instead of only py2
* Require updated intervaltree in setup.py
* Update .travis.yml
* Update .travis.yml
* Tubes fixed
* Typo
* Typo + default value
* Fixes on and on
* Many many more
* Missing lines and tests
* Refactoring
* Missing line again
* Missing again
* Copypaste design patterns
Use ==/!= to compare str, bytes, and int literals: __fmt is not 'raw'__ --> __fmt != 'raw'__
$ __python__
```python
>>> raw = 'ra'
>>> raw += 'w'
>>> raw
'raw'
>>> raw == 'raw'
True
>>> raw is 'raw'
False
```
coredumpctl is a tool that can be used to retrieve and process
core dumps and metadata which were saved by systemd-coredump(8).
systemd-coredump works by setting a piped core pattern. For example:
$ cat /proc/sys/kernel/core_pattern
|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e
This commit detects `systemd-coredump` in the pattern and then uses
`coredumpctl`, which is the CLI utility for reading the coredumps.
`coredumpctl` is instructed to write the coredump to a file. The
coredump is specified using the process ID and the basename of the
executable.
This commit also updates some doc strings to fix them and make them
more useful.
This fixes#1177, #1178
Ubuntu 16.04 finally includes `binutils-mips-linux-gnu`,
which is required. So there is no need to download
packages manually anymore. This can help supporting
python 3.
* corefile: Update NT_XXX type names for pyelftools==0.25
* travis: Do not use Sphinx version 1.8.0, it fails with recursive errors
* corefile: Fix stack detection and Mapping.__contains__ logic
* corefile: Better detection of AT_EXECFN when NT_AUXV does not have it
* corefile: Avoid trying to parse corrupt stacks
* Fix doctests for new warning
* log: Fix warning_once caching
* corefile: also expose page_offset
This is useful to load the code from the original files.
* Undo constructor field reordering
* Fix ordering in Mapping instantiation
* Commas are important
* corefile: also put page_offset as last parameter
* Improve pwnup template
* Don't want to break at main again if its been hit already
* Always start the program
* Handle extra newline printed by gdb
* Remove moved recvline
* Remove controversial set sysroot
* Update gdb.py
* Update pwnup.mako
* fix wrong gadget selection in setRegisters() (#1118)
* Fix detection logic
* Update rop.py
* add test cases for filtering out the bad instructions in rop.py
* Replace division with floor division as of PEP-238
This will also get us closer to python3.
See https://www.python.org/dev/peps/pep-0238/
* Add from __future__ import division to source files
* Fix tests relying on broken autocontrib version (#1121)
* Removed tuple parameters unpacking due to PEP-3113
As a step towards python3 compatibility, removed tuple parameters
in :mod:`pwnlib.term.term`. They are now replaced with * unpacking.
See https://www.python.org/dev/peps/pep-3113/
Check for sha256 and sha256sum before trying openssl sha256 as "openssl sha256" doesn't return an error code in some old versions of openssl (OpenSSL 0.9.8g 19 Oct 2007 at least) that doesn't support sha256.
* Add sysroot= argument to gdb.debug, expose via 'pwn debug'
Also sets a sane default sysroot for Android debugging
* Add missing newline for "set deatch-on-fork off"
Introduced in e89a9b66ac
* Add support for extracting Android boot.img
* Fix incorrect variable in 'pwn debug'
* Ensure "file /path/to/file" is the first command
When one passes a `LD_PRELOAD` which cannot be preloaded the `process_created` string might end up looking like this:
```
"ERROR: ld.so: object '/bin/bash' from LD_PRELOAD cannot be preloaded (cannot dynamically load executable): ignored.\n"
```
When we end up in such situation, pwndbg shown unintuitive error:
```
$ python -c 'from pwn import *; gdb.debug("/bin/ls", env={"LD_PRELOAD": "/bin/bash"})'
[+] Starting local process '/usr/bin/gdbserver': pid 20691
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "pwnlib/context/__init__.py", line 1349, in setter
return function(*a, **kw)
File "pwnlib/gdb.py", line 397, in debug
port = _gdbserver_port(gdbserver, ssh)
File "pwnlib/gdb.py", line 234, in _gdbserver_port
gdbserver.pid = int(process_created.split()[-1], 0)
ValueError: invalid literal for int() with base 0: 'ignored.'
```
After this patch it will show:
```
$ python -c 'from pwn import *; gdb.debug("/bin/ls", env={"LD_PRELOAD": "/bin/bash"})'
[+] Starting local process '/usr/bin/gdbserver': pid 20863
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "pwnlib/context/__init__.py", line 1349, in setter
return function(*a, **kw)
File "pwnlib/gdb.py", line 402, in debug
port = _gdbserver_port(gdbserver, ssh)
File "pwnlib/gdb.py", line 236, in _gdbserver_port
'Failed to spawn process under gdbserver. gdbserver error message: %s' % process_created
ValueError: Failed to spawn process under gdbserver. gdbserver error message: ERROR: ld.so: object '/bin/bash' from LD_PRELOAD cannot be preloaded (cannot dynamically load executable): ignored.
[*] Stopped process '/bin/ls' (pid 20863)
```
As @bennofs noticed, our detection logic was incorrect:
"This is not a good test if I understand the issue correctly.
From my tests, it seems that fault_addr was only null before this patch if it
would've been >= 2**47.
This is because on 64 bit, that's where kernel memory is mapped so we get
"protection fault" instead of "segfault".
A protection fault does not have the same siginfo as a normal segfault:
si_code will be SI_KERNEL = 0x80 and there won't be a fault_addr
(see http://elixir.free-electrons.com/linux/v4.14-rc8/source/kernel/signal.c#L1052)"
* Handle debugging under Android better with 'pwn debug'
Previously, when using --process or --pid, we would terminate the gdbserver instance
when pwntools exited. Now we wait for the user to tell us to exit.
* Handle launching Android processes by filename
* Use the --multi option with gdbserver; disable detach-on-fork
* Intentionally break doctest to make sure its running
* Revert "Intentionally break doctest to make sure its running"
This reverts commit 00578ad177.
* Copy env in process._validate using .copy method to keep the order in case of OrderedDict
* Use supplied env object to preserve order in case of OrderedDict
* Fix corefile environment variables and speed up ELF.string()
* Even more doctests
* Fix doctest arch
* No longer split on = since it isn't necessary
* Fix arch again
* Delete intermediate corefiles
* Do not flag for extended testing when the commit SHA is what matches
* Add support for finding QEMU's LD_PREFIX
* Refactor pwnlib.qemu and add official documentation
* Add more documentation, support for sysroot
* More docs, binfmt info
* Fix variable name overload
* Fix missed refactor
* Check for empty/null env
* Add qemu.rst to index
* Add cyclic default args to context
Specifically, this adds context.cyclic_alphabet and context.cyclic_size
This helps cut down on writing "n=8" all the time for users who use 8-byte
subsequences on 64-bit architectures.
Exposing the default alphabet means that users can avoid "bad bytes" in
cyclic patterns which are generated by pwntools internally (e.g. ROP padding)
Some pwnlib.util.packing docstrings were changed to clarify that word_size is
in bits.
The doctests in pwnlib.util.cyclic were updated and extended to have more
and better doctests, as well as better exaples.
* Add an example showing an alphabet with NUL bytes
This is important because old code would use pack(..., "all") when an integer
value was provided, instead of using the correct width.
* Overhaul ROP.setRegisters to support more complex cases needed by AMD64
Additionally, have a separate cache for ROPs composed of multiple ELF files
* Better presentation for stack padding slots
* Remove unnecessary stack adjustment on amd64
For amd64, it is possible that len(stackArguments) == 0.
* Add a doctest for amd64 register dependency stuff
* More i386 adjust optimizations
* Use nextGadgetAddr as the adjust value
* Always use padding for the last "ret" adjustment
* Better dump display
* Update doctests for new ROP dump() enhancements
* Add badchars support for ROP gadgets
* More doctest fixes, also ensure ROP.search does not return gadgets ending in e.g. int 0x80
* Slightly enhance template and advertise for the feature itself in scripts
Add single-dash to the "very good" characters in shell escaping, to avoid
the need to single-quote things like --host and --port.
* Add --quiet option for pwn template, move gdbscript down a bit
* Slightly better docs for 'pwn template'
* Add checksec information directly to 'pwn template' output
* Prevent Sphinx from dumping all level=INFO logging output to stdout.
See Gallopsled/pwntools#1036 for more information.
* Filter out Paramiko logs too, apply filter to all root handlers during testing
* Remove unused import
Technically this introduces issues with e.g. 'unhex ffdf | python foo.py' if
foo.py uses .interactive(). However, that was already broken for other reasons.
Fixes#1004
* fix typo in comment
* new config option: context.gdbinit
This allows setting the default gdbinit for GDB, which is helpful when you want
to enable some GDB extension only when using pwntools and not by default.
- Detect bootloaders that are already unklocked
- Detect bootloader unlock success
- Detect bootloaders that cannot be unlocked
- Detect devices which have not enabled bootloader unlocking
2017-07-10 15:54:22 -05:00
770 changed files with 14306 additions and 6020 deletions
When reporting an issue, be sure that you are running the latest released version of pwntools (`pip install --upgrade pwntools`).
## Debug Output
Having the extra debug output really helps us, and might help you diagnose the problem yourself.
When submitting an issue that has output from Pwntools, make sure to run your script as shown below, to enable the extra debugging data.
```sh
$ python exploit.py DEBUG LOG_FILE=log.txt
```
You should see `[DEBUG]` statements that show what's happening behind the scenes:
```
[+] Starting local process '/bin/sh' argv=['sh'] : pid 16823
[DEBUG] Sent 0x7 bytes:
'whoami\n'
[DEBUG] Sent 0x5 bytes:
'exit\n'
[+] Receiving all data: Done (11B)
[DEBUG] Received 0xb bytes:
'crashheap\n'
[*] Process '/bin/sh' stopped with exit code 0 (pid 16823)
```
## Verify on Ubuntu
If possible, please verify that your issue occurs on 64-bit Ubuntu 18.04. We provide a Dockerfile based on Ubuntu 18.04 via `docker.io` to make this super simple, no VM required!
Thanks for using Pwntools! If you have a question, here are some tips that will make it easier for us to help you.
## Check the Documentation
Pwntools has REALLY extensive documentation. Please check there first! https://docs.pwntools.com
## Debug Output
If you're having an issue with an exploit, run your script with full debugging output, i.e. `python exploit.py DEBUG`. This may help you to see what's happening under the hood, and the debug output will help to diagnose any issues you're encountering.
- [#1242][1242] Use IntervalTree 2.xx, disallow use of 3.xx
- [#1243][1243] Fix a typo that caused an exception when executing a binary with `process()` which returns `-ENOEXEC` and the system does not have `qemu-user` binaries installed.
- [#1083][1083] Better error messages for `gdb` when `LD_PRELOAD` is incorrect
- [#1085][1085] Add support for extracting Android `BOOTLDR!` images
- [#1075][1075] Add support for detecting GNU Screen for `run_in_new_terminal`
- [#1074][1074] Add support for running `pwntools-gdb` wrapper script instead of `gdb`
- [#1068][1068] Work around very old OpenSSL versions which don't have sha256 support *AND* don't exit with an error code when trying to use it
- [#1067][1067] Add `pwnlib.tubes.server` module, which adds a reusable `server` listener
- [#1063][1063] Add support for labels in `fit()`, allowing dynamic contents to be injected. (This feature is really cool, check out the pull request!)
+ Much better support for 64-bit Intel (amd64) ROP
+ ROP gadget selection is optimized to favor multi-pops instead of multiple single-pop gadgets
+ Added support for blacklisting byte values in ROP gadget addresses
- [#1049][1049] Enhancements to `cyclic`
+ `context` now has two additional attributes, `cyclic_alphabet` and `cyclic_length`, which correspond to the arguments `alphabet` and `n` to `cyclic()` and `cyclic_find()` and related routines.
+ The motivation for this change is to allow setting the `alphabet` globally, so that any padding / patterns generated internally to pwntools can be controlled. The specific motivation is blacklisting values in ROP padding.
- [#1052][1052] Enhancements for detecting `QEMU_LD_PREFIX` used by QEMU user-mode emulation for sysroots
- [#1035][1035] Minor documentation changes
- [#1032][1032] Enhancements to `pwn template`
- [#1031][1031] More accurate `Coredump.fault_addr` on amd64
- [#1084][1084] Fix broken tests due to `ftp.debian.org` going down
Pwntools is a CTF framework and exploit development library. Written in Python, it is designed for rapid prototyping and development, and intended to make exploit writing as simple as possible.
@@ -20,27 +21,25 @@ r.send(asm(shellcraft.sh()))
r.interactive()
```
# Try It Now!
You can now do a live demo of Pwntools, [right in your browser](https://demo.pwntools.com).
# Documentation
Our documentation is available at [docs.pwntools.com](https://docs.pwntools.com/)
A series of tutorials is also [available online](https://github.com/Gallopsled/pwntools-tutorial#readme)
To get you started, we've provided some example solutions for past CTF challenges in our [write-ups repository](https://github.com/Gallopsled/pwntools-write-ups).
# Installation
Pwntools is best supported on 64-bit Ubuntu LTE releases (12.04, 14.04, and 16.04). Most functionality should work on any Posix-like distribution (Debian, Arch, FreeBSD, OSX, etc.). Python 2.7 is required.
Pwntools is best supported on 64-bit Ubuntu LTE releases (14.04, 16.04, 18.04, and 20.04). Most functionality should work on any Posix-like distribution (Debian, Arch, FreeBSD, OSX, etc.). Python >= 2.7 is required (Python 3 suggested as best).
Most of the functionality of pwntools is self-contained and Python-only. You should be able to get running quickly with
However, some of the features (assembling/disassembling foreign architectures) require non-Python dependencies. For more information, see the [complete installation instructions here](https://docs.pwntools.com/en/stable/install.html).
Many settings in ``pwntools`` are controlled via the global variable :data:`.context, such as the selected target operating system, architecture, and bit-width.
Many settings in ``pwntools`` are controlled via the global variable :data:`.context`, such as the selected target operating system, architecture, and bit-width.
In general, exploits will start with something like:
@@ -8,6 +8,7 @@ This module also fixes a the issue that exceptions raised by an exit handler is
printed twice when the standard :mod:`atexit` is used.
"""
from__future__importabsolute_import
from__future__importdivision
importsys
importthreading
Loaded 100 of 770 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Arusekk