* Test on Python 3.15
* Use `RLIM_INFINITY` instead of `-1` constant for rlimit modification
Python 3.15 warns against using negative values instead of the constant:
DeprecationWarning: Use RLIM_INFINITY instead of negative limit value.
It has been available since Python 2.
* Update CHANGELOG
* Support paginated libc.rip API requests
libc.rip used to truncate the result set but supports pagination now. Fetch all results reliably now by requesting all pages.
Fixes#2651
* Update CHANGELOG
The nginx docs say that you have to specify the $request_uri manually when a variable is used in the `proxy_pass` value.
Since we just put `/` at the end, all requests were rewritten to `/` instead of the intended e.g. `/api/find` endpoints.
https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass
This broke in #2718 but didn't manifest right away since the responses were still cached in CI. Now after the cache expired the fetch failed.
* ssh: Discard stderr of `lsb_release`
The "No LSB modules are available." spam interferes with parsing the output for the initial checksec distro parsing when connecting to a new host.
* Update CHANGELOG
Use nginx variable pattern (set $upstream + proxy_pass https://$upstream)
to defer DNS resolution until first request instead of at config parse time.
This fixes the occasional 'host not found in upstream' error in CI when
nginx container starts before Docker DNS is fully available.
Fixes#2712
Instead of trying to resolve all of them when nginx starts and exit
if any of them fail to resolve.
[emerg] 1#1: host not found in upstream "debuginfod.elfutils.org" in /etc/nginx/nginx.conf:23
The reponses set a cloudflare `_cfuvid` cookie and had a cache time of 60 seconds.
nginx doesn't cache responses with a `Set-Cookie` header by default,
so ignore the cookie and override the cache time to 12 weeks.
Fixes#2494
* Close SSH client when authentication failed
The TCP connection stayed open after an exception was thrown
during authentication, even when a `with`-statement guarding the call.
Close the connection before re-raising the exception.
Fixes#2678
* Update CHANGELOG
* Fix `tty` and `raw` arguments in ssh.process
The arguments were unused and not passed through to the underlying `ssh_process` class.
Fixes#2038
* Add tty tests
* Update CHANGELOG
* Fix buffered process environment output without tty
* asm: try native binutils before fallback architectures
Some binutils builds support multiple architectures,
like the x86_64 assembler supports i386 too in most
package builds and vice versa. We support this by trying
the other binutils as well when looking for the right tool.
That support is optional though and there are packages
that don't include the "cross" compilation.
When assembling for the host's native architecture, we
still prefer the explicitly named `x86_64-linux-gnu-as`
instead of the native `as` binary. That native binary is
tried as a last resort after looking for any other explictly
named binary. If the fallback package is installed but
compiled without the cross-compiler support, we'd still
find the `i386-as` binary while looking for `x86_64-as`
before trying the native `as` binary.
That would fail to assemble.
This patch moves the native `as` binary forward in the list
to try it before checking the other fallback architecture ones.
Fixes#2509
* Update CHANGELOG
It looks like this was done to work around a bug in an old pip version years ago.
It caused problems when freezing dependencies which unnecessarily forced pip to the version used at freeze time when installing again.
Fixes#2625
* Fix `log.progress` ignoring `context.log_console`
When setting `context.log_console` to something other than stdout, spinners while using `log.progress` would still be printed to stdout.
Redirect them to the stream specified by `context.log_console` and don't animate if it's not a tty.
Fixes#1702
* Update CHANGELOG
* ROP: fix `self.leave is None` in some libc
When `pop rbp; pop rsp; ret;` exists in libc, it is chosen instead of
`leave`. Add a specific `order` to filter out the former one.
* add changelog for #2506
* ROP: add sp_move when `pop rsp`
* CHANGELOG: mv #2506 to `5.0.0`
I often use `checksec *` to lazily avoid typing filenames in a directory.
If the directory contains any other sub-dirs, the command fails.
With this patch, checksec will silently skip dir paths. There's still
TOCTOU issue but I don't think checksec do anything important enough
to explicitly use try/catch to account for that.
Co-authored-by: peace-maker <[email protected]>
* Windows: Raise EOFError during process.recv when stdout closes
If the receiving thread terminates while we're waiting for data to arrive we'd be waiting endlessly. Raise EOFError when .recv()ing on a process with stdout closed and no more data queued up from the receiver thread.
* Update CHANGELOG
* fix(regsort): drop self-assignments through register aliases
The self-assignment drop compared register names only (k != v), so an
assignment through an alias (e.g. {'ax': 'eax'} with registers.native32)
was not recognized as a no-op. It then surfaced as a spurious
1-register dependency cycle, which either raised PwnlibException with
xchg=False or emitted a useless 'mov' of a register into its own alias.
With xchg=False and other registers present, the bogus cycle also
forced the temporary-register code path, reordering unrelated moves and
silently leaving registers with wrong values (see the linked issue).
Compare the mapped names instead, like the rest of the function.
Fixes#2794
Co-authored-by: rahul05ranjan <[email protected]>
* docs(changelog): add regsort register-alias self-assignment fix entry
---------
Co-authored-by: Rahul Ranjan <[email protected]>
* gdb.corefile: Run gdb directly to prevent detach race
`attach()` waits for the debugger to attach by polling the `TracerPid` regularly. `corefile()` runs `detach` in gdb after dumping the core, so the debugger might attach and detach between two `TracerPid` polls and `wait_for_debugger()` thinks gdb exited before attaching and returns `None` as gdb's pid is no longer valid.
Later `corefile()` ran `psutil.Process(pid).wait()` with `pid = None` which effectively waited on the Python process itself to exit.
This caused a hang in the doctest CI sometimes, which was killed by the watchdog timer eventually.
Run `gdb` directly and just observe the exit code instead of relying on the more advanced `attach` logic for longer running debug sessions.
* Update CHANGELOG
* fix(packing): dd() raises TypeError on a bytearray destination
dd() documents bytearray as a supported destination type, but
dd(bytearray(b'Hello!'), b'?', skip=5) raises
"TypeError: 'bytes' object cannot be interpreted as an integer".
pwnlib/util/packing.py:1108 handled list and bytearray destinations in a
single branch and filled both with list(map(p8, bytearray(src))). p8()
returns a length-1 bytes object, which is what a list destination is
documented to hold (see the existing dd(list('Hello!'), (63,), skip=5)
doctest), but a bytearray slice assignment needs an iterable of int, so
every bytearray destination crashed.
Assign the byte string src directly when the destination is a bytearray.
Every src branch above the destination dispatch (packing.py:1039-1089)
has already reduced src to a byte string, so the slice assignment is
always valid. This also makes a bytearray destination agree with the
bytes and str destinations, which already copy src verbatim.
Add a doctest covering the returned value and the in-place update of the
caller's bytearray.
* Add changelog entry for #2772
---------
Co-authored-by: peace-maker <[email protected]>
* Disable debuginfod tests in libcdb.unstrip_libc
The external debuginfod servers are really slow or won't respond at all like debuginfod.ubuntu.com.
The caching in CI doesn't help since we only cache it for 3 month and the query fails afterwards most of the time. We cache the negative response too which doesn't help if the service is flakey itself :D
(cherry picked from commit 86cafce196)
* Add changelog entry for #2773
---------
Co-authored-by: Peace-Maker <[email protected]>
* Allow running with Unicorn 2.1.[34] but throw when emulating MIPS
Unicorn 2.1.5 wasn't released for a while and 2.1.2 doesn't have binary wheels for Python 3.14.
This made installing pwntools on newer Python versions more annoying since you had to build Unicorn yourself.
The only problem with the blocked versions is MIPS emulation, so only avoid that while allowing to use Unicorn 2.1.3 or 2.1.4 for other architectures.
Refs #2621
* Update CHANGELOG
* Fix _build_date() crash on non-standard locale date strings (#2513)
Some Android devices return locale-dependent strings in ro.build.date
(e.g. '2017年 11月 14日 星期二 09:55:07 CST' for Chinese locale) that
dateutil.parser.parse() cannot handle, causing wait_for_device() to
crash with a ParserError.
Fix: prefer ro.build.date.utc (integer epoch timestamp) which is
locale-independent and available on all Android devices. Fall back
to ro.build.date with error handling if the UTC property is missing.
* Add CHANGELOG entry for #2513 fix
* Simplify _build_date(): prefer ro.build.date.utc, keep minimal fallback
Address review: ro.build.date.utc is set by the AOSP build system
and available on all standard Android devices. Simplified comments
to reflect this is the primary path, with ro.build.date kept only
as a fallback for non-standard builds.
* Move CHANGELOG entry
* Fix Python 2.7 compatibility
* Disable intro doctest connecting to `ftp.ubuntu.com`
The FTP server is throttled in CI or just really slow. We already test the basic tcp tubes in the `remote` tests.
---------
Co-authored-by: coder999999999 <[email protected]>
Co-authored-by: peace-maker <[email protected]>
* fix: pad bytes fields to correct field size in FileStructure
When setting bytes fields (e.g. _IO_save_base) on FileStructure, the
value was padded to context.bytes instead of the actual field size from
self.length[val]. This caused struct.pack errors for fields whose size
differs from the pointer size.
Fixes#2693
* Cleanup test and add CHANGELOG
---------
Co-authored-by: nightcityblade <[email protected]>
Co-authored-by: Peace-Maker <[email protected]>
* Fix regression: gdbserver can't handle command-line argument containing whitespace
The `--no-startup-with-shell` argument was passed in an attempt to
fix gdbserver startup with `SHELL=/bin/fish`. (#2377) It caused a regression
which disallowed passing arguments containing whitespace to the debuggee in `gdb.debug`.
gdbserver would exit with a message:
"can't handle command-line argument containing whitespace"
There is a 3 second timeout waiting for gdbserver to startup and
printing a message with a workaround already suggesting to set
`SHELL=/bin/bash` to allow gdbserver to launch. This has to be
our best effort to guide users of such shells until we find time
to investigate a better solution.
Fixes#2550
* Update CHANGELOG
* Fix lookup of binutils for `mipsel` architecture
The endianness is encoded in the architecture name for mips targets which wasn't respected while looking for the correct binutils binary.
Sync the CI and Dockerfile binutils packages while we're at it.
Fixes#2591
* Update CHANGELOG
* still fall back to the other endianness
---------
Co-authored-by: Arusekk <[email protected]>
We could try to lookup the mapped libraries before they are actually loaded in the process.
Wait until the program is up and running before trying to fetch the memory maps.
* Fix log level of child remotes of `server` tube
The log level of the `server` tube wasn't passed
on to the `remote` wrappers created upon receiving
a new child connection. This caused the connection
closed message to be logged even when the server
object was created with `level='error'`.
* Update CHANGELOG
When trying multiple debuginfod servers, the first one to fail would save the failed attempt and block the remaining servers to be tried.
Only save that we couldn't find the debuginfo if we tried all the configured servers.
* Ignore colored_traceback error when TERM is unset
Fixes using pwntools in non-interactive sage scripts with newer colored_traceback versions.
Fixes#2594
* Update CHANGELOG
* Support longer function names in Windows getexport shellcode
Previously, an arbitrary name length limit of 8 characters was in place due to the lack of a proper strcmp loop. Implement a cheap strcmp for longer function names.
This trashes a bunch of registers and we might be able to optimize it further to reduce the damage.
Use a unique label too to avoid redefinition when using the shellcraft template twice in the same shellcode. Use the mov shellcraft template for the destination register to avoid an instruction if the result is already in the desired register.
* Update CHANGELOG
* Add Ubuntu and Debian debuginfod servers
Most CTF challenges are hosted on Ubuntu or Debian [citation needed], so check their debuginfod servers before trying the federating one as a fallback.
This is required since the elfutils.org server seems down for maintenance occasionally.
* Update CHANGELOG
* Add automatic detection of statically linked binaries in `detect_missing_binaries` function.
* Add entry for PR #2570 to CHANGELOG.md: fix `pwn template` for statically linked binaries.
* Remove partial test implementation
The commands should be tested end-to-end instead of using doctests (later).
---------
Co-authored-by: Peace-Maker <[email protected]>
For example, on my machine:
> ./main_no_flag
./main_no_flag: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by ./main_no_flag)
Co-authored-by: Arusekk <[email protected]>
Enable DOT_ALL next to the default MULTILINE regex flags to match newlines too with the `.*?`.
```
/* this works */
/*
this is not highlighted as a comment
*/
```
Lets say we have an ELF with the following symbols
```
Symbol table '.symtab' contains 5 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000000035 0 NOTYPE LOCAL DEFAULT 2 aaaa
2: 0000000000000022 0 NOTYPE LOCAL DEFAULT 2 bbbb
3: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 2 cccc
4: 0000000000000054 0 NOTYPE GLOBAL DEFAULT 2 dddd
```
Then pwnlib's ELF(binary).symbols will be `{'aaaa': 53, 'bbbb': 34, 'dddd': 84}`.
This is missing the symbol `cccc`, because its value is 0.
This change checks the name instead of the value, because the value can
be 0 if the symbol points to the beginning.
The new and correct value of pwnlib's ELF(binary).symbols will be
`{'aaaa': 53, 'cccc': 0, 'bbbb': 34, 'dddd': 84}`.
* Do not unresolve zeros
* Do not add undefined symbols
Co-authored-by: peace-maker <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* Fix installation on Python 3.5 and lower on stable
The unix-ar package requires Python 3.6. Only install it on Python 3.6 then and use the Python 2 fallback for Python <= 3.5 too.
* Update CHANGELOG
* Replace 'keepends' with 'drop'
Add 'drop' argument in place of 'keepends' for tubes. 'keepends' still
works but raises depreciation warning.
* changelog
* Ignore line numbers in PyLint check
---------
Co-authored-by: peace-maker <[email protected]>
There were additional `timeout=X` additions in #2382 which caused tests to fail randomly when a timeout was reached. The tests run through occationally, so it's not an infinite loop. But flakey tests are annoying and I don't see a reason for the timeout additions to the doctests for the core patch of that PR other than failing early during manual tests of new functionality. CI running smoothly is more important than fast failing during development on the gdb module imo.
* Fix sphinx warnings in docstrings
* Update sphinx for Python 3.13 support
Could not import extension sphinx.builders.epub3 (exception: No module named 'imghdr')
`imghdr` was removed in Python 3.13.
* Update fmtstr.py documentation
Clarified that the `value` in the `writes` dict (for `fmtstr_payload()`) should be a bytestring if the value to be written is shorter than a long
* Update fmtstr.py documentation for FmtStr class
Updated example and definition for `write()` function in `FmtStr` class to include bytes as the value.
* Fix code formatting
---------
Co-authored-by: Peace-Maker <[email protected]>
* Add `tube.upload_manually`
Upload data in chunks when having a tube connected to a shell.
This is useful when doing kernel or qemu challenges where you can't use the ssh tube's file upload features.
* Add tests and fix corner cases
* Update CHANGELOG
* Fix character escaping in tests
* Fix gzip compression on Python 2
* Patch for #issues/2290
* Return listening process on `pidof(("0.0.0.0", 1234))`
Instead of returning the process which is connected to port 1234,
return the process which is listening on that port.
* Update CHANGELOG
---------
Co-authored-by: Peace-Maker <[email protected]>
This reverts #2346 which apparently didn't do what was expected. The logger has to be initialized for the checksec output to show up, so some other file imported * from pwn before too.
With the double registration of subparsers removed in f28d3ebb6c this didn't happen anymore making checksec print nothing.
* Allow to disable caching
By setting `context.cache_dir = None`. Generate default cache dir again by setting `context.cache_dir = True`.
* Disable cache in `asm` caching test
* Update CHANGELOG
* Fix asm cache test
* Throw error when using `sni` and setting `server_hostname` manually in `remote`
Prevents silently replacing the `server_hostname` that was provided in `ssl_args`.
Fixes#2425
* Update CHANGELOG
* Bump tested python version to >= 3.10
Some workflows were using 3.8 still. Start testing on 3.12 too.
* Fix installing rpyc in CI on ubuntu 24.04
`ubuntu-latest` now points to 24.04 which requires the --break-system-packages dance.
* Test Python 2 on Ubuntu 22.04
Ubuntu 24.04 dropped the python2.7 packages.
* Test on android-34
* Fix ARM binutils disassembly output test expectation
It used to print "; 0x4" but changed to "@ 0x4" in some version.
* Skip QEMU LD_PREFIX path test
Ubuntu 24.04 seemed to have switched the qemu-user --help output to show `/usr/gnemul/qemu-$ARCH` instead of `/etc/qemu-binfmt`. Ignore the actual path in the test.
* Fix x86 ascii shellcode encoder test
The alphabet was using `\` escape sequences badly.
<stdin>:1: SyntaxWarning: invalid escape sequence '\]'
* Fix mips xor encoder unaligned memory access
qemu throws a SIGBUS error when the shellcode tries to access unaligned memory since some version. Align the "stack" properly.
* Fix util.lists.partition test output on Python 3.12
The __repr__ output of OrderedDict was changed to look like ordinary {} dicts.
* Fix safeeval test on Python 3.12
There is a 151 RESUME opcode in Python 3.12 now.
* Fix registering commandline subparsers multiple times for Python 3.11
Python 3.11 added a sanity check to argparse to prevent registering the same subparser multiple times.
argparse.ArgumentError: argument command: conflicting subparser: cyclic
Avoid importing the command twice.
`python -m pwnlib.commandline.cyclic` failed on Python 3.11 triggered in CI when running the commandline tools while collecting coverage.
* Fix pip cache in CI
* Fix returning wrong pid in WSL for run_in_new_terminal
We run cmd.exe which and other Windows processes before going back into WSL. The returned pid would be the one for the ephemeral cmd.exe process instead of the real command we wanted to launch.
I don't know how to properly trace the execution and get the command pid, so scan for newer pids matching the command for a while instead as a workaround.
We wrap the command in a script so psutil.Process.exe() returns the path to the shell instead of the real command. Look at psutil.Process.cmdline() too which contains the real program running right now.
* Make `proc.wait_for_debugger` return the debugger pid
If we fail to get the pid when launching gdb, grab it after tracing the debugger at least. gdb won't be closed when the exploit exits but at least we have the correct pid.
* Fix working directory of run_in_new_terminal in WSL
The process' cwd would be %WINDIR% due to cmd.exe not supporting WSL paths.
* Update CHANGELOG
The upload-artifact action v4.4.0 excluded hidden files from uploads by default. We want to upload only hidden `.coverage*` files though. Enable hidden files again using the `include-hidden-files` input.
https://github.com/actions/upload-artifact/releases/tag/v4.4.0
* Add ELF.close() to release resources
`ELFFile.close()` just closes the mmap'd file, but our own wrapper keeps the file handle open.
This is annoying when using a temporary file on Windows since they can't be deleted if there is
a dangling handle. This can be used together with the inherited context manager from ELFFile.
* Update CHANGELOG
* Fix parsing the PLT on Windows
The Unicorn Engine 1GB workaround doesn't work since `mmap` has different semantics on Windows.
* Update CHANGELOG
* Catch ImportError instead
* Remove unused sys import
* Cache output of `asm()`
To speed up repeated runs of an exploit, cache the assembled output.
Use a sha1 hash of the shellcode as well as relevant context values
like `context.arch` and `context.bits` to see if the exact same
shellcode was assembled for the same context before.
Fixes#2312
* Return path to cache file if `not extract`
* Update CHANGELOG
* Create temporary copy of cached file
* Add debug log about using the cache
* Include full assembler and linker commandlines in hash
This should catch any changes across pwntools updates and system environment changes.
* Include pwntools version in hash
* Support asm/disasm on Windows
Find binutils toolchains like https://gnutoolchains.com/ if they exist in the PATH.
* Update CHANGELOG
* Update binutils install docs
* cpp reads from stdin when no infile is given
No need for a tempfile here.
On Ubuntu 22.04 gdb.debug() takes at least 2 seconds. This is because
it prints only "Remote debugging from host 127.0.0.1, port 33398", but
the code expects 2 lines.
It's very unclear what the second line is supposed to be; the only lead
is "* Handle extra newline printed by gdb" in commit 338fbeba ("Improve
pwnup template, gdbserver detection (#1148)"), but I could not trace it
back to the GDB source code, both historic and modern. Perhaps, there
is a non-upstreamed patch in some distro that introduces it.
It should still be safe to skip waiting for the second line if the
first one already starts with "Remote debugging ...", so do it.
* added two arguments to gdb.debug:
+ port: specifies the port that should be used by the gdbserver (this is useful for alpine)
+ gdb_args: allows forwarding arguments to the gdb binary spawned by gdb.attach() (just a passthrough)
aslo downgraded "GDB Python API is supported only for local processes" to a warning, because it seems to work on some systems.
* added changelog entry and fixed python2.7 compatibility
* also added optional gdbserver_args parameter to gdb.debug()
* implemented requested fixes for #2382:
+ gdb.debug port is now also used for qemu
+ GDB Python API is now tested for tubes.process.process a warning for ssh.process and an error for everything else
+ updated docs to use mention that gdbserver ports are randomized by default
+ now using gdbserver_port to check if the correct port was set
+ fixed CHANGELOG.md structure
* implemented requested fixes for #2382:
+ gdb.debug port is now also used for qemu
+ GDB Python API is now tested for tubes.process.process a warning for ssh.process and an error for everything else
+ updated docs to use mention that gdbserver ports are randomized by default
+ now using gdbserver_port to check if the correct port was set
+ fixed CHANGELOG.md structure
+ added timeouts for recvline() gdb tests
+ also run gdb api tests for ssh runner
---------
Co-authored-by: gfelber <@users.noreply.github.com>
Co-authored-by: peace-maker <[email protected]>
* Fix access of undefined `ssh._tried_sftp` in `raw` mode
The attribute was never set when setting `ssh(raw=True)` which caused the code to run `ssh.__getattr__` which raised an AttributeError due to `_tried_sftp` starting with an `_` on Python 3.12.
* Document all `ssh` attributes
Only add it when we don't want to manipulate the environment or argv[0].
This will cause the problem with fish shell to reappear in those cases, but
it's another step for compatibility at least.
#2378
* Return buffered data on first EOF in tube.readline()
When there is still data available in the tube buffer when an EOFError occurs in
`tube.recvline()`, return that data even though it doesn't contain a newline.
The next time `tube.recvline()` is called afterwards will raise EOFError normally.
This behavior is in line with the GNU readline implementation and avoids
loss of data. It allows `tube.stream()` to print everything that's received before
the receiving end terminates.
A new warning is logged when data is returned due to an EOF informing
about the lack of the trailing newline character.
Fixes#2366
* Update CHANGELOG
* Add context.throw_eof_on_incomplete_line
Allow to control the behavior of `tube.recvline` and
be able to suppress the new warning.
* Cleanup docs
* Don't go though a shell in `gdb.debug`
gdbserver starts a shell and runs the target process through it.
This behavior was added in gdbserver 8.0 together with the
commandline flag --no-startup-with-shell to disable it.
* Update CHANGELOG
* Recompute text.has_bright when num_colors changes
The number of simultaneous colors in the current terminal are queried
later during initialization and the text.num_colors attribute is updated accordingly.
The text.has_bright attribute wasn't updated when that happened and always
remained set to False. Calculate those properties based on num_colors dynamically.
* Update CHANGELOG
* Enable bright colors on Windows too
Only get "colors" on Windows to avoid #1201 again.
* Fix support for old terminals on windows
* Change from pop to keys for reporting proper error message
* update changelog
* add doctest for xor
* Add newline to run xor doctests
* Switch to `dict.popitem()`
---------
Co-authored-by: marinelay <[email protected]>
Co-authored-by: Peace-Maker <[email protected]>
* Added Missing MacOS Requirements
* Updated Binutils Install Docs
Due to security reasons Homebrew has removed direct formula references, this is the workaround.
* Updated changelog for #2392
* reverting changelog
* Change newline when setting `context.os`
Windows uses `b'\r\n'` for newlines while unix uses `b'\n'`. Change the expected newline to `b'\r\n'` when setting `context.os = "windows"` automatically for convenience.
* Update CHANGELOG
* Fix other tests setting os
* Add basic RISCV64 shellcraft support
The `mov` template isn't 100% null-byte and newline free
for all inputs. Certain larger values are just emitted using
the `li` pseudo-instruction by the assembler which might
contain null-bytes.
Co-authored-by: LevitatingLion <[email protected]>
* Generate RISCV syscall constants from musl-libc
diet-libc doesn't support RISCV, so borrow from musl.
Need to consider their license.
* Update CHANGELOG
* Fix CHANGELOG
---------
Co-authored-by: LevitatingLion <[email protected]>
* Add basic support to debug processes on Windows
Currently only `windbg.debug()` and `windbg.attach()` are implemented,
which open a WinDbg instance and attach to the process.
* Update CHANGELOG
* Cleanup CheckRemoteDebuggerPresent call
Only require PROCESS_QUERY_INFORMATION access and check for errors when opening the process.
* process.close: Move closing of std fds after kill
Windows processes would block on fd.close() when the main thread is suspended.
using the "--no-cache-dir" flag in pip install, make sure downloaded packages by pip don't cache on the system. This is a best practice that makes sure to fetch from a repo instead of using a local cached one. Further, in the case of Docker Containers, by restricting caching, we can reduce image size. In terms of stats, it depends upon the number of python packages multiplied by their respective size. e.g for heavy packages with a lot of dependencies it reduces a lot by don't cache pip packages.
Further, more detailed information can be found at
https://medium.com/sciforce/strategies-of-docker-images-optimization-2ca9cc5719b6
Signed-off-by: Pratik Raj <[email protected]>
* Add local libc database provider for libcdb
* Remove unnecessary assert
* Add docstring for local_libcdb
* Suppress warning output while `context.libdb` sets default
* Testing the local system's libc first
* Set falsely `context.lcoal_libcdb` to turn off local libc-database integration
* Fix docstring
* Make path check in validator
* Fix doctests
* Add CHANGELOG
* Fix gdb.debug doctests
* Accept bytes in ssh.which
But still return a `str` return for backwards-compatibility.
* CI: Make rpyc visible in gdb's python
* Fix not running all doctests
They appear to have to be a free standing block of `>>>` with an empty line above.
* Remove unnecessary text type check in ssh.which
* Bugfix gdb.debug: exe parameter now respected
This commit now properly supports the exe parameter in
`pwnlib/gdb.py:debug()`, allowing a different argv[0] than the
executable.
It achieves this by leveraging the gdbsever`--wrapper` argument
with a python script that calls execve with the specified args.
* Backintegration for python2
* Update Changelog.md
* pwnlib gdb.py, try to pass these tests...
* Encode script in ssh.process(run=False) for tests
* Re-trigger Pull request tests
* gdb.py: easier script if argv[0] == exe
* gdb.py: Add test case for LD_Preload
* Add check that "=" not in misc.normalize_argv_env
This check checks prevents the use of "=" in the
key of an environment variable, which is generally
impossible.
* gdb.py Correct handling of LD_ env-variables
* Update pwnlib/gdb.py
Co-authored-by: peace-maker <[email protected]>
* gdb.py address comments
1. explicit ctypes.CDLL('libc.so.6'), handle execve failing
2. consistent namedTempFile
3. drop packing._encode() since it's done earlier
4. testcases solve argv-args confusion
* gdb.py: Fix Namedtempfile-prefix + Bugfix
* Restore prefix for gdbscript
* Unify execve wrapper script under one function
* gdb.py, Remove leftover script
* Fix logging scope and ignore_environ argument
---------
Co-authored-by: Youheng Lü <[email protected]>
Co-authored-by: peace-maker <[email protected]>
Co-authored-by: peace-maker <[email protected]>
* Fix readline omitting a trailing \n
This is a regression from #2129 which was based on a bogus test which mixed `input` and `readline` calls.
Only `input` (and `raw_input`) are documented to strip the newline, but `readline` itself should include it.
Fixes#2342
* Update CHANGELOG
* Fix Unicorn Engine 1GB limit exit(): raise OSError instead (Fixes#2343)
This commit fixes#2343: an issue where `pwn checksec <binary>` would fail with a bogus error of:
```
$ pwn checksec /root/x/bin/main
Could not allocate dynamic translator buffer
```
This actually comes from Unicorn Engine which tries to allocate a 1GB RWX mapping:
```
root@pwndbg:~# strace -e openat,mmap pwn checksec /root/x/bin/main 2>&1 | tail -n 10
openat(AT_FDCWD, "/usr/lib/python3/dist-packages/mpmath-0.0.0.egg-info/PKG-INFO", O_RDONLY|O_CLOEXEC) = 7
openat(AT_FDCWD, "/usr/local/lib/python3.10/dist-packages/unicorn/lib/libunicorn.so.2", O_RDONLY|O_CLOEXEC) = 7
mmap(NULL, 22447520, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 7, 0) = 0x7f2604f9d000
mmap(0x7f2605339000, 13496320, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x39c000) = 0x7f2605339000
mmap(0x7f2606018000, 3039232, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x107b000) = 0x7f2606018000
mmap(0x7f26062fe000, 1601536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x1360000) = 0x7f26062fe000
mmap(0x7f2606485000, 525728, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2606485000
mmap(NULL, 1073741824, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
Could not allocate dynamic translator buffer
+++ exited with 1 +++
```
...and if it fails, it calls `exit()`. This can be seen in Unicorn Engine code: https://github.com/unicorn-engine/unicorn/blob/56f3bdedb42d26bee1532cc01baf5eaf44a9aa23/qemu/accel/tcg/translate-all.c#L960-L963
This issue has been reported to Unicorn Engine in https://github.com/unicorn-engine/unicorn/issues/1766 but since it hasn't been fixed, this commit applies a workaround for it.
* CI: add test for pwn checksec with 500MB limit for UE 1GB limit
* Add changelog entry
* Update .github/workflows/ci.yml
---------
Co-authored-by: peace-maker <[email protected]>
* Add a `flatten` argument to `ssh.libs`
This option makes us able to avoid getting the file tree of the remote
server and just downloads the desired files in the output folder
* Update changelog
* Add a warning in ssh.libs for duplicate filenames
* Add documentation in ssh.libs for duplicate filenames fallback
* [ssh.libs] Fix flatten's duplicate check
Fix this: https://github.com/Gallopsled/pwntools/pull/2268#discussion_r1325404026
* [ssh.libs] Add better warning for flatten's duplicates
Fix this: https://github.com/Gallopsled/pwntools/pull/2268#discussion_r1325405234
* [ssh.libs] Add doctests
This commit adds:
- The `no_duplicate` binary that depends on
- `a/lib.so`
- `b/lib2.so`
- The `duplicate` binary that depends on
- `a/lib.so`
- `b/lib.so`
Then, the doctest tries to pull the libs from both binaries with
different `flatten` values.
* [ssh.libs] Add `remote` arg to documentation
* [ssh.libs] Remove fallback to unflattened when flatten fails
* [ssh.libs] Fix doctests
* [ssh.libs] Import pwnlib.data.elf.ssh_libs in pwnlib.data.elf
* [ssh.libs] Fix corrupted binaries
* [ssh.libs] We actually need to set the cwd since libs paths are relative
* [ssh.libs] Fix permission issues for CI
* [ssh.libs] Fix doctest
* Remove doctest attempts
---------
Co-authored-by: peace-maker <[email protected]>
Co-authored-by: Arusekk <[email protected]>
Co-authored-by: Peace-Maker <[email protected]>
* Optimized dynelf when using an ELF object
* Fixed minor bugs, added a new way to lookup symbols
Improved _rel_lookup and added Elf64_Rel datatype
Added support for _rel_lookup in x86 binaries
* Spelling fix
* Converted prints to python2 style
+ changelog entry
* Fixed a bug with real leaker not being resotred
---------
Co-authored-by: Bl4ckC4t <[email protected]>
* fix: split current iterm window during gdb.debug process
* add change to changelog
* escape cmd before writing osascript file
* use the previously sanitized command to run the osascript
---------
Co-authored-by: peace-maker <[email protected]>
* Fix pwn constgrep when it matches a non-constant type
This commit fixes the following issue:
```
root@pwndbg:~# pwn constgrep a
Traceback (most recent call last):
File "/usr/local/bin/pwn", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.10/dist-packages/pwnlib/commandline/main.py", line 58, in main
commands[args.command](args)
File "/usr/local/lib/python3.10/dist-packages/pwnlib/commandline/constgrep.py", line 110, in main
for _, k in sorted(out):
TypeError: '<' not supported between instances of 'Constant' and 'type'
```
Note that it was caused because of the following type object being matched and fetched from the module object:
```
ipdb> out[25:27]
[(Constant('CS', 0xd), 'CS'), (<class 'pwnlib.constants.constant.Constant'>, 'Constant')]
ipdb> sorted(out[24:27])
*** TypeError: '<' not supported between instances of 'type' and 'Constant'
```
* Add test for `pwn constgrep C` to the CI
* Add changelog entry
* Add `ELF.stripped` and `ELF.debuginfo`
Show status in checksec output too.
* Update CHANGELOG
* Show stripped and debuginfo status in red in checksec
* Match against local system libc first in libcdb
Don't do any requests if the libc currently in use on the system
running the exploit matches already. This is a small short circuit
optimization when the remote target uses the same libc as the
local one.
This looks at the libc loaded by the local shell binary. This appears
more dynamic than hardcoding library paths.
Refs #983
* Update CHANGELOG
* Handle missing SHELL envvar
* Fix hash lookup
* Retry failed lookups after one week in libcdb
The libc databases might be updated to include the searched version,
so a request that failed once might work in the future.
Refs #983
* Update CHANGELOG
Visual Studio Code sets the $TERM_PROGRAM environment variable to `"vscode"` in its terminal pane,
but doesn't support opening new panes from the command line and there is no "vscode" binary.
Make sure the target binary exists before trying to launch it.
* Explicitly define p64/u64 functions for IDE support
Instead of dynamically generating the packing and unpacking helper functions using `setattr(module, name, routine)` at runtime, unroll the loop and explicitly declare all 8 helpers.
This allows IDEs to know about the function statically without running the code.
Fixes #1657
* Update CHANGELOG
* Fix u16 comment
* Lookup process executable using PATHEXT file extensions
Windows uses some file extensions to determine which file to run when given
a non-existing file.
This allows to run `calc` instead of `calc.exe`.
* Try PATHEXT in `which` directly
* Update CHANGELOG
* Add support to start a process on Windows
This is heavily inspired by https://github.com/masthoon/pwintools
Allows to use the `process` tube on a Windows host running local
processes. I've tried to keep the changes at a minimum. Windows doesn't
support non-blocking reads, so I've opted to handle the reading in a
separate thread to simulate the non-blocking check.
* Fix `misc.which()` on Windows
`os.getuid()` is only supported on unix.
* Support [read|write]mem, libs and cwd
* Update CHANGELOG
* Don't cache loaded modules
* Wait output on stdout in `process.can_recv_raw`
Instead of handling the timeout for reading poorly in `process.recv_raw`,
wait for any bytes to be available in `process.can_recv_raw` like on linux.
This prevents a 100% spin of one core in `tube.interactive()` since it
expects the thread to block for `timeout` seconds while we would return
immediately previously.
* Remove PythonForWindows dependency
This loses the `readmem` and `writemem` feature, but that's not
necessary for this basic process startup support.
* Fix cyclic imports
* Ignore errors in reading thread
This could cause a deadlock / fastfail on shutdown when stdout is closed, thus read(1) fails
and the exception is tried to be printed to the console while stderr is in the process of getting
closed.
* Make pwn template always set context.binary
* Update CHANGELOG.md
* Allow args.EXE override in all templates
* Fall back to old behavior for remote only challenges
---------
Co-authored-by: peace-maker <[email protected]>
* Bump unicorn dependency to >=2.0.1
We're using riscv constants now which were added in unicorn2.
Fixes#2303
* Add RISCV64 plt emulation test binaries
* Detect challenge binary and libc in `pwn template`
When the `args.exe` or `args.libc` arguments are missing, look for the
files in the current working directory.
This allows to quickly get a template by just running `pwn template`
(and maybe --host --port for remote instances).
This is inspired by https://github.com/io12/pwninit
* Update CHANGELOG
* Add --no-auto argument to disable automatic detection
Allow to keep the old static template behavior.
* Add logging about detection outcome
Inform the user that we're looking for the binaries and if we found one.
* Be more strict about libc filename detection
Check about common name patterns instead of plainly the presence of "libc" in the name.
Co-authored-by: Arusekk <[email protected]>
* Change --no-auto argument to reduce double negation
Co-authored-by: Arusekk <[email protected]>
---------
Co-authored-by: Arusekk <[email protected]>
When you set the `context.terminal` to be tmux related but then try
to run the exploit outside a tmux session, you'd get a cryptic ValueError
while trying to parse the pid of the new pane in tmux.
Print a nicer reminder to start tmux first.
* term rework
* towards floating cells
* redraw when screen clears
* term: working completion
* Display non-printable characters
* ui: fix options with new term
* term: Clean up and fix examples/spinners
* changelog and lint
* Fix SSH in NOTERM mode
* defer termios setup
* term: py2 compat
* py2: compat fixup
* appease pylint
* fix scroll on winch
* term: fix hang on winch during readline
* next fails if string is not found in binary
fixes#2306
* Only display "str_bin_sh" symbol if it's available
* Update CHANGELOG
---------
Co-authored-by: Peace-Maker <[email protected]>
* Add env_add to the ssh module
* Use ignore_environ like process
---------
Co-authored-by: peace-maker <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* Add env_add to the process module
* Don't mutate passed `env` parameter dictionary
Co-authored-by: Arusekk <[email protected]>
* Update CHANGELOG
* Use a boolean flag instead of env_add
There should be preferably only one obvious way to pass env.
* Update CHANGELOG
---------
Co-authored-by: peace-maker <[email protected]>
Co-authored-by: peace-maker <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* libcdb.py - python 3.12
Python 3.12 would complain if the \d atom is not escaped in the binary string.
/usr/lib/python3.12/site-packages/pwnlib/commandline/libcdb.py:224: SyntaxWarning: invalid escape sequence '\d'
libc_version = re.search(b'libc[ -](\d+\.\d+)', exe.data)
* Use raw string
---------
Co-authored-by: peace-maker <[email protected]>
So far only relocations pointing to external symbols were considered
when filling GOT. Relocations in libc are a different thing, often
found in CTF challenges (libc with partial RELRO has many interesting
overridable function pointers, like strlen & strchrnul).
Co-authored-by: Michał Radwański <[email protected]>
* Add x86 CET status to checksec output
The ELF class now features two properties:
- ibt for indirect branch tracking `-fcf-protection=branch`
- shstk for shadow stack `-fcf-protection=return`
Both checking the presence of the respective feature bit in the
note property x86 feature segment.
* Check if user shadowstack is enabled on SSH target
Look for "user_shstk" in /proc/cpuinfo for ssh.checksec()
* Add ELF.iter_notes and ELF.iter_properties helpers
Allows to easily iterate the PT_NOTE segment and the properties in "GNU"
notes.
* Check for ibt on remote ssh machine
* Update CHANGELOG
* Suppress error when /proc/cpuinfo cannot be read
* Python 2: Fix installing from source
`python2.7 -m pip install git+https://github.com...` doesn't install
any packages.
The auto-discovery doesn't work for the old setuptools version used in
Python 2.
https://setuptools.pypa.io/en/latest/userguide/package_discovery.html#flat-layout
* Use Python 3 commandline tools in Docker images
Since the Python 2 version was installed first, the Python 3 install
didn't touch the existing shell script wrapper files.
* Fix tube.clean_and_log not logging buffered data
Only new data received after the `clean_and_log` call is printed while
data already received and buffered earlier is not.
Print the buffered data first before waiting for more so we don't miss
anything.
* Only show buffered data if DEBUG is off
* Update CHANGELOG.md
---------
Co-authored-by: Arusekk <[email protected]>
* Fix bug at ssh.py:process() - empty argv[0] Error
Before this, process.py relied on `os.execve` which disallows an empty
argv or argv[0] == "".
This commit replaces `os.execve` with `ctypes.CDLL(None).execve` to
call the C-Library function directly which allows an empty argv.
* Add #2225 to stable changelog
* Better ctypes syntax
* Add error message if cytpes.execve fails.
* Updata CHANGELOG.md
* ssh.py: python2 compatibility for os.environb
* Add check that "=" not in misc.normalize_argv_env
This check checks prevents the use of "=" in the
key of an environment variable, which is generally
impossible.
* ssh.process: Seperate cases for empty argv[0]
This commit seperates the cases for an empty argv[0]
from normal cases.
* ssh.py delete leftover comment
---------
Co-authored-by: Youheng Lü <[email protected]>
Co-authored-by: Arusekk <[email protected]>
Co-authored-by: peace-maker <[email protected]>
* Fix corefile module after pyelftools update
The `bytes2str` internal method was moved while Python 2 support was
removed in pyelftools 0.3.0.
Just use our own python aware decoder wrapper.
Fixes#2260
* Pin pyelftools < 0.3 for Python 2
* Fix pyelftools version number scheme
* Update CHANGELOG
* Fix pushing docker images on manual workflow run
Running the docker workflow manually would associate the run with the
`dev` branch. The generated tags would all point to the `dev` tag, so
tag the image manually to the correct branch.
* Fix building ci image
It's called `python2.7` instead of `python`.
* Add riscv64 binutils to base image
* Fix ssh.process not setting ssh_process.cwd
The cwd wasn't passed to the ssh_process created in ssh.process,
so the ssh_process.cwd attribute was never set.
This passes the cwd along now.
```
s = ssh(..)
io = s.process("pwd", cwd="/tmp")
io.cwd # "."
io.recvall() # b"/tmp\n"
```
* Update changelog
* Add ELF.set_runpath()
Shells out to the `patchelf` tool to patch the ELF's RUNPATH. This lets the dynamic loader look for needed shared libraries in the given path first before the system libraries when running the binary.
* Add ELF.set_interpreter()
Shells out to the `patchelf` tool to patch the ELF's PT_INTERP segment. This allows to change the ld.so used when running the binary.
* Add convenience wrapper to set runpath & interpreter
A helper function to patch the ELF such that it uses the dynamic loader and other libraries in the given folder.
* Add method to download libraries matching a libc
Download the matching libraries for the given libc binary and cache them in a local directory using `libcdb.download_libraries()`. The libraries are looked up using libc.rip and fetched from the official package repositories if available.
Only .deb and .pkg.tar.* packages are currently supported (Debian/Ubuntu, Arch).
* Add --libc argument to pwnup template
This generates code into the template which allows you to run the binary using the given libc.
The foreign libc is used by default, but you can choose to run the binary against your system's local libc using the `LOCAL_LIBC` command line argument when executing the exploit script.
* Work around Python 2 tarfile not supporting xz
* Fix crash when libc wasn't found on libc.rip
* Update README
* Use subprocess.check_output instead of pwnlib.process
* Special case Python 2 instead of 3
* Only catch Exceptions instead of everything
Co-authored-by: Arusekk <[email protected]>
* Check launchpad.net for Ubuntu libcs
This mimics the way io12/pwninit obtains the ld.so.
If the download from libc.rip fails, try launchpad.net.
* Hide comment in template when --quiet
* Please confused pylint in PY2 context
Co-authored-by: Arusekk <[email protected]>
---------
Co-authored-by: Arusekk <[email protected]>
Two minor changes were made in the commit:
- `which` was replaced with `command -v`
- Minor bug fix
## Why bother with `which` ?
- `POSIX` compatible
- `which` is an external process you're launching for doing very little (meaning builtins like hash, type or command are way cheaper), you can also rely on the builtins to actually do what you want, while the effects of external commands can easily vary from system to system.
- Many operating systems have a which that doesn't even set an exit status, meaning the if which foo won't even work there and will always report that foo exists, even if it doesn't (note that some POSIX shells appear to do this for hash too).
- Many operating systems make which do custom and evil stuff like change the output or even hook into the package manager.
Reference [here](https://stackoverflow.com/a/677212)
Co-authored-by: Debjeet Banerjee <[email protected]>
This commit adds information that the sleep shellcraft function does not check if it returned as part of an interrupt and it does not retry the syscall if this occured.
This appears to be fixed in setuptools 62, but Python 2 is stuck at
version 44. Use a workaround which overrides the disabled user installs
temporarily.
This fixes our develop Dockerfile too!
* Don't change log level for Corefile._parse_stack()
If we don't want to see the log, don't print it. There are no spammy
debug logs anymore.
Fixes#1666
* Fix doctest
From: Peace-Maker <[email protected]>
* Shellcraft: Fix spilling syscall args to the stack
There was a typo in the generated syscall template causing only the last
stack argument to be generated.
$ shellcraft -f asm mips.linux.sendto 3 0x123456 0x100 0 0xabcdefff 0x10
The 0xabcdefff argument was missing.
* Shellcraft: Generate 6 syscall arguments for unknown functions
There are syscalls with 6 arguments like mmap and sendto which couldn't
be called with all 6 arguments set.
$ shellcraft -f asm mips.linux.mmap2 0 0x1000
'PROT_READ | PROT_WRITE | PROT_EXEC' 'MAP_PRIVATE | MAP_ANONYMOUS' -1 0
* Shellcraft: Fix typo in error message
The `syscalls` variable is only present in the generate.py, not the
generated template.
* Regenerate syscall templates
* MIPS shellcraft: Push last two syscall arguments to the stack
The mips.linux.syscall template was only handling 4 syscall arguments
and silently discarded the remaining ones. Push the arguments to the stack
instead.
$ shellcraft -f asm mips.linux.syscall SYS_sendto 3 0x123456 0x100 0 0xabcdefff 0x10
Fixes#2153
* Fix mips syscall doctest
* Update CHANGELOG
Co-authored-by: Peace-Maker <[email protected]>
* Add shellcraft.sleep template wrapping SYS_nanosleep
Accepts the time in seconds as a float argument and calls SYS_nanosleep.
Fixes#1428
* Update CHANGELOG
* Use nanosleep wrapper
Co-authored-by: Arusekk <[email protected]>
* Fix bug at ssh.py:get() with relative path
* Add doctest for bug at ssh.py:get()
* doctest for download and download_file
* remove os.path.join, since _download_raw already seem
handle remote paths
* Add #2214 to stable changelog
* Fix Doctest for ssh.py to not be dependent on user dir
* Update CHANGELOG.md
* Fix typo in ssh trying to access log.Progress.error
That function doesn't exist and .failure() is used everywhere else.
* Use log.error instead of manually raising
The log.Progress logger doesn't print the exception when exiting the scope.
---------
Co-authored-by: Arusekk <[email protected]>
* Fix `remote` and `listen` in sagemath
sage's Integer doesn't seem to inherit from python's base int.
```
----> 1 listen(Integer(1337))
OSError: Int or String expected
```
Fixes#2201
* Update CHANGELOG
remote_tar is a `bytes`, so the string interpolation resulted in the
command `cd ... && tar -xzf b'/tmp/...tar.gz`. It also would have failed
had either path contained spaces.
To fix this, make everything a bytes and use proper escaping.
Co-authored-by: Arusekk <[email protected]>
"1001" would highlight the "00" even though it's not a null byte in the
input. Group the bytes correctly to look for real null bytes and
newlines only ignoring accidental matches of adjacent bytes.
Co-authored-by: Arusekk <[email protected]>
This commit forces the `path` class variable of the `ELF` class to be of
type string, even if a byte string argument is passed. This also makes
the implementation consistent with the documentation, which states that
`ELF.path` is of type `str`.
Fixes#2166
Cherry-picked from 60af3a26d0
Signed-off-by: James Raphael Tiovalen <[email protected]>
This commit forces the `path` class variable of the `ELF` class to be of
type string, even if a byte string argument is passed. This also makes
the implementation consistent with the documentation, which states that
`ELF.path` is of type `str`.
Fixes#2166
Signed-off-by: James Raphael Tiovalen <[email protected]>
* Fix libcdb.unstrip_libc tests
The old debugsymbols seem to have been pruned.
Update hashes to recent Ubuntu 22.04 libcs.
* Switch wget test to httpbingo.org
httpbin.org is very slow to respond.
Cherry-picked from 1998ff0920
* Handle `context.newline` correctly in `tube.interactive()`
The `context.newline` or `self.newline` variable isn't obeyed when typing in interactive mode. It is used when sending and receiving lines through `tube.sendline` though, causing a mismatch.
The receiving end of the `tube.interactive()` already has handling of newlines, but the sending side does not.
Example:
```python
from pwn import *
io = process('cat')
io.newline = b'\r\n'
io.sendline(b'auto')
io.interactive()
```
```
$ python testinteractive.py DEBUG
[x] Starting local process '/usr/bin/cat' argv=[b'cat']
[+] Starting local process '/usr/bin/cat' argv=[b'cat'] : pid 19060
[DEBUG] Sent 0x6 bytes:
b'auto\r\n'
[*] Switching to interactive mode
[DEBUG] Received 0x6 bytes:
b'auto\r\n'
auto
$ test
[DEBUG] Sent 0x5 bytes:
b'test\n'
[DEBUG] Received 0x5 bytes:
b'test\n'
test
```
The expected outcome would be to send `b'test\r\n'.
The same problem arises in non-term mode (`PWNLIB_NOTERM=1`), where stdin is read in binary mode causing the OS line seperators to come through. Correctly replacing them with the `context.newline` setting allows to use the interactive input on windows hosts as well, without constantly sending `\r\n`.
* Update CHANGELOG.md
* Only replace newlines in TTYs
Don't mess with the line endings when piping data through.
* Always send bytes immediately on receive
Only replace newlines in a tty if NOTERM wasn't explicitly set or
the `newline` setting was set.
* Don't return a newline after term.readline lines
The builtin `input()` doesn't include the newline.
* Remove bogous newline from CHANGELOG
* Fix libcdb.unstrip_libc tests
The old debugsymbols seem to have been pruned.
Update hashes to recent Ubuntu 22.04 libcs.
* Switch wget test to httpbingo.org
httpbin.org is very slow to respond.
* added feature to the fmtstr module, now able to generate payload without the dollar syntax.
* made some improvements in the no_dollar feature implemented
* reduced different lines
* removed some blank lines and debug stuff
* (fix) missing check for no_dollar option
* (fix) updatet argument type for documentation
* updated CHANGELOG.md
* (fix) typo in the CHANGELOG.md version
* Update pwnlib/fmtstr.py
suggested change, dont need comparison in calling the make_payload_dollar
Co-authored-by: Arusekk <[email protected]>
* Update pwnlib/fmtstr.py
suggestion; better use a non null byte thing to fill the values used by the %c to pad before writing.
Co-authored-by: Arusekk <[email protected]>
* (fix) minor fixes, typos on comments
* update pwnlib/fmtstr.py : suggestion from Arusekk, made improvements to save some bytes in the generation of the payload
* added tests to fmtstr_payload with no dollar flag
* update CHANGELOG.md with correct version
---------
Co-authored-by: Arusekk <[email protected]>
* Move most settings to pyproject.toml
* py2 minimal compat
* Make it not crash nor include spurious files
* Use PEP517
* Update workflows
* Limit workflows
Closes#1587
* Fix debugging and assembly for RISCV
They are broken since the riscv architecture was split into riscv32
and riscv64 internally in #2177.
* Always look for riscv32. riscv64 or riscv toolchains
* Look for riscv64 for riscv64 first
* Determine elf arch by e_machine and bits
Separate 32/64 for riscv and mips
Trim some trailing space
* Fix missing comma
Co-authored-by: Arusekk <[email protected]>
* Add CHANGELOG
* unicorn: Use fixed version
* Cosmetic enhancements
---------
Co-authored-by: Arusekk <[email protected]>
* libcdb: Add option to search by function offsets
When you're able to leak addresses of the libc library, use `libcdb.search_by_symbol_offsets()` to find and download the matching libc library from https://libc.rip.
If there are multiple matches, the user is prompted to select one interactively. The selection can be saved in the code for future executions of the script.
Fixes#1867
* libcdb: Handle multiple results when looking up by hash
Sometimes the same library appears to be indexed multiple times (see 0b52d2e713d243f0f65d808fcd3fbe372bb3cd32). Handle that situation by selecting the first in the list, since they should all be identical given the same hash value.
* Update CHANGELOG
* Fix off-by-one when pre-selecting a libc
* Do .json() only once
---------
Co-authored-by: Arusekk <[email protected]>
* Fixes for download_dir.
* Adds support to use SFTP if available
* Update pwnlib/tubes/ssh.py
Co-authored-by: Arusekk <[email protected]>
* Removal of unnecessary parens after 'if' keyword
Co-authored-by: Juuso Käenmäki <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* Make template respect args.EXE
To be able to execute a script using an executable with another name and/or location than specified at template generation, `args.EXE` will be used as path for ELF file loading, only using the preconfigured value when no arg was supplied
* Update CHANGELOG.md
* FIXUP: properly test for EXE arg
* Style fixes and revert changelog
Co-authored-by: Arusekk <[email protected]>
* Support adding lists in `ROP.raw()`
Allow to add multiple raw entries on a rop chain using an array instead of multiple calls to `raw()`.
```py
rop = ROP('/bin/ls')
# before
rop.raw(1)
rop.raw(2)
rop.raw(rop.ret)
# after
rop.raw([1, 2, rop.ret])
```
Fixes#2017
* Update CHANGELOG.md
* Update pwnlib/rop/rop.py
Co-authored-by: Arusekk <[email protected]>
The base image was missing the patchelf utility and the develop image
didn't update the apt cache. That caused building the image to fail when
pulling the outdated pwntools:base image from dockerhub.
The docker Makefile was using an invalid `-depth 1` argument which appear
to have been replaced by `-mindepth` and `-maxdepth` arguments recently.
The testing container is based on the develop image's Dockerfile which
copies new files into the container. Grab them from the appropriate folder.
pwndbg was installed twice when running the testing container for the
doctests.
* Support non-bytes argument to `packing._need_text()`
Allow to pass everything to `_need_text` and be sure to get a string back. This allows to log any object without the need for a format string by calling `repr` on it.
```py
# before:
log.info('%r', 5)
# now
log.info(5)
```
Fixes#2018
This is why CI should always be green.
Make it green, dear contributor, because the maintainers might merge
something without checking if it is green. Please.
The old API was a hack apparently, but it looks like GitHub Actions slowly gets less and less sketchy over time, and even goes as far as providing well-designed APIs.
Ref. https://github.com/badges/shields/issues/8671
This commit fixes a small bug with PwntoolsLexer where it matched negative numbers incorrectly in Arm assembly.
See https://github.com/pwndbg/pwndbg/pull/1367 for more information
* Fix tube.recvpred() timout argument
The `timeout` argument wasn't implemented correctly causing the function to hang instead of timing out.
* Update CHANGELOG.md
You have to call `listen.wait_for_connection()` before using the listen socket or you face raceconditions with the thread accepting the client connection.
Fixes#2100
* Allow tube.recvregex to return capture groups
When wanting to extract a value from the received data, one had to receive up until the value and the value itself in two steps without carving the data out of a large blob like this:
```
io.recvuntil(b'The address is: ')
address = int(io.recvline(), 16)
```
or use a regex twice:
```
pattern = re.compile(rb'The address is: (0x[0-9a-fA-F]+)')
data = io.recvregex(pattern)
address = int(pattern.search(data).group(1), 16)
```
This patch allows to pass `capture=True` to `tube.recvregex()` to return a `re.Match` object instead of the raw bytes.
```
addr = int(io.recvregex(rb'The address is: (0x[0-9a-fA-F]+)', capture=True).group(1), 16)
```
This allows for slicker data extraction!
Fixes#1962
* Update CHANGELOG.md
* Fix ROP without a writeable cache directory
`context.cache_dir` can be None, but the ROP class assumed it would always be a valid path and crash. Disable rop-cache if we can't save it.
Fixes#2072
* Update CHANGELOG.md
* Switch to federated debuginfod service
elfutils.org offers a federating proxy server which forwards the requests to all other debuginfod servers of the different linux distributions.
https://sourceware.org/elfutils/Debuginfod.html
Use that instead by default to increase our successrate and potentially include new servers automatically once the distro sets one up.
* Use DEBUGINFOD_URLS envvar for server selection
The `DEBUGINFOD_URLS` environment variable is what you use for your normal debuginfod setup to tell it where to look for debug symbols.
Parse it as well and prepend it to the list of servers to try to allow for dynamic adjusting of the target servers.
* Fix for overwritten env when parsing core file
* warn_once message when dealing with overwritten pointer to env variable in core file
* Check for sha256 and sha256sum before trying openssl sha256 as "openssl sha256" doesn't return an error code in some old versions of openssl (OpenSSL 0.9.8g 19 Oct 2007 at least) that doesn't support sha256.
* ret2csu: fix a silent failure when rdx is populated from r14.
One such binary can be found at https://github.com/nobodyisnobody/write-ups/tree/main/FCSC.2022/pwn/Deflation
* Clean up the code a bit
Co-authored-by: Zach Riggle <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* make pwn cyclic -l work with entry larger than 4 bytes
* Do not assume py3, do not assume little-endian
* Update changelog
* pack all bytes instead
Co-authored-by: Arusekk <[email protected]>
* Change prefix in install script to $PREFIX
On special envrionments like Termux, /usr/local is not the proper path.
* Change prefix in binutils install script to $PREFIX
* PREFIX not everywhere
Co-authored-by: Arusekk <[email protected]>
coredumpctl saves the coredumps with full path to the binary. Passing only the basename makes it fail to find the coredump when it is actually there.
Closes#1879
Fix wrong branch taken when trying to debug non native arch via ssh
(resulting qemu_port being referenced without being set).
This is done by setting the condition in line 571 to be the same
as in line 534
Signed-off-by: Aviya Erenfeld <[email protected]>
* Add the 'raw' option to SSH shell initialization
This permits the use of the SSH shell as an arbitrary TCP tunnel by
bypassing the startup environmental checks.
Co-authored-by: LMS57 <[email protected]>
* Add changelog items
* Add assignment to self variable
Co-authored-by: LMS57 <[email protected]>
* Add js_escape and js_unescape
* Add entry for js packing functions
* Use hardcoded data for doctest
* Fix js_escape data iteration for Python 2
* Move js packing functions to util.fiddling
* Apply suggestions from code review
Co-authored-by: Arusekk <[email protected]>
* Add libcdb commandline tool
Allows to dump information about a local libc binary or lookup matching libc versions based on a build hash or symbol offsets (querying https://libc.rip).
It's possible to download and unstrip a matching libc binary from the command line.
* Fix Python 2 compatibility
* Remove main_arena heuristic
* Update CHANGELOG
Co-authored-by: Arusekk <[email protected]>
* Add badge packaging status badge
Add a repology badge to show packaging status.
* Update README.md
Use shields.io everywhere
Co-authored-by: Arusekk <[email protected]>
This fixes a misbehaviour in environments where `XDG_CACHE_DIR` is not
set and `~/.cache` does not exist, e.g. in the docker image. We simply
attempt to create a temporary directory which will be registered with
`atexit` to be deleted on exit using `shutil.rmtree()`. A Python 3
approach for future would be to use a `TemporaryDirectory` instead which
would get cleaned up when the held object gets dropped out of the TLS
stack.
It is still possible for the `context.cache_dir` to be `None` if the
default directory exists but is not writable, which means some code
which uses this property is liable to blow up in such a circumstance.
When the last gadget has no arguments and the gadget before has stack
arguments, put the last gadget directly in its return address slot
(instead of a stack adjustment gadget).
* Fix CI after Groovy Gorilla went away for libc unstrip test
Build elfutils 0.181 from source since we can't use builds
from a newer ubuntu version anymore.
* Install python wheels in CI
* Fix ELF.libc_start_main_return with glibc 2.34
The actual call to `main` was split out into a new `__libc_start_call_main` function in glibc 2.34.
Update the heuristic to find main's return address by looking into all direct calls in `__libc_start_main` until the old heuristic matches.
The other direct calls could be e.g. `atexit` which doesn't call `exit`.
* Update CHANGELOG.md
* Fix CI after Groovy Gorilla went away for libc unstrip test
Build elfutils 0.181 from source since we can't use builds
from a newer ubuntu version anymore.
* Install python wheels in CI
Just stick to the replaced names everywhere.
This actually only broke linkat(2) and symlinkat(2),
but it touches many len -> length renames as well
for the sake of consistency.
Fixes#1964
Ubuntu, why are you doing this to us?
I seriously mean that Ubuntu is the most silly distro.
Like, they probably updated their docker (or VM) images last week,
then Github got it, and we started getting failures.
They totally changed the behaviour of that 'apport' thing,
and it now drops core files in /var/lib/apport/coredump
instead of the current working directory.
Why is that even... ugh. Nevermind. It works now, at least.
But for how long?
Ubuntu, why are you doing this to us?
I seriously mean that Ubuntu is the most silly distro.
Like, they probably updated their docker (or VM) images last week,
then Github got it, and we started getting failures.
They totally changed the behaviour of that 'apport' thing,
and it now drops core files in /var/lib/apport/coredump
instead of the current working directory.
Why is that even... ugh. Nevermind. It works now, at least.
But for how long?
* Fix: TypeError: can't concat str to bytes
sh_string returns a string, but two lines try to concat it with bytes, which throws a TypeError. Either convert them all to bytes or all to string.
* Update to use packing._need_bytes
I just followed the format from surrounding conversions. Hopefully that works correctly.
* Update pwnlib/tubes/ssh.py
Co-authored-by: Arusekk <[email protected]>
* Conversion to bytes
* Going for higher level conversion
Co-authored-by: Arusekk <[email protected]>
* libcdb: Load debug info and unstrip libc binary
The debug info is obtained from a debuginfod server maintained by systemtap.org. elfutils is used to unstrip the binary and merge the debug info back.
Try to unstrip every libc binary downloaded by the libcdb module by default. #1818
* Install elfutils on CI
* Update CHANGELOG.md
* Add more tests
* Fix negative test
* Raise an exception when eu-unstrip fails
This will hopefully get logged during doctests.
* Install elfutils 0.181 from groovy
Use APT Pinning to select the newer elfutils version from groovy instead of the buggy 0.176 version available in focal.
* Fix missing import of requests
pylint?
Co-authored-by: Arusekk <[email protected]>
Apparently it is us who used unicorn The Wrong Way,
e.g. trying to fetch instructions @ 0xdbdbdbdb,
which is both unaligned and high-address-space.
Unicorn just changed the exception to throw exception with the address
of an offending jump instruction, instead of the incorrect address.
Making the debugs actually work and changing address to 0x7c7c7c7c
solves the issue.
Closes#1538
Ping pentoo/pentoo-overlay#734 @blshkv
* Fix i386 encoder issue #1761
Don't encode `avoid` parameter in `fiddler.py` if type is `bytearray`
* Pass `avoid` as `bytes` instead of `set` when `encoder.encode()` is called
* Requested change: Increase readability
* Update doctest in XOR encoder
Add my failing testcase to ensure the code is fixed
* Defer import of requests and urllib
This saves 100ms-150ms of startup time on "from pwn import *"
* Update CHANGELOG
* More deferred imports to save ~20ms
* Show magic gadgets and setRegsiters in the top of the documentation
* Show the cool way to set registers magically, with fewer lines of code
* Try simple fix for doctest
* ssh: Use paramiko allow_agent parameter
* ssh: Add ignore_config parameter to opt out of using ~/.ssh
* [changelog] Add `ignore_config` argument to `pwnlib.tubes.ssh` and improve `allow_agent` implementation (#1566)
Co-authored-by: Arusekk <[email protected]>
* [process] Add writemem with tests, alias readmem == leak
* Update CHANGELOG
* Explicitly show result for easier debugging on CI
* Explicitly set arch to i386
* Troubleshooting and avoid overwriting top-level asm
* Update pwnlib/tubes/process.py
* Fix nasty race condition
Co-authored-by: Arusekk <[email protected]>
* [android] Split Android AVD testing into its own Workflow
* [android] Modify to trigger tests
* [android] Make paths more succinct and explicit
* Remove environment variable
* Rename to test
* Always run Android jobs for debugging
* Enhance Android workflow
* Disable gdbserver download
* Make the workflow actually work
* Travis missing gdbserver
* Try to fix travis
* Add good ol' SIGALRM-based debugs for tests
* Revert changes made to Travis
* But fix GHA
* Actually enable the android tests
* Actually enable
* Not necessary anymore
* Actually run the tests with AVD installation
Also fix incorrect text-vs-bytes usages in protocols.adb.
Co-authored-by: Arusekk <[email protected]>
* Cleanup unconditional imports
This may aid #1725
* Import explicitly
* Add good ol' SIGALRM-based debugs for tests
* Fix py2 GDB tests somehow
XXX: see why this matters
* Update CHANGELOG.md
* Make gdb.attach work with arrays instead of string concatenation
This results in a simpler set of code that is more resilient to
issues with quoting and files with spaces in their name.
* Remove detach_and_quit for now, this was for Docker CI testing
* Fix typo
* Fix lint errors
* shlex.quote is unavailable on Python3
* Update CHANGELOG.md
* With no escaping comes no escaping
Co-authored-by: Arusekk <[email protected]>
* Add rop.ret2csu()
* Removed magic numbers
* Replaced if elf.sym.has_key(u'__libc_csu_init') with if u'__libc_csu_init' in elf.symbols
* Consistently used unicode strings to search for symbols
* Used the Capstone SKIPDATA option to deal with endbr64 instructions
* Added comment to clarify why we set rbp and rbx
* Account for newer versions of gcc populating registers in a different order
* Code style issues
* Add padding functionality
* Split files and add tests at last
* Implement flat interface in ROP
* Make linters happy
* Update changelog
* Add a more detailed doctest
* Actually set uid and hopefully solve all the corefile problems
* Show me the bug
* More verbosity
BTW fix py3 sockets incosistency
* The most debug
* Should finally work
* It must work now
Co-authored-by: CptGibbon <[email protected]>
* Prefer Python3 for the remotely-executed Python script
This should be done as part of our general transition to Python3,
but should also be done because some systems with e.g. Python2.6
do not support OrderedDict, but DO have a Python3 version that works.
* Make the script actually work in py2 and py3
* Fix & update CHANGELOG.md
This actually is a behavior change, and can be a breaking change for some,
so must be stated prominently in the changelog.
Co-authored-by: Arusekk <[email protected]>
* Fix atexit on Python 3
`sys.exitfunc` was removed in Python 3, meaning that any handlers
registered with the `pwnlib/atexit.py` module wouldn't run. This PR
reigsters the handlers with the stdlib atexit.
* Update changelog
* fixed wrong description for shellcraft.amd64.linux.kill(). new desciption copied from pwntools/pwnlib/shellcraft/templates/common/linux/syscalls/kill.asm
* updated changelog with PR #1739
* Fix more syscalls and generify
Co-authored-by: Arusekk <[email protected]>
* mips: do not use $t0 temporary variable in dupio
dupio currently uses $t0 which is a temporary callee saved register.
The dup2 call is thus allowed to trash the register.
Use $v0 instead. $v0 is the return value register.
Either dup2 returns the provided $v0 (newfd) or -1 if it fails,
and therefore $v0 can be used safely here.
Reference:
https://courses.cs.washington.edu/courses/cse410/09sp/examples/MIPSCallingConventionsSummary.pdf
* Update changelog
* Update dupio.asm
Make a small change to kick Github Actions
* Fix CHANGELOG
Co-authored-by: Heap Crash <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* Try to catch more errors by using python -bb
* Apply changes to travis dockerfiles as well
* Introduce context._need_bytes and context._need_text
...and resolve many issues
/bin/sh seems to be sending the cd command to the background, instead of
changing directories and then running the selected command.
This has worked forever, not sure why it's failing in CI now
* [io] Introduce first draft of IO module
This is designed to make it easier to do file management when a piece of code
has to deal with creating a file that might need to be local, OR might need
to reside on some SSH server somewhere.
Specifically, the GDB code needs this for future enhancements
* [docs] Add documentation and doctests
* Update CHANGELOG
* Add pathlib2 requirement for Python2
* Use Python3 documentation
* Remove pwnlib.io module
* [filesystem] Add filesystem module, with Path and SSHPath
* Use @classmethod monkey patching instead of subclassing
This means that isinstance(Path, ssh_path_instance) returns True
* Make doctests work again
* Remove debugging code
* Rename context.ssh to context.ssh_session
This is needed because @LocalContext will pop off 'ssh' and some functions
take 'ssh' as an argument, and it would all break.
* [debug] Try to figure out why CI is failing
* [ssh] Fix failing doctest
* Fix README
* Fix README
* Fix failing doctest harder
* Remove unneeded internal file
- ssh.runner() now accepts mixed bytes and strings, by forcing everything to a string
- ssh.runner() now takes e.g. log_level='debug' as a kwarg
- Simplify ">/dev/null 2>&1" into "&>/dev/null"
* Remove unnecessary pwn.toplevel.__all__
The default behavior of `from import *` is enough and better, to make Pylance work
fix#1810
* Update CHANGELOG.md
* Add a comment for removed pwn.toplevel.__all__
Co-authored-by: Heap Crash <[email protected]>
* [travis] Switch to a different server that still has the magic gdbserver
* [travis] Install rpyc using pwndbg's method for finding the GDB-built Python
* [travis] Forgot pip dropped support for Python 3.5
* [gdb] Rewrite timeout logic for GDB API and fix doctests to use "skipif"
* [gdb] I'm not sure how this GDB API stuff ever worked in the first place
But it works now, at least locally
* [gdb] Fix broken doctests
* Update timeout to 10 seconds
* [corefile] Fix some broken tests, add several more
Notably, some were outright incorrect (corefile.libc for a statically-linked ELF)
Also fixed up some of the documentation to add more links, and added examples
of reading / writing / saving / reloading a corefile.
* [corefile] Some fixes for things but also some canaries for finding out what is happening on GHA
* [testing] More canaries to pull files down from GHA
* That was a bad idea
* Fix implementation bug in Corefile.exe
Basically we returned the segment that contained the entry point,
but need to return the first segment which shares the same filename
We also no longer handle the case where the mapping is anonymous
and set the name to AT_EXECFN. This probably breaks some QEMU stuff.
* Yep, the code used AT_EXECFN for the QEMU case
For an ARM corefile generated by qemu-user, the mappings are:
In [10]: print(core.maps)
10000000-10001000 r-xp 1000
ff7ef000-ff7f0000 ---p 1000
ff7f0000-ffff0000 rw-p 800000 [stack]
So we need to pull the name out of AT_EXECFN
* [core] Eagerly fetch self.exe so that the Mapping always has the right name
This means that Corefile.maps and Corefile.mappings will be updated for the single mapping
that contains the AT_ENTRY
* Attempt to debug getenv failure
* Fix some temporary edits and broken tests
* [ssh] Lets try and see if this works
* Remove unnecessary test and disable diagnostic logging
* [misc] Refactor kill_at_exit code for simplicity
Also catch the OSError from when the process has already died, which
was showing up at the end of CI runs
* Remove debug print
* Hail mary debugging attempt
* Try workaround that manually sets SHELL, since it may not be set on GHA
* [corefile] Remove debugging code, and consume setdefault result
* [corefile] Hide the bash-static fuckery in the RST file setup
* Update docs/source/elf/corefile.rst
Co-authored-by: Arusekk <[email protected]>
* Update pwnlib/elf/corefile.py
Co-authored-by: Arusekk <[email protected]>
Co-authored-by: Arusekk <[email protected]>
This allows piping with color into tools like ansifilter, which take
ANSI-escape-coded colorized text and emits data in other formats,
notably HTML.
This is useful for writing blog posts with PWNLIB_DEBUG=1 to display
hex-dumped data written to stdout that would not normally be colorized.
Currently this ONLY works for the environment variable PWNLIB_COLOR,
and does not make use of pwnlib.args due to cyclic imports.
* Remove likely-unused Dockerfiles, which are not being supported in the future anyway
This leaves base, stable, beta, and dev as the only Dockerfile images
Simplify extra/Docker so that Pwntools is already checked out, and build precedence works
* Base image always gets built first
* Branch-tracking Dockerfiles now require base to be built
* Use locally-checked-out-repo instead of e.g. git+https://github.com/Gallopsled/pwntools@beta
* Install tested with both Python2 and Python3
* Add Python3 user-script directory to $PATH
Update CHANGELOG with Docker changes
Add develop to Makefile, restore old Dockerfiles
Make development target separate, and automatically launch "docker run -it" with suggested arguments
Add develop Dockerfile
* Add README for the Dockerfiles
* Merge dependencies, add NOPASSWD, use hostname localhost to avoid sudo errors
* Better Dockerfile for testing, working on making all of the tests work
* wip
* WORK-IN-PROGRESS
* Remove from pwn import *
* Fix incorrect usage of progress() in SSH
* WIPWIPWIP
* MORE WIP
* [srop] Fix SROP unit tests by allocating enough stack
* [todo] Test canary in MIPS pushstr to make sure tests are actually being run
* [ROP] Make ROP doctests work from Docker and more minimal
* [srop] Simplify SROP imports
* [mips] Simplify MIPS doctest imports
* [corefile] Handle race condition against kernel / qemu
* [web] Handle file permission failure in Docker
* [core] Actually implement timeout
* Revert "[corefile] Handle race condition against kernel / qemu"
This reverts commit e02a3dce19.
* [context] Handle file permission issue in Docker
* [aarch64] Just run the shellcode, dont dump it out
* [tests] Remove test canary from mips pushstr
* [timeout] Use ELLIPSIS for time-flaky tests
* [update] Remove DONT_CHECK_FOR_UPDATES
* [context] Handle another path access error in Docker
* [timeout] Just leave it the way it was, hardly ever fails
* [context] Fix broken test because of quotes
* WIP save Docker stuff just in case
* [mips] Do not replace "asm" in the global scope with doctest
* Clear context between invocations so we dont end up with e.g. 64-bit i386
* [process] Fix timeout to actually work
* [mips] Do not replace "asm" in the global scope with doctest (Part 2)
* [rop] Fix ROP doctests
* [mips] Do not replace "asm" in the global scope with doctest (Part 3)
* Re-add all of the Py2OutputChecker stuff so I can start testing Python2
* Restore libcdb checks
* Tests work on Python2 now with "make -C travis/docker doctest2"
* Handle issue caught by Pylint
* Install pwndbg for the pwntools user
* Add trailing newline
* Remove un-necessary ELLIPSIS
* Add trailing newline
* Convert assert to simple statement
* This is an empty commit for testing webhook integrations
* [travis] Use bash as the default shell
* Commit compiled Dockerfile for use on hub.docker.com
* [gdb] Restore gdb.rst and manually disable doctests from inside Docker container
* Note newer Ubuntu versions
* State preference for Python3
* Provide working instructions for Python2 installation due to pip deprecation
* Provide information on command-line tools not being in $PATH
* Fix test to use ellipsis, since which() may not return /bin/sh
* Fix another case where /bin/true may not be used given $PATH
* Fix another absolute path failure in doctests
* Fix another issue where /usr/bin may intercede
Updated or alternative versions of binutils cannot be relied upon to create
very small versions of ELFs (e.g. when running on macos).
However, we can ensure that the non-symbolized version is SMALLER.
* Add baremetal os
This allows to run and debug a raw binary, e.g. an MBR, with
the help of qemu-system-$(arch).
Tested with:
pwn.context.arch = "i386"
pwn.context.os = "baremetal"
pwn.gdb.debug(["./mbr.bin"])
* Add changelog entry for 'baremetal' os
* Add PATH argument to which function
* [changelog] Add info on the path argument for which
* Move comment back to original location
* Sort CHANGELOG.md
Co-authored-by: Arusekk <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* run_in_new_terminal: Prefer tmux/screen over TERM_PROGRAM and x-terminal-emulator.
* run_in_new_terminal: Kill running commands when the main process exits.
Currently, when a gdb window is opened via gdb.debug(), terminating the
main pwntools process leaves the gdb window/process around. It is
inconvenient to have to close these leftover gdb windows when iterating
on an exploit.
Add a kill_at_exit parameter to run_in_new_terminal() which attempts to
terminate the command at main process exit.
This only works for terminals which do not daemonize, or for tmux, which
supports returning the command's PID.
This change also removes the special case for closing
stdin/stdout/stderr on Mac OS X, as the problem no longer reproduces
under tmux 2.8.
* Fix regression and add CHANGELOG
Co-authored-by: Arusekk <[email protected]>
* Don't introduce null bytes in MIPS shellcode.
As a verification, you can run:
```sh
pwn shellcraft mips.linux.sh |pwn phd
```
Closes#807
* Add a test
* Fix doctest under py2
* Try libc.rip for libc lookup
As of #1171 the libcdb repository isn't seeing any updates anymore. When looking up a libc by a hash or build_id try the libc database maintained by @niklasb as well.
The database uses a REST api documented at https://github.com/niklasb/libc-database/blob/master/searchengine/api.yml
* Catch exceptions properly
* Update CHANGELOG
* Add doctests
Co-authored-by: Arusekk <[email protected]>
* [elf.py] convert linux string banner into an actual str
``self.string(address)`` actually return a bytes object containing the raw bytes, not a string in a pythonesque sense !
Since quite some time on Python3, ``re.search`` only works on proper str so we need to convert it before feeding it to ``re.search``
``.decode()`` works both on Python2 and Python3
* Update kernel ELF handling
Co-authored-by: Arusekk <[email protected]>
* Implement property getter/setter for context.newline, and make tubes honor it
The default newline can now be set context-wide, or per-tube.
* [changelog] Update for 1541
* [docs] Use r-prefix for embedded newlines
Co-authored-by: Arusekk <[email protected]>
* use non-greedy operator to handle corner cases where the program output contains END in it
* END can appear in the string leak
Co-authored-by: Arusekk <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* Keep ROP cache in user's home like regular cache
I don't have statistics numbers on which would be more likely to be writable, an user's home directory or the OS's temp directory, but seems to me it'd make sense to try and write all caches to the same location.
Optimally this would be easily configurable to allow adjusting to different set-ups, obviously.
* Use cache_dir instead
Co-authored-by: Arusekk <[email protected]>
* fix --multibr issue
* modify changelong
* In which I learn how to count
* put doctest for ROP().syscall in unfortunate place
* fix context malapropism
* fix ROP gadget loading when binary contains a gadget with add rsp, <register>
* add documentation regarding the fact that loading gadget "add $sp, <reg>" will fail
All three instructions: `int 0x80`, `sysenter`, and `syscall` require
different stack frames and registers to pass arguments, but
`ROP._srop_call` uses them interchangeably.
This fix limits syscall instruction lists to gadgets that work.
Co-authored-by: Arusekk <[email protected]>
* Change lookup algorithm of `adb.which`.
This is done because the current algorithm does not work when the binary (e.g. `su`) is installed in the last element of $PATH.
Example:
```shell
$ echo $PATH
/sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin
$ echo $PATH | while read -r -d: directory; do
> echo "$directory/su";
> done
/sbin/su
/vendor/bin/su
/system/sbin/su
/system/bin/su
$ (IFS=:
> for directory in $PATH; do
> echo "$directory/su";
> done
> )
/sbin/su
/vendor/bin/su
/system/sbin/su
/system/bin/su
/system/xbin/su
```
* Update CHANGELOG
Turns out that python 2 does not raise correct socket.timeout, but
SSLError instead. With no errno. Just a message that might be
localized in some circumstances.
**THIS IS NOT THE ONLY TIME PYTHON 2 IS CAUSING PROBLEMS.**
Time to stop using it, dear exploit developers. It is dead now.
But here you are: the simple fix.
Closes#1356
* [docker] Upgrade Docker images of Pwntools to use Python3
Also installs qemu-user-static and various binutils packages for building (binutils)
and running (qemu) cross-arch binaries.
Fixes#1500
* Update CHANGELOG
* Switch to Python3 for stable/beta/dev Docker images
* [changelog] Add missing links
* [shellcraft] Avoid recursive walk of all templates for command line
Fixes#1650
* Avoid double-fetch
* Add changelog
* [pylint] Only warn for additions
* tubes/remote: Support Server Name Indication and enable by default
This allows customizing the server_name via sni="custom" or sending the
name specified in the host parameter via sni=True.
We switch to the SSLContext.wrap_socket API because ssl.wrap_socket is
deprecated and does not support SNI.
Unfortunately, SSLContext.wrap_socket dropped some of the parameters
that are supported by ssl.wrap_socket. These parameters have been moved
to the SSLContext API and can now be used by passing a custom SSLContext
to the ssl_context parameter of `remote`.
* update CHANGELOG.md
* Enhance gdb error message when unable to launch
A common issue is when the binary is not executable,
or the binary is simply missing.
* Use six.ensure_str
* [tests] Change test slightly so that failing output will be emitted
* Disable flaky UI test on Github Actions
* [ui] Try a sleep rather than disabling tests
* Add a bfdname for msp430 to make asm and disasm work
* Add a bfdname for s390 to make asm and disasm work for all architectures
* Add tests for new bfdnames
* ci: Add binutils for newly-tested architectures
Co-authored-by: Arusekk <[email protected]>
* [packing] Support negative indices for flat() and fit()
Fixes#1531
* [changelog] Add Changelog for PR
* [packing] Only take negative path if there are any
min([]) is not valid Python
* [packing] Move out_negative to a higher scope
* [stable] Fix incorrectly-merged code
* [aarch64] Use byte-strings instead of strings
* [aarch64] Have different tests for str vs bytes
* [aarch64] Convert all items to binary before processing
* [aarch64] Fix doctest results
* [aarch64] Stopgap to fix Py3 CI tests
* [aarch64] Remove failing test, will address in a future release
* [aarch64] Fix aarch64.pushstr and pushstr_array
* pushstr
* We were pushing the correct values, and each 16-byte span was correct, but the 16-byte segments were ordered wrong
* pushstr_array:
* Was entirely broken. This commit has a non-optmimized version but it works
Closes#1549
* [packing] Better alternative to #1580
pack() was failing for dictionaries that are not ordered
Closes#1582Closes#1579
* [docs] Add better documentation for flat()
* [changelog] Add 4.1.4
* [release] Bump version to 4.1.4
Co-authored-by: Arusekk <[email protected]>
* [docker] Get Travis CI dockerfile working more
* [docker] Do not upgrade all packages, only update if required
* [docker] Move requirements installation higher up before the copy
* Fix several sphinx warnings
* Several code style fixes
Mostly grep -Er '[=!]= (None|True|False)'
* Compile-time str concat
also reduced redundant characters at line ends
* Fix python2 toplevel regression
* No equality comparison with singletons again
* Clarification: lambda: [] is just list
* Reintroduce reverted fragment
* sock-based GDB connections didn't set the PID, so it didn't work
* Now uses a real Timeout object
* Added special case handling for socat, since we were attaching to the wrong process
* ssh-based doctests improved
* Works locally on everything I tested
* One test is ignored, since it doesnt work on Travis CI
* Added libc6-dbg to Github Actions
* run_in_new_terminal supports unicode strings now
* This fixes SSH-based debugging
* wait_for_debugger now has a timeout
* Fixes the case where the debugger never comes up, mostly for tests
If passed an empty dictionary defining the environment variables, the process invoked by gdb.debug() has environment variable 'PWD' set to the directory containing the executable. This behaviour is not observered when NOPTRACE is enabled: the program runs without any environment variables set. Similarly, if an empty environment variable dictionary is passed to process(), it appears to also run under a truly empty environment. Additionally, attempts to overwrite the "PWD" environment variable with a dictionary entry are not successful under gdb.debug().
This fix brings the gdb.debug() treatment of environment variables in line with and process() and gdb.debug() with NOPTRACE enabled.
It allows both empty environments, and overwriting of the 'PWD' environment variable.
Both Debian and Ubuntu provide a `binutils-avr` package, which contains
an assembler for AVR as `avr-as`. However, pwntools was looking for
something with `linux` in the name, such as `avr-linux-gnu-as`. Of course,
such a target triplet is nonsense, Linux does not run on AVRs
(apart from that one hack that actually emulated some other CPU which
therefore doesn't count).
* Disable corefile tests for Github Actions
* Disable the UI tests, as theyre broken
* Install gdbserver
* Rename step
* Remove SIGALARM to see if it fixes tests
* Set PWNLIB_NOTERM for command-line runs
`ELF.functions` wasn't updated after the binary base address was changed using `ELF.address` like `ELF.symbols` is. The documentation mentions `ELF.functions` getting updated as well, the update was missing though.
Closes#1512
Co-authored-by: Arusekk <[email protected]>
* Update Corefile warnings and replace asserts with normal checks
This means that most of the Corefile will still work, even if
we encounter issues while parsing data from the stack.
* Update pwnlib/elf/corefile.py
* Add missing return statement
Co-authored-by: Arusekk <[email protected]>
Detect if pwntools is running in a Windows Subsystem for Linux "VM" and set the terminal to open a new `cmd.exe` window on the host.
This replaces the need to add `context.terminal = ['cmd.exe', '/c', 'start', 'ubuntu', 'run']` to every script.
Debugging using `gdb.attach` under WSL with gdb in a new window out of the box!
This is a WONTFIX, but default gcc on ubuntu compiles .bss (and other RW
segments) so far away from .text that ret2dlresolve mathematically
just has to segfault unless .text and .rodata contain crafted structures
already.
Compiling and working fine on gentoo, though. Hope this proves useful.
Also closes#1448 by the way, it was really annoying.
Closes#1476.
* Enable text colors in Windows console
Consoles on Windows 10 support ANSI escape sequences including color codes.
Emulate unix curses `tigetstr` terminal capabilities by hardcoding the given escape sequences for used curses capabilities.
ANSI escape sequences have to be enabled using `kernel32.SetConsoleMode(hndl, ENABLE_VIRTUAL_TERMINAL_PROCESSING)` for the current console.
* Use object identity operator
* Import ABC from collections.abc for Python 3.
* Fix deprecation warnings due to invalid escape sequences.
* Use raw strings instead of escaping individually.
Previously, the rop._srop_call function would append the args without checking whether their value would work
directly in a register. It now uses AppendedArgument to determine whether the value is pushed to the end of
the ROP chain.
* Update proc.py
reversed([p.pid for p in processes]) only return a iter.
if use gdb.attach('some process name') will cause error like :
File "/home/tsj/.local/lib/python2.7/site-packages/pwnlib/gdb.py", line 657, in attach
pid = pids[0]
TypeError: 'listreverseiterator' object has no attribute '__getitem__'
In this case ,it treat 'pids' as a list,so here should return a list as well
* Apply suggestions from code review
Co-authored-by: Arusekk <[email protected]>
if the value returned from the leak function is a str, it converts it to a bytearray for pytohn3 compatibility. Assuming a string is returned is flaky and likely wrong.
removed the double quotes around the temp filename of the gdbscript as this broke the gdbscript functionality.
The doublequotes would be considered part of the filename and it would fail to read the file..
Tested this fix on Kali.
* The SSH tube now keeps the environment variables of processes it creates in order.
The way of achieving that is through passing the env to the script as a list of tuples,
since Python3's dictionaries are ordered by default, or you can use `OrderedDict`,
the result would be that the spawned process on the remote machine will keep the environment variables in order.
This is very important to exploits that are based on the stack's order and especially the environment variables.
For instance, a common challenge is comparing argc == 0, and afterwards accessing the argv array, as an out-of-bounds array.
The shellcode needs to be at a certain index consistently and cannot rely on the stack moving around.
* Relocated the code.
* Try GitHub actions instead of Travis CI.
This is just a proof-of-concept until we can
have it the clean way it should be done.
More commits will likely follow.
* Travis CI: Use flake8 to detect syntax errors (#1385)
* Travis CI: Use flake8 to detect syntax errors
Repeat #1305 on the Python 3 branch. Also, remove the hardcoding of default values for dist and Py2.
https://flake8.pycqa.org/en/latest/user/error-codes.html
On the flake8 test selection, this PR does _not_ focus on "_style violations_" (the majority of flake8 error codes that [__psf/black__](https://github.com/psf/black) can autocorrect). Instead these tests are focus on runtime safety and correctness:
* E9 tests are about Python syntax errors usually raised because flake8 can not build an Abstract Syntax Tree (AST). Often these issues are a sign of unused code or code that has not been ported to Python 3. These would be compile-time errors in a compiled language but in a dynamic language like Python they result in the script halting/crashing on the user.
* F63 tests are usually about the confusion between identity and equality in Python. Use ==/!= to compare str, bytes, and int literals is the classic case. These are areas where __a == b__ is True but __a is b__ is False (or vice versa). Python >= 3.8 will raise SyntaxWarnings on these instances.
* F7 tests logic errors and syntax errors in type hints
* F82 tests are almost always _undefined names_ which are usually a sign of a typo, missing imports, or code that has not been ported to Python 3. These also would be compile-time errors in a compiled language but in Python a __NameError__ is raised which will halt/crash the script on the user.
* Fail fast, exclude generated source files
* --select=E9,F63,F7
* 2to3 -f print -wn examples
* No more <> in python 3.
* Use ==/!= to compare str, bytes, and int literals (#1386)
Identity is not the same thing as equality in Python so use ==/!= to compare str, bytes, and int literals. In Python >= 3.8, these instances will raise _SyntaxWarnings_ so it is best to fix them now. https://docs.python.org/3.8/whatsnew/3.8.html#porting-to-python-3-8
$ python
```python
>>> raw = 'ra'
>>> raw += 'w'
>>> raw == 'raw'
True
>>> raw is 'raw'
False
```
* Use print() function in both Python 2 and Python 3 (#1387)
$ __pip install black future isort__
$ __futurize -f libfuturize.fixes.fix_print_with_import -w android.py__
$ __isort android.py__
$ __black android.py__
* Add cache and building artifacts
* Enable caching
* Actually enable caching for all of it
* Mistake
* Try different ABI on CI
* Try different ABI + version
* Try it harder
* Update SDK version
* Try building using newer SDK
* Accept licenses
* Ugh android emulator documentation is unfriendly
* Environment variables for AVD
* Add QEMU-system for android virtual devices
* Use the correct emulator
* New android image
* Was actually too new image
* Update ssh credentials to reflect GHA setup
* Fix spelling
* Try finding endless loops
* Actually use the timeout
* Reduce possible hang surface in doctests
* Try SSH immediately when set up
* SSH strict host key checking disable for localhost
* Try stackoverflow.com research
* Try to fix failing doctests
* Add required gcc for cross-arch tests
* Ensure gcc not failing in fmtstr
* Fix issues with shellcraft
* Try different libs
* Try again, with verbosity
* Why do these conflict???
* Fix issues related to PIE binaries
* Add bash-static
* Verbose 'could not set transport: any'
* Add AT_SECURE in auxv to prevent SEGV on libc bug
* Killing ADB server actually kills it
* Android image has /data/user/0 -> /data/data
* Set coverage mode from dev
* Try new actions
* Fix coveralls complaining
* A little time.sleep would not hurt or would it
Co-authored-by: Christian Clauss <[email protected]>
* Allow pwntools to load on Windows
Fixes loading of pwntools on Windows. `from pwn import *` works.
It allows to use some operating system agnostic features of pwntools out of the box.
This is inspired by #996.
Some features fail or even crash (local processes, gdb, assemble/disassemble)
and console colors are still not displayed correctly, but those can be addressed separately.
* Use `os.devnull` instead of manual selection
* Add heuristic to find return address into __libc_start_main
Find the return address from main into __libc_start_main in a libc shared binary. Mimic's niklasb's heuristic in libc-database.
* Add support for different `call` instructions
* Introduce tests for pwnlib.ui
This architecture can be changed later, but I'm quite proud
of how it's done currently.
* Try to debug CI
* Ugly hack to get results back
* Let's just keep it half-broken
Currently, python 3 installed libraries
and sphinx doctree cache are not retained across builds.
This PR addresses this issue, hopefully resulting in a major speedup.
* Travis CI: Use flake8 to detect syntax errors
Repeat #1305 on the Python 3 branch. Also, remove the hardcoding of default values for dist and Py2.
https://flake8.pycqa.org/en/latest/user/error-codes.html
On the flake8 test selection, this PR does _not_ focus on "_style violations_" (the majority of flake8 error codes that [__psf/black__](https://github.com/psf/black) can autocorrect). Instead these tests are focus on runtime safety and correctness:
* E9 tests are about Python syntax errors usually raised because flake8 can not build an Abstract Syntax Tree (AST). Often these issues are a sign of unused code or code that has not been ported to Python 3. These would be compile-time errors in a compiled language but in a dynamic language like Python they result in the script halting/crashing on the user.
* F63 tests are usually about the confusion between identity and equality in Python. Use ==/!= to compare str, bytes, and int literals is the classic case. These are areas where __a == b__ is True but __a is b__ is False (or vice versa). Python >= 3.8 will raise SyntaxWarnings on these instances.
* F7 tests logic errors and syntax errors in type hints
* F82 tests are almost always _undefined names_ which are usually a sign of a typo, missing imports, or code that has not been ported to Python 3. These also would be compile-time errors in a compiled language but in Python a __NameError__ is raised which will halt/crash the script on the user.
* Fail fast, exclude generated source files
* --select=E9,F63,F7
* 2to3 -f print -wn examples
* No more <> in python 3.
* Adding support for FileStructure exploitation
* Actually run the tests + basic py3 fixes
also rename the module to lower case to keep consistency
* Python 3 working
Identity is not the same thing as equality in Python so use ==/!= to compare str, bytes, and int literals. In Python >= 3.8, these instances will raise _SyntaxWarnings_ so it is best to fix them now. https://docs.python.org/3.8/whatsnew/3.8.html#porting-to-python-3-8
$ python
```python
>>> raw = 'ra'
>>> raw += 'w'
>>> raw == 'raw'
True
>>> raw is 'raw'
False
```
* feat(fmtstr): place pointers after format operations
This means that the format string can now also work in certain
cases even if the pointers contain null bytes.
That's especially important on 64 bit where most pointers contain null bytes.
* fix(fmtstr): more stuff
* feat(fmtstr): optimize
* perf(fmtstr): make sorting run in reasonable time (not perfect yet)
* perf(fmtstr): be faster
* style(fmtstr): more docs, some cleanup
* fix(fmtstr): small fixes + more docs
* style(fmtstr): cleanup small style issues
* fix(fmtstr): fix handling of szmax in merge_atoms_overlapping
* fix(fmtstr): fix Atom __getitem__ for out-of-bounds index
* refactor(fmtstr): docs and cleaner code
* fix(fmtstr): small test/doc fixes
* tests(fmtstr): add example as tests for other arches
* doc(fmtstr): more docstrings
* fix(fmtstr): fix cyclic_find warnings
* refactor(fmtstr): remove another nested function
* tests(fmtstr): fix doctest output for wrong arch
* style(fmtstr): remove unused imports
* tests(fmtstr): add binaries for tests
* Updates to requirements.txt.
Removed the < 1.8.0 requirement for sphinx. It appeared no longer
necessary.
Alphabetized the listing for easier searching.
Moved the requirements file under the docker folder. Docker does not
like relative paths to parent directories due to security restrictions.
* Updated Dockerfile to use Python3.
* Fixes to srop test errors.
Moving the imports to the beginning of the file resolved the issue.
The following errors were popping up during testing within Docker:
Document: rop/srop
------------------
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.recvline()
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.5/doctest.py", line 1321, in __run
compileflags, 1), test.globs)
File "<doctest default[18]>", line 1, in <module>
p.recvline()
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 467, in recvline
return self.recvuntil(self.newline, drop = not keepends, timeout = timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 310, in recvuntil
res = self.recv(timeout=self.timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 82, in recv
return self._recv(numb, timeout) or b''
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 160, in _recv
if not self.buffer and not self._fillbuffer(timeout):
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 131, in _fillbuffer
data = self.recv_raw(self.buffer.get_fill_size())
File "/home/travis/pwntools/pwnlib/tubes/process.py", line 707, in recv_raw
raise EOFError
EOFError
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.poll(block=True)
Expected:
0
Got:
-11
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.recvline()
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.5/doctest.py", line 1321, in __run
compileflags, 1), test.globs)
File "<doctest default[18]>", line 1, in <module>
p.recvline()
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 467, in recvline
return self.recvuntil(self.newline, drop = not keepends, timeout = timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 310, in recvuntil
res = self.recv(timeout=self.timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 82, in recv
return self._recv(numb, timeout) or b''
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 160, in _recv
if not self.buffer and not self._fillbuffer(timeout):
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 131, in _fillbuffer
data = self.recv_raw(self.buffer.get_fill_size())
File "/home/travis/pwntools/pwnlib/tubes/process.py", line 707, in recv_raw
raise EOFError
EOFError
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.poll(block=True)
Expected:
0
Got:
-11
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.recvline()
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.5/doctest.py", line 1321, in __run
compileflags, 1), test.globs)
File "<doctest default[18]>", line 1, in <module>
p.recvline()
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 467, in recvline
return self.recvuntil(self.newline, drop = not keepends, timeout = timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 310, in recvuntil
res = self.recv(timeout=self.timeout)
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 82, in recv
return self._recv(numb, timeout) or b''
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 160, in _recv
if not self.buffer and not self._fillbuffer(timeout):
File "/home/travis/pwntools/pwnlib/tubes/tube.py", line 131, in _fillbuffer
data = self.recv_raw(self.buffer.get_fill_size())
File "/home/travis/pwntools/pwnlib/tubes/process.py", line 707, in recv_raw
raise EOFError
EOFError
**********************************************************************
File "rop/srop.rst", line ?, in default
Failed example:
p.poll(block=True)
Expected:
0
Got:
-11
**********************************************************************
1 items had failures:
6 of 159 in default
159 tests in 1 items.
153 passed and 6 failed.
***Test Failed*** 6 failures.
* Fix for docstring warning in dynelf.py.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/dynelf.py:docstring of pwnlib.dynelf.DynELF._resolve_symbol_sysv:11: WARNING: Definition list ends without a blank line; unexpected unindent.
* Fix for docstring warnings in gdb.py.
The following warnings are resolved with this fix:
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_assembly:8: WARNING: Inline strong start-string without end-string.
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_assembly:21: WARNING: Block quote ends without a blank line; unexpected unindent.
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_shellcode:4: WARNING: Inline strong start-string without end-string.
/home/travis/pwntools/pwnlib/gdb.py:docstring of pwnlib.gdb.debug_shellcode:17: WARNING: Block quote ends without a blank line; unexpected unindent.
* Fix for adb/__init__.py.
The following error is resolved with this fix:
Document: protocols/adb
-----------------------
**********************************************************************
File "../../pwnlib/protocols/adb/__init__.py", line ?, in default
Failed example:
pprint(AdbClient().list('/data/user'))
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.5/doctest.py", line 1321, in __run
compileflags, 1), test.globs)
File "<doctest default[0]>", line 1, in <module>
pprint(AdbClient().list('/data/user'))
TypeError: 'module' object is not callable
**********************************************************************
1 items had failures:
1 of 10 in default
10 tests in 1 items.
9 passed and 1 failed.
***Test Failed*** 1 failures.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/protocols/adb/__init__.py:docstring of pwnlib.protocols.adb.AdbClient.stat:6: WARNING: Inline literal start-string without end-string.
* Fix for docstring warning in rop.py.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/rop/rop.py:docstring of pwnlib.rop.rop.DescriptiveStack.descriptions:3: WARNING: Inline literal start-string without end-string.
* Fix for docstring warning in ssh.py.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/tubes/ssh.py:docstring of pwnlib.tubes.ssh.ssh._init_remote_platform_info:6: WARNING: Block quote ends without a blank line; unexpected unindent.
* Doctest dictionary ordering problem in misc.py.
There was an ordering problem while running the doctest in misc.py.
Sorting the dictionary with pprint and updating the results fixed the
issue.
Document: util/misc
-------------------
**********************************************************************
File "../../pwnlib/util/misc.py", line ?, in default
Failed example:
sizes
Expected:
{'eax': 32, 'ax': 16, 'al': 8, 'ah': 8, 'ebx': 32, 'bx': 16, 'bl': 8, 'bh': 8, 'ecx': 32, 'cx': 16, 'cl': 8, 'ch': 8, 'edx': 32, 'dx': 16, 'dl': 8, 'dh': 8, 'edi': 32, 'di': 16, 'esi': 32, 'si': 16, 'ebp': 32, 'bp': 16, 'esp': 32, 'sp': 16}
Got:
{'si': 16, 'cx': 16, 'cl': 8, 'dl': 8, 'dh': 8, 'eax': 32, 'bx': 16, 'sp': 16, 'bl': 8, 'bh': 8, 'di': 16, 'esi': 32, 'edx': 32, 'dx': 16, 'ebx': 32, 'ebp': 32, 'ax': 16, 'ch': 8, 'al': 8, 'ah': 8, 'edi': 32, 'esp': 32, 'bp': 16, 'ecx': 32}
**********************************************************************
File "../../pwnlib/util/misc.py", line ?, in default
Failed example:
bigger
Expected:
{'eax': ['eax'], 'ax': ['eax', 'ax'], 'al': ['eax', 'ax', 'al'], 'ah': ['eax', 'ax', 'ah'], 'ebx': ['ebx'], 'bx': ['ebx', 'bx'], 'bl': ['ebx', 'bx', 'bl'], 'bh': ['ebx', 'bx', 'bh'], 'ecx': ['ecx'], 'cx': ['ecx', 'cx'], 'cl': ['ecx', 'cx', 'cl'], 'ch': ['ecx', 'cx', 'ch'], 'edx': ['edx'], 'dx': ['edx', 'dx'], 'dl': ['edx', 'dx', 'dl'], 'dh': ['edx', 'dx', 'dh'], 'edi': ['edi'], 'di': ['edi', 'di'], 'esi': ['esi'], 'si': ['esi', 'si'], 'ebp': ['ebp'], 'bp': ['ebp', 'bp'], 'esp': ['esp'], 'sp': ['esp', 'sp']}
Got:
{'si': ['esi', 'si'], 'cx': ['ecx', 'cx'], 'cl': ['ecx', 'cx', 'cl'], 'dl': ['edx', 'dx', 'dl'], 'dh': ['edx', 'dx', 'dh'], 'eax': ['eax'], 'bx': ['ebx', 'bx'], 'sp': ['esp', 'sp'], 'bl': ['ebx', 'bx', 'bl'], 'bh': ['ebx', 'bx', 'bh'], 'di': ['edi', 'di'], 'esi': ['esi'], 'edx': ['edx'], 'dx': ['edx', 'dx'], 'ebx': ['ebx'], 'ebp': ['ebp'], 'ax': ['eax', 'ax'], 'ch': ['ecx', 'cx', 'ch'], 'al': ['eax', 'ax', 'al'], 'ah': ['eax', 'ax', 'ah'], 'edi': ['edi'], 'esp': ['esp'], 'bp': ['ebp', 'bp'], 'ecx': ['ecx']}
**********************************************************************
File "../../pwnlib/util/misc.py", line ?, in default
Failed example:
smaller
Expected:
{'eax': ['ax', 'al', 'ah'], 'ax': ['al', 'ah'], 'al': [], 'ah': [], 'ebx': ['bx', 'bl', 'bh'], 'bx': ['bl', 'bh'], 'bl': [], 'bh': [], 'ecx': ['cx', 'cl', 'ch'], 'cx': ['cl', 'ch'], 'cl': [], 'ch': [], 'edx': ['dx', 'dl', 'dh'], 'dx': ['dl', 'dh'], 'dl': [], 'dh': [], 'edi': ['di'], 'di': [], 'esi': ['si'], 'si': [], 'ebp': ['bp'], 'bp': [], 'esp': ['sp'], 'sp': []}
Got:
{'si': [], 'cx': ['cl', 'ch'], 'cl': [], 'dl': [], 'dh': [], 'eax': ['ax', 'al', 'ah'], 'bx': ['bl', 'bh'], 'sp': [], 'bl': [], 'bh': [], 'di': [], 'esi': ['si'], 'edx': ['dx', 'dl', 'dh'], 'dx': ['dl', 'dh'], 'ebx': ['bx', 'bl', 'bh'], 'ebp': ['bp'], 'ax': ['al', 'ah'], 'ch': [], 'al': [], 'ah': [], 'edi': ['di'], 'esp': ['sp'], 'bp': [], 'ecx': ['cx', 'cl', 'ch']}
**********************************************************************
1 items had failures:
3 of 20 in default
20 tests in 1 items.
17 passed and 3 failed.
***Test Failed*** 3 failures.
* Fix for docstring warning in packing.py.
The following warning is resolved with this fix:
/home/travis/pwntools/pwnlib/util/packing.py:docstring of pwnlib.util.packing.flat:2: WARNING: Inline emphasis start-string without end-string.
* Removed commented out test in Dockerfile.
* Forgot to update location of requirements.txt.
Forgot to update the location of requirements.txt in .travis.yml.
* Made recommended changes from Arusekk.
pwnlib.shellcraft installs its custom import finder (LazyImporter) into
sys.meta_path.
According to PEP 302, finder is expected to have prototype
finder.find_module(fullname, path=None)
Note that the second argument is optional. pwnlib's LazyImporter has second
argument required, which leads to compatibility problems, surfacing
when trying to use pwnlib with other third-party modules:
>>> import pwnlib
>>> import faker
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/tmp/.venv/lib/python2.7/site-packages/faker/__init__.py", line 2, in <module>
from faker.factory import Factory # noqa F401
File "/tmp/.venv/lib/python2.7/site-packages/faker/factory.py", line 12, in <module>
from faker.config import AVAILABLE_LOCALES, DEFAULT_LOCALE, PROVIDERS
File "/tmp/.venv/lib/python2.7/site-packages/faker/config.py", line 15, in <module>
AVAILABLE_LOCALES = find_available_locales(PROVIDERS)
File "/tmp/.venv/lib/python2.7/site-packages/faker/utils/loading.py", line 45, in find_available_locales
provider_module = import_module(provider_path)
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/tmp/.venv/lib/python2.7/site-packages/faker/providers/internet/__init__.py", line 4, in <module>
from text_unidecode import unidecode
File "/tmp/.venv/lib/python2.7/site-packages/text_unidecode/__init__.py", line 6, in <module>
_replaces = pkgutil.get_data(__name__, 'data.bin').decode('utf8').split('\x00')
File "/usr/lib64/python2.7/pkgutil.py", line 576, in get_data
loader = get_loader(package)
File "/usr/lib64/python2.7/pkgutil.py", line 462, in get_loader
return find_loader(fullname)
File "/usr/lib64/python2.7/pkgutil.py", line 473, in find_loader
loader = importer.find_module(fullname)
TypeError: find_module() takes exactly 3 arguments (2 given)
* Removed old dependency for intervaltree and sortedcontainers
Also make the pwn command show usage if no arguments supplied
* Fix errno subcommand
The errno library moved out of os and into its own library
* Change template script to use system env python instead of only py2
* Require updated intervaltree in setup.py
* Update .travis.yml
* Update .travis.yml
* Tubes fixed
* Typo
* Typo + default value
* Fixes on and on
* Many many more
* Missing lines and tests
* Refactoring
* Missing line again
* Missing again
* Copypaste design patterns
Use ==/!= to compare str, bytes, and int literals: __fmt is not 'raw'__ --> __fmt != 'raw'__
$ __python__
```python
>>> raw = 'ra'
>>> raw += 'w'
>>> raw
'raw'
>>> raw == 'raw'
True
>>> raw is 'raw'
False
```
coredumpctl is a tool that can be used to retrieve and process
core dumps and metadata which were saved by systemd-coredump(8).
systemd-coredump works by setting a piped core pattern. For example:
$ cat /proc/sys/kernel/core_pattern
|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e
This commit detects `systemd-coredump` in the pattern and then uses
`coredumpctl`, which is the CLI utility for reading the coredumps.
`coredumpctl` is instructed to write the coredump to a file. The
coredump is specified using the process ID and the basename of the
executable.
This commit also updates some doc strings to fix them and make them
more useful.
This fixes#1177, #1178
Ubuntu 16.04 finally includes `binutils-mips-linux-gnu`,
which is required. So there is no need to download
packages manually anymore. This can help supporting
python 3.
* corefile: Update NT_XXX type names for pyelftools==0.25
* travis: Do not use Sphinx version 1.8.0, it fails with recursive errors
* corefile: Fix stack detection and Mapping.__contains__ logic
* corefile: Better detection of AT_EXECFN when NT_AUXV does not have it
* corefile: Avoid trying to parse corrupt stacks
* Fix doctests for new warning
* log: Fix warning_once caching
* corefile: also expose page_offset
This is useful to load the code from the original files.
* Undo constructor field reordering
* Fix ordering in Mapping instantiation
* Commas are important
* corefile: also put page_offset as last parameter
* Improve pwnup template
* Don't want to break at main again if its been hit already
* Always start the program
* Handle extra newline printed by gdb
* Remove moved recvline
* Remove controversial set sysroot
* Update gdb.py
* Update pwnup.mako
* fix wrong gadget selection in setRegisters() (#1118)
* Fix detection logic
* Update rop.py
* add test cases for filtering out the bad instructions in rop.py
* Replace division with floor division as of PEP-238
This will also get us closer to python3.
See https://www.python.org/dev/peps/pep-0238/
* Add from __future__ import division to source files
* Fix tests relying on broken autocontrib version (#1121)
* Removed tuple parameters unpacking due to PEP-3113
As a step towards python3 compatibility, removed tuple parameters
in :mod:`pwnlib.term.term`. They are now replaced with * unpacking.
See https://www.python.org/dev/peps/pep-3113/
Check for sha256 and sha256sum before trying openssl sha256 as "openssl sha256" doesn't return an error code in some old versions of openssl (OpenSSL 0.9.8g 19 Oct 2007 at least) that doesn't support sha256.
* Add sysroot= argument to gdb.debug, expose via 'pwn debug'
Also sets a sane default sysroot for Android debugging
* Add missing newline for "set deatch-on-fork off"
Introduced in e89a9b66ac
* Add support for extracting Android boot.img
* Fix incorrect variable in 'pwn debug'
* Ensure "file /path/to/file" is the first command
When one passes a `LD_PRELOAD` which cannot be preloaded the `process_created` string might end up looking like this:
```
"ERROR: ld.so: object '/bin/bash' from LD_PRELOAD cannot be preloaded (cannot dynamically load executable): ignored.\n"
```
When we end up in such situation, pwndbg shown unintuitive error:
```
$ python -c 'from pwn import *; gdb.debug("/bin/ls", env={"LD_PRELOAD": "/bin/bash"})'
[+] Starting local process '/usr/bin/gdbserver': pid 20691
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "pwnlib/context/__init__.py", line 1349, in setter
return function(*a, **kw)
File "pwnlib/gdb.py", line 397, in debug
port = _gdbserver_port(gdbserver, ssh)
File "pwnlib/gdb.py", line 234, in _gdbserver_port
gdbserver.pid = int(process_created.split()[-1], 0)
ValueError: invalid literal for int() with base 0: 'ignored.'
```
After this patch it will show:
```
$ python -c 'from pwn import *; gdb.debug("/bin/ls", env={"LD_PRELOAD": "/bin/bash"})'
[+] Starting local process '/usr/bin/gdbserver': pid 20863
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "pwnlib/context/__init__.py", line 1349, in setter
return function(*a, **kw)
File "pwnlib/gdb.py", line 402, in debug
port = _gdbserver_port(gdbserver, ssh)
File "pwnlib/gdb.py", line 236, in _gdbserver_port
'Failed to spawn process under gdbserver. gdbserver error message: %s' % process_created
ValueError: Failed to spawn process under gdbserver. gdbserver error message: ERROR: ld.so: object '/bin/bash' from LD_PRELOAD cannot be preloaded (cannot dynamically load executable): ignored.
[*] Stopped process '/bin/ls' (pid 20863)
```
As @bennofs noticed, our detection logic was incorrect:
"This is not a good test if I understand the issue correctly.
From my tests, it seems that fault_addr was only null before this patch if it
would've been >= 2**47.
This is because on 64 bit, that's where kernel memory is mapped so we get
"protection fault" instead of "segfault".
A protection fault does not have the same siginfo as a normal segfault:
si_code will be SI_KERNEL = 0x80 and there won't be a fault_addr
(see http://elixir.free-electrons.com/linux/v4.14-rc8/source/kernel/signal.c#L1052)"
* Handle debugging under Android better with 'pwn debug'
Previously, when using --process or --pid, we would terminate the gdbserver instance
when pwntools exited. Now we wait for the user to tell us to exit.
* Handle launching Android processes by filename
* Use the --multi option with gdbserver; disable detach-on-fork
* Intentionally break doctest to make sure its running
* Revert "Intentionally break doctest to make sure its running"
This reverts commit 00578ad177.
* Copy env in process._validate using .copy method to keep the order in case of OrderedDict
* Use supplied env object to preserve order in case of OrderedDict
* Fix corefile environment variables and speed up ELF.string()
* Even more doctests
* Fix doctest arch
* No longer split on = since it isn't necessary
* Fix arch again
* Delete intermediate corefiles
* Do not flag for extended testing when the commit SHA is what matches
* Add support for finding QEMU's LD_PREFIX
* Refactor pwnlib.qemu and add official documentation
* Add more documentation, support for sysroot
* More docs, binfmt info
* Fix variable name overload
* Fix missed refactor
* Check for empty/null env
* Add qemu.rst to index
* Add cyclic default args to context
Specifically, this adds context.cyclic_alphabet and context.cyclic_size
This helps cut down on writing "n=8" all the time for users who use 8-byte
subsequences on 64-bit architectures.
Exposing the default alphabet means that users can avoid "bad bytes" in
cyclic patterns which are generated by pwntools internally (e.g. ROP padding)
Some pwnlib.util.packing docstrings were changed to clarify that word_size is
in bits.
The doctests in pwnlib.util.cyclic were updated and extended to have more
and better doctests, as well as better exaples.
* Add an example showing an alphabet with NUL bytes
This is important because old code would use pack(..., "all") when an integer
value was provided, instead of using the correct width.
* Overhaul ROP.setRegisters to support more complex cases needed by AMD64
Additionally, have a separate cache for ROPs composed of multiple ELF files
* Better presentation for stack padding slots
* Remove unnecessary stack adjustment on amd64
For amd64, it is possible that len(stackArguments) == 0.
* Add a doctest for amd64 register dependency stuff
* More i386 adjust optimizations
* Use nextGadgetAddr as the adjust value
* Always use padding for the last "ret" adjustment
* Better dump display
* Update doctests for new ROP dump() enhancements
* Add badchars support for ROP gadgets
* More doctest fixes, also ensure ROP.search does not return gadgets ending in e.g. int 0x80
* Prevent Sphinx from dumping all level=INFO logging output to stdout.
See Gallopsled/pwntools#1036 for more information.
* Filter out Paramiko logs too, apply filter to all root handlers during testing
* Remove unused import
(cherry picked from commit 54c47b3daa)
Technically this introduces issues with e.g. 'unhex ffdf | python foo.py' if
foo.py uses .interactive(). However, that was already broken for other reasons.
Fixes#1004
(cherry picked from commit ebbe917713)
* Slightly enhance template and advertise for the feature itself in scripts
Add single-dash to the "very good" characters in shell escaping, to avoid
the need to single-quote things like --host and --port.
* Add --quiet option for pwn template, move gdbscript down a bit
* Slightly better docs for 'pwn template'
* Add checksec information directly to 'pwn template' output
* Prevent Sphinx from dumping all level=INFO logging output to stdout.
See Gallopsled/pwntools#1036 for more information.
* Filter out Paramiko logs too, apply filter to all root handlers during testing
* Remove unused import
* DynELF: trust ELF base address
* Bug fixes for DynELF:
* Fixed lookup when not using libcdb
* Fixed _find_linkmap returning 0 instead of None when linkmap doesn't
exist
* Fixed _lookup_build_id crashing if linkmap cannot be found
* Misc cleanup
* Changes based on feedback
Technically this introduces issues with e.g. 'unhex ffdf | python foo.py' if
foo.py uses .interactive(). However, that was already broken for other reasons.
Fixes#1004
* Force use of Ubuntu 12 builds on Travis CI until we can migrate to 14
* Fix Travis Dockerfile
* Add bash command logging to Travis scripts, since Travis removed its own logging
* curl should follow redirects when grabbing Ubuntu packages
* Update Travis cache directories to newer version of Python, as seen in Travis logs
* Use ::1 instead of ip6-localhost since Travis 12.04 changes broke it
* fix typo in comment
* new config option: context.gdbinit
This allows setting the default gdbinit for GDB, which is helpful when you want
to enable some GDB extension only when using pwntools and not by default.
- Detect bootloaders that are already unklocked
- Detect bootloader unlock success
- Detect bootloaders that cannot be unlocked
- Detect devices which have not enabled bootloader unlocking
* Fix Full RELRO detection in ELFs.
This add supports for the newer DF_BIND_NOW and DF_1_NOW flags.
These are used instead of DT_BIND_NOW when linking with gold.
Also fixed the order of checks, now it reports "No RELRO" when
BIND_NOW is present but GNU_RELRO is not (in which case the bindings
are done at start but they will not be marked as read-only).
* Add x86 test binaries for relro check.
The check for when certain ELF fields contain offsets instead of pointers can be
made more precise by checking for the type field in the ELF header instead of
comparing the value to some hard-coded range.
* Clean up socket-class and socket-type code in sock/remote/listen
* Slight change to docs
* Add doctests for listen and attribute documentation
* Fix bug in listen.close()
* Fix regression on ipv6 sockets
* Simplify documentation
* Fix doctest
* Do not use UC_MODE_BIG_ENDIAN, its support is poor
* Ghetto-swap endianness for emulation of PLT instructions
* Reduce instruction alignment so that all ARM PLT entries are found
* Add tests for ELF PLT loading
* Fix canary detection on PowerPC
* Fix x32 emulation
* Fix two failing doctests
* Add AArch64 to supported big-endian architectures
* Add MIPS GOT parsing code, fix unpack macros to use ELF settings
* Why, MIPS, are you so complicated?
* Catch weird edge cases
* I don't know what's real anymore
* Fix synthetic symbol generation and add tests
* Remove canary test
* Guard against errors in MIPS GOT
* Fix doctests
* Do not fail with gdb.debug() over ssh when given an absolute path
FixesGallopsled/pwntools#885
* Prevent failing on stable since this directory doesn't exist
* Doctest all members
* Fix doctest failures in asm and remove canary in context
* Fix failing doctest
* Use automodule for pwnlib.context
* Fix failing tests in context module
* Use automodule for pwnlib.memleak
* Fix checksec nx, execstack, relro reporting
Also add a disable_nx function
* Add disablenx command-line utility
Also fix bug where invoking command-line utilities directly by module name
(python -m pwnlib.commandline.foo) would fail
This adds functionality for user configuration files at ~/.pwn.conf
and /etc/pwn.conf. Previously this was only used by the pwnlib.log
module, and was entirely undocumented.
This is now documented, and offers an easy mechanism for other parts
of the code to have extension points.
* Remove unavailable syscalls from AArch64
The Linux kernel has apparently stopped supporting these
* Rewrite shellcraft.aarch64.linux.cat, add open which uses openat
FixesGallopsled/pwntools#896
* Fix doctests to use openat
* Add templates from generator
* Add abi.stack to all ABIs
* Enable shellcraft.arch.os.foobar.template
* Add pwnlib.registers.current()
* Add syscalls symlinks
* Add syscall generator
* Enable shellcraft.foo() shortcut for shellcraft.arch.os.syscalls.foo()
* Make templates more verbose
* Remove old templates
* Add makefile
* Delete old templates, make shellcraft --list work
This also speeds up boot time by like 0.05 seconds
* Remove unnecessary execve wrappers
* Check for common arch as well
* Don't explode on string arguments that are register names
* Catch bad argument names
* Remove accidental import
* Add aarch64 ABIs since we enabled aarch64 tests
* Fix mmap vs mmap2 syscall
* Make shellcraft command-line work for syscalls
* Remove unnecessary wrapper templates
* Set all default values to zero
* Add example of automatic determination of types
* Add missing register architectures
* Add PowerPC ABIs
* More syscall examples
* Handle mmap vs mmap2
* Add missing import, use full path
* Just use the string name
* Fix rt_sigreturn on amd64
* __NR_ --> SYS_ because of old expectations
* rstrip to remove blankline
* Fix Thumb doctests which uses unpredictable jump labels
* Import additional syscalls by updating the function source.
Support variable-argument syscalls.
Support spilling to the stack.
* Simplified constants.eval logic means no extra parens
* Simplify vararg representation; fix oepn() argument name explicitly
* Fix syscall examples
* Fix reordering of arguments, since we have to support stack spill
* Fix argument name in doctest
* Make Codacy happy
* Restore connect template accidentally removed in 9fceb12b
* Scrape all architectures for syscall names
* Enable void* and void** to be pushed
* Make Codacy happy
* Add __doc__ to sdist
* Follow shellcraft template symlinks
* Detect syscall shellcodes in shellcraft based on path
(cherry picked from commit 94462845fa)
Adds a large amount of functionality around Corefiles, to support smarter exploitation
Adds a ".corefile" property to both local and remote (ssh) processes. See the
documentation for the corefile module for extensive examples and tests.
Additionally, context.delete_corefiles and context.rename_corefiles have been added,
to control the behavior of the ".corefile" property.
* Add aarch64 documentation, which enables doctests
* Fix failing doctest
* Attempt to figure out whats going on with Travis
* Use the Xenial version of qemu-static since the Zesty version is broken for AArch64
- Show the PID when a process is created
- Make process.cwd into a property, which fetches the real CWD from procs
- Record the UID/GID and SUID/SGID of the executed file
- Add a few doctests
- Make _stop_noticed a timestamp
- Break out ssh_channel into subclass ssh_process
- Make ssh.process() directly instantiate ssh_process
- Add ssh_process().ELF
- Add ssh_process().corefile
- Record the UID and GID of the executed binary
- Add the created PID to the ssh_process message
* Fix chunked reading of ELF files
This introduces a new dependency on 'intervaltree', which is used
to automatically select segments which are relevant for a given
virtual-address range.
This also introduces a new ELF.memory attribute, which is the tree
of memory ranges, which point at the segment objects that describe
the memory range.
These are recalculated when ELF.address is adjusted, and we have
tests for that.
This also optimizes some of the other routines to avoid the wrapped
file-seeking, and to just use the mmap.
* Rewrite install scripts to use more modern versions of binutils
* Load segments in file-order, chop any overlaps
* Add pwnlib.protocols module
Migrate pwnlib.adb.protocol into that directory, and add compatibility shims
* Add documentation links, and run doctests
* Make doctests run
* pwnlib.protocols.adb.Client.transport should work with no device
We cannot determine the length of the file if passed e.g. a tube or a pipe or PTY or character device (e.g. stdin or /dev/null).
Supercedes: 9c2f0f4e
(cherry picked from commit 14f717b7b4)
This code wasnt updated when the code was changed to operate on file descriptors.
Additionally, there was a functional regression, that the first line would be skipped.
Fixes: Gallopsled/pwntools#836
Caused by: Gallopsled/pwntools#695 (d65991d)
Looks like this:
[x] Connecting to example.pwnme on port 22
[+] Connecting to example.pwnme on port 22: Done
[*] [email protected]:
Distro Ubuntu 14.04
OS: linux
Arch: amd64
Version: 3.11.0
ASLR: Enabled
Note: Susceptible to ASLR ulimit trick (CVE-2016-3672)
FixesGallopsled/pwntools#753
* Defer attempts to start ADB server until we actually fail to connect
* Upgrade to a warning message; fix .version() multiple calls with _autoclose
* Add doctest which tests the kill/restart behavior
* Add PR_SET_PTRACER to initialization, and add doctests for process.leak/corefile
* Add PR_SET_PTRACER to SSH processes
* Do not warn about YAMA if we dont need to
* Fix doctest to use raw strings
* Fix typo in doctest
* Apparently GDB is not installed on Travis by default
* Add warning message to process("foo") vs process("./foo")
* Add gdb.corefile; make run_in_new_terminal return a PID
* Intentionally break doctest because I dont believe that it worked
* Fix codacy issues
* Fix gdb.attach return value
* Add missing import
* Fix typo, use waitpid
* Refactor some code into gdb.version / gdb.binary
* Permit GDB with multipart versions
* Fix repr() on Corefile
* Silence run-in-new-terminal message for gdb.corefile
* Fix version number
* Remove Yama warning message entirely, it should no longer be necessary
* Version detection is hard. Regular expressions to the rescue!
* Remove process.kill/stop/resume, those will be a different PR
* Fix doctests
* The GDB used by Travis must be so old!
* Apparently Travis is entirely borked. Use a less robust test.
* Enable leaking memory relatively with MemLeak
FixesGallopsled/pwntools#758
* Add RelativeMemLeak alias for MemLeak(..., relative=True)
* Make codacy happy
If pandoc is not installed locally, pwntools' setup.py
tells pypandoc to download one.
Unfortunately, it fails on non-amd64 arches because
pypandocs provides precompiled binaries only for amd64.
Since pypandoc is not hard dependency for pwntools,
this error can be skipped without major harm.
In fact, pwntools' setup.py already handles ImportError
gracefully. The problem is, arch mismatch causes
a completely different exception.
* Remove support for installation without psutil.
OpenBSD was the edge case we cared about, it is supported for over a year (giampaolo/psutil#615)
* Bump psutil version to when they introduced OpenBSD support
* Fix ADB wait_for_device with ANDROID_SERIAL set, and fix performance
* Fix QEMU usage for native binaries
* Fix adb.write and adb.listdir operating on the wrong type of object
* Fix failing unit test with new behavior in adb.protocol.Client.list
* Fix adb server startup issue
* Rework deferred initialization to work around class-scope superclass attributes, and add test for AdbDevice
* Add adb.{unlink,mkdir,makedirs,isdir,exists}
* Change unlink() to throw an exception rather than return a string
* Fix doctest and code for isdir
* Remove unused variable
* Fix adb.partitions listing
* Add adb.walk and adb.find functions to ensure adb.partitions works on devices without a find binary
* Add adb.readlink to simplify logic for reading symlinks
Support custom buffer-read sizes for `tube` objects via their internal `Buffer` object. Default buffer size is 4096. Buffer size can be set per-object at instantiation-time (`tube(buffer_fill_size=...)`), by attribute (`tube.buffer_fill_size=...`), or in a context (`context.buffer_size=...`).
Previously, if getaddrinfo or socket.socket threw an exception,
_connect would return without ever calling h.success or h.failure. As a
result, progress indicators would continue to update indefinitely.
Fix this by using a with statement instead.
(Also remove an unused variable and fix a typo.)
* adding output format for python hex-notation
* cleaned up implementation
* changed format identifier to {a} --escaped
* {a] was taken by asm, changed format identifier to {d} --escaped
* adding {e}lf to help
* Extract copy-pasted callback routine, add better errors and doctests for adb.{push,pull}
* Add protocol submodule to documentation and doctests
* Add and fix doctests in adb.protocol
* Use pprint to sort dicts
* Re-implement sh_string to use embedded literals, add lots of tests
* Add docs for sh_string so that unit tests get picked up
* Mark the docstrings as raw
* Rewrite doctests to not "print" since this breaks with invisible characters (issue with doctest itself)
* Add test which uses 16KB of random non-NUL data
An error message will be printed like this when a package is found:
>>> context.arch='powerpc'
>>> pwnlib.asm.print_binutils_instructions('objdump', context)
[ERROR] Could not find 'objdump' installed for ContextType(arch = 'powerpc', bits = 32, endian = 'big')
Try installing binutils for this architecture:
$ sudo apt-get install binutils-powerpc-linux-gnu
* Add Android emulator to Travis
* More robust handling of ADB device status, and lack of devices
* Add platform-tools to PATH for adb, fastboot, etc.
* Add basic tests for some ADB code
* Add with_device annotation to auto-detect a device
* Skip Android emulator installation if we dont need it
* Do not install emulator in all cases (e.g. OSX)
Adds an install-time option to disable legacy commands, and only use 'pwn':
$ pip install --install-option='--only-use-pwn-command'
Documentation is updated to show 'pwn' entry point and sub-commands.
Fixes#404Fixes#660
Adds an install-time option to disable legacy commands, and only use 'pwn':
$ pip install --install-option='--only-use-pwn-command'
Documentation is updated to show 'pwn' entry point and sub-commands.
Fixes#404Fixes#660
* Add MIPS documentation, which should also trigger doctests
* Fix broken and missing doctests for MIPS pushstr / syscall
* Add optimization for mips.mov to fix doctest
* Exit cleanly so QEMU doesn't complain
In two instances, the file was not removed because of an incorrect lambda.
In two further instances, use of shell concatenation via run_in_new_terminal
were changed into GDB 'shell' commands.
This fixesGallopsled/pwntools#629
The clear is necessary because vars(context) will only retrieve the set keys. Any other context bits set later will not be correctly set to default values.
This bug happened because we no longer disable input processing in the terminal.
In particular `\r` is now translated to `\n`. The fix is to interpret both `\r`
and `\n` as <enter> in `term/key.py`.
(cherry picked from commit 383f245082)
When installing pwntools, setup.py required a very new version of
requests -- even though we never used any new features.
This is a problem, since the required version is much never than what is
in e.g. ubuntu's package repo. While this by itself is not much of a
problem, it is a problem if one installs python-pip from the distro's
repo as well, since python-pip will try to use the newest version of the
requests library, which caused pip to break.
The solution: Do not force such a new version of python, instead live
with a much older version.
(cherry picked from commit f91299986e)
Code says context.bits should be > 0, but error message says >= 0.
Since 0 bit architecture doesn't make sense at all, make them all be > 0.
(cherry picked from commit def9400d48)
When installing pwntools, setup.py required a very new version of
requests -- even though we never used any new features.
This is a problem, since the required version is much never than what is
in e.g. ubuntu's package repo. While this by itself is not much of a
problem, it is a problem if one installs python-pip from the distro's
repo as well, since python-pip will try to use the newest version of the
requests library, which caused pip to break.
The solution: Do not force such a new version of python, instead live
with a much older version.
Changes:
- `pwn/init` initialization code simplified, and some was moved to `pwnlib/args`
- `scramble` script added to `pwnlib.commandline`; support for encoders added to relevant scripts
- Added shellcode encoders for several architectures
- Added CGC architecture shellcode and constants
- Added Shellcraft templates for every syscall
- Added `xor_key` helper for generating a 4-byte XOR key for a data stream
- Added `getdents` for use with the relevant shellcraft scripts
- Add ELF loaders for all architectures
- Lots of MIPS and AArch64 shellcode
This bug happened because we no longer disable input processing in the terminal.
In particular `\r` is now translated to `\n`. The fix is to interpret both `\r`
and `\n` as <enter> in `term/key.py`.
Add information about various ARM coprocessors that might be present.
Correct typo
Add in comments
Change how magic numbers and offsets are stored
Use namedtuple
Renaming variables
srop.py doc modified
ARM sp check added in
Cleanup
Changed to detect X11 by the presence of the ``DISPLAY`` environment variable. Also added the field `terminal` to `context`, which allows the user to specify a default terminal.
- rop.py does not import _registers_* anymore; instead imports get_registers from srop.py
- rop.py now has a self.arch that is set to context.arch inside __init__
- (_registers_32, _registers_64) => (_registers_i386, _registers_amd64)
- get_registers and get_regpos_mapping added in
- _initialize_values now handles all archs
- get_stackpointer_index changed to get_spindex; made cleaner
- get_frame made cleaner
So now we can populate the accumulator register with the sigreturn syscall number
rop.py now finds "int 0x80" instructions.
Adding in more to srop skeleton; integrating with finding int 0x80 instruction
The following changes were made :-
- s/find/index correction for list
- required imports added in
- None as argument removed
Now generates srop frames whenever function cannot be resolved
Add in code that pivots only if the gadget is not a "pop"
dump is now modified to print out the register information from srop
Uses constants.linux.i386 for syscall numbers now
Added in some exception handling code
Rely on constants to get us the right syscall number based on context.arch
The following changes were made :-
- getattr when getting syscall_number now returns None by default
- ropgadget now looks for syscall and sysenter
- added in getattr("syscall") and getattr("sysenter")
rop.search now returns a namedtuple
The following changes were made :-
1. self.int80, self.syscall and self.sysenter now return namedtuples.
2. We have functions self.raw_address and self.raw_data to specify
how the raw values that are added in, need to be interpreted so that
the chain can be built accordingly.
Added in a function _get_syscall_inst that returns the address of a syscall instruction
Checking if travis builds it properly/
The following changes were made :-
- Change the body of _get_syscall_inst, make it cleaner
- Add in a self._syscall_instruction that caches the address of
a syscall instruction that was found
Removed raw_address and raw_data; modified srop implementation to use raw()
Cleanup getattr
Now the stack pointer is modified so that we can continue constructing the ROP chain if rop.base is specified
No need of enumerate; removed
doctest for rop-srop integration added in
Dropping the doctest ifmain statements
Correct underline in rst
Add testsetup to srop.rst
Now uses packing.pack instead of struct.pack
Added in context.arch information to srop.rst and srop.py doctest
Added "context.clear()" to the doctest
Cleaned up _reg_pos_mapping up a bit
Corrected reference link
Replace "print repr()" with "unpack_many"
Added in import to srop.rst
- Request a TTY by default, and disable echoing
- This turns off stdout buffering
- Only negative effect is that you lose Ctrl+C
- Further improvements to ssh.process to allow e.g. file handle dup'ing
- Greatly improved documentation of members/attributes
- Added .pid attribute to ssh_channel
- Uses SCP by default, falls back to old method of cat'ing files
- Switched to use exclusively SHA256 instead of multiple hashes.
- We lose no flexibility or support
- Sorted imports
In this commit, I only fix 'i386/linux/acceptloop_ipv4.asm' and
'i386/linux/mprotect_all.asm'.
BUG=None
TEST=confirmed the behavior is still the same under
gdb and strace.
- Refactor i386/amd64 mov code to use new register info
- Mostly this is to clean up the code and remove things like bigger[0][-1]
- A small enhancement to the ARM shellcode
- Resolve hosts for i386.linux.connect
- Add arm.trap
- Add gethostbyname to i386.connect
- Keep their name so that we have relevant info in shellcode
- Can be manipulated as integers as you'd expect
- Update generation scripts to emit new-style constants
- Sorted imports
- We can now build whole, compact ELF files.
- This is useful for e.g. ioarm level 1
- Various fixes for endianness support, bit sizes when assembling
- Don't catch KeyboardInterrupt
- Make stdout a PTY
- This disables buffering on stdout
- Add options to specify how stdin/stdout/stderr are created
- Expose preexec_fn
- Protect the user from common Python-specific pitfalls
- e.g. argv elements cannot contain or even end with '\x00'
Specify versions of all requirements. Had some complaints about old psutils, and this will cause new installs to upgrade.
Update Capstone installation docs. The 3.0.1 target is self-installing.
Avoid Travis CI build time penalty by using older, pre-built Capstone 2.1
Before `log.Logger.getEffectiveLevel` would return the log level of the current context or the parent logger depending on which was more verbose. This lead to some surprising behavior. Example:
```
from pwn import *
context(log_level = 'DEBUG')
log.setLevel(logging.WARNING)
log.debug('debug')
log.info('info')
log.warning('warning')
```
We would expect to see only a warning, but since `context.log_level` is a more verbose level we will see all three messages.
This commit changes `log.Logger.getEffectiveLevel` to work exactly as `logging.Logger.getEffectiveLevel` except if the current logger is named 'pwnlib', in which case the log level of the current context is returned.
Make `pwnlib.term.readline()` return a utf8 encoded `str` instead of a `unicode`. This is the behavoir of the built-in readline, so we should mimic that.
Otherwise throws this
```py
File "/home/user/pwntools/pwnlib/tubes/ssh.py", line 17, in __init__
h = log.waitfor('Opening new channel: %r' % (process or 'shell'), log_level = self.log_level)
TypeError: not all arguments converted during string formatting
```
When reporting an issue, be sure that you are running the latest released version of pwntools (`pip install --upgrade pwntools`).
Please verify that your issue occurs on 64-bit Ubuntu 14.04. You can use the Dockerfile on `docker.io` for quick testing.
```
$ docker pull pwntools/pwntools:stable
$ docker run -it pwntools/pwntools:stable
```
If possible, provide a proof-of-concept which demonstrates the problem. Include any binaries or scripts necessary to reproduce the issue, and please include the full debug output via setting the environment variable `PWNLIB_DEBUG=1`.
When reporting an issue, be sure that you are running the latest released version of pwntools (`pip install --upgrade pwntools`).
## Debug Output
Having the extra debug output really helps us, and might help you diagnose the problem yourself.
When submitting an issue that has output from Pwntools, make sure to run your script as shown below, to enable the extra debugging data.
```sh
$ python exploit.py DEBUG LOG_FILE=log.txt
```
You should see `[DEBUG]` statements that show what's happening behind the scenes:
```
[+] Starting local process '/bin/sh' argv=['sh'] : pid 16823
[DEBUG] Sent 0x7 bytes:
'whoami\n'
[DEBUG] Sent 0x5 bytes:
'exit\n'
[+] Receiving all data: Done (11B)
[DEBUG] Received 0xb bytes:
'crashheap\n'
[*] Process '/bin/sh' stopped with exit code 0 (pid 16823)
```
## Verify on Ubuntu
If possible, please verify that your issue occurs on 64-bit Ubuntu 22.04. We provide a Dockerfile based on Ubuntu 22.04 via `docker.io` to make this super simple, no VM required!
Thanks for using Pwntools! If you have a question, here are some tips that will make it easier for us to help you.
## Check the Documentation
Pwntools has REALLY extensive documentation. Please check there first! https://docs.pwntools.com
## Debug Output
If you're having an issue with an exploit, run your script with full debugging output, i.e. `python exploit.py DEBUG`. This may help you to see what's happening under the hood, and the debug output will help to diagnose any issues you're encountering.
Thanks for contributing to Pwntools! Take a moment to look at [`CONTRIBUTING.md`][contributing] to make sure you're familiar with Pwntools development.
Please provide a high-level explanation of what this pull request is for.
## Testing
Pull Requests that introduce new code should try to add doctests for that code. See [`TESTING.md`][testing] for more information.
## Target Branch
Depending on what the PR is for, it needs to target a different branch.
You can always [change the branch][change] after you create the PR if it's against the wrong branch.
In general, we like to keep things documented. You should add documentation to any new functionality, and update it for any changed functionality. Our docstrings use the [Google Style Python Docstrings](https://sphinxcontrib-napoleon.readthedocs.org/en/latest/example_google.html#example-google).
After you have documentation, you should add a [doctest](https://docs.python.org/2/library/doctest.html).
Finally, it is probably a good idea to run the test suite locally before doing
the pull-request to make sure everything works, however this is not a
requirement.
Once you are ready to do a pull-request, you should figure out if your changes
constitutes a new feature or a bugfix in stable or beta. If it is a bugfix in
stable or beta, you should do the pull-request against the branch in question,
and otherwise your pull-request should be against the dev branch.
Once you do the pull-request Travis CI will run the test-suite on it. Once it
passes one of the core developers will look at your pull request, possibly
comment on it and then hopefully merge it into the branch in question.
## Automated Testing
Pull requests against Pwntools require at a minimum that no tests have been broken, and ideally each pull request will include new tests to ensure that all of the functionality works as intended.
You can find more information on testing in [TESTING.md](TESTING.md).
Sometimes it's annoying to set up Pwntools on your workstation, and you want something that Just Works (TM).
[Docker](https://www.docker.com/) is here to the rescue! Using Docker means that you get a nice, standardized Linux environment and don't need to worry about pip or installing dependencies.
## Quick Start
First, install Docker for your OS, which you can find on their [Getting Started](https://www.docker.com/get-started) page.
Next, download and run the Pwntools stable docker image.
```sh
$ docker run -it pwntools/pwntools:stable
```
## Recommended Settings
In order to get the most from your docker image, we need to enable debugging of processes (`--privileged`) and expose the network ports from the guest to the host (`--net=host`).
```sh
$ docker run -it \
--privileged \
--net=host \
--hostname localhost \
--ulimit core=-1:-1 \
pwntools/pwntools:stable
```
## Sharing a Folder
It's really nice to be able to use your preferred native editor, and have the changes show up live inside your Docker image. This is easy to add, thanks to Docker's bind mounts (`--mount type=bind`).
With the command below, your `~/exploits` directory will magically show up inside the Docker image at `/home/pwntools/exploits` so that you can easily run them (from Docker) and edit them (from outside Docker).
If you're a Windows user `$HOME` doesn't exist in the same way as on Linux, instead it is `%UserProfile%`. The command from above would look like this, assuming your editor is Visual Studio Code and you have code.exe in your `%PATH%`.
[](https://github.com/Gallopsled/pwntools/actions/workflows/ci.yml?query=branch%3Adev)
Most code is inside the pwnlib folder with some functionality inside pwn or
bin. It is typically used as:
Pwntools is a CTF framework and exploit development library. Written in Python, it is designed for rapid prototyping and development, and intended to make exploit writing as simple as possible.
```python
frompwnimport*
context(arch='i386',os='linux')
# EXPLOIT HERE
r=remote('exploitme.example.com',31337)
# EXPLOIT CODE GOES HERE
r.send(asm(shellcraft.sh()))
r.interactive()
```
However we have made command-line frontends for some of the functionality
inside the pwnlib. These are:
*`asm`/`disasm`: Small wrapper for various assemblers
*`constgrep`: Tool for finding constants defined in header files
*`cyclic`: De Bruijn sequence generator and lookup tool
*`hex`/`unhex`: Command line tools for doing common hexing/unhexing operations
*`shellcraft`: Frontend to our shellcode
# Documentation
Our documentation is available on http://pwntools.readthedocs.org/
Our documentation is available at [docs.pwntools.com](https://docs.pwntools.com/)
A series of tutorials is also [available online](https://github.com/Gallopsled/pwntools-tutorial#readme)
To get you started, we've provided some example solutions for past CTF challenges in our [write-ups repository](https://github.com/Gallopsled/pwntools-write-ups).
# Installation
Pwntools is available as a pip package. You can install it by running
`pip install pwntools`.
Alternatively if you prefer to have the latest version in git, you can
simply clone this repository, run `pip install -r requirements.txt`
and add entries in your `PATH` and `PYTHONPATH` variables. The script
`install_local.sh` will help you do so, in case you are using bash.
Pwntools is best supported on 64-bit Ubuntu LTS releases (18.04, 20.04, 22.04, and 24.04). Most functionality should work on any Posix-like distribution (Debian, Arch, FreeBSD, OSX, etc.).
# Contact
If you have any questions not worthy of a bug report, feel free to join us
at `#gallopsled` on Freenode and ask away.
Python3 is suggested, but Pwntools still works with Python 2.7. Most of the functionality of pwntools is self-contained and Python-only. You should be able to get running quickly with
However, some of the features (assembling/disassembling foreign architectures) require non-Python dependencies. For more information, see the [complete installation instructions here](https://docs.pwntools.com/en/stable/install.html).
# Contribution
See [CONTRIBUTING.md](CONTRIBUTING.md)
# Contact and Community
If you have any questions not worthy of a [bug report](https://github.com/Gallopsled/pwntools/issues), join the Discord server at https://discord.gg/96VA2zvjCB
The aim of pwntools is exploiting software vulnerabilities, which is an unusual position, but it nevertheless can have its own security issues.
Especially that an attacker (=re-victim) is usually not prepared to be attacked back (by the re-attacker).
The first question to ask yourself is: is this an actual vulnerability?
- can it be triggered by a re-attacker (malicious honeypot pretending to be a vulnerable service)?
- does it impact the attacker (=re-victim)?
- is it serious?
* *availability: medium* means *at least* exhausting RAM or disk space of the attacker (=re-victim)
* *confidentiality: medium* means *at least* reading the filesystem of the attacker (=re-victim)
* *integrity: medium* means *at least* performing uncontrolled actions or data corruption on behalf of the attacker (=re-victim)
* if crucial for some sophisticated exploit chain, it is always serious
*`safe_eval` bypasses **are** serious.
* an example of what was **kind of** serious: [#1732](https://github.com/Gallopsled/pwntools/pull/1732)
- can it be fixed without compromising on Pwntools' usability?
If at least one of the answers is no, then this is NOT a vulnerability, so just file a bug report or feature request, without the weird confidential disclosure dance.
Just e-mail the maintainers. Arusekk is the one that is currently the most excited to fix vulnerabilities.
Or create a CTF task! Prove a point the good old hacker way!
Pwntools makes extensive use of unit tests and integration tests to ensure everything is in working order, and no regressions occur.
## Test Suite
To run the test suite, it is best to use Ubuntu 12.04 or 14.04, and run the following commands. **Be aware** that this will add a user to the machine, and create a public key for SSH login!
```sh
bash travis/install.sh
bash travis/ssh_setup.sh
pip install --upgrade --editable .
PWNLIB_NOTERM=1 make -C docs doctest
```
## Testing in Docker
A `Dockerfile` has been provided which has a clean testing environment with Ubuntu Xenial. It is very similar to the online Travis CI testing environment, but uses a more modern version of Ubuntu.
See `travis/docker/README.md` for more information.
## New Tests
To add a new test to an existing module, just add an inline doctest. If the test needs access to an external module, add the import statement to the `testsetup` block in the corresponding file in `docs/source/<module>.rst`.
To add an entirely new module, create a new `module.rst` and add it to the list in `index.rst`. The best way to see if your tests are actually being run is to add an intentionally-failing test like:
```py
>>>assertFalse
```
## Example Test Module
The module `pwnlib.testexample` exists to demonstrate how everything is tied together. The only files which were modified to add this example module and run its tests automatically are:
-`pwnlib/testexample.py`
-`docs/source/testexample.rst`
-`docs/source/index.rst`
## Shellcode and ROP
These are both less easy to test, as they require actually executing code, or loading real binaries. In order to make the process simpler, the `runner` library was created to wrap common tasks. For an example of testing shellcode with these helpers, see [exit.asm](pwnlib/shellcraft/templates/i386/linux/exit.asm).
Additionally, for loading ELF files on-the-fly, the helpers `ELF.from_bytes` and `ELF.from_assembly` are available.
Many settings in ``pwntools`` are controlled via the global variable :data:`.context`, such as the selected target operating system, architecture, and bit-width.
:mod:`pwnlib.context`
---------------------
In general, exploits will start with something like:
..code-block::python
frompwnimport*
context.arch='amd64'
Which sets up everything in the exploit for exploiting a 64-bit Intel binary.
The recommended method is to use ``context.binary`` to automagically set all of the appropriate values.
The most common way that you'll see pwntools used is
>>> from pwn import *
Which imports a bazillion things into the global namespace to make your life easier.
This is a quick list of most of the objects and routines imported, in rough order of importance and frequency of use.
-:mod:`pwnlib.context`
-:data:`pwnlib.context.context`
- Responsible for most of the pwntools convenience settings
- Set `context.log_level = 'debug'` when troubleshooting your exploit
- Scope-aware, so you can disable logging for a subsection of code via :meth:`.ContextType.local`
-``remote``, ``listen``, ``ssh``, ``process``
- :mod:`pwnlib.tubes`
- Super convenient wrappers around all of the common functionality for CTF challenges
- Connect to anything, anywhere, and it works the way you want it to
- Helpers for common tasks like ``recvline``, ``recvuntil``, ``clean``, etc.
- Interact directly with the application via ``.interactive()``
-``p32`` and ``u32``
- :mod:`pwnlib.util.packing`
- Useful functions to make sure you never have to remember if ``'>'`` means signed or unsigned for ``struct.pack``, and no more ugly ``[0]`` index at the end.
- Set ``signed`` and ``endian`` in sane manners (also these can be set once on ``context`` and not bothered with again)
- Most common sizes are pre-defined (``u8``, ``u64``, etc), and :func:`pwnlib.util.packing.pack` lets you define your own.
-``log``
- :mod:`pwnlib.log`
- Make your output pretty!
-``cyclic`` and ``cyclic_func``
- :mod:`pwnlib.util.cyclic`
- Utilities for generating strings such that you can find the offset of any given substring given only N (usually 4) bytes. This is super useful for straight buffer overflows. Instead of looking at 0x41414141, you could know that 0x61616171 means you control EIP at offset 64 in your buffer.
-``asm`` and ``disasm``
- :mod:`pwnlib.asm`
- Quickly turn assembly into some bytes, or vice-versa, without mucking about
- Supports any architecture for which you have a binutils installed
- Over 20 different architectures have pre-built binaries at `ppa:pwntools/binutils <https://launchpad.net/~pwntools/+archive/ubuntu/binutils>`_.
-``shellcraft``
- :mod:`pwnlib.shellcraft`
- Library of shellcode ready to go
- ``asm(shellcraft.sh())`` gives you a shell
- Templating library for reusability of shellcode fragments
-``ELF``
- :mod:`pwnlib.elf`
- ELF binary manipulation tools, including symbol lookup, virtual memory to file offset helpers, and the ability to modify and save binaries back to disk
-``DynELF``
- :mod:`pwnlib.dynelf`
- Dynamically resolve functions given only a pointer to any loaded module, and a function which can leak data at any address
-``ROP``
- :mod:`pwnlib.rop`
- Automatically generate ROP chains using a DSL to describe what you want to do, rather than raw addresses
-``gdb.debug`` and ``gdb.attach``
- :mod:`pwnlib.gdb`
- Launch a binary under GDB and pop up a new terminal to interact with it. Automates setting breakpoints and makes iteration on exploits MUCH faster.
- Alternately, attach to a running process given a PID, :mod:`pwnlib.tubes` object, or even just a socket that's connected to it
-``args``
- Dictionary containing all-caps command-line arguments for quick access
- Run via ``python foo.py REMOTE=1`` and ``args['REMOTE'] == '1'``.
- Can also control logging verbosity and terminal fanciness
- `NOTERM`
- `SILENT`
- `DEBUG`
-``randoms``, ``rol``, ``ror``, ``xor``, ``bits``
- :mod:`pwnlib.util.fiddling`
- Useful utilities for generating random data from a given alphabet, or simplifying math operations that usually require masking off with `0xffffffff` or calling `ord` and `chr` an ugly number of times
-``net``
- :mod:`pwnlib.util.net`
- Routines for querying about network interfaces
-``proc``
- :mod:`pwnlib.util.proc`
- Routines for querying about processes
-``pause``
- It's the new ``getch``
-``safeeval``
- :mod:`pwnlib.util.safeeval`
- Functions for safely evaluating python code without nasty side-effects.
These are all pretty self explanatory, but are useful to have in the global namespace.
-``hexdump``
-``read`` and ``write``
-``enhex`` and ``unhex``
-``more``
-``group``
-``align`` and ``align_down``
-``urlencode`` and ``urldecode``
-``which``
-``wget``
Additionally, all of the following modules are auto-imported for you. You were going to do it anyway.
Pwntools is best supported on 64-bit Ubuntu LTS releases (14.04, 16.04, 18.04, and 20.04). Most functionality should work on any Posix-like distribution (Debian, Arch, FreeBSD, OSX, etc.).
Prerequisites
-------------
In order to get the most out of ``pwntools``, you should have the
following system libraries installed.
..toctree::
:maxdepth:3
:glob:
install/*
Note: For Mac OS X you will need to have cmake ``brew install cmake`` and pkg-config ``brew install pkg-config`` installed.
Released Version
-----------------
pwntools is available as a ``pip`` package for both Python2 and Python3.
Assembly of foreign architectures (e.g. assembling Sparc shellcode on
Mac OS X) requires cross-compiled versions of ``binutils`` to be
installed. We've made this process as smooth as we can.
In these examples, replace ``$ARCH`` with your target architecture (e.g., arm, aarch64, mips64, vax, etc.).
Building `binutils` from source takes about 60 seconds on a modern 8-core machine.
Ubuntu
^^^^^^^^^^^^^^^^
For Ubuntu 12.04 through 15.10, you must first add the pwntools `Personal Package Archive repository <https://launchpad.net/~pwntools/+archive/ubuntu/binutils>`__.
Ubuntu Xenial (16.04) has official packages for most architectures, and does not require this step.
..code-block::bash
$ sudo apt-get install software-properties-common
$ sudo apt-add-repository ppa:pwntools/binutils
$ sudo apt-get update
Then, install the binutils for your architecture.
..code-block::bash
$ sudo apt-get install binutils-$ARCH-linux-gnu
Mac OS X
^^^^^^^^^^^^^^^^
Mac OS X is just as easy, but requires building binutils from source.
However, we've made ``homebrew`` recipes to make this just two commands.
After installing `brew <https://brew.sh>`__, grab the appropriate
Loaded 100 of 1413 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
peace-maker
