`pwn errno` only ever consulted the host's errno module, so numbers from a
target of a different architecture were reported as unknown (131 is
ECONNRESET on mips but unused here).
Add -c/--context, built on the same context_arg helper as asm, constgrep and
cyclic: number and name lookups now resolve against pwnlib.constants for the
requested os/arch, and --list/--search honour the flag too. --search also
matches errno names, so the flag stays useful on hosts whose strerror cannot
spell out the target's codes.
Covered with three inline doctests, as TESTING.md asks for.
* Test on Python 3.15
* Use `RLIM_INFINITY` instead of `-1` constant for rlimit modification
Python 3.15 warns against using negative values instead of the constant:
DeprecationWarning: Use RLIM_INFINITY instead of negative limit value.
It has been available since Python 2.
* Update CHANGELOG
* docs: fix argument names in regsort, ROP.raw and SSHPath docstrings
regsort documented random instead of randomize, ROP.raw documented data
instead of value, and SSHPath documented name instead of path.
* Add PR number to CHANGELOG entry
* docs: remove changelog entry for docstring fixes
* gdb.corefile: Run gdb directly to prevent detach race
`attach()` waits for the debugger to attach by polling the `TracerPid` regularly. `corefile()` runs `detach` in gdb after dumping the core, so the debugger might attach and detach between two `TracerPid` polls and `wait_for_debugger()` thinks gdb exited before attaching and returns `None` as gdb's pid is no longer valid.
Later `corefile()` ran `psutil.Process(pid).wait()` with `pid = None` which effectively waited on the Python process itself to exit.
This caused a hang in the doctest CI sometimes, which was killed by the watchdog timer eventually.
Run `gdb` directly and just observe the exit code instead of relying on the more advanced `attach` logic for longer running debug sessions.
* Update CHANGELOG
* fix(packing): dd() raises TypeError on a bytearray destination
dd() documents bytearray as a supported destination type, but
dd(bytearray(b'Hello!'), b'?', skip=5) raises
"TypeError: 'bytes' object cannot be interpreted as an integer".
pwnlib/util/packing.py:1108 handled list and bytearray destinations in a
single branch and filled both with list(map(p8, bytearray(src))). p8()
returns a length-1 bytes object, which is what a list destination is
documented to hold (see the existing dd(list('Hello!'), (63,), skip=5)
doctest), but a bytearray slice assignment needs an iterable of int, so
every bytearray destination crashed.
Assign the byte string src directly when the destination is a bytearray.
Every src branch above the destination dispatch (packing.py:1039-1089)
has already reduced src to a byte string, so the slice assignment is
always valid. This also makes a bytearray destination agree with the
bytes and str destinations, which already copy src verbatim.
Add a doctest covering the returned value and the in-place update of the
caller's bytearray.
* Add changelog entry for #2772
---------
Co-authored-by: peace-maker <[email protected]>
* feat(cli): add 'pwn errno --list' / -l + perror-string fallback (#2657)
- '--list / -l': dump every known errno (value, name, strerror message).
- '--search / -s': force perror-substring match (case-insensitive).
- Bare 'pwn errno <text>' falls back to a perror substring search when the
argument is neither an integer nor a known errno macro name. So
'pwn errno "Cannot allocate memory"' now prints ENOMEM directly.
- Positional 'error' arg is now optional so '--list' can be used alone;
argparse explains the requirement when neither is supplied.
Closes#2657
* Update CHANGELOG.md for #2727
* Add type hints to errno commandline tool
---------
Co-authored-by: Chris (ChrisJr404) <[email protected]>
Co-authored-by: Peace-Maker <[email protected]>
* fix: don't truncate the last byte of FileStructure.struntil
* changelog: add entry for #2761
---------
Co-authored-by: peace-maker <[email protected]>
* fix: Fix DeprecationWarning when setting ssl=True in remote()
The current code initializes SSLContext with `ssl.PROTOCOL_TLSv1_2`, which has been deprecated for a while now
This should be the correct fix according to the documentation (https://docs.python.org/3/library/ssl.html#ssl.SSLContext)
* Update CHANGELOG
* Fix the new ssl thing, document a way to disable cert verification
---------
Co-authored-by: peace-maker <[email protected]>
* Disable debuginfod tests in libcdb.unstrip_libc
The external debuginfod servers are really slow or won't respond at all like debuginfod.ubuntu.com.
The caching in CI doesn't help since we only cache it for 3 month and the query fails afterwards most of the time. We cache the negative response too which doesn't help if the service is flakey itself :D
(cherry picked from commit 86cafce196)
* Add changelog entry for #2773
---------
Co-authored-by: Peace-Maker <[email protected]>
* docs: put plain doctest blocks in the default group for Sphinx 9
doctest_test_doctest_blocks is the group name plain >>> blocks are
assigned to. Sphinx < 9 ignored it and always used "default"; Sphinx 9.0
honours it (sphinx-doc/sphinx#13858), so our 'true' moved every plain
block into a group named "true" while `.. doctest:: :hide:` blocks stayed
in "default". The two groups run separately, so hidden context.clear()
calls no longer ran between the examples around them.
Fixes#2711
* Update CHANGELOG for #2784
* feat(libcdb): use local debuginfod client cache before hitting servers (#2623)
If gdb / eu-debuginfod-find / abrt has already downloaded the matching
debuginfo for our libc's build-id (cached at $DEBUGINFOD_CACHE_PATH or
$XDG_CACHE_HOME/debuginfod_client/ or ~/.cache/debuginfod_client/),
unstrip_libc() now reuses it directly instead of doing another network
round-trip via debuginfod servers. Validated by ELF magic so non-ELF
junk in the cache is ignored.
Closes#2623
* Update CHANGELOG.md for #2726
* Check our own cache before looking at debuginfod-client
---------
Co-authored-by: Chris (ChrisJr404) <[email protected]>
Co-authored-by: Peace-Maker <[email protected]>
* Add Type Annotations for parts of pwnlib/util
* fix: use Callable in pwnlib/util/lists.py
* fix: Import Errors
* fix: use getLogger in pwnlib/util/net
* feat: use collections.abc.Buffer where appropriate
* fix: remove collections.abc.Buffer to use Sequence
* add CHANGELOG.md entry
* Improve packing and fiddling types
* Introduce own `BytesLike` type instead of 3.12 Buffer
* Improve web.wget types
* Use TypedDict in util.net
Specify generics properly.
* Improve pwnlib.proc types
* Improve pwnlib.util.lists types
* Fix util.splash types
* Make TypeVars private
* Improve util.hexdump types
* Remove partial types from packing.fit/flat
Those require more work to be useful.
* Fix implicit wget return
* Fix term.text type export
* Fix js_escape return type
* Fix sh_string doctest types
* Fix usage of ssh_channel as ssh_process
We access the pid, so a plain ssh_channel is not enough.
* Improve util.lists types
* Improve packing.dd types
* Fix relying on socket.AddressFamily type at runtime
The type is only in typeshed and not the real socket module. Don't access it at runtime, only at type checking.
* Fix lists.findall with string needle
* Fix dd on two files
* Update mypy-baseline.txt
* Remove testing artifacts from mypy-baseline.txt
---------
Co-authored-by: peace-maker <[email protected]>
* Support paginated libc.rip API requests
libc.rip used to truncate the result set but supports pagination now. Fetch all results reliably now by requesting all pages.
Fixes#2651
* Update CHANGELOG
* docs(args): clarify reserved args map to context, not args.NOASLR
Reserved magic arguments such as DEBUG and NOASLR are consumed as hooks
that adjust context (context.log_level, context.aslr) and are never
stored in the args dict, so args.NOASLR is always the empty string.
The pwnup template comment previously claimed these settings 'show up
in args', which reinforced the wrong mental model and led users to
check args.NOASLR instead of context.aslr. Clarify both the template
comment and the pwnlib.args module docstring.
Fixes#2564
* docs: add changelog entry for args/pwnup.mako docstring clarification
CI requires a CHANGELOG.md entry for changes touching pwnlib/args.py
and pwnlib/data/templates/pwnup.mako.
* Add support for debugging with x64dbg on Windows
Add x64dbg option to `context.debugger` to attach using x64dbg in `windbg.debug()`.
The path to the x96dbg.exe is looked up in the path and through the "Debug with x64dbg" shell extension you can install optionally when first launching the x96dbg.exe launcher.
You can specify the path using `context.x64dbg_binary` too.
* Refactor exclusion of `context.debugger="auto"` while handling `"auto"`
Co-authored-by: Christian Clauss <[email protected]>
* Update CHANGELOG
* Remove absolute path workaround
x64dbg 2026.05.27 fixed passing absolute paths to a script file using -cf.
* Remove bare raise
log.error raises already.
* Fix linting error of winreg import on linux
The module is only available on Windows. So ignore the error when linting on another platform.
---------
Co-authored-by: Christian Clauss <[email protected]>
The external debuginfod servers are really slow or won't respond at all like debuginfod.ubuntu.com.
The caching in CI doesn't help since we only cache it for 3 month and the query fails afterwards most of the time. We cache the negative response too which doesn't help if the service is flakey itself :D
* ROP: Add labels to reference gadgets dynamically
Allow to assign a name to a point in the ROP chain using `ROP.setlabel(name)`
and reference it somewhere else using `ROP.uselabel(name, offset)`.
You have to know the location of your chain in memory to resolve the
right addresses. This allows to specify addresses relative to your ROP chain payload in your chain.
Refs #2583
* Update ret2dlresolve test output
* Update CHANGELOG
* Rename functions to `label` and `ref`
This is less verbose than `setlabel` and `uselabel` but still descriptive enough to be able to guess what they do.
* fix: improve exception handling in process.py and adb.py
- process.py: restore bare except with pylint disable (intentional: catches
non-Exception errors during Python interpreter shutdown)
- adb.py: use specific (OSError, EOFError) instead of broad except Exception
* fix: add missing import for remote in adb.py
The bare except was silently catching NameError from the missing
remote import. Added 'from pwnlib.tubes.remote import remote' to
fix the actual underlying bug.
* docs: add changelog entry for PR #2743
* docs: move changelog reference link near 5.0.0 release section
* fix: catch PwnlibException in emulator AVD name lookup
The bare except was catching PwnlibException raised by remote() when
connection fails. Replacing with except (OSError, EOFError) caused
PwnlibException to propagate, potentially causing retry loops in
Android CI.
Fix by adding PwnlibException to the exception tuple.
* fix: prevent __getattr__ infinite recursion with context.device setter
When context.device setter accesses device.arch, it triggers AdbDevice's
__getattr__ (instead of the arch property). __getattr__ then calls
context.local(device=self), which triggers the setter again, causing
infinite recursion.
Fix: check if the attribute exists on the class via MRO.__dict__
before entering the context.local() path. This lets Python's normal
property resolution handle class-defined attributes.
* revert: remove __getattr__ MRO guard
Peace-maker correctly points out that __getattr__ is only called when
normal attribute lookup fails, so the MRO guard is unnecessary.
The infinite recursion is a pre-existing bug unrelated to this PR.
* Assign to `_avd` instead of `avd`
---------
Co-authored-by: mshzy <[email protected]>
Co-authored-by: peace-maker <[email protected]>
* fix(ssh): use POSIX separator for path detection in ssh.which
On Windows, `os.path.sep` is '\\', which fails to detect POSIX paths.
Hardcoding '/' ensures consistent path detection since the `ssh`
module targets POSIX-compliant systems.
* Update CHANGELOG.md
* feat(context): add path_sep and pathlist_sep to context, make the path seperator dependent on the context.os
* fix(ssh): declare path separator as private member in ssh
* fix(ssh): remote path handling with pathlib
* fix(ssh): replace os.path.basename functions with pathlib for path handling
* fix(ssh): ensure remote paths are decoded to str
* guide users to libc_start_main_return for __libc_start_main_ret
looking up __libc_start_main_ret on ELF(libc).sym just threw a plain
KeyError/AttributeError even though libcdb shows that symbol, which is
confusing (issue 2563). it's not a real symbol so we can't just add it,
and computing it means disassembling __libc_start_main which we don't
want to do for everyone by default.
so for now the symbols dotdict raises a message pointing at the existing
ELF.libc_start_main_return property when you ask for that key. lazy auto
calc can come later. added a doctest covering both sym['...'] and
sym.__libc_start_main_ret access, tests pass.
* use bare super() in symboldict
peace-maker noted new code can use py3 super(), so drop the
redundant (symboldict, self) args.
* add changelog entry for 2746
---------
Co-authored-by: peace-maker <[email protected]>
The nginx docs say that you have to specify the $request_uri manually when a variable is used in the `proxy_pass` value.
Since we just put `/` at the end, all requests were rewritten to `/` instead of the intended e.g. `/api/find` endpoints.
https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass
This broke in #2718 but didn't manifest right away since the responses were still cached in CI. Now after the cache expired the fetch failed.
* Fix some global type hint errors
Errors in global scope should be fixed before we're enabling strict mode in mypy and tackle the local problems.
* Remove pwnlib.util.pairwise since it's in itertools now
pairwise was added to itertools itself in 3.10.
https://docs.python.org/3/library/itertools.html#itertools.pairwise
* Fix atexception handlers in term mode
The colored-traceback module registers its own `sys.excepthook` but doesn't call the original handler.
Reorder the includes such that the colored-traceback is setup before and our own hooks afterwards.
We always call the old excepthook and forward the exception to the colored-traceback after firing our own handlers.
It worked before in non-term mode when stderr wasn't a tty and colors were disabled.
* Update CHANGELOG
* Import atexception in doctests
* feat(libcdb): add extra_mirrors arg + PWN_EXTRA_LIBC_MIRRORS env to download_libraries (#2666)
When the libc.rip URL points at a deb.debian.org / archive.ubuntu.com /
security.debian.org URL whose package has been pulled (e.g. older
vulnerable libc versions like 'GLIBC 2.36-9+deb12u6'), download_libraries
now also tries the same /pool/... path against any user-supplied mirror
prefix(es). Mirrors can be passed via the new keyword argument
extra_mirrors=... (str or list) or the comma/whitespace-separated
PWN_EXTRA_LIBC_MIRRORS environment variable.
Closes#2666
* Update CHANGELOG.md for #2725
* Rename environment variable to `PWNLIB_EXTRA_LIBC_MIRRORS`
All our other user-facing environment variables are `PWNLIB_` instead of `PWN_`.
* Move `import re` to toplevel
* Add type hints to touched functions
* Add example usage for download_libraries with extra_mirrors parameter
* Fix envvar in CHANGELOG
---------
Co-authored-by: Chris (ChrisJr404) <[email protected]>
Co-authored-by: Peace-Maker <[email protected]>
* Add typehints to timeout module
Co-authored-by: Copilot <[email protected]>
* Add typehints to device class
* Add typehints to PwnlibException
* Add general typehints for term.text module
Since the available module attributes are handled dynamically like `text.green()` or `text.green_on_white()` or `text.underline_red()`, it's not feasible to type out all possible combinations.
Just tell the type checker that all module attibutes have the same type and thus avoid the `error: Module has no attribute "bold_blue" [attr-defined]` errors.
* Add typehints for config module
* Add typehints for hashes module
The guaranteed algorithms in hashlib are pretty stable and we can generate the list out proper type checking and auto complete.
Still add a check in CI that notifies us if there are changes in the future.
* Add atexception and atexit types
atexception.unregister was not documented correctly since the function expects an identifier returned by `atexception.register` instead of the handler function.
* Fix toplevel type hints in shellcraft.internal
* Fix toplevel type hints in term.key
---------
Co-authored-by: Copilot <[email protected]>
* add ":" to Distro string
* Update cache file path to include username
Include username in cache file path for better uniqueness.
* Added public_key+fingerprint for ssh class & Modified ssh checksec_cache path to be more unique & checksec is per ssh server and not attached to a user
* Fix typeo
* Revert indentation to 10 chars
* ssh: Fix downloading data to tmp on Windows
Allows the checksec output of the target server to be generated.
Filenames cannot contain a colon ":".
* Update CHANGELOG
---------
Co-authored-by: Peace-Maker <[email protected]>
* refactor: optimize completion script
And deprecate an interface
* chore: update changelog
* refactor: full patch of zsh comp script
* fix: add missing message in output
* fix: a typo and a missing help
* feat: allow -l to match multiple arguments
* feat: optimize completion logic
1. move all complicated functions in front of pwn()
2. add file completion for libcdb subcommands
3. add completion for pwn errno
* Allow running with Unicorn 2.1.[34] but throw when emulating MIPS
Unicorn 2.1.5 wasn't released for a while and 2.1.2 doesn't have binary wheels for Python 3.14.
This made installing pwntools on newer Python versions more annoying since you had to build Unicorn yourself.
The only problem with the blocked versions is MIPS emulation, so only avoid that while allowing to use Unicorn 2.1.3 or 2.1.4 for other architectures.
Refs #2621
* Update CHANGELOG
* fix(ssh): resolve PermissionError on Windows during SFTP upload
On Windows, `tempfile.NamedTemporaryFile()` holds an exclusive lock on the file.
Previously, `sftp.put()` attempted to re-open the file by its path, causing a
`PermissionError: [Errno 13]`. This commit replaces `put` with `putfo`, passing a file object directly to bypass the tempfile entirely.
* Update CHANGELOG.md
Update CHANGELOG.md to include new entries and links.
* Start to gradually add type hints
Use mypy-baseline to ignore all existing errors for now while avoiding adding new ones in new code.
* Remove Python 2 fallback imports
* Remove py2compat layer
* Avoid redefining a tube helper function
* Remove explicit new-style class (object) syntax
old-style classes were removed in Python 3 so there is no distinction anymore.
* Ignore type error in shellcraft helper script
* Fix repeatedly changing `linux_dirent.d_name`
* Avoid typechecking windows_termcap on Linux
We run mypy on Linux in CI, ignore Windows related import errors
* Run mypy-baseline in CI
* Add type hint plan to CONTRIBUTING
* Sort the mypy baseline
Run with `--sort-baseline` for better diffs.
`mypy | mypy-baseline sync --sort-baseline`
* Switch dev dependencies to dependency-groups
The dependency groups aren't baked into build metadata and won't appear on Pypi.
Use nginx variable pattern (set $upstream + proxy_pass https://$upstream)
to defer DNS resolution until first request instead of at config parse time.
This fixes the occasional 'host not found in upstream' error in CI when
nginx container starts before Docker DNS is fully available.
Fixes#2712
* Remove python-dateutil dependency
It is only used for parsing the `ro.build.date` string in Android adb.
Since we use `ro.build.date.utc` now as primary build time source,
don't require `python-dateutil` anymore, but keep optional support.
* Update CHANGELOG
The coverage fluctuated when the linux tests finished before the windows or android tests since their coverage would be missing.
So the coverage for the `adb` module went up/down randomly.
Require the windows tests to pass now too!
* Fix _build_date() crash on non-standard locale date strings (#2513)
Some Android devices return locale-dependent strings in ro.build.date
(e.g. '2017年 11月 14日 星期二 09:55:07 CST' for Chinese locale) that
dateutil.parser.parse() cannot handle, causing wait_for_device() to
crash with a ParserError.
Fix: prefer ro.build.date.utc (integer epoch timestamp) which is
locale-independent and available on all Android devices. Fall back
to ro.build.date with error handling if the UTC property is missing.
* Add CHANGELOG entry for #2513 fix
* Simplify _build_date(): prefer ro.build.date.utc, keep minimal fallback
Address review: ro.build.date.utc is set by the AOSP build system
and available on all standard Android devices. Simplified comments
to reflect this is the primary path, with ro.build.date kept only
as a fallback for non-standard builds.
* Move CHANGELOG entry
* Fix Python 2.7 compatibility
* Disable intro doctest connecting to `ftp.ubuntu.com`
The FTP server is throttled in CI or just really slow. We already test the basic tcp tubes in the `remote` tests.
---------
Co-authored-by: coder999999999 <[email protected]>
Co-authored-by: peace-maker <[email protected]>
* Enable basic tubes/process test on Windows
Many tests are Linux specific using qemu or /proc etc. So only test the basic process interaction for now.
* Cleanup docs without using ellipsis for Windows output
* Set newline to b'\r\n' for local Windows processes
But only change it automatically if `context.newline` wasn't set manually.
* Disable process.terminate() test on Windows
Windows doesn't support sending SIGTERM to other processes.
https://docs.python.org/3/library/subprocess.html#subprocess.Popen.send_signal
* libcdb CI cache: add another dns server
[emerg] 1#1: host not found in upstream "debuginfod.elfutils.org" in /etc/nginx/nginx.conf:26
The cache should still start despite not being able to resolve some domain.
* Rename `windbg.debug(windbgscript=)` parameter to dbgscript
Prepare to support more debuggers than windbg.
* Pass dbgscript in file instead of inline
This way we don't have to deal with shell escaping the dbgscript commands.
* Add `context.debugger` to select which debugger to use
Allow to choose the debugger to start when using `windbg.attach`.
By default it autodetects which debugger is available.
You can specify the path to the debuggers using
`context.windbg_binary` and `context.windbgx_binary`.
* Check if default windbg installation location exists
* Update CHANGELOG
* fix: pad bytes fields to correct field size in FileStructure
When setting bytes fields (e.g. _IO_save_base) on FileStructure, the
value was padded to context.bytes instead of the actual field size from
self.length[val]. This caused struct.pack errors for fields whose size
differs from the pointer size.
Fixes#2693
* Cleanup test and add CHANGELOG
---------
Co-authored-by: nightcityblade <[email protected]>
Co-authored-by: Peace-Maker <[email protected]>
* ssh: Discard stderr of `lsb_release`
The "No LSB modules are available." spam interferes with parsing the output for the initial checksec distro parsing when connecting to a new host.
* Update CHANGELOG
Instead of trying to resolve all of them when nginx starts and exit
if any of them fail to resolve.
[emerg] 1#1: host not found in upstream "debuginfod.elfutils.org" in /etc/nginx/nginx.conf:23
The reponses set a cloudflare `_cfuvid` cookie and had a cache time of 60 seconds.
nginx doesn't cache responses with a `Set-Cookie` header by default,
so ignore the cookie and override the cache time to 12 weeks.
Fixes#2494
* Add `reveal_ptr_same_page`
Reveals a pointer mangled using PROTECT_PTR without knowing where it is stored.
Only works when the pointer points to the same page as it is stored on, which happens regularly in normal tcache next-ptrs.
* Update CHANGELOG
* Close SSH client when authentication failed
The TCP connection stayed open after an exception was thrown
during authentication, even when a `with`-statement guarding the call.
Close the connection before re-raising the exception.
Fixes#2678
* Update CHANGELOG
* ssh: support raw string input for 'key' argument
Implement support for passing the actual private key as a string or bytes, as specified in the documentation. This prevents an AttributeError by converting the string to the appropriate Paramiko PKey object (RSA/ECDSA/Ed25519).
* update changelog
* Add tests for ssh connection using a key
* Cleanup docs
---------
Co-authored-by: Your Name <[email protected]>
Co-authored-by: Peace-Maker <[email protected]>
* Fix `server.close()` not closing the listen socket
It would block on `accept` forever, not freeing the listening port.
Break out of `accept()` regularly to check if the socket was closed in the meantime and stop the thread.
* Update CHANGELOG
* Fix `listen.close()` not closing the listen socket
If `close()` was called before a connection was accepted,
the listen socket would not be closed.
* Fix closing listen socket on Windows
Windows throws an ENOTSOCK instead of an EBADF.
* Keep timeout on child socket tube
* Fix `tty` and `raw` arguments in ssh.process
The arguments were unused and not passed through to the underlying `ssh_process` class.
Fixes#2038
* Add tty tests
* Update CHANGELOG
* Fix buffered process environment output without tty
* Add (un)pack shorthands for 40-56 bit numbers
`u40`/`p40`, `u48`/`p48`, and `u56`/`p56` can be used to quickly unpack shorter integers.
`u48` can be used to unpack addresses leaked using a c-string function which stops at the first null-byte.
Instead of doing `u64(data.ljust(8, b'\x00"))` you can now just do `u48(data)`.
Refs #2447
* Update CHANGELOG
* Fix typo in CHANGELOG
* doctest: Print duration after each document
* Disable intro doctest connecting to `ftp.ubuntu.com`
The FTP server is throttled in CI or just really slow. We already test the basic tcp tubes in the `remote` tests.
* New glibc module to include libc specific funcs
* chore: add changelog entry
* refactor: move libc module
* feat: add libc in docs
* feat: add doctest for glibc module
* fix: remove `libc` ref in `util`
* fix: resolve doc test errors and warnings
* feat: multi-arch support for `ptr_(de)?mangle`
Consulting glibc source code, I found that only on x86 glibc performs
ror/rol, on other platforms, glibc only performs a simple xor.
* chore: cleanup docs
* asm: try native binutils before fallback architectures
Some binutils builds support multiple architectures,
like the x86_64 assembler supports i386 too in most
package builds and vice versa. We support this by trying
the other binutils as well when looking for the right tool.
That support is optional though and there are packages
that don't include the "cross" compilation.
When assembling for the host's native architecture, we
still prefer the explicitly named `x86_64-linux-gnu-as`
instead of the native `as` binary. That native binary is
tried as a last resort after looking for any other explictly
named binary. If the fallback package is installed but
compiled without the cross-compiler support, we'd still
find the `i386-as` binary while looking for `x86_64-as`
before trying the native `as` binary.
That would fail to assemble.
This patch moves the native `as` binary forward in the list
to try it before checking the other fallback architecture ones.
Fixes#2509
* Update CHANGELOG
* Remove useless coding line
And fix a side-effect: the leading ' ' cause neovim to add a space when
press enter, which breaks indent
* feat: add `overlap` to overlap structures easily
* chore: update changelog
* feat(overlap): throws BufferError and complete docs
* Declare return value clearly and refactor error
---------
Co-authored-by: peace-maker <[email protected]>
* Fix args for kitty launch
For gdb to work correctly it needs to get the same cwd cuze it's where the bin is located, if --cwd isn't specified gdb doesn't find the binary being debugged and show an error
* fix args
* Run vermin targetting at most Python 3.10 in CI
Run https://github.com/netromdk/vermin to verify we don't use any Python features
of newer versions than what we want to support.
* Fix getdents docs
/home/runner/work/pwntools/pwntools/pwnlib/util/getdents.py:docstring of pwnlib.util.getdents.linux_dirent:5: ERROR: Unexpected indentation. [docutils]
/home/runner/work/pwntools/pwntools/pwnlib/util/getdents.py:docstring of pwnlib.util.getdents.linux_dirent:9: WARNING: Block quote ends without a blank line; unexpected unindent. [docutils]
/home/runner/work/pwntools/pwntools/pwnlib/util/getdents.py:docstring of pwnlib.util.getdents.linux_dirent:19: WARNING: Definition list ends without a blank line; unexpected unindent. [docutils]
/home/runner/work/pwntools/pwntools/pwnlib/util/getdents.py:docstring of pwnlib.util.getdents.linux_dirent:24: ERROR: Unexpected indentation. [docutils]
/home/runner/work/pwntools/pwntools/pwnlib/util/getdents.py:docstring of pwnlib.util.getdents.linux_dirent:34: WARNING: Block quote ends without a blank line; unexpected unindent. [docutils]
* Refactor getdents.py, add support for SYS_getdents64
* remove enum import and format the code
* fix doctest and use try before decode
* use global context setting in doctest
* use IntEnum for d_type instead
* add error handling, use log to give warnings
* rewrite doctest result, add comment for functions
* complete changelog for getdents
* set context in local, so it won't affect the later tests
* Add explicitly word_size
Co-authored-by: peace-maker <[email protected]>
* Use context.bytes directly instead of context.bits
Co-authored-by: peace-maker <[email protected]>
* add linux source code of struct
* separate dirents into dirents and dirents64
* remove unhex. complete a comment.
* fix test_case. use fromhex instead of unhex.
* add __len__ method
* add DT_WHT and DT_SUBVOL
---------
Co-authored-by: peace-maker <[email protected]>
* support preexec_args in process
Previously, preexec_args where supported by ssh.process only. Add support in process too, to allow easier ssh.process/process workflows
* update changelog
* Add terminate method to process class that sends SIGTERM
* Always use `Popen.terminate()` on all operating systems
And check if the process exited right away.
* Update CHANGELOG
* Fix terminate test to show SIGTERM handler
---------
Co-authored-by: Peace-Maker <[email protected]>
It looks like this was done to work around a bug in an old pip version years ago.
It caused problems when freezing dependencies which unnecessarily forced pip to the version used at freeze time when installing again.
Fixes#2625
* support preexec_fn in debug()
Previously, preexec_fn was mistakengly run is gdbserver's context.
Run it inside the debuggee instead to allow run/debug workflows.
* update Changelog
---------
Co-authored-by: peace-maker <[email protected]>
* feat: add disable_corefiles context option
Add new context option that automatically disables core file generation
when enabled. This sets RLIMIT_CORE to (0, -1) to prevent core dump creation,
which is particularly useful for brute-force scenarios and repeated segfault
crashes where core files consume excessive disk space.
- Add disable_corefiles context variable with boolean type.
- Add @_validator decorator for disable_corefiles context variable
- Default value is False to maintain existing behavior.
Fixes#2566
* Update CHANGELOG
* Add doctest for `context.disable_corefiles`
Added doctest for `disable_corefiles` in corefile.py
Added doc for `disable_corefiles` in process.py
When `context.corefiles` is enabled:
For crashed process, `io.corefile` will return None.
For running process, gdb still dumps the corefile.
* ROP: Remove stdout and argv workaround in ROPgadget invocation
ROPgadget supports usage as a library since v5.3 in 2015.
The hacks are no longer needed.
This also fixes running multiple exploits in parallel in the same process. They could race and output was lost.
* Update CHANGELOG
* Fix race while creating rop-cache folder
There was a TOCTOU when two ROP classes were interleaved just wrong in a multi-threaded script.
* Fix regression: gdbserver can't handle command-line argument containing whitespace
The `--no-startup-with-shell` argument was passed in an attempt to
fix gdbserver startup with `SHELL=/bin/fish`. (#2377) It caused a regression
which disallowed passing arguments containing whitespace to the debuggee in `gdb.debug`.
gdbserver would exit with a message:
"can't handle command-line argument containing whitespace"
There is a 3 second timeout waiting for gdbserver to startup and
printing a message with a workaround already suggesting to set
`SHELL=/bin/bash` to allow gdbserver to launch. This has to be
our best effort to guide users of such shells until we find time
to investigate a better solution.
Fixes#2550
* Update CHANGELOG
The ubuntu/24.04 image contains a default user "ubuntu" with uid 1000.
We add our own "pwntools" user which has uid 1001 now causing mismatches
with the default host uid of 1000 for the bind-mounts in the develop/testing image.
https://bugs.launchpad.net/cloud-images/+bug/2005129
* Fix lookup of binutils for `mipsel` architecture
The endianness is encoded in the architecture name for mips targets which wasn't respected while looking for the correct binutils binary.
Sync the CI and Dockerfile binutils packages while we're at it.
Fixes#2591
* Update CHANGELOG
* still fall back to the other endianness
---------
Co-authored-by: Arusekk <[email protected]>
* tube/process: Fix redirecting stderr to stdout on Windows
Use the `subprocess` module's built-in redirection capabilities instead
of manually redirecting file descriptors, which uses the correct
pipes on Windows.
Fixes#2614
* Update CHANGELOG
* Fix `log.progress` ignoring `context.log_console`
When setting `context.log_console` to something other than stdout, spinners while using `log.progress` would still be printed to stdout.
Redirect them to the stream specified by `context.log_console` and don't animate if it's not a tty.
Fixes#1702
* Update CHANGELOG
* Cleanup `pwnlib.lexer` exports and imports
The exported lexers don't really exist. It has been this way since this code was first included in the repo, so I'd assume the intention was to have aliases for the same lexer since the disassembly looks similar between all those tools.
Many imports are unused too, so remove them.
Fixes#1309
* Update CHANGELOG
We could try to lookup the mapped libraries before they are actually loaded in the process.
Wait until the program is up and running before trying to fetch the memory maps.
* Fix log level of child remotes of `server` tube
The log level of the `server` tube wasn't passed
on to the `remote` wrappers created upon receiving
a new child connection. This caused the connection
closed message to be logged even when the server
object was created with `level='error'`.
* Update CHANGELOG
* Decode `_IO_*` flags in FileStructure member
The `_flags` and `_flags2` bitmasks are decoded to a readable string when printing the struct.
Next to setting the flags as an integer like before, you can set individual bits now and check set bits.
```
fileStr = FileStructure(null=0xdeadbeeef)
fileStr.flags = 0xfbad1807
fileStr.flags = IO_flags._IO_MAGIC | IO_flags._IO_USER_BUF | IO_flags._IO_UNBUFFERED
fileStr.flags._IO_USER_BUF = 1
```
Printing the FileStructure shows the flags in human readable form too
```
{ flags: 0xfbad0401 (_IO_USER_BUF | _IO_UNBUFFERED | _IO_MAGIC)
_IO_read_ptr: 0x0
...
```
* Update CHANGELOG
When trying multiple debuginfod servers, the first one to fail would save the failed attempt and block the remaining servers to be tried.
Only save that we couldn't find the debuginfo if we tried all the configured servers.
* Ignore colored_traceback error when TERM is unset
Fixes using pwntools in non-interactive sage scripts with newer colored_traceback versions.
Fixes#2594
* Update CHANGELOG
* Support longer function names in Windows getexport shellcode
Previously, an arbitrary name length limit of 8 characters was in place due to the lack of a proper strcmp loop. Implement a cheap strcmp for longer function names.
This trashes a bunch of registers and we might be able to optimize it further to reduce the damage.
Use a unique label too to avoid redefinition when using the shellcraft template twice in the same shellcode. Use the mov shellcraft template for the destination register to avoid an instruction if the result is already in the desired register.
* Update CHANGELOG
* Add support for finding corefiles under WSL2
The `wsl-capture-crash` tool stores the crash dumps in the Windows filesystem under `%TEMP%\wsl-crashes`. Look in that directory.
This allows to use the `process.corefile` infrastructure when debugging locally under WSL2.
* Update CHANGELOG
* Add Ubuntu and Debian debuginfod servers
Most CTF challenges are hosted on Ubuntu or Debian [citation needed], so check their debuginfod servers before trying the federating one as a fallback.
This is required since the elfutils.org server seems down for maintenance occasionally.
* Update CHANGELOG
* Add automatic detection of statically linked binaries in `detect_missing_binaries` function.
* Add entry for PR #2570 to CHANGELOG.md: fix `pwn template` for statically linked binaries.
* Remove partial test implementation
The commands should be tested end-to-end instead of using doctests (later).
---------
Co-authored-by: Peace-Maker <[email protected]>
The debugging step isn't critical and only helps debugging corefile doctest failures on Github runners if they occur.
See if we can find a coredump from the systemd-corefile handler for good measure.
Closes#2536
For example, on my machine:
> ./main_no_flag
./main_no_flag: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by ./main_no_flag)
Co-authored-by: Arusekk <[email protected]>
* Add `ssh -L` / `ssh.connect_remote()` workaround when `AllowTcpForwarding` is disabled
Use a netcat process on the remote to connect to the specified host:port and tunnel the traffic using normal `ssh.process` I/O.
This was inspired by the "Circumventing Disabled SSH Port-Forwarding with a Multiplexer" article by @guysv in the Paged Out! zine no. 5.
It allows to use `gdb.debug(arg, ssh=ssh)` to debug processes on pwn.college.
* Update CHANGELOG
* Lookup `netcat` command too
Co-authored-by: Xeonacid <[email protected]>
* Try connecting using bash /dev/tcp too
---------
Co-authored-by: Xeonacid <[email protected]>
Enable DOT_ALL next to the default MULTILINE regex flags to match newlines too with the `.*?`.
```
/* this works */
/*
this is not highlighted as a comment
*/
```
Lets say we have an ELF with the following symbols
```
Symbol table '.symtab' contains 5 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000000035 0 NOTYPE LOCAL DEFAULT 2 aaaa
2: 0000000000000022 0 NOTYPE LOCAL DEFAULT 2 bbbb
3: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 2 cccc
4: 0000000000000054 0 NOTYPE GLOBAL DEFAULT 2 dddd
```
Then pwnlib's ELF(binary).symbols will be `{'aaaa': 53, 'bbbb': 34, 'dddd': 84}`.
This is missing the symbol `cccc`, because its value is 0.
This change checks the name instead of the value, because the value can
be 0 if the symbol points to the beginning.
The new and correct value of pwnlib's ELF(binary).symbols will be
`{'aaaa': 53, 'cccc': 0, 'bbbb': 34, 'dddd': 84}`.
* Do not unresolve zeros
* Do not add undefined symbols
Co-authored-by: peace-maker <[email protected]>
Co-authored-by: Arusekk <[email protected]>
Pip doesn't have a RECORD file anymore and fails to uninstall / upgrade
on latest Ubuntu now. --force-reinstall on pwntools causes pip to
upgrade too since it's in our dependencies, which fails too.
As a workaround uninstall pwntools manually before upgrading.
* ssh: allow passing `disabled_algorithms` keyword argument from `ssh` to paramiko
* update changelog
* point to paramiko documentation
* point sphinx docs to latest stable version of paramiko
* ROP: fix `self.leave is None` in some libc
When `pop rbp; pop rsp; ret;` exists in libc, it is chosen instead of
`leave`. Add a specific `order` to filter out the former one.
* add changelog for #2506
* ROP: add sp_move when `pop rsp`
* CHANGELOG: mv #2506 to `5.0.0`
* doc: add example case for `tuple`
* update changelog
* doc: fix typos and test errors
* gdb: complete attach doc
* gdb: sleep in doc test to pass test
---------
Co-authored-by: peace-maker <[email protected]>
I often use `checksec *` to lazily avoid typing filenames in a directory.
If the directory contains any other sub-dirs, the command fails.
With this patch, checksec will silently skip dir paths. There's still
TOCTOU issue but I don't think checksec do anything important enough
to explicitly use try/catch to account for that.
Co-authored-by: peace-maker <[email protected]>
* Fix installation on Python 3.5 and lower on stable
The unix-ar package requires Python 3.6. Only install it on Python 3.6 then and use the Python 2 fallback for Python <= 3.5 too.
* Update CHANGELOG
* Drop Python 2.7 support / Require Python 3.10
Only test on Python 3 and bump minimal required python version to 3.10.
* Update CHANGELOG
* Allow to install on Python 3.4
* State intend to not artifially raise the required Python version
* Allow to passthru kwargs on `ssh.__getattr__` convenience function
Instead of only passing the command arguments themselves to `ssh.system` when using `ssh.somecommand('args')`, allow to specify keyword arguments as well `ssh.somecommand('args', tty=False)`.
* ssh: Don't allocate a tty for internal state fetching
If the remote server is printing some motd agressively, this could lead to interpreting the motd instead of the requested output.
* Use `ssh.system` internally too instead of deprecated `ssh.run`
* Update CHANGELOG
* packing: Do use extra arguments in `p*` and `u*`
* packing: Force keyword syntax for `sign` argument
---------
Co-authored-by: peace-maker <[email protected]>
* Windows: Raise EOFError during process.recv when stdout closes
If the receiving thread terminates while we're waiting for data to arrive we'd be waiting endlessly. Raise EOFError when .recv()ing on a process with stdout closed and no more data queued up from the receiver thread.
* Update CHANGELOG
* support starting a kitty window with the 'kitten' command
* get kitty pid and sigterm instead of close-window
---------
Co-authored-by: peace-maker <[email protected]>
* Add `+LINUX` and `+WINDOWS` doctest options
This allows to selectively run tests only on a single platform. We can add `# doctest: +LINUX` comments to tests that cannot work on Windows and the other way around.
To easily skip a lot of tests the `doctest_additional_flags` global variable can be defined in a `testsetup`.
This is achieved by monkey patching sphinx doctest's DocTestBuilder to use our own DocTestRunner which removes examples from the tests that have flags that don't match the platform we're running on.
* Limit Sphinx version to secure platform patches
Avoid major versions which might change the API. We have to check if the platform optionflags still work on newer versions once they are available.
* CI: Run doctests with coverage on Windows
Disable all non-trivial tests on Windows for now. The goal is to reduce the amount of linux-only tests.
* Only apply platform patch on Python 3
* Disable uploading coverage on Windows
The handrolled coveralls upload cannot handle mixed operating systems.
Refs #2480
* Use threading.Timer for doctest timeout
To interrupt the code running on the main thread, we send a signal using `_thread.interrupt_main()`. By default this causes a KeyboardInterrupt exception, which might be handled explicitly.
To raise an explicit EndlessLoop exception inside the code that is taking too long, register a SIGABRT signal handler which raises the EndlessLoop exception. The exception from the signal handler is added to the call stack and handled by the code currently running. This allows to print a better stack trace on timeout.
It is the same concept as the old implementation using `signal.alarm` but platform agnostic.
https://anonbadger.wordpress.com/2018/12/15/python-signal-handlers-and-exceptions/
* Add POSIX optionflag
Run test on other UNIX systems too if they don't use Linux specifics.
Add a TODO optionflag too to mark platform restrictions that might be too strict and should be looked at.
* Enable tube and tube/sockets tests on Windows
* Use `signal.alarm` for timeouts if it's available
* Update CHANGELOG
* Replace 'keepends' with 'drop'
Add 'drop' argument in place of 'keepends' for tubes. 'keepends' still
works but raises depreciation warning.
* changelog
* Ignore line numbers in PyLint check
---------
Co-authored-by: peace-maker <[email protected]>
There were additional `timeout=X` additions in #2382 which caused tests to fail randomly when a timeout was reached. The tests run through occationally, so it's not an infinite loop. But flakey tests are annoying and I don't see a reason for the timeout additions to the doctests for the core patch of that PR other than failing early during manual tests of new functionality. CI running smoothly is more important than fast failing during development on the gdb module imo.
* Fix sphinx warnings in docstrings
* Update sphinx for Python 3.13 support
Could not import extension sphinx.builders.epub3 (exception: No module named 'imghdr')
`imghdr` was removed in Python 3.13.
* Update fmtstr.py documentation
Clarified that the `value` in the `writes` dict (for `fmtstr_payload()`) should be a bytestring if the value to be written is shorter than a long
* Update fmtstr.py documentation for FmtStr class
Updated example and definition for `write()` function in `FmtStr` class to include bytes as the value.
* Fix code formatting
---------
Co-authored-by: Peace-Maker <[email protected]>
* Add `tube.upload_manually`
Upload data in chunks when having a tube connected to a shell.
This is useful when doing kernel or qemu challenges where you can't use the ssh tube's file upload features.
* Add tests and fix corner cases
* Update CHANGELOG
* Fix character escaping in tests
* Fix gzip compression on Python 2
* Patch for #issues/2290
* Return listening process on `pidof(("0.0.0.0", 1234))`
Instead of returning the process which is connected to port 1234,
return the process which is listening on that port.
* Update CHANGELOG
---------
Co-authored-by: Peace-Maker <[email protected]>
This reverts #2346 which apparently didn't do what was expected. The logger has to be initialized for the checksec output to show up, so some other file imported * from pwn before too.
With the double registration of subparsers removed in f28d3ebb6c this didn't happen anymore making checksec print nothing.
* Allow to disable caching
By setting `context.cache_dir = None`. Generate default cache dir again by setting `context.cache_dir = True`.
* Disable cache in `asm` caching test
* Update CHANGELOG
* Fix asm cache test
* Throw error when using `sni` and setting `server_hostname` manually in `remote`
Prevents silently replacing the `server_hostname` that was provided in `ssl_args`.
Fixes#2425
* Update CHANGELOG
* Bump tested python version to >= 3.10
Some workflows were using 3.8 still. Start testing on 3.12 too.
* Fix installing rpyc in CI on ubuntu 24.04
`ubuntu-latest` now points to 24.04 which requires the --break-system-packages dance.
* Test Python 2 on Ubuntu 22.04
Ubuntu 24.04 dropped the python2.7 packages.
* Test on android-34
* Fix ARM binutils disassembly output test expectation
It used to print "; 0x4" but changed to "@ 0x4" in some version.
* Skip QEMU LD_PREFIX path test
Ubuntu 24.04 seemed to have switched the qemu-user --help output to show `/usr/gnemul/qemu-$ARCH` instead of `/etc/qemu-binfmt`. Ignore the actual path in the test.
* Fix x86 ascii shellcode encoder test
The alphabet was using `\` escape sequences badly.
<stdin>:1: SyntaxWarning: invalid escape sequence '\]'
* Fix mips xor encoder unaligned memory access
qemu throws a SIGBUS error when the shellcode tries to access unaligned memory since some version. Align the "stack" properly.
* Fix util.lists.partition test output on Python 3.12
The __repr__ output of OrderedDict was changed to look like ordinary {} dicts.
* Fix safeeval test on Python 3.12
There is a 151 RESUME opcode in Python 3.12 now.
* Fix registering commandline subparsers multiple times for Python 3.11
Python 3.11 added a sanity check to argparse to prevent registering the same subparser multiple times.
argparse.ArgumentError: argument command: conflicting subparser: cyclic
Avoid importing the command twice.
`python -m pwnlib.commandline.cyclic` failed on Python 3.11 triggered in CI when running the commandline tools while collecting coverage.
* Fix pip cache in CI
* Fix returning wrong pid in WSL for run_in_new_terminal
We run cmd.exe which and other Windows processes before going back into WSL. The returned pid would be the one for the ephemeral cmd.exe process instead of the real command we wanted to launch.
I don't know how to properly trace the execution and get the command pid, so scan for newer pids matching the command for a while instead as a workaround.
We wrap the command in a script so psutil.Process.exe() returns the path to the shell instead of the real command. Look at psutil.Process.cmdline() too which contains the real program running right now.
* Make `proc.wait_for_debugger` return the debugger pid
If we fail to get the pid when launching gdb, grab it after tracing the debugger at least. gdb won't be closed when the exploit exits but at least we have the correct pid.
* Fix working directory of run_in_new_terminal in WSL
The process' cwd would be %WINDIR% due to cmd.exe not supporting WSL paths.
* Update CHANGELOG
The upload-artifact action v4.4.0 excluded hidden files from uploads by default. We want to upload only hidden `.coverage*` files though. Enable hidden files again using the `include-hidden-files` input.
https://github.com/actions/upload-artifact/releases/tag/v4.4.0
* Add ELF.close() to release resources
`ELFFile.close()` just closes the mmap'd file, but our own wrapper keeps the file handle open.
This is annoying when using a temporary file on Windows since they can't be deleted if there is
a dangling handle. This can be used together with the inherited context manager from ELFFile.
* Update CHANGELOG
* Fix parsing the PLT on Windows
The Unicorn Engine 1GB workaround doesn't work since `mmap` has different semantics on Windows.
* Update CHANGELOG
* Catch ImportError instead
* Remove unused sys import
* Cache output of `asm()`
To speed up repeated runs of an exploit, cache the assembled output.
Use a sha1 hash of the shellcode as well as relevant context values
like `context.arch` and `context.bits` to see if the exact same
shellcode was assembled for the same context before.
Fixes#2312
* Return path to cache file if `not extract`
* Update CHANGELOG
* Create temporary copy of cached file
* Add debug log about using the cache
* Include full assembler and linker commandlines in hash
This should catch any changes across pwntools updates and system environment changes.
* Include pwntools version in hash
* Support asm/disasm on Windows
Find binutils toolchains like https://gnutoolchains.com/ if they exist in the PATH.
* Update CHANGELOG
* Update binutils install docs
* cpp reads from stdin when no infile is given
No need for a tempfile here.
On Ubuntu 22.04 gdb.debug() takes at least 2 seconds. This is because
it prints only "Remote debugging from host 127.0.0.1, port 33398", but
the code expects 2 lines.
It's very unclear what the second line is supposed to be; the only lead
is "* Handle extra newline printed by gdb" in commit 338fbeba ("Improve
pwnup template, gdbserver detection (#1148)"), but I could not trace it
back to the GDB source code, both historic and modern. Perhaps, there
is a non-upstreamed patch in some distro that introduces it.
It should still be safe to skip waiting for the second line if the
first one already starts with "Remote debugging ...", so do it.
* added two arguments to gdb.debug:
+ port: specifies the port that should be used by the gdbserver (this is useful for alpine)
+ gdb_args: allows forwarding arguments to the gdb binary spawned by gdb.attach() (just a passthrough)
aslo downgraded "GDB Python API is supported only for local processes" to a warning, because it seems to work on some systems.
* added changelog entry and fixed python2.7 compatibility
* also added optional gdbserver_args parameter to gdb.debug()
* implemented requested fixes for #2382:
+ gdb.debug port is now also used for qemu
+ GDB Python API is now tested for tubes.process.process a warning for ssh.process and an error for everything else
+ updated docs to use mention that gdbserver ports are randomized by default
+ now using gdbserver_port to check if the correct port was set
+ fixed CHANGELOG.md structure
* implemented requested fixes for #2382:
+ gdb.debug port is now also used for qemu
+ GDB Python API is now tested for tubes.process.process a warning for ssh.process and an error for everything else
+ updated docs to use mention that gdbserver ports are randomized by default
+ now using gdbserver_port to check if the correct port was set
+ fixed CHANGELOG.md structure
+ added timeouts for recvline() gdb tests
+ also run gdb api tests for ssh runner
---------
Co-authored-by: gfelber <@users.noreply.github.com>
Co-authored-by: peace-maker <[email protected]>
* Fix access of undefined `ssh._tried_sftp` in `raw` mode
The attribute was never set when setting `ssh(raw=True)` which caused the code to run `ssh.__getattr__` which raised an AttributeError due to `_tried_sftp` starting with an `_` on Python 3.12.
* Document all `ssh` attributes
Only add it when we don't want to manipulate the environment or argv[0].
This will cause the problem with fish shell to reappear in those cases, but
it's another step for compatibility at least.
#2378
* Return buffered data on first EOF in tube.readline()
When there is still data available in the tube buffer when an EOFError occurs in
`tube.recvline()`, return that data even though it doesn't contain a newline.
The next time `tube.recvline()` is called afterwards will raise EOFError normally.
This behavior is in line with the GNU readline implementation and avoids
loss of data. It allows `tube.stream()` to print everything that's received before
the receiving end terminates.
A new warning is logged when data is returned due to an EOF informing
about the lack of the trailing newline character.
Fixes#2366
* Update CHANGELOG
* Add context.throw_eof_on_incomplete_line
Allow to control the behavior of `tube.recvline` and
be able to suppress the new warning.
* Cleanup docs
* Don't go though a shell in `gdb.debug`
gdbserver starts a shell and runs the target process through it.
This behavior was added in gdbserver 8.0 together with the
commandline flag --no-startup-with-shell to disable it.
* Update CHANGELOG
* Recompute text.has_bright when num_colors changes
The number of simultaneous colors in the current terminal are queried
later during initialization and the text.num_colors attribute is updated accordingly.
The text.has_bright attribute wasn't updated when that happened and always
remained set to False. Calculate those properties based on num_colors dynamically.
* Update CHANGELOG
* Enable bright colors on Windows too
Only get "colors" on Windows to avoid #1201 again.
* Fix support for old terminals on windows
* Change from pop to keys for reporting proper error message
* update changelog
* add doctest for xor
* Add newline to run xor doctests
* Switch to `dict.popitem()`
---------
Co-authored-by: marinelay <[email protected]>
Co-authored-by: Peace-Maker <[email protected]>
* Added Missing MacOS Requirements
* Updated Binutils Install Docs
Due to security reasons Homebrew has removed direct formula references, this is the workaround.
* Updated changelog for #2392
* reverting changelog
* Change newline when setting `context.os`
Windows uses `b'\r\n'` for newlines while unix uses `b'\n'`. Change the expected newline to `b'\r\n'` when setting `context.os = "windows"` automatically for convenience.
* Update CHANGELOG
* Fix other tests setting os
* Add basic RISCV64 shellcraft support
The `mov` template isn't 100% null-byte and newline free
for all inputs. Certain larger values are just emitted using
the `li` pseudo-instruction by the assembler which might
contain null-bytes.
Co-authored-by: LevitatingLion <[email protected]>
* Generate RISCV syscall constants from musl-libc
diet-libc doesn't support RISCV, so borrow from musl.
Need to consider their license.
* Update CHANGELOG
* Fix CHANGELOG
---------
Co-authored-by: LevitatingLion <[email protected]>
* Add basic support to debug processes on Windows
Currently only `windbg.debug()` and `windbg.attach()` are implemented,
which open a WinDbg instance and attach to the process.
* Update CHANGELOG
* Cleanup CheckRemoteDebuggerPresent call
Only require PROCESS_QUERY_INFORMATION access and check for errors when opening the process.
* process.close: Move closing of std fds after kill
Windows processes would block on fd.close() when the main thread is suspended.
using the "--no-cache-dir" flag in pip install, make sure downloaded packages by pip don't cache on the system. This is a best practice that makes sure to fetch from a repo instead of using a local cached one. Further, in the case of Docker Containers, by restricting caching, we can reduce image size. In terms of stats, it depends upon the number of python packages multiplied by their respective size. e.g for heavy packages with a lot of dependencies it reduces a lot by don't cache pip packages.
Further, more detailed information can be found at
https://medium.com/sciforce/strategies-of-docker-images-optimization-2ca9cc5719b6
Signed-off-by: Pratik Raj <[email protected]>
* Add local libc database provider for libcdb
* Remove unnecessary assert
* Add docstring for local_libcdb
* Suppress warning output while `context.libdb` sets default
* Testing the local system's libc first
* Set falsely `context.lcoal_libcdb` to turn off local libc-database integration
* Fix docstring
* Make path check in validator
* Fix doctests
* Add CHANGELOG
* Fix gdb.debug doctests
* Accept bytes in ssh.which
But still return a `str` return for backwards-compatibility.
* CI: Make rpyc visible in gdb's python
* Fix not running all doctests
They appear to have to be a free standing block of `>>>` with an empty line above.
* Remove unnecessary text type check in ssh.which
* Bugfix gdb.debug: exe parameter now respected
This commit now properly supports the exe parameter in
`pwnlib/gdb.py:debug()`, allowing a different argv[0] than the
executable.
It achieves this by leveraging the gdbsever`--wrapper` argument
with a python script that calls execve with the specified args.
* Backintegration for python2
* Update Changelog.md
* pwnlib gdb.py, try to pass these tests...
* Encode script in ssh.process(run=False) for tests
* Re-trigger Pull request tests
* gdb.py: easier script if argv[0] == exe
* gdb.py: Add test case for LD_Preload
* Add check that "=" not in misc.normalize_argv_env
This check checks prevents the use of "=" in the
key of an environment variable, which is generally
impossible.
* gdb.py Correct handling of LD_ env-variables
* Update pwnlib/gdb.py
Co-authored-by: peace-maker <[email protected]>
* gdb.py address comments
1. explicit ctypes.CDLL('libc.so.6'), handle execve failing
2. consistent namedTempFile
3. drop packing._encode() since it's done earlier
4. testcases solve argv-args confusion
* gdb.py: Fix Namedtempfile-prefix + Bugfix
* Restore prefix for gdbscript
* Unify execve wrapper script under one function
* gdb.py, Remove leftover script
* Fix logging scope and ignore_environ argument
---------
Co-authored-by: Youheng Lü <[email protected]>
Co-authored-by: peace-maker <[email protected]>
Co-authored-by: peace-maker <[email protected]>
* Fix readline omitting a trailing \n
This is a regression from #2129 which was based on a bogus test which mixed `input` and `readline` calls.
Only `input` (and `raw_input`) are documented to strip the newline, but `readline` itself should include it.
Fixes#2342
* Update CHANGELOG
* Fix Unicorn Engine 1GB limit exit(): raise OSError instead (Fixes#2343)
This commit fixes#2343: an issue where `pwn checksec <binary>` would fail with a bogus error of:
```
$ pwn checksec /root/x/bin/main
Could not allocate dynamic translator buffer
```
This actually comes from Unicorn Engine which tries to allocate a 1GB RWX mapping:
```
root@pwndbg:~# strace -e openat,mmap pwn checksec /root/x/bin/main 2>&1 | tail -n 10
openat(AT_FDCWD, "/usr/lib/python3/dist-packages/mpmath-0.0.0.egg-info/PKG-INFO", O_RDONLY|O_CLOEXEC) = 7
openat(AT_FDCWD, "/usr/local/lib/python3.10/dist-packages/unicorn/lib/libunicorn.so.2", O_RDONLY|O_CLOEXEC) = 7
mmap(NULL, 22447520, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 7, 0) = 0x7f2604f9d000
mmap(0x7f2605339000, 13496320, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x39c000) = 0x7f2605339000
mmap(0x7f2606018000, 3039232, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x107b000) = 0x7f2606018000
mmap(0x7f26062fe000, 1601536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x1360000) = 0x7f26062fe000
mmap(0x7f2606485000, 525728, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2606485000
mmap(NULL, 1073741824, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
Could not allocate dynamic translator buffer
+++ exited with 1 +++
```
...and if it fails, it calls `exit()`. This can be seen in Unicorn Engine code: https://github.com/unicorn-engine/unicorn/blob/56f3bdedb42d26bee1532cc01baf5eaf44a9aa23/qemu/accel/tcg/translate-all.c#L960-L963
This issue has been reported to Unicorn Engine in https://github.com/unicorn-engine/unicorn/issues/1766 but since it hasn't been fixed, this commit applies a workaround for it.
* CI: add test for pwn checksec with 500MB limit for UE 1GB limit
* Add changelog entry
* Update .github/workflows/ci.yml
---------
Co-authored-by: peace-maker <[email protected]>
* Add a `flatten` argument to `ssh.libs`
This option makes us able to avoid getting the file tree of the remote
server and just downloads the desired files in the output folder
* Update changelog
* Add a warning in ssh.libs for duplicate filenames
* Add documentation in ssh.libs for duplicate filenames fallback
* [ssh.libs] Fix flatten's duplicate check
Fix this: https://github.com/Gallopsled/pwntools/pull/2268#discussion_r1325404026
* [ssh.libs] Add better warning for flatten's duplicates
Fix this: https://github.com/Gallopsled/pwntools/pull/2268#discussion_r1325405234
* [ssh.libs] Add doctests
This commit adds:
- The `no_duplicate` binary that depends on
- `a/lib.so`
- `b/lib2.so`
- The `duplicate` binary that depends on
- `a/lib.so`
- `b/lib.so`
Then, the doctest tries to pull the libs from both binaries with
different `flatten` values.
* [ssh.libs] Add `remote` arg to documentation
* [ssh.libs] Remove fallback to unflattened when flatten fails
* [ssh.libs] Fix doctests
* [ssh.libs] Import pwnlib.data.elf.ssh_libs in pwnlib.data.elf
* [ssh.libs] Fix corrupted binaries
* [ssh.libs] We actually need to set the cwd since libs paths are relative
* [ssh.libs] Fix permission issues for CI
* [ssh.libs] Fix doctest
* Remove doctest attempts
---------
Co-authored-by: peace-maker <[email protected]>
Co-authored-by: Arusekk <[email protected]>
Co-authored-by: Peace-Maker <[email protected]>
* Optimized dynelf when using an ELF object
* Fixed minor bugs, added a new way to lookup symbols
Improved _rel_lookup and added Elf64_Rel datatype
Added support for _rel_lookup in x86 binaries
* Spelling fix
* Converted prints to python2 style
+ changelog entry
* Fixed a bug with real leaker not being resotred
---------
Co-authored-by: Bl4ckC4t <[email protected]>
* fix: split current iterm window during gdb.debug process
* add change to changelog
* escape cmd before writing osascript file
* use the previously sanitized command to run the osascript
---------
Co-authored-by: peace-maker <[email protected]>
* Fix pwn constgrep when it matches a non-constant type
This commit fixes the following issue:
```
root@pwndbg:~# pwn constgrep a
Traceback (most recent call last):
File "/usr/local/bin/pwn", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.10/dist-packages/pwnlib/commandline/main.py", line 58, in main
commands[args.command](args)
File "/usr/local/lib/python3.10/dist-packages/pwnlib/commandline/constgrep.py", line 110, in main
for _, k in sorted(out):
TypeError: '<' not supported between instances of 'Constant' and 'type'
```
Note that it was caused because of the following type object being matched and fetched from the module object:
```
ipdb> out[25:27]
[(Constant('CS', 0xd), 'CS'), (<class 'pwnlib.constants.constant.Constant'>, 'Constant')]
ipdb> sorted(out[24:27])
*** TypeError: '<' not supported between instances of 'type' and 'Constant'
```
* Add test for `pwn constgrep C` to the CI
* Add changelog entry
* Add `ELF.stripped` and `ELF.debuginfo`
Show status in checksec output too.
* Update CHANGELOG
* Show stripped and debuginfo status in red in checksec
* Match against local system libc first in libcdb
Don't do any requests if the libc currently in use on the system
running the exploit matches already. This is a small short circuit
optimization when the remote target uses the same libc as the
local one.
This looks at the libc loaded by the local shell binary. This appears
more dynamic than hardcoding library paths.
Refs #983
* Update CHANGELOG
* Handle missing SHELL envvar
* Fix hash lookup
* Retry failed lookups after one week in libcdb
The libc databases might be updated to include the searched version,
so a request that failed once might work in the future.
Refs #983
* Update CHANGELOG
Visual Studio Code sets the $TERM_PROGRAM environment variable to `"vscode"` in its terminal pane,
but doesn't support opening new panes from the command line and there is no "vscode" binary.
Make sure the target binary exists before trying to launch it.
* Explicitly define p64/u64 functions for IDE support
Instead of dynamically generating the packing and unpacking helper functions using `setattr(module, name, routine)` at runtime, unroll the loop and explicitly declare all 8 helpers.
This allows IDEs to know about the function statically without running the code.
Fixes #1657
* Update CHANGELOG
* Fix u16 comment
* Lookup process executable using PATHEXT file extensions
Windows uses some file extensions to determine which file to run when given
a non-existing file.
This allows to run `calc` instead of `calc.exe`.
* Try PATHEXT in `which` directly
* Update CHANGELOG
* Add support to start a process on Windows
This is heavily inspired by https://github.com/masthoon/pwintools
Allows to use the `process` tube on a Windows host running local
processes. I've tried to keep the changes at a minimum. Windows doesn't
support non-blocking reads, so I've opted to handle the reading in a
separate thread to simulate the non-blocking check.
* Fix `misc.which()` on Windows
`os.getuid()` is only supported on unix.
* Support [read|write]mem, libs and cwd
* Update CHANGELOG
* Don't cache loaded modules
* Wait output on stdout in `process.can_recv_raw`
Instead of handling the timeout for reading poorly in `process.recv_raw`,
wait for any bytes to be available in `process.can_recv_raw` like on linux.
This prevents a 100% spin of one core in `tube.interactive()` since it
expects the thread to block for `timeout` seconds while we would return
immediately previously.
* Remove PythonForWindows dependency
This loses the `readmem` and `writemem` feature, but that's not
necessary for this basic process startup support.
* Fix cyclic imports
* Ignore errors in reading thread
This could cause a deadlock / fastfail on shutdown when stdout is closed, thus read(1) fails
and the exception is tried to be printed to the console while stderr is in the process of getting
closed.
* Make pwn template always set context.binary
* Update CHANGELOG.md
* Allow args.EXE override in all templates
* Fall back to old behavior for remote only challenges
---------
Co-authored-by: peace-maker <[email protected]>
* Bump unicorn dependency to >=2.0.1
We're using riscv constants now which were added in unicorn2.
Fixes#2303
* Add RISCV64 plt emulation test binaries
* Detect challenge binary and libc in `pwn template`
When the `args.exe` or `args.libc` arguments are missing, look for the
files in the current working directory.
This allows to quickly get a template by just running `pwn template`
(and maybe --host --port for remote instances).
This is inspired by https://github.com/io12/pwninit
* Update CHANGELOG
* Add --no-auto argument to disable automatic detection
Allow to keep the old static template behavior.
* Add logging about detection outcome
Inform the user that we're looking for the binaries and if we found one.
* Be more strict about libc filename detection
Check about common name patterns instead of plainly the presence of "libc" in the name.
Co-authored-by: Arusekk <[email protected]>
* Change --no-auto argument to reduce double negation
Co-authored-by: Arusekk <[email protected]>
---------
Co-authored-by: Arusekk <[email protected]>
When you set the `context.terminal` to be tmux related but then try
to run the exploit outside a tmux session, you'd get a cryptic ValueError
while trying to parse the pid of the new pane in tmux.
Print a nicer reminder to start tmux first.
* term rework
* towards floating cells
* redraw when screen clears
* term: working completion
* Display non-printable characters
* ui: fix options with new term
* term: Clean up and fix examples/spinners
* changelog and lint
* Fix SSH in NOTERM mode
* defer termios setup
* term: py2 compat
* py2: compat fixup
* appease pylint
* fix scroll on winch
* term: fix hang on winch during readline
* next fails if string is not found in binary
fixes#2306
* Only display "str_bin_sh" symbol if it's available
* Update CHANGELOG
---------
Co-authored-by: Peace-Maker <[email protected]>
* Add env_add to the ssh module
* Use ignore_environ like process
---------
Co-authored-by: peace-maker <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* Add env_add to the process module
* Don't mutate passed `env` parameter dictionary
Co-authored-by: Arusekk <[email protected]>
* Update CHANGELOG
* Use a boolean flag instead of env_add
There should be preferably only one obvious way to pass env.
* Update CHANGELOG
---------
Co-authored-by: peace-maker <[email protected]>
Co-authored-by: peace-maker <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* libcdb.py - python 3.12
Python 3.12 would complain if the \d atom is not escaped in the binary string.
/usr/lib/python3.12/site-packages/pwnlib/commandline/libcdb.py:224: SyntaxWarning: invalid escape sequence '\d'
libc_version = re.search(b'libc[ -](\d+\.\d+)', exe.data)
* Use raw string
---------
Co-authored-by: peace-maker <[email protected]>
So far only relocations pointing to external symbols were considered
when filling GOT. Relocations in libc are a different thing, often
found in CTF challenges (libc with partial RELRO has many interesting
overridable function pointers, like strlen & strchrnul).
Co-authored-by: Michał Radwański <[email protected]>
* Add x86 CET status to checksec output
The ELF class now features two properties:
- ibt for indirect branch tracking `-fcf-protection=branch`
- shstk for shadow stack `-fcf-protection=return`
Both checking the presence of the respective feature bit in the
note property x86 feature segment.
* Check if user shadowstack is enabled on SSH target
Look for "user_shstk" in /proc/cpuinfo for ssh.checksec()
* Add ELF.iter_notes and ELF.iter_properties helpers
Allows to easily iterate the PT_NOTE segment and the properties in "GNU"
notes.
* Check for ibt on remote ssh machine
* Update CHANGELOG
* Suppress error when /proc/cpuinfo cannot be read
* Python 2: Fix installing from source
`python2.7 -m pip install git+https://github.com...` doesn't install
any packages.
The auto-discovery doesn't work for the old setuptools version used in
Python 2.
https://setuptools.pypa.io/en/latest/userguide/package_discovery.html#flat-layout
* Use Python 3 commandline tools in Docker images
Since the Python 2 version was installed first, the Python 3 install
didn't touch the existing shell script wrapper files.
* Fix tube.clean_and_log not logging buffered data
Only new data received after the `clean_and_log` call is printed while
data already received and buffered earlier is not.
Print the buffered data first before waiting for more so we don't miss
anything.
* Only show buffered data if DEBUG is off
* Update CHANGELOG.md
---------
Co-authored-by: Arusekk <[email protected]>
* Fix bug at ssh.py:process() - empty argv[0] Error
Before this, process.py relied on `os.execve` which disallows an empty
argv or argv[0] == "".
This commit replaces `os.execve` with `ctypes.CDLL(None).execve` to
call the C-Library function directly which allows an empty argv.
* Add #2225 to stable changelog
* Better ctypes syntax
* Add error message if cytpes.execve fails.
* Updata CHANGELOG.md
* ssh.py: python2 compatibility for os.environb
* Add check that "=" not in misc.normalize_argv_env
This check checks prevents the use of "=" in the
key of an environment variable, which is generally
impossible.
* ssh.process: Seperate cases for empty argv[0]
This commit seperates the cases for an empty argv[0]
from normal cases.
* ssh.py delete leftover comment
---------
Co-authored-by: Youheng Lü <[email protected]>
Co-authored-by: Arusekk <[email protected]>
Co-authored-by: peace-maker <[email protected]>
* Fix corefile module after pyelftools update
The `bytes2str` internal method was moved while Python 2 support was
removed in pyelftools 0.3.0.
Just use our own python aware decoder wrapper.
Fixes#2260
* Pin pyelftools < 0.3 for Python 2
* Fix pyelftools version number scheme
* Update CHANGELOG
* Fix pushing docker images on manual workflow run
Running the docker workflow manually would associate the run with the
`dev` branch. The generated tags would all point to the `dev` tag, so
tag the image manually to the correct branch.
* Fix building ci image
It's called `python2.7` instead of `python`.
* Add riscv64 binutils to base image
* Fix ssh.process not setting ssh_process.cwd
The cwd wasn't passed to the ssh_process created in ssh.process,
so the ssh_process.cwd attribute was never set.
This passes the cwd along now.
```
s = ssh(..)
io = s.process("pwd", cwd="/tmp")
io.cwd # "."
io.recvall() # b"/tmp\n"
```
* Update changelog
* Add ELF.set_runpath()
Shells out to the `patchelf` tool to patch the ELF's RUNPATH. This lets the dynamic loader look for needed shared libraries in the given path first before the system libraries when running the binary.
* Add ELF.set_interpreter()
Shells out to the `patchelf` tool to patch the ELF's PT_INTERP segment. This allows to change the ld.so used when running the binary.
* Add convenience wrapper to set runpath & interpreter
A helper function to patch the ELF such that it uses the dynamic loader and other libraries in the given folder.
* Add method to download libraries matching a libc
Download the matching libraries for the given libc binary and cache them in a local directory using `libcdb.download_libraries()`. The libraries are looked up using libc.rip and fetched from the official package repositories if available.
Only .deb and .pkg.tar.* packages are currently supported (Debian/Ubuntu, Arch).
* Add --libc argument to pwnup template
This generates code into the template which allows you to run the binary using the given libc.
The foreign libc is used by default, but you can choose to run the binary against your system's local libc using the `LOCAL_LIBC` command line argument when executing the exploit script.
* Work around Python 2 tarfile not supporting xz
* Fix crash when libc wasn't found on libc.rip
* Update README
* Use subprocess.check_output instead of pwnlib.process
* Special case Python 2 instead of 3
* Only catch Exceptions instead of everything
Co-authored-by: Arusekk <[email protected]>
* Check launchpad.net for Ubuntu libcs
This mimics the way io12/pwninit obtains the ld.so.
If the download from libc.rip fails, try launchpad.net.
* Hide comment in template when --quiet
* Please confused pylint in PY2 context
Co-authored-by: Arusekk <[email protected]>
---------
Co-authored-by: Arusekk <[email protected]>
Two minor changes were made in the commit:
- `which` was replaced with `command -v`
- Minor bug fix
## Why bother with `which` ?
- `POSIX` compatible
- `which` is an external process you're launching for doing very little (meaning builtins like hash, type or command are way cheaper), you can also rely on the builtins to actually do what you want, while the effects of external commands can easily vary from system to system.
- Many operating systems have a which that doesn't even set an exit status, meaning the if which foo won't even work there and will always report that foo exists, even if it doesn't (note that some POSIX shells appear to do this for hash too).
- Many operating systems make which do custom and evil stuff like change the output or even hook into the package manager.
Reference [here](https://stackoverflow.com/a/677212)
Co-authored-by: Debjeet Banerjee <[email protected]>
This commit adds information that the sleep shellcraft function does not check if it returned as part of an interrupt and it does not retry the syscall if this occured.
This appears to be fixed in setuptools 62, but Python 2 is stuck at
version 44. Use a workaround which overrides the disabled user installs
temporarily.
This fixes our develop Dockerfile too!
* Don't change log level for Corefile._parse_stack()
If we don't want to see the log, don't print it. There are no spammy
debug logs anymore.
Fixes#1666
* Fix doctest
From: Peace-Maker <[email protected]>
* Shellcraft: Fix spilling syscall args to the stack
There was a typo in the generated syscall template causing only the last
stack argument to be generated.
$ shellcraft -f asm mips.linux.sendto 3 0x123456 0x100 0 0xabcdefff 0x10
The 0xabcdefff argument was missing.
* Shellcraft: Generate 6 syscall arguments for unknown functions
There are syscalls with 6 arguments like mmap and sendto which couldn't
be called with all 6 arguments set.
$ shellcraft -f asm mips.linux.mmap2 0 0x1000
'PROT_READ | PROT_WRITE | PROT_EXEC' 'MAP_PRIVATE | MAP_ANONYMOUS' -1 0
* Shellcraft: Fix typo in error message
The `syscalls` variable is only present in the generate.py, not the
generated template.
* Regenerate syscall templates
* MIPS shellcraft: Push last two syscall arguments to the stack
The mips.linux.syscall template was only handling 4 syscall arguments
and silently discarded the remaining ones. Push the arguments to the stack
instead.
$ shellcraft -f asm mips.linux.syscall SYS_sendto 3 0x123456 0x100 0 0xabcdefff 0x10
Fixes#2153
* Fix mips syscall doctest
* Update CHANGELOG
Co-authored-by: Peace-Maker <[email protected]>
* Add shellcraft.sleep template wrapping SYS_nanosleep
Accepts the time in seconds as a float argument and calls SYS_nanosleep.
Fixes#1428
* Update CHANGELOG
* Use nanosleep wrapper
Co-authored-by: Arusekk <[email protected]>
* Fix bug at ssh.py:get() with relative path
* Add doctest for bug at ssh.py:get()
* doctest for download and download_file
* remove os.path.join, since _download_raw already seem
handle remote paths
* Add #2214 to stable changelog
* Fix Doctest for ssh.py to not be dependent on user dir
* Update CHANGELOG.md
* Fix typo in ssh trying to access log.Progress.error
That function doesn't exist and .failure() is used everywhere else.
* Use log.error instead of manually raising
The log.Progress logger doesn't print the exception when exiting the scope.
---------
Co-authored-by: Arusekk <[email protected]>
* Fix `remote` and `listen` in sagemath
sage's Integer doesn't seem to inherit from python's base int.
```
----> 1 listen(Integer(1337))
OSError: Int or String expected
```
Fixes#2201
* Update CHANGELOG
remote_tar is a `bytes`, so the string interpolation resulted in the
command `cd ... && tar -xzf b'/tmp/...tar.gz`. It also would have failed
had either path contained spaces.
To fix this, make everything a bytes and use proper escaping.
Co-authored-by: Arusekk <[email protected]>
"1001" would highlight the "00" even though it's not a null byte in the
input. Group the bytes correctly to look for real null bytes and
newlines only ignoring accidental matches of adjacent bytes.
Co-authored-by: Arusekk <[email protected]>
This commit forces the `path` class variable of the `ELF` class to be of
type string, even if a byte string argument is passed. This also makes
the implementation consistent with the documentation, which states that
`ELF.path` is of type `str`.
Fixes#2166
Cherry-picked from 60af3a26d0
Signed-off-by: James Raphael Tiovalen <[email protected]>
This commit forces the `path` class variable of the `ELF` class to be of
type string, even if a byte string argument is passed. This also makes
the implementation consistent with the documentation, which states that
`ELF.path` is of type `str`.
Fixes#2166
Signed-off-by: James Raphael Tiovalen <[email protected]>
* Fix libcdb.unstrip_libc tests
The old debugsymbols seem to have been pruned.
Update hashes to recent Ubuntu 22.04 libcs.
* Switch wget test to httpbingo.org
httpbin.org is very slow to respond.
Cherry-picked from 1998ff0920
* Handle `context.newline` correctly in `tube.interactive()`
The `context.newline` or `self.newline` variable isn't obeyed when typing in interactive mode. It is used when sending and receiving lines through `tube.sendline` though, causing a mismatch.
The receiving end of the `tube.interactive()` already has handling of newlines, but the sending side does not.
Example:
```python
from pwn import *
io = process('cat')
io.newline = b'\r\n'
io.sendline(b'auto')
io.interactive()
```
```
$ python testinteractive.py DEBUG
[x] Starting local process '/usr/bin/cat' argv=[b'cat']
[+] Starting local process '/usr/bin/cat' argv=[b'cat'] : pid 19060
[DEBUG] Sent 0x6 bytes:
b'auto\r\n'
[*] Switching to interactive mode
[DEBUG] Received 0x6 bytes:
b'auto\r\n'
auto
$ test
[DEBUG] Sent 0x5 bytes:
b'test\n'
[DEBUG] Received 0x5 bytes:
b'test\n'
test
```
The expected outcome would be to send `b'test\r\n'.
The same problem arises in non-term mode (`PWNLIB_NOTERM=1`), where stdin is read in binary mode causing the OS line seperators to come through. Correctly replacing them with the `context.newline` setting allows to use the interactive input on windows hosts as well, without constantly sending `\r\n`.
* Update CHANGELOG.md
* Only replace newlines in TTYs
Don't mess with the line endings when piping data through.
* Always send bytes immediately on receive
Only replace newlines in a tty if NOTERM wasn't explicitly set or
the `newline` setting was set.
* Don't return a newline after term.readline lines
The builtin `input()` doesn't include the newline.
* Remove bogous newline from CHANGELOG
* Fix libcdb.unstrip_libc tests
The old debugsymbols seem to have been pruned.
Update hashes to recent Ubuntu 22.04 libcs.
* Switch wget test to httpbingo.org
httpbin.org is very slow to respond.
* added feature to the fmtstr module, now able to generate payload without the dollar syntax.
* made some improvements in the no_dollar feature implemented
* reduced different lines
* removed some blank lines and debug stuff
* (fix) missing check for no_dollar option
* (fix) updatet argument type for documentation
* updated CHANGELOG.md
* (fix) typo in the CHANGELOG.md version
* Update pwnlib/fmtstr.py
suggested change, dont need comparison in calling the make_payload_dollar
Co-authored-by: Arusekk <[email protected]>
* Update pwnlib/fmtstr.py
suggestion; better use a non null byte thing to fill the values used by the %c to pad before writing.
Co-authored-by: Arusekk <[email protected]>
* (fix) minor fixes, typos on comments
* update pwnlib/fmtstr.py : suggestion from Arusekk, made improvements to save some bytes in the generation of the payload
* added tests to fmtstr_payload with no dollar flag
* update CHANGELOG.md with correct version
---------
Co-authored-by: Arusekk <[email protected]>
* Move most settings to pyproject.toml
* py2 minimal compat
* Make it not crash nor include spurious files
* Use PEP517
* Update workflows
* Limit workflows
Closes#1587
* Fix debugging and assembly for RISCV
They are broken since the riscv architecture was split into riscv32
and riscv64 internally in #2177.
* Always look for riscv32. riscv64 or riscv toolchains
* Look for riscv64 for riscv64 first
* Determine elf arch by e_machine and bits
Separate 32/64 for riscv and mips
Trim some trailing space
* Fix missing comma
Co-authored-by: Arusekk <[email protected]>
* Add CHANGELOG
* unicorn: Use fixed version
* Cosmetic enhancements
---------
Co-authored-by: Arusekk <[email protected]>
* libcdb: Add option to search by function offsets
When you're able to leak addresses of the libc library, use `libcdb.search_by_symbol_offsets()` to find and download the matching libc library from https://libc.rip.
If there are multiple matches, the user is prompted to select one interactively. The selection can be saved in the code for future executions of the script.
Fixes#1867
* libcdb: Handle multiple results when looking up by hash
Sometimes the same library appears to be indexed multiple times (see 0b52d2e713d243f0f65d808fcd3fbe372bb3cd32). Handle that situation by selecting the first in the list, since they should all be identical given the same hash value.
* Update CHANGELOG
* Fix off-by-one when pre-selecting a libc
* Do .json() only once
---------
Co-authored-by: Arusekk <[email protected]>
* Fixes for download_dir.
* Adds support to use SFTP if available
* Update pwnlib/tubes/ssh.py
Co-authored-by: Arusekk <[email protected]>
* Removal of unnecessary parens after 'if' keyword
Co-authored-by: Juuso Käenmäki <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* Make template respect args.EXE
To be able to execute a script using an executable with another name and/or location than specified at template generation, `args.EXE` will be used as path for ELF file loading, only using the preconfigured value when no arg was supplied
* Update CHANGELOG.md
* FIXUP: properly test for EXE arg
* Style fixes and revert changelog
Co-authored-by: Arusekk <[email protected]>
* Support adding lists in `ROP.raw()`
Allow to add multiple raw entries on a rop chain using an array instead of multiple calls to `raw()`.
```py
rop = ROP('/bin/ls')
# before
rop.raw(1)
rop.raw(2)
rop.raw(rop.ret)
# after
rop.raw([1, 2, rop.ret])
```
Fixes#2017
* Update CHANGELOG.md
* Update pwnlib/rop/rop.py
Co-authored-by: Arusekk <[email protected]>
The base image was missing the patchelf utility and the develop image
didn't update the apt cache. That caused building the image to fail when
pulling the outdated pwntools:base image from dockerhub.
The docker Makefile was using an invalid `-depth 1` argument which appear
to have been replaced by `-mindepth` and `-maxdepth` arguments recently.
The testing container is based on the develop image's Dockerfile which
copies new files into the container. Grab them from the appropriate folder.
pwndbg was installed twice when running the testing container for the
doctests.
* Support non-bytes argument to `packing._need_text()`
Allow to pass everything to `_need_text` and be sure to get a string back. This allows to log any object without the need for a format string by calling `repr` on it.
```py
# before:
log.info('%r', 5)
# now
log.info(5)
```
Fixes#2018
This is why CI should always be green.
Make it green, dear contributor, because the maintainers might merge
something without checking if it is green. Please.
The old API was a hack apparently, but it looks like GitHub Actions slowly gets less and less sketchy over time, and even goes as far as providing well-designed APIs.
Ref. https://github.com/badges/shields/issues/8671
This commit fixes a small bug with PwntoolsLexer where it matched negative numbers incorrectly in Arm assembly.
See https://github.com/pwndbg/pwndbg/pull/1367 for more information
* Fix tube.recvpred() timout argument
The `timeout` argument wasn't implemented correctly causing the function to hang instead of timing out.
* Update CHANGELOG.md
You have to call `listen.wait_for_connection()` before using the listen socket or you face raceconditions with the thread accepting the client connection.
Fixes#2100
* Allow tube.recvregex to return capture groups
When wanting to extract a value from the received data, one had to receive up until the value and the value itself in two steps without carving the data out of a large blob like this:
```
io.recvuntil(b'The address is: ')
address = int(io.recvline(), 16)
```
or use a regex twice:
```
pattern = re.compile(rb'The address is: (0x[0-9a-fA-F]+)')
data = io.recvregex(pattern)
address = int(pattern.search(data).group(1), 16)
```
This patch allows to pass `capture=True` to `tube.recvregex()` to return a `re.Match` object instead of the raw bytes.
```
addr = int(io.recvregex(rb'The address is: (0x[0-9a-fA-F]+)', capture=True).group(1), 16)
```
This allows for slicker data extraction!
Fixes#1962
* Update CHANGELOG.md
* Fix ROP without a writeable cache directory
`context.cache_dir` can be None, but the ROP class assumed it would always be a valid path and crash. Disable rop-cache if we can't save it.
Fixes#2072
* Update CHANGELOG.md
* Switch to federated debuginfod service
elfutils.org offers a federating proxy server which forwards the requests to all other debuginfod servers of the different linux distributions.
https://sourceware.org/elfutils/Debuginfod.html
Use that instead by default to increase our successrate and potentially include new servers automatically once the distro sets one up.
* Use DEBUGINFOD_URLS envvar for server selection
The `DEBUGINFOD_URLS` environment variable is what you use for your normal debuginfod setup to tell it where to look for debug symbols.
Parse it as well and prepend it to the list of servers to try to allow for dynamic adjusting of the target servers.
* Fix for overwritten env when parsing core file
* warn_once message when dealing with overwritten pointer to env variable in core file
* Check for sha256 and sha256sum before trying openssl sha256 as "openssl sha256" doesn't return an error code in some old versions of openssl (OpenSSL 0.9.8g 19 Oct 2007 at least) that doesn't support sha256.
* ret2csu: fix a silent failure when rdx is populated from r14.
One such binary can be found at https://github.com/nobodyisnobody/write-ups/tree/main/FCSC.2022/pwn/Deflation
* Clean up the code a bit
Co-authored-by: Zach Riggle <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* make pwn cyclic -l work with entry larger than 4 bytes
* Do not assume py3, do not assume little-endian
* Update changelog
* pack all bytes instead
Co-authored-by: Arusekk <[email protected]>
* Change prefix in install script to $PREFIX
On special envrionments like Termux, /usr/local is not the proper path.
* Change prefix in binutils install script to $PREFIX
* PREFIX not everywhere
Co-authored-by: Arusekk <[email protected]>
coredumpctl saves the coredumps with full path to the binary. Passing only the basename makes it fail to find the coredump when it is actually there.
Closes#1879
Fix wrong branch taken when trying to debug non native arch via ssh
(resulting qemu_port being referenced without being set).
This is done by setting the condition in line 571 to be the same
as in line 534
Signed-off-by: Aviya Erenfeld <[email protected]>
* Add the 'raw' option to SSH shell initialization
This permits the use of the SSH shell as an arbitrary TCP tunnel by
bypassing the startup environmental checks.
Co-authored-by: LMS57 <[email protected]>
* Add changelog items
* Add assignment to self variable
Co-authored-by: LMS57 <[email protected]>
* Add js_escape and js_unescape
* Add entry for js packing functions
* Use hardcoded data for doctest
* Fix js_escape data iteration for Python 2
* Move js packing functions to util.fiddling
* Apply suggestions from code review
Co-authored-by: Arusekk <[email protected]>
* Add libcdb commandline tool
Allows to dump information about a local libc binary or lookup matching libc versions based on a build hash or symbol offsets (querying https://libc.rip).
It's possible to download and unstrip a matching libc binary from the command line.
* Fix Python 2 compatibility
* Remove main_arena heuristic
* Update CHANGELOG
Co-authored-by: Arusekk <[email protected]>
* Add badge packaging status badge
Add a repology badge to show packaging status.
* Update README.md
Use shields.io everywhere
Co-authored-by: Arusekk <[email protected]>
This fixes a misbehaviour in environments where `XDG_CACHE_DIR` is not
set and `~/.cache` does not exist, e.g. in the docker image. We simply
attempt to create a temporary directory which will be registered with
`atexit` to be deleted on exit using `shutil.rmtree()`. A Python 3
approach for future would be to use a `TemporaryDirectory` instead which
would get cleaned up when the held object gets dropped out of the TLS
stack.
It is still possible for the `context.cache_dir` to be `None` if the
default directory exists but is not writable, which means some code
which uses this property is liable to blow up in such a circumstance.
When the last gadget has no arguments and the gadget before has stack
arguments, put the last gadget directly in its return address slot
(instead of a stack adjustment gadget).
* Fix CI after Groovy Gorilla went away for libc unstrip test
Build elfutils 0.181 from source since we can't use builds
from a newer ubuntu version anymore.
* Install python wheels in CI
* Fix ELF.libc_start_main_return with glibc 2.34
The actual call to `main` was split out into a new `__libc_start_call_main` function in glibc 2.34.
Update the heuristic to find main's return address by looking into all direct calls in `__libc_start_main` until the old heuristic matches.
The other direct calls could be e.g. `atexit` which doesn't call `exit`.
* Update CHANGELOG.md
* Fix CI after Groovy Gorilla went away for libc unstrip test
Build elfutils 0.181 from source since we can't use builds
from a newer ubuntu version anymore.
* Install python wheels in CI
Just stick to the replaced names everywhere.
This actually only broke linkat(2) and symlinkat(2),
but it touches many len -> length renames as well
for the sake of consistency.
Fixes#1964
Ubuntu, why are you doing this to us?
I seriously mean that Ubuntu is the most silly distro.
Like, they probably updated their docker (or VM) images last week,
then Github got it, and we started getting failures.
They totally changed the behaviour of that 'apport' thing,
and it now drops core files in /var/lib/apport/coredump
instead of the current working directory.
Why is that even... ugh. Nevermind. It works now, at least.
But for how long?
Ubuntu, why are you doing this to us?
I seriously mean that Ubuntu is the most silly distro.
Like, they probably updated their docker (or VM) images last week,
then Github got it, and we started getting failures.
They totally changed the behaviour of that 'apport' thing,
and it now drops core files in /var/lib/apport/coredump
instead of the current working directory.
Why is that even... ugh. Nevermind. It works now, at least.
But for how long?
* Fix: TypeError: can't concat str to bytes
sh_string returns a string, but two lines try to concat it with bytes, which throws a TypeError. Either convert them all to bytes or all to string.
* Update to use packing._need_bytes
I just followed the format from surrounding conversions. Hopefully that works correctly.
* Update pwnlib/tubes/ssh.py
Co-authored-by: Arusekk <[email protected]>
* Conversion to bytes
* Going for higher level conversion
Co-authored-by: Arusekk <[email protected]>
* libcdb: Load debug info and unstrip libc binary
The debug info is obtained from a debuginfod server maintained by systemtap.org. elfutils is used to unstrip the binary and merge the debug info back.
Try to unstrip every libc binary downloaded by the libcdb module by default. #1818
* Install elfutils on CI
* Update CHANGELOG.md
* Add more tests
* Fix negative test
* Raise an exception when eu-unstrip fails
This will hopefully get logged during doctests.
* Install elfutils 0.181 from groovy
Use APT Pinning to select the newer elfutils version from groovy instead of the buggy 0.176 version available in focal.
* Fix missing import of requests
pylint?
Co-authored-by: Arusekk <[email protected]>
Apparently it is us who used unicorn The Wrong Way,
e.g. trying to fetch instructions @ 0xdbdbdbdb,
which is both unaligned and high-address-space.
Unicorn just changed the exception to throw exception with the address
of an offending jump instruction, instead of the incorrect address.
Making the debugs actually work and changing address to 0x7c7c7c7c
solves the issue.
Closes#1538
Ping pentoo/pentoo-overlay#734 @blshkv
* Fix i386 encoder issue #1761
Don't encode `avoid` parameter in `fiddler.py` if type is `bytearray`
* Pass `avoid` as `bytes` instead of `set` when `encoder.encode()` is called
* Requested change: Increase readability
* Update doctest in XOR encoder
Add my failing testcase to ensure the code is fixed
* Defer import of requests and urllib
This saves 100ms-150ms of startup time on "from pwn import *"
* Update CHANGELOG
* More deferred imports to save ~20ms
* Show magic gadgets and setRegsiters in the top of the documentation
* Show the cool way to set registers magically, with fewer lines of code
* Try simple fix for doctest
* ssh: Use paramiko allow_agent parameter
* ssh: Add ignore_config parameter to opt out of using ~/.ssh
* [changelog] Add `ignore_config` argument to `pwnlib.tubes.ssh` and improve `allow_agent` implementation (#1566)
Co-authored-by: Arusekk <[email protected]>
* [process] Add writemem with tests, alias readmem == leak
* Update CHANGELOG
* Explicitly show result for easier debugging on CI
* Explicitly set arch to i386
* Troubleshooting and avoid overwriting top-level asm
* Update pwnlib/tubes/process.py
* Fix nasty race condition
Co-authored-by: Arusekk <[email protected]>
* [android] Split Android AVD testing into its own Workflow
* [android] Modify to trigger tests
* [android] Make paths more succinct and explicit
* Remove environment variable
* Rename to test
* Always run Android jobs for debugging
* Enhance Android workflow
* Disable gdbserver download
* Make the workflow actually work
* Travis missing gdbserver
* Try to fix travis
* Add good ol' SIGALRM-based debugs for tests
* Revert changes made to Travis
* But fix GHA
* Actually enable the android tests
* Actually enable
* Not necessary anymore
* Actually run the tests with AVD installation
Also fix incorrect text-vs-bytes usages in protocols.adb.
Co-authored-by: Arusekk <[email protected]>
* Cleanup unconditional imports
This may aid #1725
* Import explicitly
* Add good ol' SIGALRM-based debugs for tests
* Fix py2 GDB tests somehow
XXX: see why this matters
* Update CHANGELOG.md
* Make gdb.attach work with arrays instead of string concatenation
This results in a simpler set of code that is more resilient to
issues with quoting and files with spaces in their name.
* Remove detach_and_quit for now, this was for Docker CI testing
* Fix typo
* Fix lint errors
* shlex.quote is unavailable on Python3
* Update CHANGELOG.md
* With no escaping comes no escaping
Co-authored-by: Arusekk <[email protected]>
* Add rop.ret2csu()
* Removed magic numbers
* Replaced if elf.sym.has_key(u'__libc_csu_init') with if u'__libc_csu_init' in elf.symbols
* Consistently used unicode strings to search for symbols
* Used the Capstone SKIPDATA option to deal with endbr64 instructions
* Added comment to clarify why we set rbp and rbx
* Account for newer versions of gcc populating registers in a different order
* Code style issues
* Add padding functionality
* Split files and add tests at last
* Implement flat interface in ROP
* Make linters happy
* Update changelog
* Add a more detailed doctest
* Actually set uid and hopefully solve all the corefile problems
* Show me the bug
* More verbosity
BTW fix py3 sockets incosistency
* The most debug
* Should finally work
* It must work now
Co-authored-by: CptGibbon <[email protected]>
* Prefer Python3 for the remotely-executed Python script
This should be done as part of our general transition to Python3,
but should also be done because some systems with e.g. Python2.6
do not support OrderedDict, but DO have a Python3 version that works.
* Make the script actually work in py2 and py3
* Fix & update CHANGELOG.md
This actually is a behavior change, and can be a breaking change for some,
so must be stated prominently in the changelog.
Co-authored-by: Arusekk <[email protected]>
* Fix atexit on Python 3
`sys.exitfunc` was removed in Python 3, meaning that any handlers
registered with the `pwnlib/atexit.py` module wouldn't run. This PR
reigsters the handlers with the stdlib atexit.
* Update changelog
* fixed wrong description for shellcraft.amd64.linux.kill(). new desciption copied from pwntools/pwnlib/shellcraft/templates/common/linux/syscalls/kill.asm
* updated changelog with PR #1739
* Fix more syscalls and generify
Co-authored-by: Arusekk <[email protected]>
* mips: do not use $t0 temporary variable in dupio
dupio currently uses $t0 which is a temporary callee saved register.
The dup2 call is thus allowed to trash the register.
Use $v0 instead. $v0 is the return value register.
Either dup2 returns the provided $v0 (newfd) or -1 if it fails,
and therefore $v0 can be used safely here.
Reference:
https://courses.cs.washington.edu/courses/cse410/09sp/examples/MIPSCallingConventionsSummary.pdf
* Update changelog
* Update dupio.asm
Make a small change to kick Github Actions
* Fix CHANGELOG
Co-authored-by: Heap Crash <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* Try to catch more errors by using python -bb
* Apply changes to travis dockerfiles as well
* Introduce context._need_bytes and context._need_text
...and resolve many issues
/bin/sh seems to be sending the cd command to the background, instead of
changing directories and then running the selected command.
This has worked forever, not sure why it's failing in CI now
* [io] Introduce first draft of IO module
This is designed to make it easier to do file management when a piece of code
has to deal with creating a file that might need to be local, OR might need
to reside on some SSH server somewhere.
Specifically, the GDB code needs this for future enhancements
* [docs] Add documentation and doctests
* Update CHANGELOG
* Add pathlib2 requirement for Python2
* Use Python3 documentation
* Remove pwnlib.io module
* [filesystem] Add filesystem module, with Path and SSHPath
* Use @classmethod monkey patching instead of subclassing
This means that isinstance(Path, ssh_path_instance) returns True
* Make doctests work again
* Remove debugging code
* Rename context.ssh to context.ssh_session
This is needed because @LocalContext will pop off 'ssh' and some functions
take 'ssh' as an argument, and it would all break.
* [debug] Try to figure out why CI is failing
* [ssh] Fix failing doctest
* Fix README
* Fix README
* Fix failing doctest harder
* Remove unneeded internal file
- ssh.runner() now accepts mixed bytes and strings, by forcing everything to a string
- ssh.runner() now takes e.g. log_level='debug' as a kwarg
- Simplify ">/dev/null 2>&1" into "&>/dev/null"
* Remove unnecessary pwn.toplevel.__all__
The default behavior of `from import *` is enough and better, to make Pylance work
fix#1810
* Update CHANGELOG.md
* Add a comment for removed pwn.toplevel.__all__
Co-authored-by: Heap Crash <[email protected]>
* [travis] Switch to a different server that still has the magic gdbserver
* [travis] Install rpyc using pwndbg's method for finding the GDB-built Python
* [travis] Forgot pip dropped support for Python 3.5
* [gdb] Rewrite timeout logic for GDB API and fix doctests to use "skipif"
* [gdb] I'm not sure how this GDB API stuff ever worked in the first place
But it works now, at least locally
* [gdb] Fix broken doctests
* Update timeout to 10 seconds
* [corefile] Fix some broken tests, add several more
Notably, some were outright incorrect (corefile.libc for a statically-linked ELF)
Also fixed up some of the documentation to add more links, and added examples
of reading / writing / saving / reloading a corefile.
* [corefile] Some fixes for things but also some canaries for finding out what is happening on GHA
* [testing] More canaries to pull files down from GHA
* That was a bad idea
* Fix implementation bug in Corefile.exe
Basically we returned the segment that contained the entry point,
but need to return the first segment which shares the same filename
We also no longer handle the case where the mapping is anonymous
and set the name to AT_EXECFN. This probably breaks some QEMU stuff.
* Yep, the code used AT_EXECFN for the QEMU case
For an ARM corefile generated by qemu-user, the mappings are:
In [10]: print(core.maps)
10000000-10001000 r-xp 1000
ff7ef000-ff7f0000 ---p 1000
ff7f0000-ffff0000 rw-p 800000 [stack]
So we need to pull the name out of AT_EXECFN
* [core] Eagerly fetch self.exe so that the Mapping always has the right name
This means that Corefile.maps and Corefile.mappings will be updated for the single mapping
that contains the AT_ENTRY
* Attempt to debug getenv failure
* Fix some temporary edits and broken tests
* [ssh] Lets try and see if this works
* Remove unnecessary test and disable diagnostic logging
* [misc] Refactor kill_at_exit code for simplicity
Also catch the OSError from when the process has already died, which
was showing up at the end of CI runs
* Remove debug print
* Hail mary debugging attempt
* Try workaround that manually sets SHELL, since it may not be set on GHA
* [corefile] Remove debugging code, and consume setdefault result
* [corefile] Hide the bash-static fuckery in the RST file setup
* Update docs/source/elf/corefile.rst
Co-authored-by: Arusekk <[email protected]>
* Update pwnlib/elf/corefile.py
Co-authored-by: Arusekk <[email protected]>
Co-authored-by: Arusekk <[email protected]>
This allows piping with color into tools like ansifilter, which take
ANSI-escape-coded colorized text and emits data in other formats,
notably HTML.
This is useful for writing blog posts with PWNLIB_DEBUG=1 to display
hex-dumped data written to stdout that would not normally be colorized.
Currently this ONLY works for the environment variable PWNLIB_COLOR,
and does not make use of pwnlib.args due to cyclic imports.
* Remove likely-unused Dockerfiles, which are not being supported in the future anyway
This leaves base, stable, beta, and dev as the only Dockerfile images
Simplify extra/Docker so that Pwntools is already checked out, and build precedence works
* Base image always gets built first
* Branch-tracking Dockerfiles now require base to be built
* Use locally-checked-out-repo instead of e.g. git+https://github.com/Gallopsled/pwntools@beta
* Install tested with both Python2 and Python3
* Add Python3 user-script directory to $PATH
Update CHANGELOG with Docker changes
Add develop to Makefile, restore old Dockerfiles
Make development target separate, and automatically launch "docker run -it" with suggested arguments
Add develop Dockerfile
* Add README for the Dockerfiles
* Merge dependencies, add NOPASSWD, use hostname localhost to avoid sudo errors
* Better Dockerfile for testing, working on making all of the tests work
* wip
* WORK-IN-PROGRESS
* Remove from pwn import *
* Fix incorrect usage of progress() in SSH
* WIPWIPWIP
* MORE WIP
* [srop] Fix SROP unit tests by allocating enough stack
* [todo] Test canary in MIPS pushstr to make sure tests are actually being run
* [ROP] Make ROP doctests work from Docker and more minimal
* [srop] Simplify SROP imports
* [mips] Simplify MIPS doctest imports
* [corefile] Handle race condition against kernel / qemu
* [web] Handle file permission failure in Docker
* [core] Actually implement timeout
* Revert "[corefile] Handle race condition against kernel / qemu"
This reverts commit e02a3dce19.
* [context] Handle file permission issue in Docker
* [aarch64] Just run the shellcode, dont dump it out
* [tests] Remove test canary from mips pushstr
* [timeout] Use ELLIPSIS for time-flaky tests
* [update] Remove DONT_CHECK_FOR_UPDATES
* [context] Handle another path access error in Docker
* [timeout] Just leave it the way it was, hardly ever fails
* [context] Fix broken test because of quotes
* WIP save Docker stuff just in case
* [mips] Do not replace "asm" in the global scope with doctest
* Clear context between invocations so we dont end up with e.g. 64-bit i386
* [process] Fix timeout to actually work
* [mips] Do not replace "asm" in the global scope with doctest (Part 2)
* [rop] Fix ROP doctests
* [mips] Do not replace "asm" in the global scope with doctest (Part 3)
* Re-add all of the Py2OutputChecker stuff so I can start testing Python2
* Restore libcdb checks
* Tests work on Python2 now with "make -C travis/docker doctest2"
* Handle issue caught by Pylint
* Install pwndbg for the pwntools user
* Add trailing newline
* Remove un-necessary ELLIPSIS
* Add trailing newline
* Convert assert to simple statement
* This is an empty commit for testing webhook integrations
* [travis] Use bash as the default shell
* Commit compiled Dockerfile for use on hub.docker.com
* [gdb] Restore gdb.rst and manually disable doctests from inside Docker container
* Note newer Ubuntu versions
* State preference for Python3
* Provide working instructions for Python2 installation due to pip deprecation
* Provide information on command-line tools not being in $PATH
* Fix test to use ellipsis, since which() may not return /bin/sh
* Fix another case where /bin/true may not be used given $PATH
* Fix another absolute path failure in doctests
* Fix another issue where /usr/bin may intercede
Updated or alternative versions of binutils cannot be relied upon to create
very small versions of ELFs (e.g. when running on macos).
However, we can ensure that the non-symbolized version is SMALLER.
* Add baremetal os
This allows to run and debug a raw binary, e.g. an MBR, with
the help of qemu-system-$(arch).
Tested with:
pwn.context.arch = "i386"
pwn.context.os = "baremetal"
pwn.gdb.debug(["./mbr.bin"])
* Add changelog entry for 'baremetal' os
* Add PATH argument to which function
* [changelog] Add info on the path argument for which
* Move comment back to original location
* Sort CHANGELOG.md
Co-authored-by: Arusekk <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* run_in_new_terminal: Prefer tmux/screen over TERM_PROGRAM and x-terminal-emulator.
* run_in_new_terminal: Kill running commands when the main process exits.
Currently, when a gdb window is opened via gdb.debug(), terminating the
main pwntools process leaves the gdb window/process around. It is
inconvenient to have to close these leftover gdb windows when iterating
on an exploit.
Add a kill_at_exit parameter to run_in_new_terminal() which attempts to
terminate the command at main process exit.
This only works for terminals which do not daemonize, or for tmux, which
supports returning the command's PID.
This change also removes the special case for closing
stdin/stdout/stderr on Mac OS X, as the problem no longer reproduces
under tmux 2.8.
* Fix regression and add CHANGELOG
Co-authored-by: Arusekk <[email protected]>
* Don't introduce null bytes in MIPS shellcode.
As a verification, you can run:
```sh
pwn shellcraft mips.linux.sh |pwn phd
```
Closes#807
* Add a test
* Fix doctest under py2
* Try libc.rip for libc lookup
As of #1171 the libcdb repository isn't seeing any updates anymore. When looking up a libc by a hash or build_id try the libc database maintained by @niklasb as well.
The database uses a REST api documented at https://github.com/niklasb/libc-database/blob/master/searchengine/api.yml
* Catch exceptions properly
* Update CHANGELOG
* Add doctests
Co-authored-by: Arusekk <[email protected]>
* [elf.py] convert linux string banner into an actual str
``self.string(address)`` actually return a bytes object containing the raw bytes, not a string in a pythonesque sense !
Since quite some time on Python3, ``re.search`` only works on proper str so we need to convert it before feeding it to ``re.search``
``.decode()`` works both on Python2 and Python3
* Update kernel ELF handling
Co-authored-by: Arusekk <[email protected]>
* Implement property getter/setter for context.newline, and make tubes honor it
The default newline can now be set context-wide, or per-tube.
* [changelog] Update for 1541
* [docs] Use r-prefix for embedded newlines
Co-authored-by: Arusekk <[email protected]>
* use non-greedy operator to handle corner cases where the program output contains END in it
* END can appear in the string leak
Co-authored-by: Arusekk <[email protected]>
Co-authored-by: Arusekk <[email protected]>
* Keep ROP cache in user's home like regular cache
I don't have statistics numbers on which would be more likely to be writable, an user's home directory or the OS's temp directory, but seems to me it'd make sense to try and write all caches to the same location.
Optimally this would be easily configurable to allow adjusting to different set-ups, obviously.
* Use cache_dir instead
Co-authored-by: Arusekk <[email protected]>
* fix --multibr issue
* modify changelong
* In which I learn how to count
* put doctest for ROP().syscall in unfortunate place
* fix context malapropism
* fix ROP gadget loading when binary contains a gadget with add rsp, <register>
* add documentation regarding the fact that loading gadget "add $sp, <reg>" will fail
All three instructions: `int 0x80`, `sysenter`, and `syscall` require
different stack frames and registers to pass arguments, but
`ROP._srop_call` uses them interchangeably.
This fix limits syscall instruction lists to gadgets that work.
Co-authored-by: Arusekk <[email protected]>
* Change lookup algorithm of `adb.which`.
This is done because the current algorithm does not work when the binary (e.g. `su`) is installed in the last element of $PATH.
Example:
```shell
$ echo $PATH
/sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin
$ echo $PATH | while read -r -d: directory; do
> echo "$directory/su";
> done
/sbin/su
/vendor/bin/su
/system/sbin/su
/system/bin/su
$ (IFS=:
> for directory in $PATH; do
> echo "$directory/su";
> done
> )
/sbin/su
/vendor/bin/su
/system/sbin/su
/system/bin/su
/system/xbin/su
```
* Update CHANGELOG
Turns out that python 2 does not raise correct socket.timeout, but
SSLError instead. With no errno. Just a message that might be
localized in some circumstances.
**THIS IS NOT THE ONLY TIME PYTHON 2 IS CAUSING PROBLEMS.**
Time to stop using it, dear exploit developers. It is dead now.
But here you are: the simple fix.
Closes#1356
* [docker] Upgrade Docker images of Pwntools to use Python3
Also installs qemu-user-static and various binutils packages for building (binutils)
and running (qemu) cross-arch binaries.
Fixes#1500
* Update CHANGELOG
* Switch to Python3 for stable/beta/dev Docker images
* [changelog] Add missing links
* [shellcraft] Avoid recursive walk of all templates for command line
Fixes#1650
* Avoid double-fetch
* Add changelog
* [pylint] Only warn for additions
* tubes/remote: Support Server Name Indication and enable by default
This allows customizing the server_name via sni="custom" or sending the
name specified in the host parameter via sni=True.
We switch to the SSLContext.wrap_socket API because ssl.wrap_socket is
deprecated and does not support SNI.
Unfortunately, SSLContext.wrap_socket dropped some of the parameters
that are supported by ssl.wrap_socket. These parameters have been moved
to the SSLContext API and can now be used by passing a custom SSLContext
to the ssl_context parameter of `remote`.
* update CHANGELOG.md
* Enhance gdb error message when unable to launch
A common issue is when the binary is not executable,
or the binary is simply missing.
* Use six.ensure_str
* [tests] Change test slightly so that failing output will be emitted
* Disable flaky UI test on Github Actions
* [ui] Try a sleep rather than disabling tests
* Add a bfdname for msp430 to make asm and disasm work
* Add a bfdname for s390 to make asm and disasm work for all architectures
* Add tests for new bfdnames
* ci: Add binutils for newly-tested architectures
Co-authored-by: Arusekk <[email protected]>
* [packing] Support negative indices for flat() and fit()
Fixes#1531
* [changelog] Add Changelog for PR
* [packing] Only take negative path if there are any
min([]) is not valid Python
* [packing] Move out_negative to a higher scope
* [stable] Fix incorrectly-merged code
* [aarch64] Use byte-strings instead of strings
* [aarch64] Have different tests for str vs bytes
* [aarch64] Convert all items to binary before processing
* [aarch64] Fix doctest results
* [aarch64] Stopgap to fix Py3 CI tests
* [aarch64] Remove failing test, will address in a future release
* [aarch64] Fix aarch64.pushstr and pushstr_array
* pushstr
* We were pushing the correct values, and each 16-byte span was correct, but the 16-byte segments were ordered wrong
* pushstr_array:
* Was entirely broken. This commit has a non-optmimized version but it works
Closes#1549
* [packing] Better alternative to #1580
pack() was failing for dictionaries that are not ordered
Closes#1582Closes#1579
* [docs] Add better documentation for flat()
* [changelog] Add 4.1.4
* [release] Bump version to 4.1.4
Co-authored-by: Arusekk <[email protected]>
* [docker] Get Travis CI dockerfile working more
* [docker] Do not upgrade all packages, only update if required
* [docker] Move requirements installation higher up before the copy
* Fix several sphinx warnings
* Several code style fixes
Mostly grep -Er '[=!]= (None|True|False)'
* Compile-time str concat
also reduced redundant characters at line ends
* Fix python2 toplevel regression
* No equality comparison with singletons again
* Clarification: lambda: [] is just list
* Reintroduce reverted fragment
* sock-based GDB connections didn't set the PID, so it didn't work
* Now uses a real Timeout object
* Added special case handling for socat, since we were attaching to the wrong process
* ssh-based doctests improved
* Works locally on everything I tested
* One test is ignored, since it doesnt work on Travis CI
* Added libc6-dbg to Github Actions
* run_in_new_terminal supports unicode strings now
* This fixes SSH-based debugging
* wait_for_debugger now has a timeout
* Fixes the case where the debugger never comes up, mostly for tests
If passed an empty dictionary defining the environment variables, the process invoked by gdb.debug() has environment variable 'PWD' set to the directory containing the executable. This behaviour is not observered when NOPTRACE is enabled: the program runs without any environment variables set. Similarly, if an empty environment variable dictionary is passed to process(), it appears to also run under a truly empty environment. Additionally, attempts to overwrite the "PWD" environment variable with a dictionary entry are not successful under gdb.debug().
This fix brings the gdb.debug() treatment of environment variables in line with and process() and gdb.debug() with NOPTRACE enabled.
It allows both empty environments, and overwriting of the 'PWD' environment variable.
Both Debian and Ubuntu provide a `binutils-avr` package, which contains
an assembler for AVR as `avr-as`. However, pwntools was looking for
something with `linux` in the name, such as `avr-linux-gnu-as`. Of course,
such a target triplet is nonsense, Linux does not run on AVRs
(apart from that one hack that actually emulated some other CPU which
therefore doesn't count).
* Disable corefile tests for Github Actions
* Disable the UI tests, as theyre broken
* Install gdbserver
* Rename step
* Remove SIGALARM to see if it fixes tests
* Set PWNLIB_NOTERM for command-line runs
`ELF.functions` wasn't updated after the binary base address was changed using `ELF.address` like `ELF.symbols` is. The documentation mentions `ELF.functions` getting updated as well, the update was missing though.
Closes#1512
Co-authored-by: Arusekk <[email protected]>
* Update Corefile warnings and replace asserts with normal checks
This means that most of the Corefile will still work, even if
we encounter issues while parsing data from the stack.
* Update pwnlib/elf/corefile.py
* Add missing return statement
Co-authored-by: Arusekk <[email protected]>
Detect if pwntools is running in a Windows Subsystem for Linux "VM" and set the terminal to open a new `cmd.exe` window on the host.
This replaces the need to add `context.terminal = ['cmd.exe', '/c', 'start', 'ubuntu', 'run']` to every script.
Debugging using `gdb.attach` under WSL with gdb in a new window out of the box!
This is a WONTFIX, but default gcc on ubuntu compiles .bss (and other RW
segments) so far away from .text that ret2dlresolve mathematically
just has to segfault unless .text and .rodata contain crafted structures
already.
Compiling and working fine on gentoo, though. Hope this proves useful.
Also closes#1448 by the way, it was really annoying.
Closes#1476.
* Enable text colors in Windows console
Consoles on Windows 10 support ANSI escape sequences including color codes.
Emulate unix curses `tigetstr` terminal capabilities by hardcoding the given escape sequences for used curses capabilities.
ANSI escape sequences have to be enabled using `kernel32.SetConsoleMode(hndl, ENABLE_VIRTUAL_TERMINAL_PROCESSING)` for the current console.
* Use object identity operator
* Import ABC from collections.abc for Python 3.
* Fix deprecation warnings due to invalid escape sequences.
* Use raw strings instead of escaping individually.
Previously, the rop._srop_call function would append the args without checking whether their value would work
directly in a register. It now uses AppendedArgument to determine whether the value is pushed to the end of
the ROP chain.
* Update proc.py
reversed([p.pid for p in processes]) only return a iter.
if use gdb.attach('some process name') will cause error like :
File "/home/tsj/.local/lib/python2.7/site-packages/pwnlib/gdb.py", line 657, in attach
pid = pids[0]
TypeError: 'listreverseiterator' object has no attribute '__getitem__'
In this case ,it treat 'pids' as a list,so here should return a list as well
* Apply suggestions from code review
Co-authored-by: Arusekk <[email protected]>
if the value returned from the leak function is a str, it converts it to a bytearray for pytohn3 compatibility. Assuming a string is returned is flaky and likely wrong.
removed the double quotes around the temp filename of the gdbscript as this broke the gdbscript functionality.
The doublequotes would be considered part of the filename and it would fail to read the file..
Tested this fix on Kali.
* The SSH tube now keeps the environment variables of processes it creates in order.
The way of achieving that is through passing the env to the script as a list of tuples,
since Python3's dictionaries are ordered by default, or you can use `OrderedDict`,
the result would be that the spawned process on the remote machine will keep the environment variables in order.
This is very important to exploits that are based on the stack's order and especially the environment variables.
For instance, a common challenge is comparing argc == 0, and afterwards accessing the argv array, as an out-of-bounds array.
The shellcode needs to be at a certain index consistently and cannot rely on the stack moving around.
* Relocated the code.
* Try GitHub actions instead of Travis CI.
This is just a proof-of-concept until we can
have it the clean way it should be done.
More commits will likely follow.
* Travis CI: Use flake8 to detect syntax errors (#1385)
* Travis CI: Use flake8 to detect syntax errors
Repeat #1305 on the Python 3 branch. Also, remove the hardcoding of default values for dist and Py2.
https://flake8.pycqa.org/en/latest/user/error-codes.html
On the flake8 test selection, this PR does _not_ focus on "_style violations_" (the majority of flake8 error codes that [__psf/black__](https://github.com/psf/black) can autocorrect). Instead these tests are focus on runtime safety and correctness:
* E9 tests are about Python syntax errors usually raised because flake8 can not build an Abstract Syntax Tree (AST). Often these issues are a sign of unused code or code that has not been ported to Python 3. These would be compile-time errors in a compiled language but in a dynamic language like Python they result in the script halting/crashing on the user.
* F63 tests are usually about the confusion between identity and equality in Python. Use ==/!= to compare str, bytes, and int literals is the classic case. These are areas where __a == b__ is True but __a is b__ is False (or vice versa). Python >= 3.8 will raise SyntaxWarnings on these instances.
* F7 tests logic errors and syntax errors in type hints
* F82 tests are almost always _undefined names_ which are usually a sign of a typo, missing imports, or code that has not been ported to Python 3. These also would be compile-time errors in a compiled language but in Python a __NameError__ is raised which will halt/crash the script on the user.
* Fail fast, exclude generated source files
* --select=E9,F63,F7
* 2to3 -f print -wn examples
* No more <> in python 3.
* Use ==/!= to compare str, bytes, and int literals (#1386)
Identity is not the same thing as equality in Python so use ==/!= to compare str, bytes, and int literals. In Python >= 3.8, these instances will raise _SyntaxWarnings_ so it is best to fix them now. https://docs.python.org/3.8/whatsnew/3.8.html#porting-to-python-3-8
$ python
```python
>>> raw = 'ra'
>>> raw += 'w'
>>> raw == 'raw'
True
>>> raw is 'raw'
False
```
* Use print() function in both Python 2 and Python 3 (#1387)
$ __pip install black future isort__
$ __futurize -f libfuturize.fixes.fix_print_with_import -w android.py__
$ __isort android.py__
$ __black android.py__
* Add cache and building artifacts
* Enable caching
* Actually enable caching for all of it
* Mistake
* Try different ABI on CI
* Try different ABI + version
* Try it harder
* Update SDK version
* Try building using newer SDK
* Accept licenses
* Ugh android emulator documentation is unfriendly
* Environment variables for AVD
* Add QEMU-system for android virtual devices
* Use the correct emulator
* New android image
* Was actually too new image
* Update ssh credentials to reflect GHA setup
* Fix spelling
* Try finding endless loops
* Actually use the timeout
* Reduce possible hang surface in doctests
* Try SSH immediately when set up
* SSH strict host key checking disable for localhost
* Try stackoverflow.com research
* Try to fix failing doctests
* Add required gcc for cross-arch tests
* Ensure gcc not failing in fmtstr
* Fix issues with shellcraft
* Try different libs
* Try again, with verbosity
* Why do these conflict???
* Fix issues related to PIE binaries
* Add bash-static
* Verbose 'could not set transport: any'
* Add AT_SECURE in auxv to prevent SEGV on libc bug
* Killing ADB server actually kills it
* Android image has /data/user/0 -> /data/data
* Set coverage mode from dev
* Try new actions
* Fix coveralls complaining
* A little time.sleep would not hurt or would it
Co-authored-by: Christian Clauss <[email protected]>
* Allow pwntools to load on Windows
Fixes loading of pwntools on Windows. `from pwn import *` works.
It allows to use some operating system agnostic features of pwntools out of the box.
This is inspired by #996.
Some features fail or even crash (local processes, gdb, assemble/disassemble)
and console colors are still not displayed correctly, but those can be addressed separately.
* Use `os.devnull` instead of manual selection
* Add heuristic to find return address into __libc_start_main
Find the return address from main into __libc_start_main in a libc shared binary. Mimic's niklasb's heuristic in libc-database.
* Add support for different `call` instructions
* Introduce tests for pwnlib.ui
This architecture can be changed later, but I'm quite proud
of how it's done currently.
* Try to debug CI
* Ugly hack to get results back
* Let's just keep it half-broken
Currently, python 3 installed libraries
and sphinx doctree cache are not retained across builds.
This PR addresses this issue, hopefully resulting in a major speedup.
* Travis CI: Use flake8 to detect syntax errors
Repeat #1305 on the Python 3 branch. Also, remove the hardcoding of default values for dist and Py2.
https://flake8.pycqa.org/en/latest/user/error-codes.html
On the flake8 test selection, this PR does _not_ focus on "_style violations_" (the majority of flake8 error codes that [__psf/black__](https://github.com/psf/black) can autocorrect). Instead these tests are focus on runtime safety and correctness:
* E9 tests are about Python syntax errors usually raised because flake8 can not build an Abstract Syntax Tree (AST). Often these issues are a sign of unused code or code that has not been ported to Python 3. These would be compile-time errors in a compiled language but in a dynamic language like Python they result in the script halting/crashing on the user.
* F63 tests are usually about the confusion between identity and equality in Python. Use ==/!= to compare str, bytes, and int literals is the classic case. These are areas where __a == b__ is True but __a is b__ is False (or vice versa). Python >= 3.8 will raise SyntaxWarnings on these instances.
* F7 tests logic errors and syntax errors in type hints
* F82 tests are almost always _undefined names_ which are usually a sign of a typo, missing imports, or code that has not been ported to Python 3. These also would be compile-time errors in a compiled language but in Python a __NameError__ is raised which will halt/crash the script on the user.
* Fail fast, exclude generated source files
* --select=E9,F63,F7
* 2to3 -f print -wn examples
* No more <> in python 3.
* Adding support for FileStructure exploitation
* Actually run the tests + basic py3 fixes
also rename the module to lower case to keep consistency
* Python 3 working
Identity is not the same thing as equality in Python so use ==/!= to compare str, bytes, and int literals. In Python >= 3.8, these instances will raise _SyntaxWarnings_ so it is best to fix them now. https://docs.python.org/3.8/whatsnew/3.8.html#porting-to-python-3-8
$ python
```python
>>> raw = 'ra'
>>> raw += 'w'
>>> raw == 'raw'
True
>>> raw is 'raw'
False
```
- Detect bootloaders that are already unklocked
- Detect bootloader unlock success
- Detect bootloaders that cannot be unlocked
- Detect devices which have not enabled bootloader unlocking
2017-07-10 15:54:22 -05:00
1080 changed files with 75002 additions and 14739 deletions
When reporting an issue, be sure that you are running the latest released version of pwntools (`pip install --upgrade pwntools`).
## Debug Output
Having the extra debug output really helps us, and might help you diagnose the problem yourself.
When submitting an issue that has output from Pwntools, make sure to run your script as shown below, to enable the extra debugging data.
```sh
$ python exploit.py DEBUG LOG_FILE=log.txt
```
You should see `[DEBUG]` statements that show what's happening behind the scenes:
```
[+] Starting local process '/bin/sh' argv=['sh'] : pid 16823
[DEBUG] Sent 0x7 bytes:
'whoami\n'
[DEBUG] Sent 0x5 bytes:
'exit\n'
[+] Receiving all data: Done (11B)
[DEBUG] Received 0xb bytes:
'crashheap\n'
[*] Process '/bin/sh' stopped with exit code 0 (pid 16823)
```
## Verify on Ubuntu
If possible, please verify that your issue occurs on 64-bit Ubuntu 24.04. We provide a Dockerfile based on Ubuntu 24.04 via `docker.io` to make this super simple, no VM required!
Thanks for using Pwntools! If you have a question, here are some tips that will make it easier for us to help you.
## Check the Documentation
Pwntools has REALLY extensive documentation. Please check there first! https://docs.pwntools.com
## Debug Output
If you're having an issue with an exploit, run your script with full debugging output, i.e. `python exploit.py DEBUG`. This may help you to see what's happening under the hood, and the debug output will help to diagnose any issues you're encountering.
In memoriam — [Zach Riggle][zach] — long time contributor and maintainer of Pwntools.
- [#2062][2062] make pwn cyclic -l work with entry larger than 4 bytes
- [#2092][2092] shellcraft: dup() is now called dupio() consistently across all supported arches
- [#2093][2093] setresuid() in shellcraft uses current euid by default
- [#2125][2125] Allow tube.recvregex to return capture groups
- [#2144][2144] Removes `p2align 2``asm()` headers from `x86-32`, `x86-64` and `mips` architectures to avoid inconsistent instruction length when patching binaries
- [1242][1242] Use IntervalTree 2.xx, disallow use of 3.xx
- [1243][1243] Fix a typo that caused an exception when executing a binary with `process()` which returns `-ENOEXEC` and the system does not have `qemu-user` binaries installed.
- [#1242][1242] Use IntervalTree 2.xx, disallow use of 3.xx
- [#1243][1243] Fix a typo that caused an exception when executing a binary with `process()` which returns `-ENOEXEC` and the system does not have `qemu-user` binaries installed.
- [Contributing to a project](https://docs.github.com/en/get-started/exploring-projects-on-github/contributing-to-a-project)
- [Fork the repository](https://docs.github.com/en/get-started/exploring-projects-on-github/contributing-to-a-project#creating-your-own-copy-of-a-project)
- [Clone your fork](https://docs.github.com/en/get-started/exploring-projects-on-github/contributing-to-a-project#cloning-a-fork-to-your-computer)
- [Creating a branch to work on](https://docs.github.com/en/get-started/exploring-projects-on-github/contributing-to-a-project#creating-a-branch-to-work-on)
- [Making and pushing changes](https://docs.github.com/en/get-started/exploring-projects-on-github/contributing-to-a-project#making-and-pushing-changes)
- [Making a Pull Request](https://docs.github.com/en/get-started/exploring-projects-on-github/contributing-to-a-project#making-a-pull-request)
## pwntools Specifics
In general, we like to keep things documented. You should add documentation to any new functionality, and update it for any changed functionality. Our docstrings use the [Google Style Python Docstrings](https://sphinxcontrib-napoleon.readthedocs.org/en/latest/example_google.html#example-google).
After you have documentation, you should add a [doctest](https://docs.python.org/2/library/doctest.html).
After you have documentation, you should add a [doctest](https://docs.python.org/3/library/doctest.html).
Finally, it is probably a good idea to run the test suite locally before doing
the pull-request to make sure everything works, however this is not a
requirement.
requirement. See the [TESTING](TESTING.md) documentation on how to do that.
Once you are ready to do a pull-request, you should figure out if your changes
constitutes a new feature or a bugfix in stable or beta. If it is a bugfix in
stable or beta, you should do the pull-request against the branch in question,
and otherwise your pull-request should be against the dev branch.
Once you do the pull-request Travis CI will run the test-suite on it. Once it
Once you do the pull-request Github Actions will run the test-suite on it. Once it
passes one of the core developers will look at your pull request, possibly
comment on it and then hopefully merge it into the branch in question.
## Python Type Hints
Since Pwntools 5 dropped support for Python 2, type hints are added gradually to the existing code base.
[Type hints in Python](https://typing.python.org/en/latest/spec/index.html) allow static analysis tools and IDEs to help catch errors and provide a better developer
experience without running the code.
New code should include type hints where applicable and shouldn't add new type warnings.
You can use [this cheat sheet](https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html) as a reference for the type hint syntax.
Pwntools uses [mypy](https://mypy.readthedocs.io/en/stable/index.html) as a static type checker. Instead of trying to
introduce type hints everywhere at once, Pwntools uses [mypy-baseline](https://mypy-baseline.orsinium.dev/usage) to gradually reduce the number of problems over time.
To run the type checker locally, you can install Pwntools with the `dev` extra and check for new errors (`--group dev` requires pip 25.1+):
```shell
$ pip install -U . --group dev
$ mypy | mypy-baseline filter
```
If you fixed some problems, you can update the list of known errors in `mypy-baseline.txt` using
```shell
$ mypy | mypy-baseline sync --sort-baseline
```
## Automated Testing
Pull requests against Pwntools require at a minimum that no tests have been broken, and ideally each pull request will include new tests to ensure that all of the functionality works as intended.
Sometimes it's annoying to set up Pwntools on your workstation, and you want something that Just Works (TM).
[Docker](https://www.docker.com/) is here to the rescue! Using Docker means that you get a nice, standardized Linux environment and don't need to worry about pip or installing dependencies.
## Quick Start
First, install Docker for your OS, which you can find on their [Getting Started](https://www.docker.com/get-started) page.
Next, download and run the Pwntools stable docker image.
```sh
$ docker run -it pwntools/pwntools:stable
```
## Recommended Settings
In order to get the most from your docker image, we need to enable debugging of processes (`--privileged`) and expose the network ports from the guest to the host (`--net=host`).
```sh
$ docker run -it \
--privileged \
--net=host \
--hostname localhost \
--ulimit core=-1:-1 \
pwntools/pwntools:stable
```
## Sharing a Folder
It's really nice to be able to use your preferred native editor, and have the changes show up live inside your Docker image. This is easy to add, thanks to Docker's bind mounts (`--mount type=bind`).
With the command below, your `~/exploits` directory will magically show up inside the Docker image at `/home/pwntools/exploits` so that you can easily run them (from Docker) and edit them (from outside Docker).
If you're a Windows user `$HOME` doesn't exist in the same way as on Linux, instead it is `%UserProfile%`. The command from above would look like this, assuming your editor is Visual Studio Code and you have code.exe in your `%PATH%`.
[](https://github.com/Gallopsled/pwntools/actions/workflows/ci.yml?query=branch%3Adev)
Pwntools is a CTF framework and exploit development library. Written in Python, it is designed for rapid prototyping and development, and intended to make exploit writing as simple as possible.
@@ -20,29 +22,28 @@ r.send(asm(shellcraft.sh()))
r.interactive()
```
# Try It Now!
You can now do a live demo of Pwntools, [right in your browser](https://demo.pwntools.com).
# Documentation
Our documentation is available at [docs.pwntools.com](https://docs.pwntools.com/)
A series of tutorials is also [available online](https://github.com/Gallopsled/pwntools-tutorial#readme)
To get you started, we've provided some example solutions for past CTF challenges in our [write-ups repository](https://github.com/Gallopsled/pwntools-write-ups).
# Installation
Pwntools is best supported on 64-bit Ubuntu LTE releases (12.04, 14.04, 16.04 and 18.04). Most functionality should work on any Posix-like distribution (Debian, Arch, FreeBSD, OSX, etc.). Python >= 2.7 is required (Python 3 suggested as best).
Pwntools is best supported on 64-bit Ubuntu LTS releases (22.04 and 24.04). Most functionality should work on any Posix-like distribution (Debian, Arch, FreeBSD, OSX, etc.).
Most of the functionality of pwntools is self-contained and Python-only. You should be able to get running quickly with
Pwntools supports Python 3.10+ since version 5.0.0. Use Pwntools 4.x for older versions as well as Python 2.7. Most of the functionality of pwntools is self-contained and Python-only. You should be able to get running quickly with
However, some of the features (assembling/disassembling foreign architectures) require non-Python dependencies. For more information, see the [complete installation instructions here](https://docs.pwntools.com/en/stable/install.html).
@@ -50,8 +51,5 @@ However, some of the features (assembling/disassembling foreign architectures) r
See [CONTRIBUTING.md](CONTRIBUTING.md)
# Contact
If you have any questions not worthy of a [bug report](https://github.com/Gallopsled/pwntools/issues), feel free to ping us
at [`#pwntools` on Freenode](irc://irc.freenode.net/pwntools) and ask away.
Click [here](https://kiwiirc.com/client/irc.freenode.net/pwntools) to connect.
There is also a [mailing list](https://groups.google.com/forum/#!forum/pwntools-users) for higher latency discussion.
# Contact and Community
If you have any questions not worthy of a [bug report](https://github.com/Gallopsled/pwntools/issues), join the Discord server at https://discord.gg/96VA2zvjCB
The aim of pwntools is exploiting software vulnerabilities, which is an unusual position, but it nevertheless can have its own security issues.
Especially that an attacker (=re-victim) is usually not prepared to be attacked back (by the re-attacker).
The first question to ask yourself is: is this an actual vulnerability?
- can it be triggered by a re-attacker (malicious honeypot pretending to be a vulnerable service)?
- does it impact the attacker (=re-victim)?
- is it serious?
* *availability: medium* means *at least* exhausting RAM or disk space of the attacker (=re-victim)
* *confidentiality: medium* means *at least* reading the filesystem of the attacker (=re-victim)
* *integrity: medium* means *at least* performing uncontrolled actions or data corruption on behalf of the attacker (=re-victim)
* if crucial for some sophisticated exploit chain, it is always serious
*`safe_eval` bypasses **are** serious.
* an example of what was **kind of** serious: [#1732](https://github.com/Gallopsled/pwntools/pull/1732)
- can it be fixed without compromising on Pwntools' usability?
If at least one of the answers is no, then this is NOT a vulnerability, so just file a bug report or feature request, without the weird confidential disclosure dance.
Just e-mail the maintainers. Arusekk is the one that is currently the most excited to fix vulnerabilities.
Or create a CTF task! Prove a point the good old hacker way!
@@ -4,18 +4,25 @@ Pwntools makes extensive use of unit tests and integration tests to ensure every
## Test Suite
To run the test suite, it is best to use Ubuntu 12.04 or 14.04, and run the following commands. **Be aware** that this will add a user to the machine, and create a public key for SSH login!
To run the test suite, it is best to use Ubuntu 22.04 or 24.04, and run the following commands. **Be aware** that this will add a user to the machine, and create a public key for SSH login!
```sh
bash travis/install.sh
bash travis/ssh_setup.sh
pip install --upgrade --editable .
pip install --upgrade -r docs/requirements.txt
PWNLIB_NOTERM=1 make -C docs doctest
```
You can run doctests for a single file for faster iteration:
A `Dockerfile` has been provided which has a clean testing environment with Ubuntu Xenial. It is very similar to the online Travis CI testing environment, but uses a more modern version of Ubuntu.
A `Dockerfile` has been provided which has a clean testing environment with Ubuntu Noble. It is very similar to the online Github Actions CI testing environment.
See `travis/docker/README.md` for more information.
pwntools is best supported on Ubuntu 12.04 and 14.04, but most
functionality should work on any Posix-like distribution (Debian, Arch,
FreeBSD, OSX, etc.).
Pwntools is best supported on 64-bit Ubuntu LTS releases (22.04 and 24.04). Most functionality should work on any Posix-like distribution (Debian, Arch, FreeBSD, OSX, etc.).
Prerequisites
-------------
@@ -17,17 +15,60 @@ following system libraries installed.
install/*
Released Version
-----------------
pwntools is available as a ``pip`` package.
Note: For Mac OS X you will need to have cmake ``brew install cmake`` and pkg-config ``brew install pkg-config`` installed.
pwntools is available as a ``pip`` package for Python3. Version v5.0.0 supports Python3.10 or later. Use v4 if you use earlier versions of Python.
Python3
^^^^^^^
The Python version required for installing Pwntools is kept as low as possible on a best-effort basis. However, new features target Python3.10 and later.
@@ -5,41 +5,56 @@ Assembly of foreign architectures (e.g. assembling Sparc shellcode on
Mac OS X) requires cross-compiled versions of ``binutils`` to be
installed. We've made this process as smooth as we can.
In these examples, replace ``$ARCH`` with your target architecture (e.g., arm, mips64, vax, etc.).
In these examples, replace ``$ARCH`` with your target architecture (e.g., arm, aarch64, mips64, vax, etc.).
Building `binutils` from source takes about 60 seconds on a modern 8-core machine.
Ubuntu
^^^^^^^^^^^^^^^^
For Ubuntu 12.04 through 15.10, you must first add the pwntools `Personal Package Archive repository <http://binutils.pwntools.com>`__.
For Ubuntu 12.04 through 15.10, you must first add the pwntools `Personal Package Archive repository <https://launchpad.net/~pwntools/+archive/ubuntu/binutils>`__.
Ubuntu Xenial (16.04) has official packages for most architectures, and does not require this step.
..code-block::bash
$ apt-get install software-properties-common
$ apt-add-repository ppa:pwntools/binutils
$ apt-get update
$ sudo apt-get install software-properties-common
$ sudo apt-add-repository ppa:pwntools/binutils
$ sudo apt-get update
Then, install the binutils for your architecture.
..code-block::bash
$ apt-get install binutils-$ARCH-linux-gnu
$ sudo apt-get install binutils-$ARCH-linux-gnu
Mac OS X
^^^^^^^^^^^^^^^^
Mac OS X is just as easy, but requires building binutils from source.
However, we've made ``homebrew`` recipes to make this a single command.
After installing `brew <http://brew.sh>`__, grab the appropriate
However, we've made ``homebrew`` recipes to make this just two commands.
After installing `brew <https://brew.sh>`__, grab the appropriate
setuid=False,# Disable setuid bit so processes can be debugged
shell=False)# Enable or disable shell evaluation
print(io.recvall())
# Show off the interactive shell
shell.interactive()
Loaded 100 of 1080 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Yahya Kayaal