DRload's `-imagelist` handling opens the list file with `fopen()` but
never
closes it after reading through it with `fgets()`, leaking the file
handle
for the lifetime of the process.
Add the missing `fclose(f)` once the read loop completes.
Fixes: #1819
- Update `core/kernel_linux/CMakeLists.txt`:
- Include source files in `core/` as dependencies for kernel module.
- Define `LINUX_KERNEL`, `DYNAMORIO_INTERNAL`,
`DR_DO_NOT_DEFINE_MAX_MIN`
- Add `-Wframe-larger-than=4096`
- Update include headers in source files under `core/`. This includes
using wrapper headers (e.g., `stdarg_wrapper.h`) or excluding headers
that don't have an equivalent in kernel (e.g., `math.h`).
- Exclude floating point arithmetic and user space specific logic (e.g.,
signal handling, rseq) from kernel build.
- Optimize frame stack in `options.c` to make frame less than 4KB.
Issue: #8021
Shares TESTANY, TESTALL, and BUFFER_SIZE_ELEMENTS and related defines,
in dr_project_wide_defines.h, eliminating duplication in many places.
Removes ext/ext_utils.h which becomes empty after this change.
Issue: #8001
In preparation of moving the TEST* defines into
dr_project_wide_headers.h, we replace all uses of TEST with the
equivalent TESTANY. Although we avoid exposing
dr_project_wide_headers.h in DR_API headers, it is exposed in
drmemtrace's headers where TEST causes collisions with third-party code
such as googletest.
Issue: #8001
Adds a new top-level header common/dr_project_wide_defines.h that is
shared across all domains to avoid duplication.
This PR moves just the ALIGN* macros into this header. More shared
defines will be moved in further installments.
The new header is installed into the drmemtrace export dir as A) we're
already exporting utils.h which includes it and B) otherwise we have to
duplicate some defines in trace_entry.h. However, I first tried
duplicating and not including utils.h in trace_entry.h, which revealed
several missing headers in clients/drcachesim/ files. I left those
additions in this PR though they could be separated out.
The new header includes stdint.h to avoid depending on DR API integer
defines, which caused a Windows build issue which was resolved by
removing defines we had added to libdwarf.h back when we supported
VS2008 which did not have stdint.h. Those added defines are removed
here.
Issue: #8001
Updates our Github Actions runners to install clang-format-21. Updates
runsuite.cmake to prefer that version.
Reformats our files with this version.
Fixes#8008
Removes non-UTF8 characters from tools/runjob.c.
These crash clang-format-diff and are blocking PR #8009.
Also fixes non-UTF8 characters found in two other files while at it.
Issue: #8008
Updates all the "FIXME" comments in the repository to instead use "TODO"
(for those about missing implementations: "NYI" or
"ASSERT_NOT_IMPLEMENTED") or "XXX" (everything else). This matches our
style guide which discourages "FIXME" as its connotations are too
negative and too easily mis-interpreted by readers and in code audits to
imply that the current state of things is broken in some way instead of
just meaning something could be improved.
This fixes a check for a file that is expected to be present at a
/debug/ path in release builds on macOS (despite files in /debug/ paths
only being present in debug builds). `drrun` (without `-nocheck`) will
error in release builds on macOS since `lib64/debug/libdrpreload.dylib`
is not present and `is_debug` is wrongly set to false. This changes
`is_debug` to be true for this file (as it should be true for all
checked files living in the `lib64/debug/` path. The corresponding
`lib64/release/` drpreload file is already present in the list and is
already checked for release builds.
The check is here:
https://github.com/DynamoRIO/dynamorio/blob/898d3fb4291706a823d19c73ff0904ea3ba4a649/tools/drdeploy.c#L576-L587
Issue: #5383
When building for Android with -DDR_COPY_TO_DEVICE=ON we can use CTest
to run the DynamoRIO tests on an attached Android device.
Previously this was done by adding `adb shell` to the beginning of the
test command. This works fine for many test commands but doesn't
properly preserve argument quoting so test commands which have spaces
inside an argument would get parsed incorrectly leading to test
failures.
For example if we run:
adb shell "foo" "bar baz"
the quotes are interpreted by the host shell so the command that the
remote Android shell sees is:
foo bar baz
The "bar baz" argument has been effectively split into two.
An easy way to solve this is to surround the test command in single
quotes:
adb shell '"foo" "bar baz"'
so the full test command is passed to the Android shell with the
argument quotes intact:
"foo" "bar baz"
I have changed the CMake scripts to use a wrapper script instead of
invoking adb directly (run_on_android_device.sh) which just applies this
extra layer of quoting and allows us to run tests on Android more
reliably.
Issues: #2154, #1874
Made some tweaks to the CMake scripts to get the test suite building for
Android:
- Drmemtrace is not currently built for Android so we need to disable
some tests which depend on it when building for Android (rseq).
- Build linux.infloop test binary because it compiles fine for Android
and other CMake code expects the target to exist.
- Removed --dynamic-list-data because it is not needed with the latest
NDK toolchains.
- Copy `run_in_bg` tool to the Android test device because it is needed
by some tests.
It should now be possible to build for Android64 with -DBUILD_TESTS=ON
using the latest Android ndk (tested with r27c).
Issues: #2154, #1874
This patch configures the github CI to spawn 3 AArch64 precommit jobs
without sve, with sve 256 and with sve 128. This is accomplished by
a small python program which makes a call to set the vector length
and an update to the job matrix to spawn the appropriate jobs.
Enable the feature of detachment on Linux.
- I setup a temporary separate stack for the nudged thread, and resume
the normal execution of this thread after detachment.
- A new variable named `nudged_sigcxt` is used to save the `sigcxt` of
nudged thread.
- And the extra code `dcontext == GLOBAL_DCONTEXT` is added to cover the
new code path that won't be executed before during the process of thread
exit.
- What's more, I turn off the option of `sigreturn_setcontext` for the
smooth resumption of nudged thread on X64 ISA.
- Finally, the frontend, automated test cases and documentation are
modified accordingly.
Issue: [#95](https://github.com/DynamoRIO/dynamorio/issues/95)
To avoid symbol conflicts with IN, OUT, and INOUT, which are defined in
our exported headers, we add DR_PARAM_ to the front.
The aarch64 codec.c was left as the shorter names since the
aarch64_check_codec_order.py assumes single-line signatures.
Issue: #3348
The old way to trigger detach on the Windows platform is the
no-longer-supported "drcontrol" front-end.
Re-exposing the detach feature in drconfig front-end on the Windows
platform.
The following briefly describes my manual testing process, and I'll
continue to submit an automated testing tool in a new PR soon.
I wrote my own continuously running example as a manual test case for
DynamoRIO, which counts the time of summing the first 1 billion numbers
in real time. I'd like to use the real-time output of the test case to
present the running state of the program.
```
#include <iostream>
#include <chrono>
#define LOOPCOUNT 100000;
// Function to perform a computationally intensive task
void performTask()
{
//calculate the sum of the first 1 billion numbers
long long unsigned sum = 0;
for (int i = 1; i <= 1000000000; ++i)
{
sum += i;
}
std::cout << "The sum of the first 1 billion numbers: " << sum << std::endl;
}
void single_loop() {
// Start the timer
auto start = std::chrono::high_resolution_clock::now();
// Perform the computationally intensive task
performTask();
// Stop the timer
auto end = std::chrono::high_resolution_clock::now();
// Calculate the elapsed time
auto duration = std::chrono::duration_cast<std::chrono::milliseconds>(end - start);
// Print the elapsed time
std::cout << "Elapsed time: " << duration.count() << " milliseconds" << std::endl;
}
int main()
{
int counts = LOOPCOUNT;
for (int i = 0; i < counts;i++)
{
single_loop();
}
return 0;
}
```
Here are the steps to perform a manual test：
1. Execute our test case: ./SumOneBillion.exe
2. Use the "ps" command to get the target process ID: ps | grep
SumOneBillion
3. Use "-attach" option to instrument the target process ID： .\drrun.exe
-attach pid -c64 D:\dynamorio\build_debug\api\bin\inscount.dll
4. Use "-detach" option to stop the instrumentation：.\drconfig.exe
-detach pid
The output of the test case and the DynamoRIO in debug version is like
following.
> The sum of the first 1 billion numbers: 500000000500000000
> Elapsed time: 192 milliseconds
> The sum of the first 1 billion numbers: 500000000500000000
> Elapsed time: 191 milliseconds
> The sum of the first 1 billion numbers: 500000000500000000
> Elapsed time: 191 milliseconds
**> <Starting application
D:\vs_demos_repos\SumOneBillion\x64\Release\SumOneBillion.exe (16300)>
> <cannot remove dll from rbtree: at root/min + can't find real tree>
> <Running on newer-than-this-build "Microsoft Windows 10-2009 x64">
> <Early threads found>
> <Initial options = -no_dynamic_options -client_lib
'D:\dynamorio\build_debug\api\bin\inscount.dll;0;' -client_lib64
'D:\dynamorio\build_debug\api\bin\inscount.dll;0;' -code_api -probe_api
-stack_size 56K -max_elide_jmp 0 -max_elide_call 0
-no_inline_ignored_syscalls -native_exec_default_list ''
-no_native_exec_managed_code -skip_terminating_threads
-no_indcall2direct >
> Client inscount is running**
> <CURIOSITY : instr_get_opcode(instr_new) !=
instr_get_opcode(instr_old) in file
D:\DynamoRIODetach\dynamorio-master\core\win32\callback.c line 2079
> version 9.93.19549, custom build
> -no_dynamic_options -client_lib
'D:\dynamorio\build_debug\api\bin\inscount.dll;0;' -client_lib64
'D:\dynamorio\build_debug\api\bin\inscount.dll;0;' -code_api -probe_api
-stack_size 56K -max_elide_jmp 0 -max_elide_call 0
-no_inline_ignored_syscalls -native_exec_default_list ''
> D:\dynamorio\build_debug\lib64\debug\dynamorio.dll=0x0000000015000000
> D:\dynamorio\build_debug\api\bin\inscount.dll=0x00007ff721f90000
> C:\WINDOWS/system32/KERNEL32.dll=0x0000026257e00000
> C:\WINDOWS/system32/KERNELBASE.dll=0x0000026257f40000
> D:\dynamorio\build_debug\ext\lib64\debug/drmgr.dll=0x00007ff721ff0000>
> <CURIOSITY : instr_new == instrlist_first(ilist) || instr_new ==
instr_get_next(instrlist_first(ilist)) in file
D:\dynamorio\core\win32\callback.c line 2082
> version 9.93.19549, custom build
> -no_dynamic_options -client_lib
'D:\dynamorio\build_debug\api\bin\inscount.dll;0;' -client_lib64
'D:\dynamorio\build_debug\api\bin\inscount.dll;0;' -code_api -probe_api
-stack_size 56K -max_elide_jmp 0 -max_elide_call 0
-no_inline_ignored_syscalls -native_exec_default_list ''
> D:\dynamorio\build_debug\lib64\debug\dynamorio.dll=0x0000000015000000
> D:\dynamorio\build_debug\api\bin\inscount.dll=0x00007ff721f90000
> C:\WINDOWS/system32/KERNEL32.dll=0x0000026257e00000
> C:\WINDOWS/system32/KERNELBASE.dll=0x0000026257f40000
> D:\dynamorio\build_debug\ext\lib64\debug/drmgr.dll=0x00007ff721ff0000>
> <Cleaning hooked Nt wrapper @0x00007ffeba790800 sysnum=0x1c2>
> The sum of the first 1 billion numbers: 500000000500000000
> Elapsed time: 417 milliseconds
> The sum of the first 1 billion numbers: 500000000500000000
> Elapsed time: 552 milliseconds
> The sum of the first 1 billion numbers: 500000000500000000
> Elapsed time: 545 milliseconds
> The sum of the first 1 billion numbers: 500000000500000000
> Elapsed time: 537 milliseconds
> The sum of the first 1 billion numbers: 500000000500000000
> Elapsed time: 543 milliseconds
> The sum of the first 1 billion numbers: 500000000500000000
> Elapsed time: 539 milliseconds
> The sum of the first 1 billion numbers: 500000000500000000
> Elapsed time: 538 milliseconds
**> <curiosity: rex.w on OPSZ_6_irex10_short4!>
> <received nudge mask=0x4 id=0x00000000 arg=0x0000000000000000>
> <Detaching from application
D:\vs_demos_repos\SumOneBillion\x64\Release\SumOneBillion.exe (16300)>
> <Detaching from process, entering final cleanup>
> Instrumentation results: 20766267822 instructions executed**
> The sum of the first 1 billion numbers: 500000000500000000
> Elapsed time: 194 milliseconds
> The sum of the first 1 billion numbers: 500000000500000000
> Elapsed time: 194 milliseconds
> The sum of the first 1 billion numbers: 500000000500000000
> Elapsed time: 194 milliseconds
>
Issue: [#725](https://github.com/DynamoRIO/dynamorio/issues/725)
Fixes issues with DR's rseq handling in glibc 2.35+.
Glibc 2.35 added support for the Linux rseq feature. See
https://lwn.net/Articles/883104/ for details. TLDR; glibc registers
its own struct rseq at init time, and stores its offset from the
thread pointer in __rseq_offset. The glibc-registered struct rseq is
present in the struct pthread. If glibc's rseq support isn't
available, either due to some issue or because the user disabled
it by exporting GLIBC_TUNABLES=glibc.pthread.rseq=0, it will
set __rseq_size to zero.
Improves the heuristic to find the registered struct rseq. For the
glibc-support case: on AArch64, it is at a -ve offset from app lib
seg base, whereas on x86 it's at a +ve offset. On both AArch64
and x86, the offset is of the opposite sign than what it would be
if the app registered the struct rseq manually in its static TLS
(which happens for older glibc and when glibc's rseq support
is disabled).
Detects whether the glibc rseq support is enabled by looking at
the sign of the struct rseq offset.
Removes the drrun -disable_rseq workaround added by #5695.
Adjusts the linux.rseq test to get the struct rseq registered by
glibc, when it's available. Also fixes some issues in the test.
Adds the Ubuntu_22 tag to rseq tests so that they are enabled.
Our Ubuntu-20 CI tests the case without rseq support in glibc,
where the app registers the struct rseq. This also helps test the
case where the app is not using glibc.
Also, our Ubuntu-22 CI tests the case with Glibc rseq support.
Manually tested the disabled rseq support case on glibc 2.35,
but not adding a CI version of it.
Fixes#5431
Adds a workaround for the SIGFPE in glibc 2.34+ __libc_early_init() by
setting two ld.so globals located via hardcoded offsets, making this
fragile and considered temporary. (Improvements might include decoding
__libc_early_init or other functions to find the offsets, which is also
fragile; making runtime options to set them for a non-rebuild fix;
disabling the call to __libc_early_init which doesn't seem to be needed
for 2.34).
Tested on glibc 2.34 where every libc-using client crashes with SIGFPE
but they work with this fix.
Adds an Ubuntu22 GA CI run but it has many failures due to the rseq
issue #5431. Adds a workaround for this by having drrun set -disable_rseq
if it detects glibc 2.35+. Even with this we have a number of test failures
so for now we use a label to just run 4 sanity-check tests. This should
be enough to detect glibc changes that break the offsets here.
Issue: #5437, #5431
Switches the signal DR uses to suspend threads from SIGSTKFLT, which
is not supported in gdb, to SIGILL, which is shared with nudges.
SIGILL is unlikely to be blocked or to be sent asynchronously.
We now use sigqueueinfo (where supported) to set a flag to distinguish
a nudge from a suspend signal,
and to set certain siginfo fields that libc does not set, further
increasing confidence that there will be no confusion with the
application's use of signals.
Since QEMU crashes if we send SIGILL, we make SUSPEND_SIGNAL a
variable instead of a cosntant and set it back to SIGSTKFLT when
-xarch_root is set and live with the downsides there.
Fixes#5651
Fixes many unused-variable warnings when building the core and
extension libraries. This enables a config without tests or docs to
build. The drsyms libraries are missing so clients/ is not yet built.
Increases the clang-format version from 12 to 14 to match local versions,
as 12 has bugs in clang-format-diff that showed up in many places in
this PR.
Adds alignment of adjusted stack size options to page sizes in options_enable_code_api_dependences().
Issue: #5383
This patch renames the nudgeunix tool to drnudgeunix to be
named more consistently with the rest of DynamoRIOs frontend tooling.
This helps for packaging as nudgeunix is somehow non-specific.
For backward compatibility, a symlink with the old name is added.
Related to issue #5153
Signed-off-by: Felix Moessbauer <[email protected]>
This patch adds a check when parsing the tool config files:
Clients for alternative archs (e.g. for cross-exec) injection
are only appended if the path to the client lib actually
exists.
This is a preparation for system-wide deployments, where this
check has to be performed at runtime as users might only
install the dr library for a single architecture.
Previously having invalid paths in the tool config file
worked if the respective client was not requested, but lead to a warning.
With this patch, the missing client is simply not registered at all.
Signed-off-by: Felix Moessbauer <[email protected]>
This patch adds a new parameter -tool_dir to specify
in which folder tool configuration files should be
looked up.
Signed-off-by: Felix Moessbauer <[email protected]>
Makes interrupting a blocking syscall on attach the default behavior.
Removes the -skip_syscall drrun/drinject parameter.
Adds a test by adding an optional blocking syscall to infloop.
Issue: #38
Implements missing functionality for ptrace attach on AArch64 and
AArch32: generated code sequences were previously x86-only;
-skip_syscall handling only supported x86; and AArch64 does not
support PTRACE_POKEUSER or PTRACE_PEEKUSER.
For AArch32, Thumb vs Arm mode require multiple steps: clearing LSB to
point at the path used as data via a call; switching to Arm mode for
DR's _start; setting the LSB of the initial app PC.
For AArch32, additionally fixes an encoder error where the opcode is
queried before copying a needs-no-encoding instruction. This is
required for the instruction used to hold data for injection.
Tweaks the disassembler to leave a level 0 instr alone, again to
better handle the data-only insruction used for injection.
Enables the client.attach test on AArch64 and AArch32. For AArch32,
it needs -skip_syscall. Long-term we want that on by default
everywhere but we want explicit tests that hit it on all platforms
first.
Tested manually on an AArch32 machine. Unfortunately the client.attach test
is not trival to set up under QEMU with its multiple command lines and
background processes so that is left as beyond the scope of this PR
and is instead considered part of #4719.
Issue: #38
Fixes a regression from PR #5443 where warnings about an incomplete
multi-bitwidth installation that used to be suppressed when -use_dll
is passed were instead enabled.
Issue: #5250
Adds a new config parameter DYNAMORIO_ALTINJECT for specifying the
path to the alternate-bitwidth DR library for cross-bitwidth execve.
If the parameter is set, DR uses it; otherwise it falls back to the
current hardcoded path assumptions.
drrun and friends automatically set the parameter if the other
bitwidth library is found.
Adds an automated test by augmenting the "linux.execve{32,64}" test
from a 64-bit build which does a test-and-build. The renamed
"linux.xarch" test now does one build (for efficiency) and runs a
script which executes both (32<->64) directions for each of the
existing assumed-layout strategy and the new
config-points-at-alt-bitwidth strategy.
Also tested manually:
$ bin64/drrun -v -use_alt_dll ~/dr/releases/DynamoRIO-Linux-9.0.0/lib32/debug/libdynamorio.so -trace_threshold 51 -- suite/tests/bin/linux.execve64 ~/dr/git/build_x86_dbg_tests/suite/tests/bin/linux.execve-sub32
Fixes#5250
This addresses issue #5130 in the following ways:
- Improve the glibc import checking code by using a regex to match all imports, and then for each import extract the version and use CMake's version comparison functionality to match the version. This allows us to write the desired glibc version as a string such as "2.4", instead of having to write a regular expression that matches everything greater than 2.4 or in general the desired version.
- Introduce a BUILD_PACKAGE option that is OFF by default, but can be turned on when invoking cmake as follows: `cmake -DBUILD_PACKAGE=ON ..`. This is important for rolling release distributions where glibc 2.32 is currently stable and these checks prevent drrun from building. By defaulting to OFF, building from source will provide a nice out-of-the-box experience on such distributions.
- When packaging up DynamoRIO, BUILD_PACKAGE is set to ON in make/package.cmake. This way we ensure that the glibc checks are performed against the distributable version, such that it works on all supported Linux distributions, i.e. those that have not yet reached EOL.
- Bump the glibc version to 2.17. After checking multiple distributions with long support times (typically Debian and CentOS, and Ubuntu LTS being a close one), The oldest versions that are still supported as of writing are Debian Stretch, CentOS 7, RHEL 7, and Ubuntu 16.04 LTS (Xenial). These ship glibc 2.24, glibc 2.17 glibc 2.17 and glibc 2.23 respectively. According to the release page of glibc the release date of glibc 2.17 is 2012-12-25.
- The AArch64 version was 2.17, while non-AArch64 was 2.4. We no longer make a distinction between architectures when selecting the minimum supported glibc version, as all non-rolling release distributions use the same glibc for a given version regardless of the architecture.
- This change is also described in the changelog for the upcoming DynamoRIO release.
Co-authored-by: Derek Bruening <[email protected]>
This addresses issue #3805 as follows:
- Remove old detection code for CMake versions older than 2.6.3 to detect `cl_path`. This was hindering CMake from finding the correct `cl_path`.
- Ensure `cl_path` is defined before `cpp2asm_support.cmake` gets included, such that `ml.exe` and `ml64.exe` for x86 and x64 respectively are found in the correct path.
- Specify `cl_path` as a hint when trying to find `lib.exe` and `dumpbin.exe`.
- The Windows SDK that is being used may be installed into `C:\Program Files (x86)\Windows Kits\10`, of which the installation path is covered by a different registry key, namely `HKEY_LOCAL_MACHINE\Software\Microsoft\Windows Kits\Installed Roots\KitsRoot10`. Check if the registry key points to an existing path, and use it as `kits_dir` if it does and then use that to derive the correct `sdk_bindir`. Otherwise fall back to the old method of detecting `sdk_bin` and `sdk_bindir`.
- When MFC is installed, this will try building drstats. Since drstats has hardcoded paths to `Uuid.Lib`, we now simply use `kits_dir` to specify the correct path to `Uuid.Lib`.
I have tested this with Tools for Visual Studio 2019 as follows:
```
git clone https://github.com/StephanvanSchaik/dynamorio -b i3805-msvc2019
cd dynamorio
mkdir build
cd build
cmake ..
cmd.exe
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build\vcvarsall.bat" amd64
msbuild.exe DynamoRIO.sln
```
This builds everything successfully. However, do note that you should not have Windows 11 SDK (version 10.0.22000.0) installed. Trying to build DynamoRIO against the Windows 11 SDK will fail with rc.exe emitting error RC2188 due to line 253 in `C:\Program Files (x86)\Windows Kits\10\Include\10.0.22000.0\um\winnt.h`. See also: https://bugs.python.org/issue45220 (EDIT: the fix for this is to ensure `_AMD64_` or `_X86_` are defined when building).
Co-authored-by: Derek Bruening <[email protected]>
Fixes#3805
Renames filter lists:
+ s/blacklist/blocklist/
+ s/whitelist/allowlist/
This renames a public-facing drcpusim option:
+ s/-blacklist/-blocklist/
The legacy option is still supported using a droption alias.
Adds a test of both option names.
This also renames some non-publicized DR options:
+ s/whitelist_company_names_default/allowlist_company_names_default/
+ s/whitelist_company_names/allowlist_company_names/
It also renames DR variables like DYNAMORIO_VAR_APP_PROCESS_ALLOWLIST_ID.
We do not bother to provide aliases to support the old names for these
options and variables as they are not documented or commonly used anymore
and maybe should be removed in some future cleanup.
Issue #4366
Fixes a number of issues with Linux attach:
+ Set xdi to zero for x86 _start relocation of libdynamorio.
+ Implement remote memset for .bss zeroing in elf_loader_map_phdrs(),
fixing a crash in some builds such as Ubuntu20 release build.
+ Don't kill target if attach fails.
+ Fix crash if no pid passed.
+ Adds a useful error message on failure to look at ptrace permissions.
+ Adds a warning to use -skip_syscall if attach hangs.
+ Adds a test by porting the Windows client.attach test to Linux.
Disables the mprotect syscall due to weird failures which need to be
examined.
Further tests of blocking syscalls and -skip_syscall are needed.
Re-enables the attach help message for drrun and the deployment docs.
Tested release build on Ubuntu20 where the .bss crash reproduced every
run and is now gone.
Tested "ctest --repeat-until-fail 100 -V -R client.attach" on Ubuntu20
and on a Debian-ish system: no failures.
Issue: #38, #5054Fixes#5054
Hides the -attach flag from the drrun -help output.
Hides the -attach example in the documentation.
Once #5054 is fixed we can re-enable these, but for now it is just too
broken and advertising it causes more confusion than good.
Issue: #5145, #5054
Adds an attach feature for Windows, marked as experimental.
Builds on the unsubmitted PR #3328.
One difference from the original PR, is not taking over threads that are terminating (otherwise attach always fails).
Added the possibility to sleep 1 millisecond between takeover attempts, and controlling the number of attempts.
Attach fails when the main thread (to which we are injecting) is blocking. To solve this, we create a new suspended thread that sleeps indefinitely and we inject into it. As part of pointing this thread at a Sleep function, generalizes find_remote_ntdll_base() to find_remote_dll_base(), and solves an infinite loop there coming from strange kernel behavior.
Adds attach documentation to the section on how to deploy an application under DR.
Adds a new test client.attach. It uses a new target rather than the existing infloop due to differences in staging the output checking steps. Unfortunately there are still some flaky failures, so the test is added to the ignore list for now in order to make progress and get this key feature into DR.
Co-authored-by: Yibai Zhang <[email protected]>
Co-authored-by: orbp <[email protected]>
Issue: #725
Enables attach injection on Linux.
Adds an -attach option to drrun.
Builds on the existing partial ptrace injection support.
Handles the problem of attaching when the target was in the middle of
an auto-restart syscall, where the kernel will report to the tracer that PC
is at the next instruction after syscall, but will set it back to syscall instruction
by subtracting PC (PC -= sizeof(syscall)) after continuation.
The default handling waits for the syscall to complete by single-stepping.
The -skip_syscall drrun option enables an alternate handling which aborts
the syscall and returns -EINTR (adjusting the kernel's ERESTARTSYS).
This may break the app as an auto-restart syscall is not expected to return,
and is only implemented for x86 for now due to lack of testing in arm/aarch64.
A regression test will be added separately.
Co-authored-by: Yibai Zhang <[email protected]>
Co-authored-by: Derek Bruening <[email protected]>
Refactors the exported content in globals.h, c_defines.h, and
globals_shared.h into a new file core/lib/globals_api.h, which is
exported as dr_defines.h.
Removes the duplicate printf macros in globals_shared.h in favor of
those in globals_api.h.
Renames mcxtx.h to mcxtx_api.h, exports it as dr_mcxtx.h, and resorts
to an ifdef to include the internal name vs the external.
Drops support for non-HAVE_FVISIBILITY. This allows us to remove
genapi.pl altogether.
Documents the _api.h naming convention.
Fixes#3092
Cleans up dr_frontend.h and sets it up for direct copying.
Moves dr_probe.h content from hotpatch.c and globals_shared.h and
places it in a new file core/lib/probe_api.h for direct copying to
dr_probe.h.
Separates exported content from core/arch/proc.h into
core/arch/proc_api.h, which is exported directly as dr_proc.h.
Issue: #3092
Moves the small non-public parts of core/ir/*/opcode.h into instr.h
and opnd.h.
Cleans up core/ir/*/opcode.h and codec.py's generated content such
that these files are now just directly copied to the public include/
directory.
Renames opcode.h to opcode_api.h for all 3 arches to make it clear
they are part of the public interface.
Issue: #3092
Removes the CLIENT_INTERFACE define. It is now always enabled.
As part of this, removes all of the IF_CLIENT_INTERFACE and related macros.
Removes SEPARATE_NONPERSISTENT_HEAP which is now always true.
Issue: #2971
Removes the old top-level modes VPS and VMSAFE, making VMAP (aka
regular code mode client interface) the default and removing
references to it as well.
Removes conditional docs for the other modes spread throughout our
documentation.
Removes test suite builds for these modes, including the vps build
from our GA CI suite.
Updates various references.
This breaks the CI due to removed code containing tabs (in comments in
CMake_doxyfile.cmake) and trailing spaces (a .fig file). In the PR I temporarily
disabled those checks and confirmed all tests were green.
Issue: #2971
The group delimiters @{ and @} in our headers were inside regular C
comments rather than Doxygen comments. This worked fine with doxygen
through 1.8.13; but 1.8.17 and up ignore these group directives. Here
we fix this by adding the extra asterisk.
Fixes#4749
Replaces all of the old lightning logos with the new elephant logo.
Updates the doxygen generation to insert the logo into the navbar
frame for doxygen version 1.9.1 which has slightly different syntax
there.
Issue: #2860
Adds a long-missing feature: following into a Windows child process of
a different bitwidth.
Switches injection from DR and from drinjectlib (including drrun and
drinject) to use -early_inject_map. This was most easily done by turning
on -early_inject by default as well. However, the -early_inject_location
default is INJECT_LOCATION_ThreadStart, a new "early" injection location
which is the same late takeover point as with thread injection (we could
also use _ImageEntry, which is only very slightly later, but that fails for
.NET and other applications). Switching all injection over to
map-from-the-parent simplifies cross-arch following, as well as making it
easier to shift the takeover point to an earlier spot in the future. This
is a step toward #607 by switching drinjectlib to use map injection; the
takeover point, as mentioned, is still the thread start.
Placing a hook at the thread start causes some stability issues, so instead
of the usual hook for -early_inject_map, for INJECT_LOCATION_ThreadStart we
set the thread context, like thread injection does. The gencode still
restores the hook as a nop, for simplicity. For parent64 child32, we can't
easily locate the thread start, so we assume it's
ntdll32!RtlUserThreadStart (which is also a fallback if anything fails in
other cases; the final fallback is a hook at the image entry, which works
nearly everywhere but not for .NET where the image entry is not reached).
Adds an -inject_x64 option to inject a 64-bit DR lib into a 32-bit
child from a 64-bit parent, but this option is only sketched out and
is not fully supported yet: #49 covers adding tests and official
support.
Adds library swapping code to find the other-bitwidth library, which
assumes a parallel directory structure. Add a new fatal error if the
library for a child is not found.
To support generating code for all 3 child-parent cases (same-same,
32-64, and 64-32), and in particular for 32-64, switches the small
gencode sequence for -early_inject_map from using IR to using raw
bytes. A multi-arch encoder (#1684) would help but we would need
cross-bitwidth support there, which is not on the horizon. Fixes what
look like bugs in the original gencode generation along the way
(s/pc/cur_local_pos/ and s/local_code_buf/remote_code_buf/): it's not
clear how it worked before.
Adds support for several system calls from a 32-bit parent to a 64-bit
child where the desired NtWow64* system call does not exist. We use
switch_modes_and_call() for NtProtectVirtualMemory and
NtQueryVirtualMemory.
Changes all types in the injection code to handle 64-bit addresses in
32-bit code. Adds UNICODE_STRING_32 and
RTL_USER_PROCESS_PARAMETERS_32 for handling 32-bit structures from
64-bit parents. Similarly, adds RTL_USER_PROCESS_PARAMETERS_64 and
PROCESS_BASIC_INFORMATION64.
Adds get_process_imgname_cmdline() capability for 64-bit remote from 32-bit.
Adds get_remote_proc_address() and uses it to look up
dynamorio_earliest_init_takeover() in a child DR.
Finds the remote ntdll base via a remote query memory walk plus remote
image header parsing. This requires adding a switch_modes_and_call()
version of NtQueryVirtualMemory (also mentioned above), which needs
64-bit args: so we refactor switch_modes_and_call() to take in a
struct of all 64-bit fields for the args.
Fixes a few bugs in other routines to properly get the image name and
image entry for 32-bit children of 64-bit parents.
Updates environment variable propagation code to handle a 32-bit
parent and a 64-bit child. Updates a 64-bit parent and 32-bit child
to insert the variables into the 32-bit PEB (64-bit does no good),
which requires finding the 32-bit PEB. This is done via the 32-bit
TEB, using a hack due to what seems like a kernel bug where it has the
TebBaseAddress 0x2000 too low.
Makes environment variable propagation failures fatal and visible,
unlike previously where errors would just result in silently letting
the child run natively. Turns some other prior soft errors into fatal
errors on child takeover.
Moves environment variable propagation to post-CreateUserProcess
instead of waiting for ResumeThread, which avoids having to get the
thread context (for which we have no other-bitwidth support) to figure
out whether it's the first thread in the process or not. We bail on
propagation for pre-Vista where we'd have to wait for ResumeThred.
Generalizes the other-bitwidth Visual Studio toolchain environment
variable setting for use in a new build-and-test other-bitwidth test
which builds dynamorio and the large_options client (to ensure options
are propagated to children; and it has convenient init and exit time
prints) for the other bitwidth, arranges parallel lib dirs, and runs
the other client.
Issue: #803, #147, #607, #49Fixes#803
Fixes multiple problems encountered with CMake 3.18+:
1) Deprecated old version support. Solved by updating our minimums
everywhere to 3.7 which was already our top-level minimum. At this
point it should be fine to require clients to use 3.7 too.
2) Missing -rdynamic for test executables, where some clients expect
to find global symbols in .dynsym. This is related to policy CMP0065.
3) Missing location property in the samples_proj test.
The fix is in DynamoRIO_add_rel_rpaths(), instead of only using the per-file-type
location property, use the actual path when the library is imported
and we know the path at config time.
4) Assembly compilation breakage from two factors:
A) The CMake-generated define -Ddynamorio_EXPORTS is being passed as
--defsym dynamorio_EXPORTS in <DEFINES> for asm targets, but we pass
that to cpp.
B) Various compiler flags like --MD are in <FLAGS> and cpp does not
like them.
My solution for assembly is to:
1) set(CMAKE_ASM_DEFINE_FLAG "-D")
2) Remove <FLAGS> from the cpp command for asm.
We do end up needing "-mavx*" to set the built-in defines __AVX__, etc.
I put in explicit setting of "-mavx*" using the existing avx support
queries, shifted earlier during config.
3) Move all of our preprocessor defines from COMPILE_FLAGS properties with
explicit -D to COMPILE_DEFINITIONS properites without -D so we get them
in <DEFINES>.
This was a little involved:
+ There were a number of places pasing defines as flags.
+ COMPILE_FLAGS is a string while COMPILE_DEFINITIONS is a list,
necessitating some conversions and changes in handling.
+ I had to split out a new function DynamoRIO_extra_defines from
DynamoRIO_extra_cflags. I maintained the get_DynamoRIO_defines
interface.
+ I had to update several places like in test setup where we remove
defines to avoid duplicates from the command line and
configure.h.
Fixes#4370Fixes#4581
Fixes a gcc 10 build warning on snprintf overlapping in drdeploy.c by
disabling the warning type. A better solution would be nice to avoid
missing warnings on real problems but this will do for now.
Fixes a warning about ignoring a return value in the client.alloc test.
Fixes#4363Fixes#4511
Adds two gdb python scripts I've developed that may be useful to others:
1) drsymload: loads DR symbols regardless of gdb's current state,
which may include having DR symbols at the wrong address. It does
this by reading /proc/self/maps and running objdump on
libdynamorio.so. Ideally this would be integrated into a revived
libdynamorio.so-gdb.py: that's part of #2100.
2) memquery: prints the /proc/self/maps line for a given address.
I'm still shocked gdb doesn't provide such a command natively.
Issue: #2100
Adds new options and interfaces to specify alternate-bitwidth client
libraries for use when the application creates a child process of the
other bitwidth.
For DR, adds -client_lib32 and -client_lib64 options. Switches main
usage to use the appropriate option, with its contents then copied
into -client_lib (to avoid the pain of removing that options).
For drconfiglib, adds dr_register_client_ex() with
dr_client_iterator_next_ex() to support querying other-bitwidth client
registrations.
Adds a new libutil.drconfig_test for drconfiglib. Fixes a UNIX bug
found by the test: existing client queries were cutting off the last
character of the path and options due to differing snprintf semantics.
Also fixes a Windows drconfig handle leak found by the test that
was preventing unregistration from deleting config files.
For drrun and drconfig, adds "-c32" and "-c64" options, with an
additional "--" separating the client options between them.
For tool files, adds CLIENT{32,64}_{REL,ABS}. Updates drcov,
drcpusim, and drcachesim to use the new syntax and drcachesim's
launcher to process it. Tested these manually:
===========================================================================
$ ninja install
$ rm *.log
$ ../exports/bin64/drrun -t drcov -- ~/dr/test/execve64 ~/dr/test/hello32
$ l -t *.log
20K drcov.execve64.109583.0000.proc.log 20K drcov.execve64.109585.0000.proc.log
20K drcov.hello32.109585.0000.proc.log
$ rm -rf drm*.dir
$ ../exports/bin64/drrun -verbose -t drcachesim -verbose 1 -offline -- ~/dr/test/execve64 ~/dr/test/hello32
$ l -td *.dir
4.0K drmemtrace.hello32.117714.7095.dir/ 4.0K drmemtrace.execve64.117714.6260.dir/
4.0K drmemtrace.execve64.117713.9314.dir/
===========================================================================
Adds tests of -c32/-c64 to the existing cross-arch linux.execve{32,64}
tests (Windows won't work until #803 is addressed).
The tests look like this:
===========================================================================
$ cmake . && ctest -V -R 'linux.execve(32|64)'
8: Running test command: "/home/bruening/dr/git/build_x64_dbg_tests/bin64/drrun" "-32" "-dr_home" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit" "-stderr_mask" "0xC" "-dumpcore_mask" "0" "-c32" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit/suite/tests/bin/libclient.large_options.dll.so" "-paramA" "foo" "-paramB" "bar" "--" "-c64" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/bin/libclient.large_options.dll.so" "-paramA" "foo" "-paramB" "bar" "--" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit/suite/tests/bin/linux.execve32" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/bin/linux.execve-sub64"
8: large_options passed: -paramA foo -paramB bar
8: parent is running under DynamoRIO
8: parent waiting for child
8: child is running under DynamoRIO
8: large_options passed: -paramA foo -paramB bar
8: it_worked
8: running under DynamoRIO
8: large_options exiting
8: child has exited
8: large_options exiting
8:
1/2 Test #8: linux.execve32 ................... Passed 3.93 sec
9: Test command: /home/bruening/dr/git/build_x64_dbg_tests/bin64/drrun "-stderr_mask" "0xC" "-dumpcore_mask" "0" "-c32" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit/suite/tests/bin/libclient.large_options.dll.so" "-paramA" "foo" "-paramB" "bar" "--" "-c64" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/bin/libclient.large_options.dll.so" "-paramA" "foo" "-paramB" "bar" "--" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/bin/linux.execve64" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit/suite/tests/bin/linux.execve-sub32"
9: Test timeout computed to be: 1500
9: large_options passed: -paramA foo -paramB bar
9: parent is running under DynamoRIO
9: parent waiting for child
9: child is running under DynamoRIO
9: large_options passed: -paramA foo -paramB bar
9: it_worked
9: running under DynamoRIO
9: large_options exiting
9: child has exited
9: large_options exiting
2/2 Test #9: linux.execve64 ................... Passed 0.86 sec
===========================================================================
Issue: #147, #803Fixes: #147
Moves all of the IR-related files (instruction generation, encoding,
decoding, disassembly, instructions, operands, instruction lists) from
core/arch to core/ir, mirroring the arch-specific subdirectories under
core/ir. This is a code cleanup step toward properly isolating the
drdecode library, as well as moving us toward the ability to build for
a separate target architecture from the host architecture and
eventually perhaps building in multiple target architectures in the
same binary for decoding and IR manipulation.
Also renames mangle_utils.c to ir_utils.c to better fit its purpose
and location.
Issue: #1684
Changes Appveyor to use VS2017 instead of VS2013. We no longer officially
maintain VS2013 support.
Includes quite a few small fixes to get all builds and as many tests as possible passing on VS2017 and Win10-1607 on Appveyor:
+ Add VS2017 support to x64 LIB env var swapping
+ Add pseudo-dll mapping for Ext-MS-Win-NtUser-* and API-MS-Win-RtCore-NtUser-*
+ Increase drmarker cmd size
+ Avoid crashing in debug-only code if PEB.KernelCallbackTable is NULL
+ Expand win32.fpe allowed output to include 'inf' for DBL_MAX+DBL_MAX
+ Pass /arch:IA32 /MD to tool.cpuid.exe to keep drcpusim tests passing w/o any new opcodes
+#4056 drsyms: allow either order of args in output; allow an extra leading underscore; allow a too-long name_available_size; give up on types matching given all the weirdness in recent dbghelps
+ client.fibers fixes for VS2017:
+ Delete the main fiber on exit, since the deletion callback is not called natively
with VS2017 otherwise.
+ Increase the max FLS slot limit to match pre-Win10.
+ Allow either order of deletion callbacks.
+ client.winxfer test:
+ Allow variations in event counts
+ Remove #4053 assert on un-suspendable un-scheduled attach thread
since it happens on win10 at exit and is not fatal
+ Fix unit_tests problems:
+ Set -msgbox_mask to 0 to avoid hangs
+ Skip static TLS from #4030 since it leads to asserts in the
LoadLibrary unit tests b/c we don't support TLS in later-loaded libs
+ Fix exit-time crashes by always swapping the PEB and swapping back at
the end of the drwinapi tests.
* Add an AUTOMATED_TESTING cmake and cpp define, set it for arg_travis
in runsuite.cmake, and use it to make -msgbox_mask 0 by default, as a
failsafe to avoid hangs in CI.
+ Use a better dbghelp:
+ Update search paths: use ProgFiles(x86) instead of PROGFILES.
+ Have drsyms make a local copy in ext/lib64/debug.
+ i#4057: Copy CTest xml files manually
CMake 3.14 dropped support for copying xml results locally.
We solve that by manually copying them ourselves.
+ Fix VPS build warnings with VS2017 and from #4050 and #4053
+ Switch to regular FindMFC; fix DRstats build warnings on VS2017
+ Disable MSVC popups in drcachesim's launcher to avoid hanging Appveyor
+ Ignore the remaining VS2017 test failures, leaving fixing those to #4058
Issue: #4030, #4053, #4001, #4058Fixes#2924Fixes#4056Fixes#4057
Adds new flags LINK_PADDED and INSTR_BRANCH_PADDED and uses them to
mark exit cti padding added for safe patching. Reads the link flag in
decode_fragment() to remove the padding during trace building. This
avoids prior trace barriers from extra basic block exits from inlined
syscall exits, rseq mangling, and client changes.
Removes the option and stats around -pad_jmps_mark_no_trace which is
no longer needed.
Tested on the linux.rseq test which now successfully traces through
the rseq mangling.
Fixes#4038
Tejash Gupta