The kernel implementation of query_time_seconds() currently returns
seconds since the Unix Epoch (1970-01-01), while DR's documented
cross-platform contract uses seconds since 1601-01-01.
This causes the kernel build to return a different time base from the
user-space implementation.
Moves UTC_TO_EPOCH_SECONDS to os_shared.h making it available to both
the Unix and Linux kernel implementations, and converts the kernel
query_time_seconds() result to the 1601-based time.
No changes are made to the existing Unix implementations of
query_time_millis() or query_time_micros().
Fixes#8076
---------
Co-authored-by: Bin Wang <[email protected]>
In preparation of moving the TEST* defines into
dr_project_wide_headers.h, we replace all uses of TEST with the
equivalent TESTANY. Although we avoid exposing
dr_project_wide_headers.h in DR_API headers, it is exposed in
drmemtrace's headers where TEST causes collisions with third-party code
such as googletest.
Issue: #8001
As we are trying to build DR files with the Linux kernel build system
(Kbuild), which has `-Wstrict-prototypes` turned on by default, we are
turning it on in this project as well. Also, the [coding style
conventions](https://dynamorio.org/page_code_style.html#autotoc_md106)
of this project already requires that declarations of functions that
take no argument should have the explicit `void` keyword in their
signatures.
- Enable `-Wstrict-prototypes` when compiling C files.
- Add explicit `void` to the signatures of functions that take no
argument.
For global functions whose declarations already have `void` (or were
added `void` in this PR), this PR also adds `void` to their definitions.
This is an optional cosmetic match, not required by the style guide or
`-Wstrict-prototypes`.
Fixed#7975
This patch fixes two issues when attaching to threads under DR control:
1) Thread blocks SUSPEND_SIGNAL (SIGILL on Linux).
2) Thread is sitting in one of the sigwait syscalls with SUSPEND_SIGNAL
in its wait set.
In both cases the usual signal based attach mechanism cannot be used.
One solution is to use ptrace for both cases. This is guarded by the
-attach_unmask_suspend_signal option and implemented on AArch64 Linux
only.
1) ptrace unmask:
- Create a DR client helper thread.
- Enumerate /proc/<pid>/task and attach to each thread via ptrace.
- Use PTRACE_{GET,SET}SIGMASK to clear SUSPEND_SIGNAL from each mask.
- Detach and continue with the normal suspend and takeover loop.
This keeps the standard signal based takeover working for threads that
only blocked SUSPEND_SIGNAL.
2) ptrace takeover:
- Detect threads waiting in rt_sigwaitinfo/rt_sigtimedwait and whose
wait set contains SUSPEND_SIGNAL (using /proc/self/task/<tid>/syscall
and reading the thread's sigset).
- For those threads use ptrace to:
- Stop and attach to them.
- Read registers and signal mask.
- Install a small takeover stub on the stack, detach and resume.
- os_thread_take_over() then initializes DR state and calls
d_r_dispatch().
A unit test, api.sigill_blocked, has been added for both cases and a
brief description of the -attach_unmask_suspend_signal option added to
the 'DynamoRIO Runtime Options' section of the documentation.
This feature exposed a bug in suite/tests/api/detach_state_shared.c on
x86-64. The memory permissions and cache consistency check in
core/vmareas.c:
ASSERT(!ok || !TEST(MEMPROT_WRITE, prot2) ||
TEST(FRAG_SELFMOD_SANDBOXED, *flags) ||
!INTERNAL_OPTION(hw_cache_consistency));
failed because the *_from_DR*() tests (self-modifying code) left the
shared page as RWX. Later, when a *_from_cache*() test (not
self-modifying code, normal RX) landed on that same page DR expected a
normal non-selfmodifying non-writeable fragment in the page, failing
the ASSERT().
The fix makes each *_from_DR*() test restore its page to RX before
returning,
so later *_from_cache*() tests see the expected permissions.
Issue: #7805
Updates all the "FIXME" comments in the repository to instead use "TODO"
(for those about missing implementations: "NYI" or
"ASSERT_NOT_IMPLEMENTED") or "XXX" (everything else). This matches our
style guide which discourages "FIXME" as its connotations are too
negative and too easily mis-interpreted by readers and in code audits to
imply that the current state of things is broken in some way instead of
just meaning something could be improved.
Fixes the private loader's redirect of libc open() to a new function
that properly handles the flags, instead of the old redirect which
treated them as DR's custom flags.
Adds a test that hits the truncation described in the issue and so fails
without the fix.
Also tested on the /dev/shm scenario described on the users list
https://groups.google.com/g/dynamorio-users/c/kZHTGiIUn8Y/m/rSurDOWeAQAJ.
Fixes#7562
Fixes two stack overflow scenarios that occur when DR delivers an app
signal to the native signal handler for a thread that is mid-detach.
First case: when a thread is handling the suspend signal and is waiting
for the detacher thread to wake it up and tell it to continue detaching.
Currently, DR unblocks signals before starting the wait. If the signal is
delivered at this point, currently execute_native_handler() incorrectly
delivers the signal to the native handler on DR's own signal stack. To
fix this: we now do not unblock signals during this wait as it
complicates native signal delivery, also for the second case described
below.
Additionally, for a detaching thread, we now do not explicitly
restore the app's sigblocked mask; DR already restores the mask on the
signal frame, which would be restored automatically when the thread
returns from the DR detach signal handler. This avoids another case
where the app may be on DR's signal stack when the native signal is
delivered.
Second case: when the thread has been woken up by the detacher thread,
executed sig_detach, and reinstated the app signal stack (if available). If
the signal is delivered at this point, execute_native_handler() adds a new
signal frame on top of DR's own signal frame on the app stack and invokes
the native signal handler. This sometimes ends up taking too much stack
space which causes a stack overflow, as observed on an internal app with
frequent profiling signals that use the stack-intensive libunwind to get a
stack trace for all threads. To fix this: we reuse the same signal frame
for delivering the signal to the native signal handler, when the app
doesn't need a non-RT frame.
The new code is exercised by the existing detach_signal test. Also
modified the test to have some threads that have a very small sigstack,
which helps reproduce the crash originally seen on a real app. (There
was already a note in detach_signal test about using a large sigstack to
avoid this stack overflow.)
Tested on an internal app where failures reduce from ~136/4000 to
~1/4000.
Issue: #6814
Enable the feature of detachment on Linux.
- I setup a temporary separate stack for the nudged thread, and resume
the normal execution of this thread after detachment.
- A new variable named `nudged_sigcxt` is used to save the `sigcxt` of
nudged thread.
- And the extra code `dcontext == GLOBAL_DCONTEXT` is added to cover the
new code path that won't be executed before during the process of thread
exit.
- What's more, I turn off the option of `sigreturn_setcontext` for the
smooth resumption of nudged thread on X64 ISA.
- Finally, the frontend, automated test cases and documentation are
modified accordingly.
Issue: [#95](https://github.com/DynamoRIO/dynamorio/issues/95)
To avoid symbol conflicts with IN, OUT, and INOUT, which are defined in
our exported headers, we add DR_PARAM_ to the front.
The aarch64 codec.c was left as the shorter names since the
aarch64_check_codec_order.py assumes single-line signatures.
Issue: #3348
SIGSTKSZ and MINSIGSTKSZ are moving away from being constants.
SIGSTKSZ is now a call to sysconf().
This was causing a build error building our tests, which is fixed
using the suggestion from Stanislaw Kardach.
For the core, we eliminate the constants to avoid invoking libc. We
look for AT_MINSIGSTKSIZE in our existing auxv search code and if not
found we fall back to a hardcoded value.
Co-authored-by: Stanislaw Kardach <[email protected]>
Fixes#5434
This is a first of a set of pull requests whose aim is to enable DynamoRIO
compilation for RISC-V architecture.
This change introduces basic architecture specific macro, enum and type
definitions that next commits will use.
It adds definitions and structures describing basic architecture specifics
such as registers, scratch registers, SIMD data, TLS, stub sizes,
syscall/signal ABI, ELF section types and calling conventions.
This will serve as a basis for later commits which will enable DynamoRIO
compilation for a RISC-V architecture.
A key subset of the changes here is "semantic register names".
When code uses register names or offsets directly it is hard to follow
whether a given register has a special purpose or any scratch register
can be used.
It also results in IF_<arch>_ELSE ladders which become less readable
with each new architecture added.
Here we choose semantic register names for the following use-cases:
1. core/translate.c: Unit tests for IBL used DR_REG_XCX/R2 directly
where TBL_TARGET_SLOT and IBL_TARGET_REG are defined. In such
scenarios - replace direct register name with the semantic one to
avoid confusion.
2. core/ir/decodelib.c and core/arch/arch.h: Two places utilizing same
register fields and requiring synchronization. Instead define
semantic name and use it in both places.
3. core/arch/arch.h: According to documentation dr_insert_call()
register r11 is clobbered. However this works only because X86 and
ARM both have a register named r11. New architectures might not, i.e.
RISC-V names registers xN. Instead add a CALL_SCRATCH_REG definition
inside arch.h and use this instead. Similarly it seems that if
documentation would use semantic names, it would be easier to produce
a single table appendix with mapping to concrete registers.
NOTE:
This code is not validated and may contain stubs as the main point was
to achieve compilation and estimate the effort required for the port.
This code is not compiled yet (see my branch for a full compilation codebase
reference) and should not impact the current X86 and ARM/Aarch32
functionality.
Issue: #3544
Signed-off-by: Stanislaw Kardach <[email protected]>
Multiple headers fed into dr_tools.h, which is no longer possible with
the simple copy-as-is export model. We split the os-specific state,
aux libs, and memory query types from os_shared.h into a new file
os_api.h locally and dr_os_utils.h exported, and move the
corresponding routines from instrument_api.h into the file as well.
The tracedump types are similarly separated into fragment_api.h,
exported as dr_tracedump.h.
Furthermore, the rest of the dr_tools.h content is moved out of
instrument_api.h to core/lib/dr_tools.h, except process-query routines
which fit better in os_api.h. This completes the refactoring of
dr_tools.h which is now completely removed from genapi.pl.
Issue: #3092
Removes the CLIENT_INTERFACE define. It is now always enabled.
As part of this, removes all of the IF_CLIENT_INTERFACE and related macros.
Removes SEPARATE_NONPERSISTENT_HEAP which is now always true.
Issue: #2971
Adds a long-missing feature: following into a Windows child process of
a different bitwidth.
Switches injection from DR and from drinjectlib (including drrun and
drinject) to use -early_inject_map. This was most easily done by turning
on -early_inject by default as well. However, the -early_inject_location
default is INJECT_LOCATION_ThreadStart, a new "early" injection location
which is the same late takeover point as with thread injection (we could
also use _ImageEntry, which is only very slightly later, but that fails for
.NET and other applications). Switching all injection over to
map-from-the-parent simplifies cross-arch following, as well as making it
easier to shift the takeover point to an earlier spot in the future. This
is a step toward #607 by switching drinjectlib to use map injection; the
takeover point, as mentioned, is still the thread start.
Placing a hook at the thread start causes some stability issues, so instead
of the usual hook for -early_inject_map, for INJECT_LOCATION_ThreadStart we
set the thread context, like thread injection does. The gencode still
restores the hook as a nop, for simplicity. For parent64 child32, we can't
easily locate the thread start, so we assume it's
ntdll32!RtlUserThreadStart (which is also a fallback if anything fails in
other cases; the final fallback is a hook at the image entry, which works
nearly everywhere but not for .NET where the image entry is not reached).
Adds an -inject_x64 option to inject a 64-bit DR lib into a 32-bit
child from a 64-bit parent, but this option is only sketched out and
is not fully supported yet: #49 covers adding tests and official
support.
Adds library swapping code to find the other-bitwidth library, which
assumes a parallel directory structure. Add a new fatal error if the
library for a child is not found.
To support generating code for all 3 child-parent cases (same-same,
32-64, and 64-32), and in particular for 32-64, switches the small
gencode sequence for -early_inject_map from using IR to using raw
bytes. A multi-arch encoder (#1684) would help but we would need
cross-bitwidth support there, which is not on the horizon. Fixes what
look like bugs in the original gencode generation along the way
(s/pc/cur_local_pos/ and s/local_code_buf/remote_code_buf/): it's not
clear how it worked before.
Adds support for several system calls from a 32-bit parent to a 64-bit
child where the desired NtWow64* system call does not exist. We use
switch_modes_and_call() for NtProtectVirtualMemory and
NtQueryVirtualMemory.
Changes all types in the injection code to handle 64-bit addresses in
32-bit code. Adds UNICODE_STRING_32 and
RTL_USER_PROCESS_PARAMETERS_32 for handling 32-bit structures from
64-bit parents. Similarly, adds RTL_USER_PROCESS_PARAMETERS_64 and
PROCESS_BASIC_INFORMATION64.
Adds get_process_imgname_cmdline() capability for 64-bit remote from 32-bit.
Adds get_remote_proc_address() and uses it to look up
dynamorio_earliest_init_takeover() in a child DR.
Finds the remote ntdll base via a remote query memory walk plus remote
image header parsing. This requires adding a switch_modes_and_call()
version of NtQueryVirtualMemory (also mentioned above), which needs
64-bit args: so we refactor switch_modes_and_call() to take in a
struct of all 64-bit fields for the args.
Fixes a few bugs in other routines to properly get the image name and
image entry for 32-bit children of 64-bit parents.
Updates environment variable propagation code to handle a 32-bit
parent and a 64-bit child. Updates a 64-bit parent and 32-bit child
to insert the variables into the 32-bit PEB (64-bit does no good),
which requires finding the 32-bit PEB. This is done via the 32-bit
TEB, using a hack due to what seems like a kernel bug where it has the
TebBaseAddress 0x2000 too low.
Makes environment variable propagation failures fatal and visible,
unlike previously where errors would just result in silently letting
the child run natively. Turns some other prior soft errors into fatal
errors on child takeover.
Moves environment variable propagation to post-CreateUserProcess
instead of waiting for ResumeThread, which avoids having to get the
thread context (for which we have no other-bitwidth support) to figure
out whether it's the first thread in the process or not. We bail on
propagation for pre-Vista where we'd have to wait for ResumeThred.
Generalizes the other-bitwidth Visual Studio toolchain environment
variable setting for use in a new build-and-test other-bitwidth test
which builds dynamorio and the large_options client (to ensure options
are propagated to children; and it has convenient init and exit time
prints) for the other bitwidth, arranges parallel lib dirs, and runs
the other client.
Issue: #803, #147, #607, #49Fixes#803
Moves all of the IR-related files (instruction generation, encoding,
decoding, disassembly, instructions, operands, instruction lists) from
core/arch to core/ir, mirroring the arch-specific subdirectories under
core/ir. This is a code cleanup step toward properly isolating the
drdecode library, as well as moving us toward the ability to build for
a separate target architecture from the host architecture and
eventually perhaps building in multiple target architectures in the
same binary for decoding and IR manipulation.
Also renames mangle_utils.c to ir_utils.c to better fit its purpose
and location.
Issue: #1684
Adds dr_get_app_args() to enable a client to retrieve the locations where command-line arguments of the app reside. C strings of the arguments are retrieved via a new function, namely dr_app_arg_as_cstring(). Also, dr_num_app_args() enables the client to get the number of arguments.
The PR adds support only for Unix and early injection for now.
The gathering of error codes of failed DR API routines is also added with dr_get_error_code().
Issue: #2662.
Adds static TLS support to the Windows private loader.
This involves the following:
Swaps the TEB->ThreadLocalStoragePointer field between app and private
states, with new dcontext fields to store the values.
Adds an os_privmod_data_t to the Windows loader to store TLS data.
Parses PE TLS data fields, records the TLS callbacks, sets the TLS
array index, and records the TLS initialization data. Calls TLS
callbacks prior to regular entry points; warns on crashes but does not
consider them fatal.
Handling TLS is simplified with a hardcoded maximum size and lack of
support for dynamicaly-loaded-library TLS. This lets us allocate an
array at thread init and not need any complex scheme to reallocate it.
Prior to calling the thread init function for a library, its TLS data
is allocated and initialized from its recorded initialization data.
Since both process init and process exit library functions expect TLS
to be set up, reorders several loader sequences:
+ Partially unifies Windows and Linux by splitting loader_init() into
loader_init_prologue() (for setting up the private PEB used by
arch_init for gencode), called early, and loader_init_epilogue(),
called after thread init so we have a dcontext and can set up the
TLS. However, Linux still needs relocs and TLS *after* thread init
while Windows is the reverse, so we have a split ordering in a
newly-divided loader_init_{prologue,epilogue}().
This undoes the #338 special casing which is now removed.
+ Splits out instrument_exit_event() from instrument_exit() and moves
instrument_exit() to after the final thread exit, to ensure we call
the thread exit library functions. Removes
instrument_exit_post_sideline() which is now merged into
instrument_exit().
+ Adds loader_make_exit_calls() to enable calling both the thread exit
and process exit library functions before TLS is freed (yes, process
exit functions blindly de-reference TLS, just like process init
functions do).
+ Delays process and thread init function calling until the
statically-imported set of libraries is fully loaded, so we have that
TLS array size.
Reverses the modlist iteration order for function calling, to properly
call independent libraries first before their dependents.
Adds static TLS tests to the client.raw_tls test. Renames the test to
client.tls, and changes the client to C++ to use std::vector and a
custom class with the C++11 'thread_local' for TLS callback testing.
However, the vector is disabled for Linux because it breaks that
loader: i#4034. Plus, VS2013 doesn't support 'thread_local', so that
is disabled until we upgrade to VS2017.
Issue: #338, #4002, #4030, #4034Fixes#4030Fixes#4002
Reverts the now-obsolete run-native approach for an older version of
the restartable sequence ("rseq") feature. That version never made it
to the mainline kernel, and the run-native approach failed to allow
tools to see rseq code. Reverts most of commits cda88be and 0935136.
Issue: #2350
Adds a new option -satisfy_w_xor_x and implements it for Linux. It
uses a dual-map approach to satisfy write-xor-execute security
policies. A writable shadow is created for the vmcode region, while
vmcode itself is read-only. A constant offset is then used to
translate between writable and executable addresses.
The dual mapping uses memfd_create if available, falling back to shm
if not, for the file backing. The scheme relies on a single large,
now-separated-from-vmheap vmcode in order to have a single mirrored
region. If vmcode is left as the default 1G size, the scheme simply
fails if vmcode runs out of space: the user must request 2G up front
(with corresponding loss of -vm_base_near_app: see #3570) for such a
huge application.
Generated code and the code cache store executable addresses,
converting to writable on every write. Special heap units store
writable addresses and convert to executable when passing to the user.
Committing new memory uses a fixed mmap of the mapping file, instead
of mprotect, since most W^X policies do not allow execution after
mprotect.
Adds a new w_xor_x test.
Updates syscall_linux_x86.h with numbers from FC27 to include SYS_memfd_create.
No support is yet provided for Windows, for Mac, for 32-bit, or for
persistent caches.
Issue: #3566, #3570
Renames multi-word global symbols with popular names in other software
to help reduce the chance of name conflicts:
s/config_{in,ex}it/d_r_config_{in,ex}it/
s/mutex_lock/d_r_mutex_lock/
s/mutex_unlock/d_r_mutex_unlock/
s/mutex_trylock/d_r_mutex_trylock/
s/mutex_delete/d_r_mutex_delete/
s/mutex_lock_app/d_r_mutex_lock_app/
s/mutex_mark_as_app/d_r_mutex_mark_as_app/
s/mutex_fork_reset/d_r_mutex_fork_reset/
s/read_lock/d_r_read_lock/
s/write_lock/d_r_write_lock/
s/write_unlock/d_r_write_unlock/
s/read_unlock/d_r_read_unlock/
s/write_trylock/d_r_write_trylock/
s/parse_word/d_r_parse_word/
s/parse_int/d_r_parse_int/
s/print_timestamp/d_r_print_timestamp/
s/get_thread_id/d_r_get_thread_id/
s/internal_error/d_r_internal_error/
s/safe_read/d_r_safe_read/
Issue: #3348
Since we want a single build of drlibc for both static and shared DR
libs, we do not want any STATIC_LIBRARY defines in there. Here we
remove the define in os_page_size_init() by adding a parameter
"env_followed_by_auxv" and a function
"is_our_environ_followed_by_auxv()" and effectively moving the
STATIC_LIBRARY to the callers.
This fixes a regression for #2122 caused by creating drlibc in
710a6898.
Issue: #1409, #2122
Renames several single-word global symbols to help reduce the chance
of name conflicts:
+ s/debugRegister/d_r_debug_register/
+ s/crc32/d_r_crc32/
+ s/dispatch/d_r_dispatch/
+ s/emulate/d_r_emulate/
+ s/extensions/base_extensions/
Issue: #3348
Allow for non-natural spill position when recognizing spill/restore if part of DR's mangling. Dependent on the app instruction's register usage, mangling is able to spill different registers than it is naturally assumed.
Adds a subtest to client.mangle_suspend to expose i#3327.
Fixes#3327
Massive clang-format of every C or C++ source file (*.h, *.c, *.cpp),
except for:
+ third_party/
+ ext/drsysm/libelftc/include/
+ ext/drsyms/demangle.cc
This is a single, large commit by design to present a single history
disruption point and bring the code base into a consistent format.
clang-format version 6.0 was used.
Tweaks the clang-format rules to indent 4 after pre-processor hashes, to
allow single-line case labels, and align trailing comments.
Disables the pp_indent vera style check as we changed the indent rule and
clang-format now covers it. Leaves the other checks, even though some are
redundant (they found clang-format errors).
Moves C++ token defines from globals_shared.h and globals.h to a new
header, core/lib/c_defines.h, to avoid a clang-format parsing error where
it fails to identify the include guard.
Adds genapi.pl removal of extra indentation inside API_EXPORT_ONLY and
CLIENT_INTERFACE regions.
Adds exclusions around large regions we don't want to format, in these
files:
+ core/win32/syscallx.h
+ core/arch/x86/decode_table.c
+ core/arch/arm/table_a32_pred.c
+ core/arch/arm/table_a32_unpred.c
+ core/arch/arm/table_t32_16.c
+ core/arch/arm/table_t32_16_it.c
+ core/arch/arm/table_t32_base.c
+ core/arch/arm/table_t32_coproc.c
+ core/arch/arm/table_encode.c
Adds smaller exclusions to work around clang-format bugs:
+ Several missing break-after-return-type
+ Several >90-char lines
+ Misc scattered issues, all listed in #2876
Issue: #2876
Refactors signal_thread_inherit() to be called from a new routine
os_thread_init_finalize() which is invoked while holding
thread_initexit_lock yet after synch_thread_init(). This eliminates
races with suspend signals arriving in newly half-initialized threads,
which then drop the signals. The refactoring rearranges several
thread initialization sequences to pass the clone record through
dynamo_thread_init().
This refactoring allows us to revert the os_thread_suspend timeout
from commit 972cddfa PR #2762 which added a timeout to
os_thread_suspend that turns out to not be safe on UNIX as the suspend
model assumes there is no retry.
Delays mask relaxing in handle_suspend_signal() to avoid timeout on
suspend due to an intervening signal.
Includes tweaks to an i#3020-related assert and i#2993-related alarm
lock retry which got in the way of testing the final solution here.
Tested by running thread creating apps that attach and detach many
times, similar to the static burst tests in our suite.
Issue: #3020, #2993Fixes: #2779
Refactors find_executable_vm_areas() to share its map entry skipping with
the re-takeover re-walk from b06a7022, but not its module list or
executable area updates. This entry skipping for vmheap turns out to make
a big performance difference when attaching.
Removes individual updates to memcache for entries inside vmheap which were
already bulk-added for the find_executable_vm_areas() walk.
Issue: #2037
Adds a feature where a client library can request that the private loader
complain if malloc & co. are called at any time other than process init or
exit, to help clients that want to support being linked statically with the
app. Because it has to be early, the feature is triggered by a variable
declaration DR_DISALLOW_UNSAFE_STATIC. It can be overridden
dynamically by a new API routine dr_allow_unsafe_static_behavior().
Fixes drcachesim to use placement new for its offline custom module data
allocations.
Issue: #975, #2006
To avoid problems with client threads starting execution before DR's signal
handlers are in place, we delay client thread execution until DR starts
executing the app.
Implements a broadcast event variant of event_t to support waking all
client threads at start time.
Adjusts the client.signal and client.thread tests to handle this new
restriction.
Removes the erroneous call to signal_reinstate_handlers() in client
threads.
Fixes#2335
On Unix, after 5 seconds of waiting for a thread to acknowledge a received
signal, os_thread_suspend now returns false so that the caller can
retry.
This fixes a thread related to creating a new application thread close to the time when
detach happens.
Issue: #2157
This PR adds a new function to support transfer/relocation while waiting for a contended application mutex lock.
As mentioned in code comments, this routine is to allow client code called via a clean call to lock application locks safely. Deadlocks can be avoided during detach/xfer using this lock.
Fixes#2725
To handle a thread exiting on attach, adds a timeout to wait_event() and
its corresponding implementations: os_wait_event() on Windows and
ksynch_wait() on UNIX. Uses the timeout to check whether a thread exited,
and if so, to move on.
Augments the test from #2600 to test this race.
Abandon the api.detach_spawn test on Windows:i#2611 covers fixing the
tricky problems on Windows. Leaves in place some fixes toward #2611:
+ Fixes a race where we put interception_code hooks in place before marking
them +x
+ Increases MAX_THREADS_WAITING_FOR_DR_INIT
Fixes clang 32-bit missing __moddi3 by adding it to third_party/libgcc and linking
that into x86 and arm.
To enable adding race checks, moves doing_detach inside the synchall and adds
started_detach for the few checks that need pre-synch querying.
Removes the dynamo_thread_init_during_process_exit flag that was added in
45dd931 for #2075, as the UNIX uninit_thread_count solution from #2600 solves
that problem on its own.
Fixes#2601
Fixes 3 problems related to the kernel placing vvar+vdso in libdynamorio's text-data gap:
1) Ensures old_libdr_base is NULL by not relying on the calling convention
during early injection.
2) Handles asserts/curiosities inside memquery code by adding
is_readable_without_exception_query_os_noblock()
and memquery_from_os_will_block() to avoid deadlock on locks
used on UNIX for memory queries.
3) Detects something in the text-data gap and reloads libdynamorio during
early injection as the rest of DR assumes there's nothing there.
Fixes#2641
Removes the STACK_GUARD_PAGE define and makes all of its code always
compiled.
Implements stack overflow detection on UNIX by using a guard page at the
bottom of the stack. Adds a new label on a crash that distinguishes a
stack overflow from a general crash. Adds a new option -stack_guard_pages
covering this feature along with a new test.
Does not support the feature on Windows where the kernel and -vm_reserve
end up not working well together: i#2595 covers addressing that.
Fixes#2592
Inserts nops during mangling after instructions that could set the trap flag to isolate where a single-step exception can occur. When we see such an exception, we build a special single-instruction basic block to emulate the single-step execution, after which we forge the exception to pass to the app.
Currently only Windows is supported: #2385 covers adding UNIX support.
Adds a test.
Fixes#2144
Adds preliminary handling for restartable sequences in Linux, a kernel
feature meant for per-cpu critical regions. The code here targets a
particular Linux kernel patch, but tries to be general to make it easy to
tweak for other flavors.
Adds a new option -rseq_sysnum to enable this support and to specify the
syscall number for the kernel extension.
For initial handling, we execute the restartable sequence natively. It is
very complex to build regular blocks and support instrumentation, so we
start with native execution to at least allow correct app execution even if
tools miss some code. We assume each sub-sequence is entered via a call
and exited via a return, and we use our existing native_exec gateway and
resumption mechanisms.
Sets the signal handler for the 3 alarm signals to SIG_IGN during attach
and detach, to reduce problems with races while we try to coordinate taking
over or sending native all of the app threads.
This is not a panacea, as timer_create can send out any signal, not just
the 3 classic alarm signals, on timer expiration. Plus, non-timer-related
signals could arrive during attach or detach. However, this will help with
the most typical cases.
Adds an itimer to the api.static_signal test, though it is not easy to
reproduce these problems in small applications.
I'm considering this to fix the filed issues despite the above-mentioned
remaining corner cases as I'm considering those to be pathological:
Fixes#2161Fixes#2270
Adds a flag DR_STATE_DR_TLS controlling the DR TLS swap added in i#2089.
This flag is *not* part of DR_STATE_ALL to avoid a regression in clients.
Review-URL: https://codereview.appspot.com/312260043
Puts in a new scheme for is_thread_tls_initialized(): rather than reading
thread id fields and comparing to expensive syscalls, and trying to
distinguish different fork and clone children, we instead use a very simple
approach. We add a magic number field to our TLS and
is_thread_tls_initialized() simply does a safe read of that field: if it
equals our magic number, it's initialized.
On a clone, the parent swaps to a separate private TLS with an invalid
magic field. The child inherits this and thus starts out correctly
uninitialized, while the parent's syscall gencode still works as it ignores
the magic field. The parent's TLS is restored first thing in dispatch().
On a fork, nothing is done, and thus the magic number remains and the child
is correctly initialized.
We avoid safe read faults on thread exit by pointing the exiting thread's
TLS at a fake TLS with an invalid magic number. This is efficient for regular
threads and required for threads missing CLONE_SIGHAND, for which we must
also skip the segment zeroing at thread exit.
On swapping to native, we also put in the private TLS. On re-take-over, we
add new queries (including using a special "invalid" magic number to
distinguish from a completely unknown thread) and TLS swaps to several
attach points to avoid thinking that a NULL dcontext or
!is_thread_tls_initialized() means an unknown thread.
Generalizes the segment swapping code to correctly swap DR's segment too
instead of what it did before where it took in a segment parameter but then
hardcoded the register to use.
The app using the aux segment is not yet supported: that's i#2088/i#107.
The new scheme is under the -safe_read_tls_init option, which is now on by
default. Once we're sure it's stable we'll remove the option and the old
code.
Adds a new test linux.clone-pie, a PIE build of linux.clone, which forces
the use of the MSR for TLS instead of the GDT and results in a crash with
-no_safe_read_tls_init.
Expands the linux.clone* tests to also test a thread without CLONE_SIGHAND.
Fixes#2089
Review-URL: https://codereview.appspot.com/314160043
Adds support for a different thread calling dr_app_start() from the thread
that called dr_app_setup(). We initialize the calling thread if it's not
already initialized, and point at the primary thread's signal handling
info.
A test is forthcoming as part of i#1729.
Fixes#2034
Review-URL: https://codereview.appspot.com/317730043
The stack pointer is passed as a third argument to relocate_dynamorio
and on Linux this is used to search the auxiliary vector for AT_PAGESZ.
Similarly, there are calls to os_page_size_init in dynamorio_app_init
and standalone_init to do the same thing. On MacOS, and on Linux if
for some reason os_page_size is called before these initialisations,
the page size is discovered using the system calls mmap and munmap,
which should work on any POSIX system.
A later commit should get Mac OSX to use sysctl_query on hw.pagesize.
Review-URL: https://codereview.appspot.com/313800043
Adds removal of all signal handler and vsyscall hooks while fully native
(during usage of the start/stop API). This simplifies handling of native
threads. This does not yet solve single-native-thread signals for
native_exec or DR_EMIT_GO_NATIVE.
The implementation uses a new routine os_process_{not_,}under_dynamorio().
A test api.static_signal is forthcoming but it still requires some other DR
attach fixes before it fully works.
Review-URL: https://codereview.appspot.com/308680043
When the page size is not a compile-time constant it is misleading to
provide the traditional PAGE_SIZE macro, so we provide a function
instead, though we provisionally define PAGE_SIZE as os_page_size()
for DynamoRIO's internal use, and, if DynamoRIO_PAGE_SIZE_COMPATIBILITY
is set, we also define PAGE_SIZE as dr_page_size() for use by clients.
Similarly, PAGE_START is removed from the API, though if
DynamoRIO_PAGE_SIZE_COMPATIBILITY is set a definition in terms of
dr_page_size() is provided.
DynamoRIO_PAGE_SIZE_COMPATIBILITY will be set automatically if the
client targets version 6.2 or earlier.
Modify non-core code to use the new API.
Currently the value returned by os_page_size() is always 4096. In future
on AArch64 it will sometimes be 16384 or 65536.
Review-URL: https://codereview.appspot.com/309340043
Merges the Linux detach code with the Windows detach code. Keeps several
OS-specific helper routines called out from the new, generalized detach
code. Enables the two new app-detach tests for Windows.
Review-URL: https://codereview.appspot.com/307310043
