Commit Graph

11201 Commits

Author SHA1 Message Date
Martin Liska
7240eb3ba9 Do not rely on that subject of ErrorAllocTypeMismatch is a heap address.
Differential Revision: https://reviews.llvm.org/D54856.

llvm-svn: 350085
2018-12-27 08:39:13 +00:00
Evgeniy Stepanov
772bf988cf Revert "[msan] Disable tail call optimization in msan rtl."
This reverts commit r350080, which breaks
* gcc as the host compiler
* some tests on ppc64

llvm-svn: 350083
2018-12-27 03:30:42 +00:00
Evgeniy Stepanov
faef9b77de [msan] Disable tail call optimization in msan rtl.
This should give us better error stack traces on the bots, in
particular, for https://bugs.llvm.org/show_bug.cgi?id=40162

llvm-svn: 350080
2018-12-27 00:21:20 +00:00
Michal Gorny
a939b40eae [xray] Detect MPROTECT and error out when it's enabled (on NetBSD)
Add a CheckMPROTECT() routine to detect when pax MPROTECT is enabled
on NetBSD, and error xray out when it is.  The solution is adapted
from existing CheckASLR().

Differential Revision: https://reviews.llvm.org/D56049

llvm-svn: 350030
2018-12-23 15:09:28 +00:00
Michal Gorny
470ce63251 [xray] Disable alignas() for thread_local objects on NetBSD
Disable enforcing alignas() for structs that are used as thread_local
data on NetBSD.  The NetBSD ld.so implementation is buggy and does
not enforce correct alignment; however, clang seems to take it for
granted and generates instructions that segv on wrongly aligned objects.
Therefore, disable those alignas() statements on NetBSD until we can
establish a better fix.

Apparently, std::aligned_storage<> does not have any real effect
at the moment, so we can leave it as-is.

Differential Revision: https://reviews.llvm.org/D56000

llvm-svn: 350029
2018-12-23 15:09:20 +00:00
Dimitry Andric
a5afbce82b Set the default SANITIZER_CXX_ABI library to libc++ for FreeBSD, and
also mark it as a system library, like on macOS.

llvm-svn: 350020
2018-12-23 11:49:47 +00:00
David Carlier
58d3823086 [Sanitizer] Enable POSIX regex api on FreeBSD.
Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D56009

M    lib/sanitizer_common/sanitizer_common_interceptors.inc
M    lib/sanitizer_common/sanitizer_platform_interceptors.h
M    lib/sanitizer_common/sanitizer_platform_limits_freebsd.cc
M    lib/sanitizer_common/sanitizer_platform_limits_freebsd.h
D    test/sanitizer_common/TestCases/NetBSD/regex.cc
A  + test/sanitizer_common/TestCases/Posix/regex.cc

llvm-svn: 350002
2018-12-22 11:17:27 +00:00
Kamil Rytarowski
edbe2b3f02 Add support for LLVM profile for NetBSD
Summary:
NetBSD uses typical UNIX interfaces.

All tests pass except instrprof-dlopen-dlclose-gcov.test, as there
is not supported semantics of atexit(3) in dlopen(3)ed+dlclose(3)d
DSO.

NetBSD also ships an older version of LLVM profile (ABI v.2 predating
ABI v.4 in upstream version) inside libc. That copy has been manually
removed during the porting and testing process of the upstream version
to NetBSD. Otherwise there were conflicts between them two.

Reviewers: joerg, vitalybuka, vsk

Subscribers: srhines, fedor.sergeev, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55997

llvm-svn: 349994
2018-12-22 06:56:19 +00:00
Dan Liew
c6027e20d4 Fix comment typo.
llvm-svn: 349961
2018-12-21 21:41:43 +00:00
Dan Liew
a8334ed571 Fix static_assert() scope in CombinedAllocator.
It should be at the class scope and not inside the `Init(...)` function
because we want to error out as soon as the wrong type is constructed.
At the function scope the `static_assert` is only checked if the
function might be called.

This is a follow up to r349957.

rdar://problem/45284065

llvm-svn: 349960
2018-12-21 21:41:37 +00:00
Dan Liew
09f6d77840 Fix static_assert() scope in SizeClassAllocator32.
It should be at the class scope and not inside the `Init(...)` function
because we want to error out as soon as the wrong type is constructed.
At the function scope the `static_assert` is only checked if the
function might be called.

This is a follow up to r349138.

rdar://problem/45284065

llvm-svn: 349959
2018-12-21 21:41:31 +00:00
Dan Liew
14e0d9ed89 Introduce AddressSpaceView template parameter to CombinedAllocator.
Summary:
This is a follow up to https://reviews.llvm.org/D55764 .

For the ASan and LSan allocatorsthe type declarations have been modified
so that it's possible to create a combined allocator type that
consistently uses a different type of `AddressSpaceView`. We intend to
use this in future patches. For the other sanitizers they just use
`LocalAddressSpaceView` by default because we have no plans to use these
allocators in an out-of-process manner.

rdar://problem/45284065

Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov, yln

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D55766

llvm-svn: 349957
2018-12-21 21:22:27 +00:00
Dan Liew
d2c6af7358 Introduce AddressSpaceView template parameter to SizeClassAllocator64.
Summary:
This is a follow up patch to r349138.

This patch makes a `AddressSpaceView` a type declaration in the
allocator parameters used by `SizeClassAllocator64`. For ASan, LSan, and
the unit tests the AP64 declarations have been made templated so that
`AddressSpaceView` can be changed at compile time. For the other
sanitizers we just hard-code `LocalAddressSpaceView` because we have no
plans to use these allocators in an out-of-process manner.

rdar://problem/45284065

Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D55764

llvm-svn: 349954
2018-12-21 21:09:31 +00:00
David Carlier
c46751593b [Sanitizer] Move the unit test in the right place.
llvm-svn: 349917
2018-12-21 15:43:32 +00:00
David Carlier
7c21d95c9d [Sanitizer] Enable strtonum in FreeBSD
Reviewers: krytarowski, vitalybuka

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D55993

llvm-svn: 349916
2018-12-21 15:42:24 +00:00
Michal Gorny
1e8e4fc834 [xray] [tests] Detect and handle missing LLVMTestingSupport gracefully
Add a code to properly test for presence of LLVMTestingSupport library
when performing a stand-alone build, and skip tests requiring it when
it is not present.  Since the library is not installed, llvm-config
reported empty --libs for it and the tests failed to link with undefined
references.  Skipping the two fdr_* test files is better than failing to
build, and should be good enough until we find a better solution.

NB: both installing LLVMTestingSupport and building it automatically
from within compiler-rt sources are non-trivial.  The former due to
dependency on gtest, the latter due to tight integration with LLVM
source tree.

Differential Revision: https://reviews.llvm.org/D55891

llvm-svn: 349899
2018-12-21 13:37:30 +00:00
Dan Liew
88fe16c56d [CMake] Print out the list of sanitizers that the sanitizer_common tests will run against.
Summary:
This is a change requested by Vitaly Buka as prerequisite to landing
https://reviews.llvm.org/D55740.

Reviewers: vitalybuka, kubamracek

Subscribers: mgorny, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D55939

llvm-svn: 349897
2018-12-21 11:57:31 +00:00
Vitaly Buka
cdecca0324 Revert "[asan] Disable test on powerpc64be"
Now the test is passing on that bot. Some incremental build issues?

This reverts commit e00b5a5229ae02088d9f32a4e328eaa08abaf354.

llvm-svn: 349852
2018-12-20 23:25:26 +00:00
Vitaly Buka
5bcd4e8876 [asan] Disable test on powerpc64be
llvm-svn: 349844
2018-12-20 22:29:54 +00:00
Evgeniy Stepanov
3b7e8b2dbb [sanitizer] Support running without fd 0,1,2.
Summary:
Support running with no open file descriptors (as may happen to
"init" process on linux).
* Remove a check that writing to stderr succeeds.
* When opening a file (ex. for log_path option), dup the new fd out of
[0, 2] range to avoid confusing the program.

(2nd attempt, this time without the sanitizer_rtems change)

Reviewers: pcc, vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D55801

llvm-svn: 349817
2018-12-20 20:36:33 +00:00
David Carlier
22594ae962 [Sanitizer] Enable vis api on FreeBSD
Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D55923

llvm-svn: 349762
2018-12-20 14:25:43 +00:00
Ilya Biryukov
ea8646ad66 Revert "[sanitizer] Support running without fd 0,1,2."
This reverts commit r349699.
Reason: the commit breaks compilation of sanitizer_rtems.cc when
building for RTEMS.

llvm-svn: 349745
2018-12-20 12:50:03 +00:00
Vitaly Buka
5538115092 [asan] Revert still Androind incompatible tests enabled in r349736
llvm-svn: 349740
2018-12-20 10:56:34 +00:00
Vitaly Buka
cd51ffa290 [asan] Fix and re-enable few test on Android
llvm-svn: 349736
2018-12-20 10:15:51 +00:00
Eugene Leviant
d3bd614856 [HWASAN] Add support for memory intrinsics
This is patch complements D55117 implementing __hwasan_mem*
functions in runtime

Differential revision: https://reviews.llvm.org/D55554

llvm-svn: 349730
2018-12-20 09:10:03 +00:00
Evgeniy Stepanov
17e705129e Remove pointless casts.
llvm-svn: 349717
2018-12-20 01:49:21 +00:00
Vitaly Buka
a938b9dd3e [asan] Disable test incompatible with new Android
llvm-svn: 349705
2018-12-20 00:24:09 +00:00
Evgeniy Stepanov
f762a9f8f0 [sanitizer] Support running without fd 0,1,2.
Summary:
Support running with no open file descriptors (as may happen to
"init" process on linux).
* Remove a check that writing to stderr succeeds.
* When opening a file (ex. for log_path option), dup the new fd out of
[0, 2] range to avoid confusing the program.

Reviewers: pcc, vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D55801

llvm-svn: 349699
2018-12-19 23:45:17 +00:00
Kostya Kortchinsky
ebaba9df3a [sanitizer] Remove spurious semi-colon
Summary:
An extra ';' at the end of a namespace triggers a pedantic warning:
```
.../sanitizer_common/sanitizer_type_traits.h:42:2: warning: extra ‘;’ [-Wpedantic]
 };  // namespace __sanitizer
```

Reviewers: eugenis, delcypher

Reviewed By: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D55849

llvm-svn: 349666
2018-12-19 19:05:29 +00:00
Michal Gorny
0f71c17e7c [sanitizer_common] Fix sha2 interceptors not to use vars in array len
Fix the sha2 interceptor macros to use a constant for array parameter
length rather than referencing the extern variable.  Since the digest
length is provided in hash name, reuse the macro parameter for it.
Verify that the calculated value matches the one provided by system
headers.

Differential Revision: https://reviews.llvm.org/D55811

llvm-svn: 349645
2018-12-19 17:22:14 +00:00
Amy Kwan
6c735b02f1 [compiler-rt][builtins][PowerPC] Enable builtins tests on PowerPC 64 bit LE
This patch aims to enable the tests for the compiler-rt builtin functions (that
currently already exist within compiler-rt) for PowerPC 64bit LE (ppc64le).
Previously when unit tests are run, these tests would be reported as
UNSUPPORTED. This patch updates the REQUIRES line for each test (to enable for
ppc64le), and each test is linked against compiler-rt when running.

Differential Revision: https://reviews.llvm.org/D54449

llvm-svn: 349634
2018-12-19 15:52:41 +00:00
Kamil Rytarowski
342d03e71f Reimplement Thread Static Data ASan routines with TLS
Summary:
Thread Static Data cannot be used in early init on NetBSD
and FreeBSD. Reuse the ASan TSD API for compatibility with
existing code with an alternative implementation using Thread
Local Storage.

New version uses Thread Local Storage to store a pointer
with thread specific data. The destructor from TSD has been
replaced with a TLS destrucutor that is called upon thread
exit.

Reviewers: joerg, vitalybuka, jfb

Reviewed By: vitalybuka

Subscribers: dim, emaste, ro, jfb, devnexen, kubamracek, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55596

llvm-svn: 349619
2018-12-19 11:11:29 +00:00
Dmitry Vyukov
2ba80d6df6 tsan: align default value of detect_deadlocks flag with actual behavior
I tricked myself into thinking that deadlock detection is off by default in TSan by looking at the default value of the detect_deadlocks flag and outdated docs. (Created a pull request to update docs.)
I even managed to confuse others: https://groups.google.com/forum/#!topic/thread-sanitizer/xYvnAYwtoDk

However, the default value is overwritten in code (TSan_flags.cc:InitializeFlags). The TSan/deadlock tests also rely on this

This changes aligns the default value of the flag with the actual default behavior.

Author: yln (Julian Lettner)
Reviewed in: https://reviews.llvm.org/D55846

llvm-svn: 349609
2018-12-19 09:34:13 +00:00
Vitaly Buka
6471f72a63 [asan] Disable ODR test on Android
llvm-svn: 349585
2018-12-19 00:07:25 +00:00
Vitaly Buka
4e4920694c [asan] Restore ODR-violation detection on vtables
Summary:
unnamed_addr is still useful for detecting of ODR violations on vtables

Still unnamed_addr with lld and --icf=safe or --icf=all can trigger false
reports which can be avoided with --icf=none or by using private aliases
with -fsanitize-address-use-odr-indicator

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: kubamracek, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D55799

llvm-svn: 349555
2018-12-18 22:23:30 +00:00
Nico Weber
cfa54fb456 Fix a gcc -Wpedantix warning
llvm-svn: 349492
2018-12-18 15:17:01 +00:00
Peter Collingbourne
44ea4f5744 hwasan: Allow range of frame descriptors to be empty.
As of r349413 it's now possible for a binary to contain an empty
hwasan frame section. Handle that case simply by doing nothing.

Differential Revision: https://reviews.llvm.org/D55796

llvm-svn: 349428
2018-12-18 00:48:07 +00:00
Dan Liew
b85b15d64a Don't trigger sanitizer initialization from sysctlbyname and sysctl interceptor.
Summary:
This fixes the `ThreadSanitizer-x86_64-iossim` testsuite which broke
when r348770 (https://reviews.llvm.org/D55473) landed.

The root cause of the problem is that early-on during the iOS simulator
init process a call to `sysctlbyname` is issued. If the TSan initializer
is triggered at this point it will eventually trigger a call to
`__cxa_at_exit(...)`. This call then aborts because the library
implementing this function is not yet had its initialization function
called.

rdar://problem/46696934

Reviewers: kubamracek, george.karpenkov, devnexen, vitalybuka, krytarowski

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D55711

llvm-svn: 349402
2018-12-17 21:42:55 +00:00
David Carlier
34144f5ec0 [Sanitizer] capsicum variadic api subset
Reviewers: markj, vitalybuka

Reviewed By: markj

Differential Revision: https://reviews.llvm.org/D55714

llvm-svn: 349392
2018-12-17 20:57:06 +00:00
Kamil Rytarowski
ca72239534 Improve the comment in previous
llvm-svn: 349296
2018-12-16 10:24:06 +00:00
Kamil Rytarowski
bb66b0dac1 Expand TSan sysroot workaround to NetBSD
https://bugs.llvm.org/show_bug.cgi?id=26651

llvm-svn: 349295
2018-12-16 10:22:30 +00:00
Kamil Rytarowski
7970eccc89 Enable test/msan/pthread_getname_np.cc for NetBSD
llvm-svn: 349263
2018-12-15 10:42:14 +00:00
Kamil Rytarowski
17751e1414 Enable SANITIZER_INTERCEPT_PTHREAD_GETNAME_NP for NetBSD
llvm-svn: 349262
2018-12-15 10:41:55 +00:00
Kamil Rytarowski
5748bb4133 Fix internal_sleep() for NetBSD
This is a follow up of a similar fix for Linux from D55692.

llvm-svn: 349257
2018-12-15 08:25:25 +00:00
Peter Collingbourne
a53bb3ac85 Fix typo in test cases as well.
llvm-svn: 349255
2018-12-15 07:08:04 +00:00
Peter Collingbourne
1ad24332d1 hwasan: Fix typo: Previosly -> Previously.
llvm-svn: 349254
2018-12-15 07:06:24 +00:00
Kostya Serebryany
1879e8d3fc [libFuzzer] make len_control less aggressive
llvm-svn: 349210
2018-12-14 23:21:31 +00:00
Kamil Rytarowski
cea79772c5 Mark interception_failure_test.cc as passing for NetBSD and asan-dynamic-runtime
llvm-svn: 349159
2018-12-14 16:26:09 +00:00
Kamil Rytarowski
2ae9783b4f Set shared_libasan_path in lit tests for NetBSD
Reuse the Linux code path.

llvm-svn: 349156
2018-12-14 15:58:05 +00:00
Dan Liew
41fec1bfc5 Introduce AddressSpaceView template parameter to SizeClassAllocator32, FlatByteMap, and TwoLevelByteMap.
Summary:
This is a follow up patch to r346956 for the `SizeClassAllocator32`
allocator.

This patch makes `AddressSpaceView` a template parameter both to the
`ByteMap` implementations (but makes `LocalAddressSpaceView` the
default), some `AP32` implementations and is used in `SizeClassAllocator32`.
The actual changes to `ByteMap` implementations and
`SizeClassAllocator32` are very simple. However the patch is large
because it requires changing all the `AP32` definitions, and users of
those definitions.

For ASan and LSan we make `AP32` and `ByteMap` templateds type that take
a single `AddressSpaceView` argument. This has been done because we will
instantiate the allocator with a type that isn't `LocalAddressSpaceView`
in the future patches. For the allocators used in the other sanitizers
(i.e. HWAsan, MSan, Scudo, and TSan) use of `LocalAddressSpaceView` is
hard coded because we do not intend to instantiate the allocators with
any other type.

In the cases where untemplated types have become templated on a single
`AddressSpaceView` parameter (e.g. `PrimaryAllocator`) their name has
been changed to have a `ASVT` suffix (Address Space View Type) to
indicate they are templated.  The only exception to this are the `AP32`
types due to the desire to keep the type name as short as possible.

In order to check that template is instantiated in the correct a way a
`static_assert(...)` has been added that checks that the
`AddressSpaceView` type used by `Params::ByteMap::AddressSpaceView` matches
the `Params::AddressSpaceView`. This uses the new `sanitizer_type_traits.h`
header.

rdar://problem/45284065

Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov

Subscribers: mgorny, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D54904

llvm-svn: 349138
2018-12-14 09:03:18 +00:00
Fangrui Song
c4e6de2e3f [sanitizer] Fix nolibc internal_sleep
Reviewers: kubamracek, vitalybuka

Reviewed By: vitalybuka

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D55692

llvm-svn: 349134
2018-12-14 08:09:43 +00:00
Vlad Tsyrklevich
137e23d536 Windows ASan: Instrument _msize_base()
Summary:
A recent update to the VS toolchain in chromium [1] broke the windows
ASan bot because the new toolchain calls _msize_base() instead of
_msize() in a number of _aligned_* UCRT routines. Instrument
_msize_base() as well.

[1] https://crbug.com/914947

Reviewers: rnk, #sanitizers, vitalybuka

Reviewed By: rnk, #sanitizers, vitalybuka

Subscribers: vitalybuka, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D55684

llvm-svn: 349115
2018-12-14 00:39:16 +00:00
Evgeniy Stepanov
e6acf2c3b4 Revert "Switch Android from TLS_SLOT_TSAN(8) to TLS_SLOT_SANITIZER(6)"
Breaks sanitizer-android buildbot.

This reverts commit 85e02baff327e7b67ea5b47897302901abb2aa5d.

llvm-svn: 349093
2018-12-13 23:47:59 +00:00
Dan Liew
c2e2dd43f0 Implement a small subset of the C++ type_traits header inside sanitizer_common so we can avoid depending on system C++ headers.
Summary:
In particular we implement the `is_same<T,U>` templated type. This is
useful for doing compile-time comparison of types in `static_assert`s.
The plan is to use this in another patch (
https://reviews.llvm.org/D54904 ).

Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov

Subscribers: mgorny, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D54951

llvm-svn: 349077
2018-12-13 19:55:36 +00:00
David Carlier
a4ee854259 [Sanitizer] capsicum further support of the API
Reviewers: vitalybuka, krytarowski, emaste

Reviewed By: emaste

Differential Revision: https://reviews.llvm.org/D55622

llvm-svn: 349042
2018-12-13 15:05:24 +00:00
Kamil Rytarowski
0bc12879fc Add a new interceptors for cdbr(3) and cdbw(3) API from NetBSD
Summary:
cdb - formats of the constant database.

cdbr, cdbr_open, cdbr_open_mem, cdbr_entries, cdbr_get, cdbr_find,
cdbr_close - constant database access methods.

cdbw_open, cdbw_put, cdbw_put_data, cdbw_put_key, cdbw_stable_seeder,
cdbw_output, cdbw_close - creates constant databases.

Add a dedicated test for this API.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55167

llvm-svn: 349021
2018-12-13 10:19:00 +00:00
Kamil Rytarowski
ce86b919da Add new interceptors for vis(3) API in NetBSD
Summary:
Add interceptors for the NetBSD style of vis(3) present inside libc:

 - vis
 - nvis
 - strvis
 - stravis
 - strnvis
 - strvisx
 - strnvisx
 - strenvisx
 - svis
 - snvis
 - strsvis
 - strsnvis
 - strsvisx
 - strsnvisx
 - strsenvisx
 - unvis
 - strunvis
 - strnunvis
 - strunvisx
 - strnunvisx

Add a dedicated test verifying the installed interceptors.

Based on original work by Yang Zheng.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54594

llvm-svn: 349018
2018-12-13 10:14:01 +00:00
Vitaly Buka
a257639a69 [asan] Don't check ODR violations for particular types of globals
Summary:
private and internal: should not trigger ODR at all.
unnamed_addr: current ODR checking approach fail and rereport false violation if
a linker merges such globals
linkonce_odr, weak_odr: could cause similar problems and they are already not
instrumented for ELF.

Reviewers: eugenis, kcc

Subscribers: kubamracek, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D55621

llvm-svn: 349015
2018-12-13 09:47:39 +00:00
Evgeniy Stepanov
1020085898 [hwasan] Link ubsan_cxx to shared runtime library.
Summary: This is needed for C++-specific ubsan and cfi error reporting to work.

Reviewers: kcc, vitalybuka

Subscribers: srhines, kubamracek, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D55589

llvm-svn: 348986
2018-12-12 22:56:00 +00:00
Ryan Prichard
b7e1cae526 Switch Android from TLS_SLOT_TSAN(8) to TLS_SLOT_SANITIZER(6)
Summary:
The TLS_SLOT_TSAN slot is available starting in N, but its location (8)
is incompatible with the proposed solution for implementing ELF TLS on
Android (i.e. bump ARM/AArch64 alignment to reserve an 8-word TCB).

Instead, starting in Q, Bionic replaced TLS_SLOT_DLERROR(6) with
TLS_SLOT_SANITIZER(6). Switch compiler-rt to the new slot.

Reviewers: eugenis, srhines, enh

Reviewed By: eugenis

Subscribers: ruiu, srhines, kubamracek, javed.absar, kristof.beyls, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D55581

llvm-svn: 348984
2018-12-12 22:45:12 +00:00
Evgeniy Stepanov
0af6e5facd [hwasan] Verify Android TLS slot at startup.
Summary:
Add a check that TLS_SLOT_TSAN / TLS_SLOT_SANITIZER, whichever
android_get_tls_slot is using, is not conflicting with
TLS_SLOT_DLERROR.

Reviewers: rprichard, vitalybuka

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D55587

llvm-svn: 348979
2018-12-12 22:10:52 +00:00
David Carlier
8ec89e6b16 [Sanitizer] Expand FSEEK interception to FreeBSD
Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D55556

llvm-svn: 348888
2018-12-11 19:08:40 +00:00
Dan Liew
a94b00513f Fix bug where we'd try symbolize a second time with the same arguments.
Summary:
Fix bug where we'd try symbolize a second time with the same arguments even though symbolization failed the first time.

This looks like a long standing typo given that the guard for trying
symbolization again is to only try it if symbolization failed using
`binary` and `original_binary != binary`.

Reviewers: kubamracek, glider, samsonov

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D55504

llvm-svn: 348841
2018-12-11 12:43:44 +00:00
Yi Kong
a140d5269e [builtins] Remove trailing whitespaces, NFC
Remove trailing whitespaces so that it is easier to diff the code between
div{s,d,t}f3.c

llvm-svn: 348807
2018-12-10 22:52:59 +00:00
David Carlier
2b26a98a0d [Sanitizer] expand sysctl/getmntinfo/nl_langinfo to Darwin
Reviewers: vitalybuka, krytarowski, kubamracek

Reviewed By: vitalybuka, krytarowski

Differential Revision: https://reviews.llvm.org/D55473

llvm-svn: 348770
2018-12-10 16:29:30 +00:00
Kamil Rytarowski
2eb1f5432f Add data types needed for md2(3)/NetBSD interceptors
Missing part of D55469.

llvm-svn: 348747
2018-12-10 09:09:18 +00:00
Kamil Rytarowski
50bd2ec198 Add interceptors for the sha2(3) from NetBSD
Summary:
SHA224_Init, SHA224_Update, SHA224_Final, SHA224_End, SHA224_File,
SHA224_FileChunk, SHA224_Data, SHA256_Init, SHA256_Update, SHA256_Final,
SHA256_End, SHA256_File, SHA256_FileChunk, SHA256_Data, SHA384_Init,
SHA384_Update, SHA384_Final, SHA384_End, SHA384_File, SHA384_FileChunk,
SHA384_Data, SHA512_Init, SHA512_Update, SHA512_Final, SHA512_End,
SHA512_File, SHA512_FileChunk, SHA512_Data – calculates the NIST Secure
Hash Standard (version 2)

Add tests for new interceptors.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54987

llvm-svn: 348745
2018-12-10 09:06:56 +00:00
Kamil Rytarowski
e7971f1bce Add interceptors for md2(3) from NetBSD
Summary:
MD2Init, MD2Update, MD2Final, MD2End, MD2File, MD2Data - calculates the
RSA Data Security, Inc., "MD2" message digest.

Add a dedicated test.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55469

llvm-svn: 348744
2018-12-10 09:01:00 +00:00
Kamil Rytarowski
14d36e6e75 Add new interceptors for FILE repositioning stream
Summary:
Add new interceptors for a set of functions to reposition a stream:
fgetpos, fseek, fseeko, fsetpos, ftell, ftello, rewind .

Add a dedicated test.

Enable this interface on NetBSD.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55471

llvm-svn: 348743
2018-12-10 08:56:14 +00:00
David Carlier
83b0754586 Fix conflict types for this FreeBSD test.
llvm-svn: 348707
2018-12-08 16:29:50 +00:00
Kamil Rytarowski
e1f966508f Fix a typo in the strtoi test
https://reviews.llvm.org/D54702

llvm-svn: 348683
2018-12-08 04:46:15 +00:00
Kamil Rytarowski
3ac7a15163 Revert a chunk of previous change in sanitizer_platform_limits_netbsd.h
Undefining INLINE breaks the build.
The invalid change in this file has been overlooked in D55386.

llvm-svn: 348680
2018-12-08 02:47:12 +00:00
Kamil Rytarowski
2e11b93f35 Add interceptors for md5(3) from NetBSD
Summary:
MD5Init, MD5Update, MD5Final, MD5End, MD5File, MD5Data - calculates the
RSA Data Security, Inc., "MD5" message digest.

Add a dedicated test.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54993

llvm-svn: 348679
2018-12-08 01:50:18 +00:00
Kamil Rytarowski
f9b8569d72 Add interceptors for the rmd160(3) from NetBSD
Summary:
RMD160Init, RMD160Update, RMD160Final, RMD160Transform, RMD160End,
RMD160File, RMD160Data - calculates the ``RIPEMD-160'' message digest.

Add a dedicated test for this API.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54991

llvm-svn: 348678
2018-12-08 01:47:29 +00:00
Kamil Rytarowski
e9bf12f82f Add interceptors for the md4(3) from NetBSD
Summary:
MD4Init, MD4Update, MD4Final, MD4End, MD4File, MD4Data - calculates the
RSA Data Security, Inc., "MD4" message digest.

Add dedicated test.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54990

llvm-svn: 348677
2018-12-08 01:43:39 +00:00
Kamil Rytarowski
3f47a6fbd7 Add interceptors for the sha1(3) from NetBSD
Summary:
Add interceptors for:

 - SHA1Init
 - SHA1Update
 - SHA1Final
 - SHA1Transform
 - SHA1End
 - SHA1File
 - SHA1FileChunk
 - SHA1Data

Add a dedicated regression test for this API.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: mgorny, llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54927

llvm-svn: 348676
2018-12-08 01:39:47 +00:00
Vitaly Buka
4ce6d4611b [sanitizer] Add lit.local.cfg for FreeBSD
llvm-svn: 348674
2018-12-08 01:00:45 +00:00
Vitaly Buka
4e23840e30 [sanitizer] Suppress lint warning conflicting with clang-format
llvm-svn: 348673
2018-12-08 01:00:37 +00:00
David Carlier
aaa95cbeea Fix style.
llvm-svn: 348672
2018-12-08 00:44:38 +00:00
David Carlier
cc3be702b0 Conflict fixes from previous commits.
llvm-svn: 348669
2018-12-08 00:21:40 +00:00
David Carlier
a0d0202d89 [Sanitizer] capsicum api subset interception
- For the moment a subset of this api dealing with file descriptors permissions and ioctls.

Reviewers: vitalybuka, krytarowski

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D55368

llvm-svn: 348668
2018-12-08 00:14:04 +00:00
Kamil Rytarowski
ae3ae31e9c Add interceptors for the strtoi(3)/strtou(3) from NetBSD
Summary:
strtoi/strtou converts string value to an intmax_t/uintmax_t integer.

Add a dedicated test.

Enable this API for NetBSD.

It's a reworked version of the original work by Yang Zheng.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, tomsun.0.7, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54702

llvm-svn: 348663
2018-12-07 22:24:35 +00:00
Dan Liew
5ab96bf57c Fix IOError exception being raised in asan_symbolize.pycrash when
using `atos` symbolizer on Darwin when the binaries don't exist.

For now we just produce an unsymbolicated stackframe when the binary
doesn't exist.

llvm-svn: 348659
2018-12-07 22:14:20 +00:00
Kamil Rytarowski
2f5fd174c9 Add a new interceptors for statvfs1(2) and fstatvfs1(2) from NetBSD
Summary:
statvfs1, fstatvfs1 - get file system statistics.

While there, use file descriptor related macros in the fstatvfs interceptor.

Add a dedicated test.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: dvyukov, kubamracek, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55031

llvm-svn: 348656
2018-12-07 22:01:16 +00:00
Kamil Rytarowski
592898b306 Add a new interceptor for fparseln(3) from NetBSD
Summary:
fparseln - returns the next logical line from a stream.

Add a dedicated test for this API.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55019

llvm-svn: 348654
2018-12-07 21:50:44 +00:00
Kamil Rytarowski
0fed92a933 Add new interceptor for strtonum(3)
Summary:
strtonum(3) reliably convertss string value to an integer.
This function is used in OpenBSD compat namespace
and is located inside NetBSD's libc.

Add a dedicated test for this interface.

It's a reworked version of the original code by Yang Zheng.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54527

llvm-svn: 348651
2018-12-07 21:47:36 +00:00
David Carlier
a742193309 Missing freebsd files.
A    lib/sanitizer_common/sanitizer_platform_limits_freebsd.cc
A    lib/sanitizer_common/sanitizer_platform_limits_freebsd.h

llvm-svn: 348635
2018-12-07 20:07:49 +00:00
David Carlier
da2a653134 [Sanitizer] Separate FreeBSD interception data structures
Reviewers: vitalybuka, krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D55386

llvm-svn: 348634
2018-12-07 20:05:55 +00:00
Dean Michael Berris
25d505953a [XRay] Use preallocated memory for XRay profiling
Summary:
This change builds upon D54989, which removes memory allocation from the
critical path of the profiling implementation. This also changes the API
for the profile collection service, to take ownership of the memory and
associated data structures per-thread.

The consolidation of the memory allocation allows us to do two things:

- Limits the amount of memory used by the profiling implementation,
  associating preallocated buffers instead of allocating memory
  on-demand.

- Consolidate the memory initialisation and cleanup by relying on the
  buffer queue's reference counting implementation.

We find a number of places which also display some problematic
behaviour, including:

- Off-by-factor bug in the allocator implementation.

- Unrolling semantics in cases of "memory exhausted" situations, when
  managing the state of the function call trie.

We also add a few test cases which verify our understanding of the
behaviour of the system, with important edge-cases (especially for
memory-exhausted cases) in the segmented array and profile collector
unit tests.

Depends on D54989.

Reviewers: mboerger

Subscribers: dschuff, mgorny, dmgreen, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D55249

llvm-svn: 348568
2018-12-07 06:23:06 +00:00
Dean Michael Berris
190c49bc8f Re-land "[XRay] Move-only Allocator, FunctionCallTrie, and Array"
This reverts commit r348455, with some additional changes:

- Work-around deficiency of gcc-4.8 by duplicating the implementation of
  `AppendEmplace` in `Append`, but instead of using brace-init for the
  copy construction, use a placement new explicitly calling the copy
  constructor.

llvm-svn: 348563
2018-12-07 03:19:13 +00:00
Michal Gorny
ef4b600301 [test] Add missing cmake include for building libFuzzer alone
Include CompilerRTCompile in fuzzer tests explicitly.  Otherwise, when
building only libFuzzer, CMake fails due to:

CMake Error at cmake/Modules/AddCompilerRT.cmake:395 (sanitizer_test_compile):
  Unknown CMake command "sanitizer_test_compile".
Call Stack (most recent call first):
  lib/fuzzer/tests/CMakeLists.txt:53 (generate_compiler_rt_tests)

Differential Revision: https://reviews.llvm.org/D55378

llvm-svn: 348524
2018-12-06 20:04:08 +00:00
David Carlier
51e820d0d8 [Sanitizer] getmntinfo support in FreeBSD
Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D55354

llvm-svn: 348500
2018-12-06 17:04:18 +00:00
Dan Liew
f101eb101d Add new __sanitizer_mz_default_zone() API which returns the address of the ASan malloc zone. This API will be used for testing in future patches.
Summary:
The name of the function is based on `malloc_default_zone()` found
in Darwin's `malloc/malloc.h` header file.

Reviewers: kubamracek, george.karpenkov

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D55064

llvm-svn: 348481
2018-12-06 12:39:00 +00:00
Dean Michael Berris
82f7b21f17 Revert "[XRay] Move-only Allocator, FunctionCallTrie, and Array"
This reverts commits r348438, r348445, and r348449 due to breakages with
gcc-4.8 builds.

llvm-svn: 348455
2018-12-06 03:28:57 +00:00
Dean Michael Berris
889d20715e [XRay] Use a local lvalue as arg to AppendEmplace(...)
This is a follow-up to D54989.

Further work-around gcc-4.8 failing to handle brace-init with temporaries.

llvm-svn: 348449
2018-12-06 02:55:47 +00:00
Dean Michael Berris
400afa0230 [XRay] Use default-constructed struct as argument to Append(...)
This is a follow-up to D54989.

Work-around gcc-4.8 failing to handle brace-init for structs to imply
default-construction of an aggregate, and treats it as an initialiser
list instead.

llvm-svn: 348445
2018-12-06 01:56:27 +00:00
Dean Michael Berris
cb447a2604 Re-land r348335 "[XRay] Move-only Allocator, FunctionCallTrie, and Array"
Continuation of D54989.

Additional changes:

  - Use `.AppendEmplace(...)` instead of `.Append(Type{...})` to appease
    GCC 4.8 with confusion on when an initializer_list is used as
    opposed to a temporary aggregate initialized object.

llvm-svn: 348438
2018-12-06 00:25:56 +00:00
Kamil Rytarowski
04d8dca7ab Remove XFAIL in get_module_and_offset_for_pc.cc for NetBSD-MSan
After updating GET_LINK_MAP_BY_DLOPEN_HANDLE() for recent NetBSD
this test no longer fails.

llvm-svn: 348378
2018-12-05 16:05:25 +00:00
David Carlier
82494cb8cf [Sanitizer] nl_langinfo forgotten bit.
M    lib/sanitizer_common/sanitizer_platform_interceptors.h

llvm-svn: 348377
2018-12-05 16:02:26 +00:00
David Carlier
f3233b2ff2 [Sanitizer] expand nl_langinfo interception to FreeBSD
Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D55327

llvm-svn: 348376
2018-12-05 16:01:22 +00:00
Kamil Rytarowski
efadb532bb Add a new interceptor for modctl(2) from NetBSD
Summary:
modctl - controls loadable kernel modules.

Skip tests as this call uses privileged operations.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55025

llvm-svn: 348370
2018-12-05 15:13:20 +00:00
Kamil Rytarowski
fab764359f Add a new interceptor for nl_langinfo(3) from NetBSD
Summary:
nl_langinfo - gets locale information.

Add a dedicated test.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55015

llvm-svn: 348369
2018-12-05 15:06:53 +00:00
Hans Wennborg
83ff22c297 Revert r348335 "[XRay] Move-only Allocator, FunctionCallTrie, and Array"
.. and also the follow-ups r348336 r348338.

It broke stand-alone compiler-rt builds with GCC 4.8:

In file included from /work/llvm/projects/compiler-rt/lib/xray/xray_function_call_trie.h:20:0,
                 from /work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.h:21,
                 from /work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:15:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget&}; T = __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71:   required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget]’
/work/llvm/projects/compiler-rt/lib/xray/xray_function_call_trie.h:517:54:   required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget&>((* & args#0))}’ from ‘<brace-enclosed initializer list>’ to ‘__xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget’
     new (AlignedOffset) T{std::forward<Args>(args)...};
     ^
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::profileCollectorService::{anonymous}::ThreadTrie&}; T = __xray::profileCollectorService::{anonymous}::ThreadTrie]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71:   required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::profileCollectorService::{anonymous}::ThreadTrie]’
/work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:98:34:   required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::profileCollectorService::{anonymous}::ThreadTrie&>((* & args#0))}’ from
‘<brace-enclosed initializer list>’ to ‘__xray::profileCollectorService::{anonymous}::ThreadTrie’
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::profileCollectorService::{anonymous}::ProfileBuffer&}; T = __xray::profileCollectorService::{anonymous}::ProfileBuffer]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71:   required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::profileCollectorService::{anonymous}::ProfileBuffer]
’
/work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:244:44:   required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::profileCollectorService::{anonymous}::ProfileBuffer&>((* & args#0))}’ from ‘<brace-enclosed initializer list>’ to ‘__xray::profileCollectorService::{anonymous}::ProfileBuffer’

> Summary:
> This change makes the allocator and function call trie implementations
> move-aware and remove the FunctionCallTrie's reliance on a
> heap-allocated set of allocators.
>
> The change makes it possible to always have storage associated with
> Allocator instances, not necessarily having heap-allocated memory
> obtainable from these allocator instances. We also use thread-local
> uninitialised storage.
>
> We've also re-worked the segmented array implementation to have more
> precondition and post-condition checks when built in debug mode. This
> enables us to better implement some of the operations with surrounding
> documentation as well. The `trim` algorithm now has more documentation
> on the implementation, reducing the requirement to handle special
> conditions, and being more rigorous on the computations involved.
>
> In this change we also introduce an initialisation guard, through which
> we prevent an initialisation operation from racing with a cleanup
> operation.
>
> We also ensure that the ThreadTries array is not destroyed while copies
> into the elements are still being performed by other threads submitting
> profiles.
>
> Note that this change still has an issue with accessing thread-local
> storage from signal handlers that are instrumented with XRay. We also
> learn that with the testing of this patch, that there will be cases
> where calls to mmap(...) (through internal_mmap(...)) might be called in
> signal handlers, but are not async-signal-safe. Subsequent patches will
> address this, by re-using the `BufferQueue` type used in the FDR mode
> implementation for pre-allocated memory segments per active, tracing
> thread.
>
> We still want to land this change despite the known issues, with fixes
> forthcoming.
>
> Reviewers: mboerger, jfb
>
> Subscribers: jfb, llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D54989

llvm-svn: 348346
2018-12-05 10:19:55 +00:00
Dean Michael Berris
9e4b7efa0e [XRay] Use uptr instead of uintptr_t
Follow-up to D54989.

llvm-svn: 348338
2018-12-05 07:14:06 +00:00
Dean Michael Berris
d49fc9c6fa [XRay] Use deallocateBuffer instead of deallocate
Follow-up to D54989.

llvm-svn: 348336
2018-12-05 07:05:44 +00:00
Dean Michael Berris
adc880467d [XRay] Move-only Allocator, FunctionCallTrie, and Array
Summary:
This change makes the allocator and function call trie implementations
move-aware and remove the FunctionCallTrie's reliance on a
heap-allocated set of allocators.

The change makes it possible to always have storage associated with
Allocator instances, not necessarily having heap-allocated memory
obtainable from these allocator instances. We also use thread-local
uninitialised storage.

We've also re-worked the segmented array implementation to have more
precondition and post-condition checks when built in debug mode. This
enables us to better implement some of the operations with surrounding
documentation as well. The `trim` algorithm now has more documentation
on the implementation, reducing the requirement to handle special
conditions, and being more rigorous on the computations involved.

In this change we also introduce an initialisation guard, through which
we prevent an initialisation operation from racing with a cleanup
operation.

We also ensure that the ThreadTries array is not destroyed while copies
into the elements are still being performed by other threads submitting
profiles.

Note that this change still has an issue with accessing thread-local
storage from signal handlers that are instrumented with XRay. We also
learn that with the testing of this patch, that there will be cases
where calls to mmap(...) (through internal_mmap(...)) might be called in
signal handlers, but are not async-signal-safe. Subsequent patches will
address this, by re-using the `BufferQueue` type used in the FDR mode
implementation for pre-allocated memory segments per active, tracing
thread.

We still want to land this change despite the known issues, with fixes
forthcoming.

Reviewers: mboerger, jfb

Subscribers: jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D54989

llvm-svn: 348335
2018-12-05 06:44:34 +00:00
Kamil Rytarowski
b9601a385a Update GET_LINK_MAP_BY_DLOPEN_HANDLE() for NetBSD x86
NetBSD 8.99.26 changed the layout of internal structure
returned by dlopen(3), switch to it.

Set new values for amd64 and i386 based on the results
of &((struct Struct_Obj_Entry*)0)->linkmap.

llvm-svn: 348329
2018-12-05 03:17:21 +00:00
Vitaly Buka
8076c57fd2 [asan] Add clang flag -fsanitize-address-use-odr-indicator
Reviewers: eugenis, m.ostapenko, ygribov

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D55157

llvm-svn: 348327
2018-12-05 01:44:31 +00:00
Vitaly Buka
d6bab09b4b [asan] Split -asan-use-private-alias to -asan-use-odr-indicator
Reviewers: eugenis, m.ostapenko, ygribov

Subscribers: mehdi_amini, kubamracek, hiraditya, steven_wu, dexonsmith, llvm-commits

Differential Revision: https://reviews.llvm.org/D55156

llvm-svn: 348316
2018-12-04 23:17:41 +00:00
Vitaly Buka
10db3f4779 [asan] Remove use_odr_indicator runtime flag
Summary:
Flag was added for testing 3 years ago. Probably it's time
to simplify code and usage by removing it.

Reviewers: eugenis, m.ostapenko

Subscribers: mehdi_amini, kubamracek, steven_wu, dexonsmith, llvm-commits

Differential Revision: https://reviews.llvm.org/D55254

llvm-svn: 348315
2018-12-04 23:17:32 +00:00
David Carlier
f3c8a00760 Fix sanitizer unit test
llvm-svn: 348297
2018-12-04 19:49:19 +00:00
David Carlier
eb5bfac7ce Unbreak build due to style.
llvm-svn: 348295
2018-12-04 19:17:26 +00:00
David Carlier
2330a24ca5 [Sanitizer] intercept part of sysctl Api
- Distringuish what FreeBSD/NetBSD can and NetBSD specifics.
- Fixing page size value collection.

Reviewers: krytarowski, vitalybuka

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D55265	

llvm-svn: 348293
2018-12-04 19:00:38 +00:00
Dan Liew
f73b782105 [SanitizerCommon] Test CombinedAllocator::ForEachChunk() in unit tests.
Summary:

Previously we weren't testing this function in the unit tests.

Reviewers: kcc, cryptoad, dvyukov, eugenis, kubamracek

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D54861

llvm-svn: 348260
2018-12-04 14:03:55 +00:00
Petr Hosek
5c43abcf2d [compiler-rt] Use the new zx_futex_wait for Fuchsia sanitizer runtime
This finishes the soft-transition to the new primitive that implements
priority inheritance.

Differential Revision: https://reviews.llvm.org/D55244

llvm-svn: 348236
2018-12-04 04:07:43 +00:00
Kamil Rytarowski
9d62f4db78 Improve the regerror(3) interceptor
The res returned value might differ with REAL(strlen)(errbuf) + 1,
as the buffer's value is limited with errbuf_size.

Hot fix for D54584.

llvm-svn: 348231
2018-12-04 02:18:18 +00:00
Kamil Rytarowski
5b8d585925 Add interceptors for the sysctl(3) API family from NetBSD
Summary:
Add new interceptors for:

 - sysctl
 - sysctlbyname
 - sysctlgetmibinfo
 - sysctlnametomib
 - asysctl
 - asysctlbyname

Cover the API with a new test file TestCases/NetBSD/sysctl.cc.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: devnexen, kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54076

llvm-svn: 348228
2018-12-04 01:51:06 +00:00
Kamil Rytarowski
175d2b8620 Add interceptors for the fts(3) API family from NetBSD
Summary:
fts(3) is API to traverse a file hierarchy.
Cover this interface with interceptors.

Add a test to validate the interface reading
the number of regular files in /etc.

Based on original work by Yang Zheng.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54247

llvm-svn: 348227
2018-12-04 01:45:52 +00:00
Kamil Rytarowski
145ac7940e Add new interceptor for regex(3) in NetBSD
Summary:
Add interceptors for the NetBSD style of regex(3) present inside libc:

 - regcomp
 - regexec
 - regerror
 - regfree
 - regnsub
 - regasub

Add a dedicated test verifying the installed interceptors.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54584

llvm-svn: 348224
2018-12-04 01:41:42 +00:00
Vedant Kumar
6b062cd694 [gcov/Darwin] Ensure external symbols are exported when using an export list
Make sure that symbols needed to implement runtime support for gcov are
exported when using an export list on Darwin.

Without the clang driver exporting these symbols, the linker hides them,
resulting in tapi verification failures.

rdar://45944768

Differential Revision: https://reviews.llvm.org/D55151

llvm-svn: 348187
2018-12-03 20:53:58 +00:00
Eugene Leviant
a50c05872e [SanitizerCommon] Remove RenameFile
This function seems to be no longer used by compiler-rt
libraries

Differential revision: https://reviews.llvm.org/D55123

llvm-svn: 348140
2018-12-03 14:03:51 +00:00
Dan Liew
8bffb63497 Introduce a way to allow the ASan dylib on Darwin platforms to be loaded via dlopen().
Summary:

The purpose of this option is provide a way for the ASan dylib
to be loaded via `dlopen()` without triggering most initialization
steps (e.g. shadow memory set up) that normally occur when the
ASan dylib is loaded.

This new functionality is exposed by

- A `SANITIZER_SUPPORTS_INIT_FOR_DLOPEN` macro which indicates if the
  feature is supported. This only true for Darwin currently.
- A `HandleDlopenInit()` function which should return true if the library
  is being loaded via `dlopen()` and
  `SANITIZER_SUPPORTS_INIT_FOR_DLOPEN` is supported. Platforms that
  support this may perform any initialization they wish inside this
  function.

Although disabling initialization is something that could potentially
apply to other sanitizers it appears to be unnecessary for other
sanitizers so this patch only makes the change for ASan.

rdar://problem/45284065

Reviewers: kubamracek, george.karpenkov, kcc, eugenis, krytarowski

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D54469

llvm-svn: 348078
2018-12-01 15:45:42 +00:00
Vitaly Buka
a7b079534a [compiler-rt] Use "ColumnLimit: 0" instead of "clang-format off" in tests
Reviewers: eugenis, jfb

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D55152

llvm-svn: 348061
2018-12-01 01:24:29 +00:00
Kamil Rytarowski
f130d111b6 Add a new interceptor for getvfsstat(2) from NetBSD
Summary:
getvfsstat - gets list of all mounted file systems.

Add a dedicated test.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55014

llvm-svn: 348027
2018-11-30 19:43:53 +00:00
Reid Kleckner
63f084bd7a Revert r346560 "[winasan] Unpoison the stack in NtTerminateThread"
This reverts r343606 again. The NtTerminateThread interceptor is causing
problems in NaCl:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/CrWinAsan/1839

I reproduced the problem locally and tried my best to debug them, but
it's beyond me.

llvm-svn: 347933
2018-11-29 23:57:17 +00:00
Julian Lettner
647782c78b (no commit message)
llvm-svn: 347788
2018-11-28 18:17:23 +00:00
Matt Morehouse
32c57553c3 [gcov] Disable instrprof-gcov-fork.test.
Test has been flaky for over a week and author hasn't fixed.

llvm-svn: 347779
2018-11-28 17:24:07 +00:00
Martin Storsjo
fec4f351d5 [CMake] Add a missing case of TO_CMAKE_PATH
This fixes building sanitizers for mingw natively.

llvm-svn: 347646
2018-11-27 09:23:15 +00:00
Kuba Mracek
265cd31fbf Fix filtering of sanitizer_common unittest architectures on Darwin.
llvm-svn: 347622
2018-11-27 01:37:08 +00:00
Benjamin Kramer
a2ada4d1ce [X86][compiler-rt] Add missing semicolon
llvm-svn: 347519
2018-11-24 20:57:03 +00:00
Craig Topper
428caa398c [X86] Make conversion of feature bits into a mask explicitly unsigned by using 1U instead of 1.
llvm-svn: 347517
2018-11-24 20:25:45 +00:00
Craig Topper
62e5924dd6 [X86][compiler-rt] Attempt to fix a warning about a shift amount being negative in a macro expansion.
llvm-svn: 347516
2018-11-24 20:14:03 +00:00
David Carlier
0a01f5d244 Unbreak FreeBSD build.
M    lib/sanitizer_common/sanitizer_platform_limits_posix.cc

llvm-svn: 347451
2018-11-22 10:31:01 +00:00
Petr Hosek
e7dec7848b [XRay] Support for Fuchsia
This extends XRay to support Fuchsia.

Differential Revision: https://reviews.llvm.org/D52162

llvm-svn: 347443
2018-11-22 02:00:44 +00:00
Peter Collingbourne
56f3bb4b36 tsan: Update measurements in check_analyze.sh.
These changed as a result of r347379. Unfortunately there was a
regression; filed PR39748 to track it.

Differential Revision: https://reviews.llvm.org/D54821

llvm-svn: 347442
2018-11-22 00:47:37 +00:00
Peter Collingbourne
c56109306b tsan: Correct the name of an executable.
llvm-svn: 347439
2018-11-22 00:00:26 +00:00
David Carlier
0c81a62d9d [Sanitizer] Adding setvbuf in supported platforms and other stream buffer functions
- Enabling setvbuf interceptions for non NetBSD platforms.
- setbuf, setbuffer, setlinebuf as well.

Reviewers: vitalybuka, krytarowski	

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D54779

llvm-svn: 347426
2018-11-21 21:17:46 +00:00
Roman Lebedev
1d0c7f563c [compiler-rt][UBSan] silence_unsigned_overflow: do *NOT* ignore *fatal* unsigned overflows
Summary:
D48660 / rL335762 added a `silence_unsigned_overflow` env flag for [[ https://github.com/google/oss-fuzz/pull/1717 | oss-fuzz needs ]],
that allows to silence the reports from unsigned overflows.
It makes sense, it is there because `-fsanitize=integer` sanitizer is not enabled on oss-fuzz,
so this allows to still use it as an interestingness signal, without getting the actual reports.

However there is a slight problem here.
All types of unsigned overflows are ignored.
Even if `-fno-sanitize-recover=unsigned` was used (which means the program will die after the report)
there will still be no report, the program will just silently die.

At the moment there are just two projects on oss-fuzz that care:
* [[ 8eeffa627f/projects/llvm_libcxx/build.sh (L18-L20) | libc++ ]]
* [[ 8eeffa627f/projects/librawspeed/build.sh | RawSpeed ]] (me)

I suppose this could be overridden there ^, but i really don't think this is intended behavior in any case..

Reviewers: kcc, Dor1s, #sanitizers, filcab, vsk, kubamracek

Reviewed By: Dor1s

Subscribers: dberris, mclow.lists, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54771

llvm-svn: 347415
2018-11-21 20:35:43 +00:00
Dmitry Vyukov
d0fb5d8b00 tsan: add pthread_tryjoin_np and pthread_timedjoin_np interceptors
Add pthread_tryjoin_np() and pthread_timedjoin_np() interceptors on Linux,
so that ThreadSanitizer can handle programs using these functions.

Author: Yuri Per (yuri)
Reviewed in: https://reviews.llvm.org/D54521

llvm-svn: 347383
2018-11-21 09:31:21 +00:00
Douglas Yung
5839abb2f9 Add header <atomic> which is needed to compile with some older library versions.
llvm-svn: 347382
2018-11-21 08:54:40 +00:00
Dean Michael Berris
2b95e3c5cf [XRay] Add a test for re-initialising FDR mode (NFC)
This change adds an end-to-end test that ensures FDR mode can be
re-initialised safely in the face of multiple threads being traced.

llvm-svn: 347368
2018-11-21 00:30:26 +00:00
David Carlier
d931c135f0 Revert "[Sanitizer] intercept setvbuf on other platforms where it is supported"
llvm-svn: 347358
2018-11-20 22:50:31 +00:00
David Carlier
a1ea566fbc [Sanitizer] Unbreak non NetBSD builds.
llvm-svn: 347357
2018-11-20 22:35:29 +00:00
David Carlier
5ffb47b3a1 [Sanitizer] intercept setvbuf on other platforms where it is supported
Unit tests enabled only in platform tested.

Reviewers: krytarowski, vitalybuka

Reviewed By: krytarowski, vitalybuka

Differential Revision: https://reviews.llvm.org/D54739

llvm-svn: 347355
2018-11-20 22:17:23 +00:00
Michal Gorny
4b0b84f4bb [cmake] Fix detecting terminfo library
Copy the fix for determining the correct terminfo library from LLVM --
use distinct variables for check_library_exists() calls.  Otherwise,
the first check (for -ltinfo) populates the variable and no other checks
are performed.  Effectively, systems with other libraries than the first
one listed are presumed not to have terminfo routines at all.

Also sync the check order to include the NetBSD fix from r347156.

This partially fixes undefined symbols when linking XRay tests.  It's
probably not the best solution to the problem there but as long
as the terminfo check stays in config-ix, I thnk it's worth fixing.

Differential Revision: https://reviews.llvm.org/D54641

llvm-svn: 347338
2018-11-20 18:41:11 +00:00
Vitaly Buka
afedb893a6 [tsan] Add __cxa_guard_acquire hooks to support cooperative scheduling
Reviewers: dvyukov

Subscribers: krytarowski, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D54664

llvm-svn: 347336
2018-11-20 18:21:42 +00:00
Dean Michael Berris
388af45f18 [XRay] Add a test for allocator exhaustion
Use a more representative test of allocating small chunks for
oddly-sized (small) objects from an allocator that has a page's worth of
memory.

llvm-svn: 347286
2018-11-20 03:56:04 +00:00
Dean Michael Berris
ba02cb58cf [XRay] Move buffer extents back to the heap
Summary:
This change addresses an issue which shows up with the synchronised race
between threads writing into a buffer, and another thread reading the
buffer.

In a lot of cases, we cannot guarantee that threads will always see the
signal to finalise their buffers in time despite the grace periods and
state machine maintained through atomic variables. This change addresses
it by ensuring that the same instance being updated to indicate how much
of the buffer is "used" by the writing thread is the same instance being
read by the thread processing the buffer to be written out to disk or
handled through the iterators.

To do this, we ensure that all the "extents" instances live in their own
the backing store, in a different contiguous page from the
buffer-specific backing store. We also take precautions to ensure that
the atomic variables are cache-line-sized to prevent false-sharing from
unnecessarily causing cache contention on unrelated writes/reads.

It's feasible that we may in the future be able to move the storage of
the extents objects into the single backing store, slightly changing the
way to compute the size(s) of the buffers, but in the meantime we'll
settle for the isolation afforded by having a different backing store
for the extents instances.

Reviewers: mboerger

Subscribers: jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D54684

llvm-svn: 347280
2018-11-20 01:00:26 +00:00
Petr Hosek
8e0e35a3f5 [compiler-rt] Use zx_futex_wait_deprecated for Fuchsia sanitizer runtime
This change is part of the soft-transition to the new synchronization
primitives which implement priority inheritance.

Differential Revision: https://reviews.llvm.org/D54727

llvm-svn: 347279
2018-11-20 00:55:20 +00:00
Kamil Rytarowski
74abaf8cdc Add interceptor for the setvbuf(3) from NetBSD
Summary:
setvbuf(3) is a routine to setup stream buffering.

Enable the interceptor for NetBSD.

Add dedicated tests for setvbuf(3) and functions
on top of this interface: setbuf, setbuffer, setlinebuf.

Based on original work by Yang Zheng.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: devnexen, tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54548

llvm-svn: 347270
2018-11-19 22:44:26 +00:00