577 Commits

Author SHA1 Message Date
Kostya Serebryany
66d69563f7 [libFuzzer] properly hide the memcmp interceptor from msan
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@294061 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-03 22:51:38 +00:00
Mike Aizatsky
0022fc7373 [libfuzzer] chromium-related compilation fixes
Reviewers: kcc

Differential Revision: https://reviews.llvm.org/D29502

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@294035 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-03 20:26:44 +00:00
Kostya Serebryany
5459106ddf [libFuzzer] reorganize the tracing code to make it easier to experiment with inlined coverage instrumentation. NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293928 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-02 19:56:01 +00:00
Marcos Pividori
ebf6feac60 [libFuzzer] Properly handle exceptions with UnhandledExceptionFilter.
Use SetUnhandledExceptionFilter instead of AddVectoredExceptionHandler.

According to the documentation on Structured Exception Handling, this is the
order for the Exception Dispatching:
+ If the process is being debugged, the system notifies the debugger.
+ The Vectored Exception Handler is called.
+ The system attempts to locate a frame-based exception handler by searching the
 stack frames of the thread in which the exception occurred.
+ If no frame-based handler can be found, the UnhandledExceptionFilter filter is
 called.
+ Default handling based on the exception type.

So, similar to what we do for asan, we should use SetUnhandledExceptionFilter
instead of AddVectoredExceptionHandler, so user's code that is being fuzzed can
execute frame-based exception handlers before we catch them . We want to catch
unhandled exceptions, not all the exceptions.

Differential Revision: https://reviews.llvm.org/D29462

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293920 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-02 19:07:53 +00:00
Marcos Pividori
f7fdc84b39 [libFuzzer] Disable afl tests on non-posix systems.
AflDriver is not supported on non posix systems.

Differential Revision: https://reviews.llvm.org/D29422

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293830 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-01 22:40:50 +00:00
Marcos Pividori
b2df6a0c9b [libFuzzer] Disable equivalence tests on non posix systems.
We can not run this test until we implement shared memory on Windows.

Differential Revision: https://reviews.llvm.org/D29421

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293829 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-01 22:40:45 +00:00
Marcos Pividori
98399aa4b2 [libFuzzer] Isolate merge tests that require posix.
Differential Revision: https://reviews.llvm.org/D29420

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293828 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-01 22:40:40 +00:00
Marcos Pividori
b928b9b13e [libFuzzer] Add features windows and posix for lit tests.
Add 2 features: posix and windows.
Sometimes we want some specific tests only for posix and we use:
 REQUIRES: posix
Sometimes we want some specific tests only for windows and we use:
 REQUIRES: windows

Differential Revision: https://reviews.llvm.org/D29418

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293827 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-01 22:40:34 +00:00
Marcos Pividori
adb057cab2 [libFuzzer] Accept different extensions.
Differential Revision: https://reviews.llvm.org/D29417

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293826 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-01 22:40:29 +00:00
Marcos Pividori
1d98b650ef [libFuzzer] Fix test because cmd prompt does not expand wildcard.
Commands should expand the wildcards on Windows, the cmd prompt doesn't.
Because of that sancov was not finding the needed file.
To deal with this, we use ls and xargs from gnu win utils.

Differential Revision: https://reviews.llvm.org/D29374

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293825 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-01 22:39:55 +00:00
Marcos Pividori
12cb357da1 [libFuzzer] Do not use llvm-objdump for disassembling a DSO.
When disassembling a DSO, for calls to functions from the PLT, llvm-objdump only
prints the offset from the PLT, like: <.plt+0x30>.

While objdump and dumpbin print the function name, like:
<__sanitizer_cov_trace_pc_guard@plt>

When analyzing the coverage in libFuzzer we dissasemble and look for the calls
to __sanitizer_cov_trace_pc_guard.

So, this fails when using llvm-objdump on a DSO.

Differential Revision: https://reviews.llvm.org/D29372

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293791 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-01 17:59:23 +00:00
Marcos Pividori
90e4124e52 [libFuzzer] Properly check if we can use dumpbin.
The flag "/sumary" is necessary, otherwise it returns a non-zero value.

Differential Revision: https://reviews.llvm.org/D29371

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293790 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-01 17:59:19 +00:00
Kostya Serebryany
63c362484c [libFuzzer] increase the default size for shmem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293722 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-01 00:07:47 +00:00
Marcos Pividori
2c531bbcf6 [libFuzzer] Implement TmpDir() for Windows.
Differential Revision: https://reviews.llvm.org/D28977

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293516 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-30 18:14:53 +00:00
Kostya Serebryany
dec0b05633 [libfuzzer] include errno.h. On Ubuntu 14.04 we got away w/o it, but other systems seem to require it
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293389 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-28 18:56:05 +00:00
Kostya Serebryany
001f56735f [libFuzzer] make shmem more robust in the presence of signals
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293339 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-27 22:41:30 +00:00
Kostya Serebryany
49c910dae1 [libFuzzer] simplify the value profiling callback further: don't use (idx MOD prime) on the hot path where it is useless anyway
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293239 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-27 00:39:12 +00:00
Kostya Serebryany
eff755f3da [libFuzzer] make sure (again) that __builtin_popcountl is compiled into popcnt
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293237 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-27 00:20:55 +00:00
Kostya Serebryany
bcc64b878f [libFuzzer] simplify the value profile code and disable asan/msan on it
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293236 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-27 00:09:59 +00:00
Kostya Serebryany
6c2c7c89da [libFuzzer] remove a bit of stale code
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293129 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-26 01:45:54 +00:00
Kostya Serebryany
c43e0682f2 [libFuzzer] further simplify __sanitizer_cov_trace_pc_guard
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293128 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-26 01:34:58 +00:00
Kostya Serebryany
df040aa50b [libFuzzer] simplify the code for __sanitizer_cov_trace_pc_guard and make sure it is not asan/msan-instrumented
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293125 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-26 01:04:54 +00:00
Kostya Serebryany
13aa2bfeaf [libFuzzer] don't call GetPreviousInstructionPc on the hot path -- only when dumping the PCs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293117 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-26 00:22:08 +00:00
Kostya Serebryany
72e0240a06 [libFuzzer] mutate empty input using the regular mutators (instead of a custom dummy one). This way when we mutate an empty input there is a chance we will get a dictionary word
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292843 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-23 22:52:13 +00:00
Kostya Serebryany
fc4ec25a17 [libFuzzer] make sure we use the feedback from std::string operator ==
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292835 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-23 22:11:04 +00:00
Kostya Serebryany
f7ac01d0a0 [libFuzzer] deflake a test
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292813 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-23 18:44:40 +00:00
Marcos Pividori
6ea436d7ac [libFuzzer] Add missing dependency for tests.
Dependency on TestBinaries was erroneously removed on r292735.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292765 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 21:55:46 +00:00
Marcos Pividori
7728ff0f87 [libFuzzer] Specify the CRT considered (MT or MD) for tests on Windows.
Differential Revision: https://reviews.llvm.org/D28439

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292749 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 02:28:12 +00:00
Marcos Pividori
2858ad7ec2 [libFuzzer] Fix test with shared libraries on Windows.
We need to set BINARY_DIR to: ${CMAKE_BINARY_DIR}/lib/Fuzzer/test , so the dll
is placed in the same directory than the test LLVMFuzzer-DSOTest, and is found
when executing that test.
As we are using CMAKE_CXX_CREATE_SHARED_LIBRARY to link the dll, we can't modify
the output directory for the import library. It will be created in the same
directory than the dll (in BINARY_DIR), no matter which value we set to
LIBRARY_DIR. So, if we set LIBRARY_DIR to a different directory than BINARY_DIR,
when linking LLVMFuzzer-DSOTest, cmake will look for the import library
LLVMFuzzer-DSO1.lib in LIBRARY_DIR, and won't find it, since it was created in
BINARY_DIR. So, for Windows, we need that LIBRARY_DIR and BINARY_DIR are the
same directory.

Differential Revision: https://reviews.llvm.org/D27870

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292748 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 02:28:08 +00:00
Marcos Pividori
61d2c04941 [libFuzzer] AlrmHandler is executed in a different thread for Windows.
Don't check for InFuzzingThread() on Windows, since the AlarmHandler() is
always executed by a different thread from a thread pool.
If we don't add these changes, the alarm handler will never execute.
Note that we decided to ignore possible problem in the synchronization.

Differential Revision: https://reviews.llvm.org/D28723

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292746 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 01:58:59 +00:00
Marcos Pividori
9c2ee32cbf [libFuzzer] Leak Sanitizer is not supported for Windows.
Differential Revision: https://reviews.llvm.org/D28709

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292745 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 01:58:55 +00:00
Marcos Pividori
44ad5c2d2c [libFuzzer] Fix OutOfMemory tests to work on 32 bits.
I add 2 changes to make the tests work on 32 bits and on 64 bits.
I change the size allocated to 0x20000000 and add the flag: -rss_limit_mb=300.
Otherwise the output for 32 bits and 64 bits is different.
For 64 bits the value 0xff000000 doesn't exceed kMaxAllowedMallocSize.
For 32 bits, kMaxAllowedMallocSize is set to 0xc0000000, so the call to
Allocate() will fail earlier printing "WARNING: AddressSanitizer failed to
allocate ..." , and wont't call malloc hooks.
So, we need to consider a size smaller than 2GB (so malloc doesn't fail on
32bits) and greater that the value provided by -rss_limit_mb.
Because of that I use: 0x20000000.

Differential Revision: https://reviews.llvm.org/D28706

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292744 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 01:58:50 +00:00
Marcos Pividori
ed3dca071a [libFuzzer] Avoid undefined behavior, properly discard output to stdout/stderr.
Fix libFuzzer when setting -close_fd_mask to a non-zero value.
In previous implementation, libFuzzer closes the file descriptors for
stdout/stderr. This has some disavantages:

For `fuzzer-fdmask.test`, we write directly to stdout and stderr using the
file streams stdout and stderr, after the file descriptors are closed, which is
undefined behavior. In Windows, in particular, this was making the test fail.

Also, if we close stdout and we open a new file in libFuzzer, we get the file
descriptor 1, which could generate problem if some code assumes file descriptors
refers to stdout and works directly writing to the file descriptor 1, but it
will be writing to the opened file (for example using std::cout).

Instead of closing the file descriptors, I redirect the output to /dev/null on
linux and nul on Windows.

Differential Revision: https://reviews.llvm.org/D28718

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292743 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 01:58:45 +00:00
Marcos Pividori
3bd9b31212 [libFuzzer] Remove lib prefix from library names on tests.
This changes is necessary on Windows, where libraries doesn't include the prefix
"lib".

Differential Revision: https://reviews.llvm.org/D28710

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292742 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 01:58:40 +00:00
Marcos Pividori
6820da1473 [libFuzzer] Fix ListFilesInDirRecursive() to do the same for Posix and Windows.
Update `ListFilesInDirRecursive` implementation on Windows to have the same
behavior than for Posix, when the directory doesn't exists and when it is empty.

Differential Revision: https://reviews.llvm.org/D28711

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292741 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 01:58:36 +00:00
Marcos Pividori
8f08df7ee1 [libFuzzer] Consider both possible separators for tests.
Differential Revision: https://reviews.llvm.org/D28636

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292740 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 01:58:31 +00:00
Marcos Pividori
55c19060a6 [libFuzzer] Portably disassemble and find calls to sanitizer_cov_trace_pc_guard.
Instead of directly using objdump, which is not present on Windows, we consider
different tools depending on the platform.
For Windows, we consider dumpbin and llvm-objdump.

Differential Revision: https://reviews.llvm.org/D28635

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292739 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 01:58:26 +00:00
Marcos Pividori
8b351cd95a [libFuzzer] Portable implementation of IsInterestingCoverageFile().
For Posix systems and Windows, we need to consider different cases.

Differential Revision: https://reviews.llvm.org/D28633

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292738 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 01:27:47 +00:00
Marcos Pividori
d2146889b5 [libFuzzer] Remove optimization flags for tests.
We need to build all the tests with -O0, otherwise optimizations may merge some
basic blocks and the tests will fail.
In this diff, I simplify the cmake implementation and I remove the flags for
Windows too (/O[123s]).

Differential Revision: https://reviews.llvm.org/D28632

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292737 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 01:27:42 +00:00
Marcos Pividori
6c13258b28 [libFuzzer] Expose Sanitizer Coverage functions from libFuzzer.
We need to expose Sanitizer Coverage's functions that are rewritten with a
different implementation, so compiler-rt's libraries have access to it.

Differential Revision: https://reviews.llvm.org/D28618

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292736 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 01:27:38 +00:00
Marcos Pividori
4509e88ac3 [libFuzzer] Remove dependencies for tests on Windows.
Remove dependency on FileCheck, sancov and not for tests on Windows.
If LLVM_USE_SANITIZER=Address and LLVM_USE_SANITIZE_COVERAGE=YES, this will
trigger the building of dependencies with sanitizer instrumentation.
This will fail in Windows, since cmake will use link.exe for linking and won't
include compiler-rt libraries.

Differential Revision: https://reviews.llvm.org/D27993

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292735 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 01:27:34 +00:00
Marcos Pividori
bcc6b36ead [libFuzzer] Disable afl tests for Windows.
On Windows, we don't have interoperability between libFuzzer and afl.

Differential Revision: https://reviews.llvm.org/D28355

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292734 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-22 01:26:18 +00:00
Vitaly Buka
fcd9eea31b [libFuzzer] Use CXX to set compiler to use
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D28976

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292697 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-21 01:21:56 +00:00
Kostya Serebryany
3a0d592eba [libFuzzer] fix gcc build
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292695 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-21 01:08:22 +00:00
Kostya Serebryany
e0c77c88b7 [libFuzzer] use print+exit(1) instead of assert to report an error
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292685 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-21 00:13:50 +00:00
Kostya Serebryany
79f0d94edf [libFuzzer] re-enable LLVMFuzzer-RepeatedMemcmp test, cleanup the test runner nearby
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292683 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-21 00:01:27 +00:00
Kostya Serebryany
02e6813300 [libFuzzer] call __sanitizer_dump_coverage via EF
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292681 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-20 23:35:29 +00:00
Marcos Pividori
38f8755c0f [libFuzzer] Don't use #ifdef for defined macros, instead use #if.
Differential Revision: https://reviews.llvm.org/D28972

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292670 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-20 22:49:13 +00:00
Marcos Pividori
cad86c75f7 [libFuzzer] Use clang as linker on Windows, to properly include sanitizer libraries.
In order to use sanitizers on Windows, we need to link against many runtime
libraries which will depend on the target being created (executable or dll) and
the c runtime library used (MT/MD).
By default, cmake uses link.exe for linking, which fails because we don't
specify the appropiate dependencies. As we don't want to consider all of that
possible situations which depends on the implementation of the compiler-rt, the
simplest option is to change the rules for linking executables and shared
libraries, using the compiler instead of link.exe.
Clang driver will consider the sanitizer flags, and automatically provide the
required libraries to the linker.

Differential Revision: https://reviews.llvm.org/D27869

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292669 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-20 22:49:08 +00:00
Marcos Pividori
40dcc15c25 [libFuzzer] Properly use compiler options supported on Windows.
Replace "-g" by "-gline-tables-only". "-g" is not supported by clang-cl.

Differential Revision: https://reviews.llvm.org/D27868

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292668 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-20 22:48:47 +00:00