Commit Graph

516 Commits

Author SHA1 Message Date
Kostya Serebryany
577674bc4f [libFuzzer] disable test that requires debug info -- it fails on the bot
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281584 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-15 05:46:58 +00:00
Kostya Serebryany
3f6d6753de [libFuzzer] move the AFL driver build rule test into the uninstrumented dir
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281583 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-15 05:17:39 +00:00
Kostya Serebryany
7e2d153c71 [libFuzzer] fix print_pcs test
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281580 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-15 04:43:06 +00:00
Kostya Serebryany
f0b5dd76ab [libFuzzer] implement print_pcs with trace-pc-guard. Change the trace-pc-guard heuristic for 8-bit counters to look more like in AFL (not that it's provable better, but the existin test preferes this heuristic)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281577 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-15 04:36:45 +00:00
Kostya Serebryany
26586a8660 [libFuzzer] add 8-bit counters to trace-pc-guard handler
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281568 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-15 01:30:18 +00:00
Kostya Serebryany
a4826c1b99 [libFuzzer] start using trace-pc-guard as an alternative source of coverage
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281435 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-14 02:13:06 +00:00
Kostya Serebryany
05082bd46a [libFuzzer] print a failed-merge warning only in the merge mode
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281130 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-10 02:17:22 +00:00
Kostya Serebryany
74ad0cfa1d [libFuzzer] don't print help for internal flags
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281124 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-10 00:35:30 +00:00
Kostya Serebryany
7942244fd1 [libFuzzer] print a visible message if merge fails due to a crash
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281122 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-10 00:15:41 +00:00
Kostya Serebryany
2c9e9a6439 [libFuzzer] use sizeof() in tests instead of 4 and 8
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281111 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-09 22:21:16 +00:00
Kostya Serebryany
fb49dafc05 [libFuzzer] one more puzzle for value profile
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281106 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-09 21:58:42 +00:00
Kostya Serebryany
9af2b5e4e4 [libFuzzer] one more puzzle, value_profile cracks it in a second
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281066 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-09 18:00:04 +00:00
Kostya Serebryany
4475d671c4 [libFuzzer] improve -print_pcs to not print new PCs coming from libFuzzer itself
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281016 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-09 02:38:28 +00:00
Kostya Serebryany
ddc2d44ce2 [libFuzzer] remove unneeded call
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281014 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-09 01:57:38 +00:00
Kostya Serebryany
05e1dea79e [libFuzzer] remove use_traces=1 since use_value_profile seems to be strictly better
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281007 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-09 01:17:03 +00:00
Kostya Serebryany
501485f6e4 [libFuzzer] add -minimize_crash flag (to minimize crashers). also add two tests that I failed to commit last time
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280332 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-01 01:22:27 +00:00
Mike Aizatsky
d7a5ad2193 [libfuzzer] simplified unit truncation; do not write trunc items to disc
Differential Revision: https://reviews.llvm.org/D24049

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280153 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-30 20:49:07 +00:00
Kostya Serebryany
72187d4993 [libFuzzer] fix a bug when running a single unit of N bytes with -max_len=M, M<N, caused a buffer overflow
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280098 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-30 14:52:05 +00:00
Kostya Serebryany
773652b49c [libFuzzer] stop using bits for memcmp's value profile -- seems to blow up the corpus too much
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280096 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-30 14:39:33 +00:00
Kostya Serebryany
c7c6f45c22 [libFuzzer] use bits instead of bytes for memcmp/strcmp value profile -- the fuzzer reaches the goal much faster, at least on the simple puzzles
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280054 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-30 03:05:50 +00:00
Kostya Serebryany
ff612bcd8e [libFuzzer] use trace-div and trace-gep for guided fuzzing, add tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280046 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-30 01:30:14 +00:00
Kostya Serebryany
be74ea271b [libFuzzer] simplify a test to make it pass on the bot
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279796 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-26 00:18:16 +00:00
Kostya Serebryany
cf34910507 [libFuzzer] make sure we have symbols on fuzzer tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279792 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-25 23:30:02 +00:00
Kostya Serebryany
7750b380af [libFizzer] rename -print_new_cov_pcs=1 into -print_pcs=1 and make it more useful: print PCs only after the initial corpus has been read and symbolize them
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279787 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-25 22:35:08 +00:00
Kostya Serebryany
cf1269a0b2 [libFuzzer] simplify the code, NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279697 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-25 01:25:03 +00:00
Kostya Serebryany
4d000ed9c5 [libFuzzer] make a test more deterministic
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279686 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-24 23:10:17 +00:00
Kostya Serebryany
0c7940eaf1 [libFuzzer] use __attribute__((target("popcnt"))) only on x86_64
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279601 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-24 01:38:42 +00:00
Kostya Serebryany
bb66a7d956 [libFuzzer] collect 64 states for value profile, not 65
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279588 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-23 23:37:37 +00:00
Kostya Serebryany
47b194388e [libFuzzer] fix the non-debug build warnings
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279321 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-19 20:57:09 +00:00
Kostya Serebryany
2fb93610ff [libFuzzer] add more __attribute__((visibility("default")))
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279143 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-18 20:52:52 +00:00
Kostya Serebryany
aa4e23e1bb [sanitizer-coverage/libFuzzer] instrument comparisons with __sanitizer_cov_trace_cmp[1248] instead of __sanitizer_cov_trace_cmp, don't pass the comparison type to save a bit performance. Use these new callbacks in libFuzzer
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279027 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-18 01:25:28 +00:00
Kostya Serebryany
ac9c0f4d45 [libFuzzer] force proper popcnt instruction
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279002 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-17 23:09:57 +00:00
Kostya Serebryany
e65e275cf2 [libFuzzer] given 0 and 255 more preference when inserting repeated bytes
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278986 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-17 21:50:54 +00:00
Kostya Serebryany
9744bb48e4 [libFuzzer] one more mutation: ChangeBinaryInteger; also fix the breakage from r278970
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278982 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-17 21:30:30 +00:00
Kostya Serebryany
7526feb22f [libFuzzer] when printing the reproducer input, also print the base input and the mutation sequence
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278975 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-17 20:45:23 +00:00
Justin Bogner
7d7a23e700 Replace a few more "fall through" comments with LLVM_FALLTHROUGH
Follow up to r278902. I had missed "fall through", with a space.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278970 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-17 20:30:52 +00:00
Kostya Serebryany
79d100891f [libFuzzer] more mutations
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278950 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-17 18:10:42 +00:00
Kostya Serebryany
f61ef6cf72 [libFuzzer] minor speed improvement
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278856 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-16 21:28:05 +00:00
Kostya Serebryany
2e1aa9cd1c [libFuzzer] new experimental feature: value profiling. Profiles values that affect control flow and treats new values as new coverage.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278839 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-16 19:33:51 +00:00
Kostya Serebryany
b4d6119096 [libFuzzer] refactoring around PCMap, NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278825 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-16 17:37:13 +00:00
Kostya Serebryany
e286c13fc8 [libFuzzer] print a verbose message after executing inputs in non-fuzzing mode
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278724 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-15 19:44:04 +00:00
Kostya Serebryany
0347ebc2df [libFuzzer] fix the bot
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278721 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-15 19:36:13 +00:00
Kostya Serebryany
137c3957ed [libFuzzer] add InsertRepeatedBytes and EraseBytes.
New mutation: InsertRepeatedBytes.
Updated mutation: EraseByte => EraseBytes.

This helps https://github.com/google/sanitizers/issues/710
where libFuzzer was not able to find a known bug.
Now it finds it in minutes.

Hopefully, the change is general enough to help other targets.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278687 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-15 17:48:28 +00:00
Dan Liew
a3e4fd51f1 [LibFuzzer] Fix -jobs=<N> where <N> > 1 and the number of workers is > 1 on macOS.
The original `ExecuteCommand()` called `system()` from the C library.
The C library implementation of this on macOS contains a mutex which
serializes calls to `system()`. This prevented the `-jobs=` flag
from running copies of the fuzzing binary in parallel which is
the opposite of what is intended.

To fix this on macOS an alternative implementation of `ExecuteCommand()`
is provided that can be used concurrently. This is provided in
`FuzzerUtilDarwin.cpp` which is guarded to only compile code on Apple
platforms. The existing implementation has been moved to a new file
`FuzzerUtilLinux.cpp` which is guarded to only compile code on Linux.

This commit includes a simple test to check that LibFuzzer is being
executed in parallel when requested.

Differential Revision: https://reviews.llvm.org/D22742

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278544 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-12 18:29:36 +00:00
Kostya Serebryany
3aa06bae18 [libFuzzer] make libFuzzer work with a bit older clang versions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277941 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-06 21:28:56 +00:00
Kostya Serebryany
1efe1f7b69 [libFuzzer] don't print bogus error message
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277940 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-06 21:23:29 +00:00
Mike Aizatsky
7e8350b8b7 [libfuzzer] do not warn about missing pcbuffer functions: they are new.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277927 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-06 17:03:22 +00:00
Mike Aizatsky
4613003000 [sanitizers] trace buffer API to use user-allocated buffer.
Differential Revision: https://reviews.llvm.org/D23185

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277859 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-05 20:09:53 +00:00
Kostya Serebryany
1f0a7ed0f3 [libFuzzer] extend the messages printed by afl_driver
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276052 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-19 23:18:28 +00:00
Kostya Serebryany
3487192114 [libFuzzer] properly intercept memmem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276006 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-19 18:29:06 +00:00
Kostya Serebryany
730992dc3b [libFuzzer] add hooks for strstr, strcasestr, strcasecmp, strncasecmp
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275648 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-15 23:27:19 +00:00
Kostya Serebryany
3d1f237171 [libFuzzer] add ThreadedLeakTest
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275582 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-15 17:19:43 +00:00
Dan Liew
1a68871c47 [LibFuzzer] Unbreak the build on macOS which was broken by r272858.
``afl_driver.cpp`` currently relies on weak symbols which doesn't
work properly under macOS. For now fix the build by providing a
dummy implementation of ``LLVMFuzzerInitialize(...)``. This is just
a temporary measure until we fix ``afl_driver.cpp`` for macOS.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@274778 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-07 18:14:11 +00:00
Mike Aizatsky
f6653a309e [libFuzzer] Let user specify extra stats file.
Summary: If AFL_DRIVER_EXTRA_STATS_FILENAME is set and valid, write to it peak_rss_mb and slowest_unit_time_sec. These are both stats that libFuzzer can print but afl cannot.

Reviewers: kcc, aizatsky, metzman

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D21742

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@274273 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-30 20:43:06 +00:00
Mike Aizatsky
8bbe924637 [libfuzzer] moving is_ascii handler inside mutation dispatcher.
Summary: It also fixes a bug, when first random might not be ascii.

Differential Revision: http://reviews.llvm.org/D21573

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@273611 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-23 20:44:48 +00:00
Vitaly Buka
07997a91ef [libFuzzer] Add standard license info and comment header to AFLDriverTest.cpp
Summary: Add license info and brief description of file to AFLDriverTest.cpp.

Reviewers: kcc, aizatsky

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D21487

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@273527 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-23 02:19:36 +00:00
Kostya Serebryany
ee71b6abb0 [libFuzzer] make the single-run output more reliable
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272998 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-17 13:07:06 +00:00
Kostya Serebryany
c0e47a82c1 [libFuzzer] use the new chainable malloc hooks instead of the old un-chainable ones, use atomic for malloc/free counters instead of a thread local counter in the main thread. This should make on-the-spot leak detection in libFuzzer more reliable
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272948 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-16 20:17:41 +00:00
Vitaly Buka
3cf4eef2a1 Fix test from D21194
Bot sets ASAN_OPTIONS=handle_abort=1 which prevents expected crash.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272866 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-16 01:52:48 +00:00
Vitaly Buka
eb7016ef82 Debugging D21194 issues on bot
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272863 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-16 01:26:46 +00:00
Vitaly Buka
35825a3ed7 Enable libFuzzer's afl_driver to append stderr to a file.
Summary:
[libFuzzer] Enable afl_driver to append stderr to a user specified file.

Append stderr of afl_driver to the file specified by the environmental variable
AFL_DRIVER_STDERR_DUPLICATE_FILENAME if it is set. This lets users see outputs
on crashes without rerunning crashing test cases (which won't work for crashes
that are difficult to reproduce). Before this patch, stderr would only be sent to afl-fuzz
and users would have no way of seeing it.

Reviewers: llvm-commits, aizatsky, kcc, vitalybuka

Subscribers: vitalybuka

Differential Revision: http://reviews.llvm.org/D21194

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272858 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-16 00:14:42 +00:00
Dan Liew
7029af9787 [LibFuzzer] Fix `FuzzerMutate.ShuffleBytes2` unit test on OSX.
The ``FuzzerMutate.ShuffleBytes2`` unit test was failing on
OSX due to the implementation of ``std::random_shuffle()``
being different between libcxx and libstdc++.

@kcc has decided (see http://reviews.llvm.org/D21218) it is acceptable
for there to be different mutation behavior on different platforms so
this commit just adjusts the test to perform the minimum number of
iterations (that is a power of 2) to see all the mutations the unit test
is looking for.

Differential Revision: http://reviews.llvm.org/D21359

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272743 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-15 01:40:02 +00:00
Vitaly Buka
93cdb4b527 Revert "Enable libFuzzer's afl_driver to append stderr to a file."
Crashes with AddressSanitizer: SEGV on unknown address
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/9924/steps/annotate/logs/stdio

This reverts commit r272706.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272726 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-14 22:09:00 +00:00
Vitaly Buka
6ff3418b63 Enable libFuzzer's afl_driver to append stderr to a file.
Summary:
[libFuzzer] Enable afl_driver to append stderr to a user specified file.

Append stderr of afl_driver to the file specified by the environmental variable
AFL_DRIVER_STDERR_DUPLICATE_FILENAME if it is set. This lets users see outputs
on crashes without rerunning crashing test cases (which won't work for crashes
that are difficult to reproduce). Before this patch, stderr would only be sent to afl-fuzz
and users would have no way of seeing it.

Reviewers: llvm-commits, aizatsky, kcc, vitalybuka

Subscribers: vitalybuka

Differential Revision: http://reviews.llvm.org/D21194

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272706 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-14 20:42:05 +00:00
Dan Liew
eac51a2bde [LibFuzzer] Disable the `fuzzer-trace-pc.test` test on non-linux platforms.
On OSX this test sometimes fails due to the
``LLVMFuzzer-FullCoverageSetTest-TracePC`` program going over the
default 2GiB memory limit. This shouldn't be happening and needs
investigating. For now just disable the test so we can set up an
OSX buildbot.

Differential Revision: http://reviews.llvm.org/D21319

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272696 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-14 19:15:13 +00:00
Dan Liew
a29300175d [LibFuzzer] Move tests in `fuzzer-traces.test` that require hooks to their own test.
The tests in ``fuzzer-traces-hooks.test`` only work on Linux because calls to hooks
(e.g. ``__sanitizer_weak_hook_memcmp()``) from inside the sanitizer
runtime are only implemented on Linux. Therefore these tests are set to
only run on Linux.

Differential Revision: http://reviews.llvm.org/D21253

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272600 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-14 00:11:34 +00:00
Dan Liew
bb05836e33 [LibFuzzer] Fix some unit test crashes on OSX.
This fixes the following unit tests:

FuzzerDictionary.ParseOneDictionaryEntry
FuzzerDictionary.ParseDictionaryFile

The issue appears to be mixing non-ASan-ified code (LibFuzzer) and
ASan-ified code (the unittest) as the tests would pass fine if
everything was built with ASan enabled.

I believe the issue is that different implementations of std::vector<>
are being used in LibFuzzer and outside LibFuzzer (in the unittests).
For Libcxx (I've not seen the issue manifest for libstdc++) we can disable
the ASanified std::vector<> by definining the ``_LIBCPP_HAS_NO_ASAN`` macro.
Doing this fixes the tests on OSX.

Differential Revision: http://reviews.llvm.org/D21049

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272374 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-10 05:33:07 +00:00
Kostya Serebryany
41a5827cd3 [libFuzzer] add one more OOM test, which we currently don't handle very well
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272240 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-09 01:20:35 +00:00
Kostya Serebryany
819b4d1da2 [libFuzzer] add 'weak' back to __sanitizer_malloc_hook and __sanitizer_free_hook
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272116 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-08 04:49:29 +00:00
Kostya Serebryany
fd9cb1a5c5 [libFuzzer] add a test that is built w/o coverage instrumentation but has the coverage rt (it should now fail with a descriptive message)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272090 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-08 01:46:13 +00:00
Dan Liew
1a1c8bea2d [LibFuzzer] Declare and use sanitizer functions in `fuzzer::ExternalFunctions`
This fixes linking problems on OSX.

Unfortunately it turns out we need to use an instance of the
``fuzzer::ExternalFunctions`` object in several places so this
commit also replaces all instances with a single global instance.

It also turns out initializing a global ``fuzzer::ExternalFunctions``
before main is entered (i.e. letting the object be initialised by the
global initializers) is not safe (on OSX the call to ``Printf()`` in the
CTOR crashes if it is called from a global initializer) so we instead
have a global ``fuzzer::ExternalFunctions*`` and initialize it inside
``FuzzerDriver()``.

Multiple unit tests depend also depend on the
``fuzzer::ExternalFunctions*`` global so a ``main()`` function has been
added that initializes it before running any tests.

Differential Revision: http://reviews.llvm.org/D20943

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272072 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-07 23:32:50 +00:00
Dan Liew
d3e1f7de2b [LibFuzzer] Split the fuzzer-oom.test into two tests.
This is necessary because the existing fuzzer-oom.test was Linux
specific due to its use of __sanitizer_print_memory_profile() which
is only available on Linux right now and so the test would fail on OSX.

Differential Revision: http://reviews.llvm.org/D20977

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272061 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-07 21:23:30 +00:00
Mike Aizatsky
b1953d8c1b [libfuzzer] custom crossover interface function.
Differential Revision: http://reviews.llvm.org/D21089

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272054 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-07 20:22:15 +00:00
Mike Aizatsky
c7f790d744 [libfuzzer] prune_corpus option for disabling pruning during the load.
Summary:
The option is very useful for testing, plus I intend to measure
its effect on fuzzer effectiveness.

Differential Revision: http://reviews.llvm.org/D21084

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272035 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-07 18:16:32 +00:00
Dan Liew
47f90f5156 [LibFuzzer] s/dataflow sanitizer/DataflowSanitizer/
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271980 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-07 04:44:49 +00:00
Dan Liew
ef07357b2f [LibFuzzer] Disable building and running LSan tests on Apple platforms because LSan is not currently supported.
Differential Revision: http://reviews.llvm.org/D20947

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271979 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-07 04:44:39 +00:00
Dan Liew
47f223bd24 [LibFuzzer] Provide stub implementation of __sanitizer_cov_trace_pc_indir
Calls to this function are currently injected by the
``SanitizerCoverageModule`` pass when the both the ``indirect-calls``
and ``trace-pc`` sanitizer coverage options are enabled and the code
being instrumented has indirect calls. Previously because LibFuzzer did
not define this function this would lead to link errors when building
some of the tests on OSX.

Differential Revision: http://reviews.llvm.org/D20946

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271938 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-06 20:27:09 +00:00
Mike Aizatsky
664513feca [libfuzzer] hiding custom mutator handling in MutationDispatcher.
Summary: Refactoring, no functional changes.

Differential Revision: http://reviews.llvm.org/D20975

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271740 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-03 21:34:29 +00:00
Mike Aizatsky
21e8db8cc1 [libfuzzer] splitting fuzzer.test
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271697 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-03 18:05:22 +00:00
Dan Liew
e8edd31970 [LibFuzzer] Disable compiling and running the LibFuzzer dataflow sanitizer tests on Apple platforms.
This fixes a broken part of the build on OSX as the dataflow sanitizer is not supported
on OSX yet.

Differential Revision: http://reviews.llvm.org/D20894

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271492 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-02 05:48:09 +00:00
Dan Liew
8bb56ccbc0 [LibFuzzer] Reimplement how the optional user functions are called.
The motivation for this change is to fix linking issues on OSX.
However this only partially fixes linking issues (the uninstrumented
tests and a few others  won't succesfully link yet).

This change introduces a struct of function pointers
(``fuzzer::ExternalFuntions``) which when initialised will point to the
optional functions if they are available.  Currently these
``LLVMFuzzerInitialize`` and ``LLVMFuzzerCustomMutator`` functions.

Two implementations of ``fuzzer::ExternalFunctions`` constructor are
provided one for Linux and one for OSX.

The OSX implementation uses ``dlsym()`` because the prior implementation
using weak symbols does not work unless the additional flags are passed
to the linker.

The Linux implementation continues to use weak symbols because the
``dlsym()`` approach does not work unless additional flags are passed
to the linker.

Differential Revision: http://reviews.llvm.org/D20741

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271491 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-02 05:48:02 +00:00
Kostya Serebryany
08b019eaf8 [libFuzzer] use __sanitizer_print_memory_profile to print the memory profile on OOM
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271465 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-02 01:33:11 +00:00
Kostya Serebryany
3886b25c4a [libFuzzer] when an invalid flag is given, warn, but don't crash
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271404 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 16:41:12 +00:00
Kostya Serebryany
a736e85343 [libFuzzer] fix a use-after-free (!) in libFuzzer caused by r270905: that CL caused a push_back in the main corpus invalidating the vector<> iterators in rare cases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271186 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-29 15:58:57 +00:00
Kostya Serebryany
8c598528c3 [libFuzzer] fix a failure that occurs when running individual inputs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271095 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-28 04:19:46 +00:00
Kostya Serebryany
dc4065fcc1 [libFuzzer] use __sanitizer_set_report_fd with -close_fd_mask. This allows us to keep asan reports when closing target's stderr
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271053 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-27 21:46:22 +00:00
Richard Smith
0513d1e203 Fix compilation with GCC, which treats this as a constructor name not a type
name. (GCC is correct here per the latest language DRs.)


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271044 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-27 21:05:35 +00:00
Dan Liew
70172d9997 [LibFuzzer] Refactor declaration of tests in CMake.
Add a new CMake function (``add_libfuzzer_test()``) to simplify
declaration of executables for testing LibFuzzer and use it to
reorganise how tests are declared.

Note that configuration of the lit configuration files has been moved
as late as possible because we are going to need to disable some tests
for some platforms and we will need to propagate this information into
the lit configuration.

Note the code for custom mains was removed because no tests are
currently written for this and Kostya seems happy to remove this.

Differential Revision: http://reviews.llvm.org/D20706

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270958 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-27 03:14:40 +00:00
Kostya Serebryany
3e20a7ae17 [libFuzzer] make check-fuzzer a bit faster
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270947 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-27 01:12:21 +00:00
Kostya Serebryany
6effb7148b [libFuzzer] make OOM-handling more portable. Instead of sending a signal to the main fuzzing thread, print the message in the getrusage thread and exit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270945 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-27 00:54:15 +00:00
Kostya Serebryany
d59f71d0d3 [libFuzzer] more refactoring: make sure CurrentUnitData is awlays a valid pointer to read from
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270942 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-27 00:21:33 +00:00
Kostya Serebryany
b987d27af9 [libFuzzer] more refactoring around CurrentUnit. Also add a threading test on which we currently have a race (when reporting bugs from multiple threads)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270929 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 22:17:32 +00:00
Dan Liew
bbd3f4216e [LibFuzzer] Add missing #include<string>
This partially fixes the compilation of the LibFuzzer unit test
on OSX using AppleClang.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270926 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 21:54:25 +00:00
Kostya Serebryany
1c5d1efdb3 [libFuzzer] refactor: hide CurrentUnitData inside an interface function. NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270922 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 21:32:30 +00:00
Dan Liew
caa716a991 [LibFuzzer] Emit error if LLVM_USE_SANITIZER is not correctly set.
Previously CMake would successfully configure and compile (with warnings
about ``-fsanitize-coverage=...`` being unused) but the tests LibFuzzer
tests would fail.

Differential Revision: http://reviews.llvm.org/D20662

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270913 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 20:55:09 +00:00
Dan Liew
b82cddc56f [LibFuzzer] Allow LibFuzzer to be built in modes other than RELEASE.
Previously the flags were only being set correctly when the
build type was "Release". Now the build should work properly
for all the supported build types. When building libFuzzer
the optimization level respects whatever is used for the
rest of LLVM but for the LibFuzzer tests we force -O0.

Differential Revision: http://reviews.llvm.org/D20558

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270912 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 20:55:05 +00:00
Kostya Serebryany
ddaf22d2ba [libFuzzer] when there is a leak in the existing corpus report the reproducer properly
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270905 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 20:25:49 +00:00
Kostya Serebryany
34fca4b9ec [libFuzzer] reimplement the way we do -only_ascii to allow more 'const' in function declarations. Add a test for -only_ascii. NFC intended
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270900 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 20:03:02 +00:00
Mike Aizatsky
215450cd0e [libfuzzer] replacing unittest for truncate_units with functional test.
Differential Revision: http://reviews.llvm.org/D20641

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270755 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 21:00:17 +00:00
Kostya Serebryany
305ed5060f [libFuzzer] print stats if we crash on empty input
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270639 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 00:15:36 +00:00
Mike Aizatsky
b0d0618635 [libfuzzer] Trying random unit prefixes during corpus load.
Differential Revision: http://reviews.llvm.org/D20301

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270632 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-24 23:14:29 +00:00
Kostya Serebryany
7e463431d7 [libFuzzer] add a license header to afl/afl_driver.cpp
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270598 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-24 19:05:25 +00:00
Dan Liew
036e9cc7c7 [LibFuzzer] Fix implementation of `GetPeakRSSMb()` on Mac OSX.
On Linux ``rusage.ru_maxrss`` is in KiB but on Mac OSX it is in bytes.

Differential Revision: http://reviews.llvm.org/D20410

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270173 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-20 01:37:54 +00:00
Dan Liew
adef8786dd [LibFuzzer] Fix `NumberOfCpuCores()` on Mac OSX.
The ``nprocs`` command does not exist under Mac OSX so use
``sysctl`` instead on that platform.

Whilst I'm here

* Use ``pclose()`` instead of ``fclose()`` which the ``popen()``
  documentation says should be used.
* Check for errors that were previously unhandled.

Differential Revision: http://reviews.llvm.org/D20409

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270172 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-20 01:30:36 +00:00
Dan Liew
86af2862c5 [LibFuzzer]
Work around crashes in ``__sanitizer_malloc_hook()`` under Mac OSX.

Under Mac OSX we intercept calls to malloc before thread local
storage is initialised leading to a crash when accessing
``AllocTracer``. To workaround this ``AllocTracer`` is only accessed
in the hook under Linux. For symmetry ``__sanitizer_free_hook()``
is also modified in the same way.

To support this change a set of new macros
LIBFUZZER_LINUX and LIBFUZZER_APPLE has been defined which can be
used to check the target being compiled for.

Differential Revision: http://reviews.llvm.org/D20402

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270145 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-19 22:00:33 +00:00
Kostya Serebryany
d8c064b2e9 [libFuzzer] do the merge faster and a bit less precise
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269497 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-13 22:11:23 +00:00
Kostya Serebryany
909ee84abf [libFuzzer] print the file name before executing the input so that if there is a crash we know which files has caused it
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269450 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-13 18:10:33 +00:00
Kostya Serebryany
22dd3bbcf0 [libFuzzer] simplify FuzzerInterface.h
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269448 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-13 18:04:35 +00:00
Kostya Serebryany
32409ab4c5 [libFuzzer] add a driver file to use AFL on LLVMFuzzerTestOneInput
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269141 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-10 23:46:50 +00:00
Mike Aizatsky
0d68393b0f [libfuzzer] Refactoring coverage state-management code.
It is now less state-dependent and will allow easier comparing of
coverages of different units.

Differential Revision: http://reviews.llvm.org/D20085

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269140 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-10 23:43:15 +00:00
Kostya Serebryany
c48930b338 [libFuzzer] add a test for libFuzzer+ubsan, extend the docs on using libFuzzer+ubsan
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268968 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-09 21:02:36 +00:00
Kostya Serebryany
2f120989e3 [libFuzzer] better document the -merge=1 flag
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268957 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-09 19:11:36 +00:00
Kostya Serebryany
f01dfdd8aa [libFuzzer] enhance -rss_limit_mb and enable by default. Now it will print the OOM reproducer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268821 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-06 23:38:07 +00:00
Kostya Serebryany
815a884f59 [libFuzzer] add exeprimental -rss_limit_mb flag to fight against OOMs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268807 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-06 21:58:35 +00:00
Kostya Serebryany
18c0216842 [libFuzzer] print stats after running individual inputs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268547 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-04 20:44:50 +00:00
Kostya Serebryany
168e51186d [libFuzzer] enable detect_leaks=1, add proper docs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268088 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-29 18:49:55 +00:00
Kostya Serebryany
cb249e3b07 [libFuzzer] disable leak detection if we have tried it for 1000 times w/o finding a leak [part 2]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267771 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-27 19:52:56 +00:00
Kostya Serebryany
30f53168fa [libFuzzer] disable leak detection if we have tried it for 1000 times w/o finding a leak
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267770 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-27 19:52:34 +00:00
Kostya Serebryany
cb05ff9241 [libFuzzer] remove dead code
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267455 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-25 19:41:45 +00:00
Kostya Serebryany
a0a13e0ee1 [libFuzzer] added -detect_leaks flag (0 by default for now). When enabled, it will help finding leaks while fuzzing
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266838 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-20 00:24:21 +00:00
Kostya Serebryany
d3f038dc3e [libFuzzer] try to print correct time in seconds when reporting a timeout. Don't report timeouts while still loading the corpus.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266693 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-18 22:50:39 +00:00
Kostya Serebryany
37e715dc57 [libFuzzer] warn if the corpus is empty
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266670 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-18 21:14:11 +00:00
Mehdi Amini
f6071e14c5 [NFC] Header cleanup
Removed some unused headers, replaced some headers with forward class declarations.

Found using simple scripts like this one:
clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap'

Patch by Eugene Kosov <claprix@yandex.ru>

Differential Revision: http://reviews.llvm.org/D19219

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266595 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-18 09:17:29 +00:00
Kostya Serebryany
ac6d266478 [libFuzzer] add a better warning for command line flags with -- (two dashes)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266480 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-15 21:56:29 +00:00
Hans Wennborg
f721b994c5 Remove redundant .c_str(), as suggested by PR25633
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265988 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-11 20:35:17 +00:00
Mike Aizatsky
5bb9d06dac [libfuzzer] defensive assert
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265866 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-08 23:32:24 +00:00
Mike Aizatsky
f4643f6392 [libfuzzer] adding license headers to cpp files
Differential Revision: http://reviews.llvm.org/D18705

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265174 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-01 18:38:58 +00:00
Kostya Serebryany
7544ffc130 [libFuzzer] more docs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@264803 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-29 23:07:36 +00:00
Kostya Serebryany
3d4018c32c [libFuzzer] use fflush after every Printf
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@264459 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-25 20:31:26 +00:00
Kostya Serebryany
d32f138d30 [libFuzzer] handle SIGTERM
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@264338 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-24 21:03:58 +00:00
Kostya Serebryany
c09d592889 [libFuzzer] don't report memory leaks if we are dying due to a timeout (just use _Exit instead of exit in the timeout callback)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@264237 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-24 01:32:08 +00:00
Kostya Serebryany
f4b00d0631 [libFuzzer] use fdopen+vfprintf instead of fsnprintf+write
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@264230 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-24 00:57:32 +00:00
Kostya Serebryany
4aa62c5d17 [libFuzzer] add a flag close_fd_mask so that we can silence spammy targets by closing stderr/stdout
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263831 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-18 20:58:29 +00:00
Benjamin Kramer
3ac2aa592d [Fuzzer] Guard no_sanitize_memory attributes behind __has_feature.
Otherwise GCC fails to build it because it doesn't know the attribute.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263787 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-18 14:19:19 +00:00
Kostya Serebryany
2b341f70ca [libFuzzer] read corpus dirs recursively
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263773 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-18 01:36:00 +00:00
Kostya Serebryany
ab641c1abd [libFuzzer] improve -merge functionality
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263769 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-18 00:23:29 +00:00
Kostya Serebryany
bcace10c40 [libFuzzer] deprecate several flags
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263739 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-17 19:59:39 +00:00
Kostya Serebryany
227603719a [libFuzzer] add __attribute__((no_sanitize_memory)) to two functions that may be called from signal handler(s) or from msan. This will hopefully avoid msan false reports which I can't reproduce
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263737 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-17 19:42:35 +00:00
Mike Aizatsky
fa4edb682f [libfuzzer] speeding up corpus load
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263591 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-15 21:47:21 +00:00
Kostya Serebryany
2f2c2e3539 [libFuzzer] use max_len exactly equal to the max size of input. Fix 32-bit build
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263518 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-15 01:28:00 +00:00
Kostya Serebryany
b30f32650e [libFuzzer] try to use max_len based on the items of the corpus instead of blindly defaulting to 64 bytes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263323 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-12 01:57:04 +00:00
Mike Aizatsky
c0d25bf9b8 [libfuzzer] adding std:string to allowed adaptable argument.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262757 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-04 23:18:01 +00:00
Kostya Serebryany
2c1ecb8c48 [libFuzzer] log less when re-loading files; fix a silly bug: when running single files actually run all of them, not just the first one
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262754 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-04 22:35:40 +00:00
Mike Aizatsky
d7ee221d4d [libfuzzer] arbitrary function adapter.
The adapter automates converting sequence of bytes into arbitrary
arguments.

Differential Revision: http://reviews.llvm.org/D17829

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262673 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-03 23:45:29 +00:00
Kostya Serebryany
2ef77db652 [libFuzzer] when interrupted, call _Exit() instead of exit()
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262667 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-03 22:36:37 +00:00
Kostya Serebryany
c2ecfd6993 [libFuzzer] add -Werror for libFuzzer build rule
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262517 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-02 21:08:16 +00:00
Dmitry Vyukov
f984411dc5 libfuzzer: fix compiler warnings
- unused sigaction/setitimer result (used in assert)
- unchecked fscanf return value
- signed/unsigned comparison



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262472 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-02 09:54:40 +00:00
Kostya Serebryany
66bb64fd43 [libFuzzer] deprecate exit_on_first flag
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262417 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-01 22:33:14 +00:00
Kostya Serebryany
7a1dcf9965 [libFuzzer] add generic signal handlers so that libFuzzer can report at least something if ASan is not handlig the signals for us. Remove abort_on_timeout flag.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262415 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-01 22:19:21 +00:00
Kostya Serebryany
8ca429497a [libFuzzer] remove FuzzerSanitizerOptions.cpp
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262354 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-01 17:46:32 +00:00
Rafael Espindola
93ed620d27 Refactor duplicated code for linking with pthread.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262344 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-01 15:54:40 +00:00
Kostya Serebryany
0b3b8cba94 [libFuzzer] fixing the bot
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262106 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-27 03:14:23 +00:00
Kostya Serebryany
a6b4efd1d0 [libFuzzer] speedup path coverage handling
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262102 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-27 01:50:16 +00:00
Kostya Serebryany
86b1b67565 [libFuzzer] add -print_final_stats=1 flag
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262084 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-26 22:42:23 +00:00
Kostya Serebryany
1ff29eb9ca [libFuzzer] initial implementation of path coverage based on -fsanitize-coverage=trace-pc. This does not scale well yet, but already cracks FullCoverageSetTest in seconds
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@262073 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-26 21:33:56 +00:00
Kostya Serebryany
a755f1bd65 [libFuzzer] only read MaxLen bytes from every file in the corpus to speedup loading the corpus
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261267 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-18 21:49:10 +00:00
Kostya Serebryany
b73d5ba466 [libFuzzer] fix the libFuzzer bot
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261184 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-18 02:02:40 +00:00
Kostya Serebryany
73b0e08885 [libFuzzer] don't timeout when loading the corpus. Be a bit more verbose when loading large corpus.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261143 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-17 19:42:34 +00:00
Kostya Serebryany
2d7392fe48 [libFuzzer] remove std::vector operations from hot paths, NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260829 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-13 17:56:51 +00:00
Kostya Serebryany
49429cee7f [libFuzzer] don't require seed in fuzzer::Mutate, instead use the global Fuzzer object for fuzzer::Mutate. This makes custom mutators fast
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260810 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-13 06:24:18 +00:00
Kostya Serebryany
5a08f1b013 [libFuzzer] remove the C++-ish variant of FuzzerDriver from the interface
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260801 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-13 03:59:26 +00:00
Kostya Serebryany
d1eacb8ffc [libFuzzer] simplify CTOR of MutationDispatcher
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260800 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-13 03:46:26 +00:00
Kostya Serebryany
1d35d47455 [libFuzzer] get rid of MutationDispatcher::Impl (simplify the code; NFC)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260799 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-13 03:37:24 +00:00
Kostya Serebryany
efb0cc7640 [libFuzzer] get rid of UserSuppliedFuzzer; NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260798 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-13 03:25:16 +00:00
Kostya Serebryany
728ca1266f [libFuzzer] simplify the code around Random. NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260797 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-13 03:00:53 +00:00
Kostya Serebryany
f71ac00d01 [libFuzzer] remove UserSuppliedFuzzer from the interface (it was a bad idea).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260796 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-13 02:39:30 +00:00
Kostya Serebryany
e6d7e3d948 [libFuzzer] provide a plain C interface for custom mutators (experimental)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260794 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-13 02:29:38 +00:00
Kostya Serebryany
14c6007ab2 [libFuzzer] make -runs=N flag also affect the simple runner (will execute every input N times)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260649 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-12 02:32:03 +00:00
Mike Aizatsky
dff48e6497 [libfuzzer] Removing coverage-related flags from asan options.
Summary:
Reasons to remove are twofold:
 - we don't really need coverage=1 for libfuzzer operation
 - makes controlling coverage for fuzzer processes non-trivial.

Differential Revision: http://reviews.llvm.org/D17168

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260611 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-11 22:20:34 +00:00
Kostya Serebryany
5a91878f44 [libFuzzer] hot fix a test
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259732 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-04 00:12:28 +00:00
Kostya Serebryany
598f7017b4 [libFuzzer] don't write the test unit when a leak is detected (since we don't know which unit causes the leak)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259731 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-04 00:02:17 +00:00
Kostya Serebryany
7cec6c634b [libFuzzer] don't create too many trace-based mutations as it may be too slow
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259600 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-02 23:17:45 +00:00
Kostya Serebryany
485551ecaf [libFuzzer] allow passing 1 or more files as individual inputs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259459 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-02 03:03:47 +00:00
Kostya Serebryany
05de8e95f6 [libFuzzer] fail if the corpus dir does not exist
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259454 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-02 02:07:26 +00:00
Kostya Serebryany
58b3c64b6b [libFuzzer] add -timeout_exitcode option
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259265 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-29 23:30:07 +00:00
Kostya Serebryany
fb614ec4d4 [libFuzzer] re-enable test for -abort_on_timeout=1, this time protecting from ASAN_OPTIONS set outside
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@259263 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-29 23:19:00 +00:00
Ivan Krasin
79890da71d Temporary disable broken fuzzer/timeout tests.
Reviewers: kcc

Differential Revision: http://reviews.llvm.org/D16543

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258702 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-25 19:05:45 +00:00
Kostya Serebryany
d75ddafc2f [libFuzzer] add -abort_on_timeout option
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258631 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-23 19:34:19 +00:00
Kostya Serebryany
21a169fad1 [libFuzzer] add more fields to DictionaryEntry to count the number of uses and successes
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258589 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-22 23:55:14 +00:00
Ivan Krasin
da57df2854 Use std::piecewise_constant_distribution instead of ad-hoc binary search.
Summary:
Fix the issue with the most recently discovered unit receiving much less attention.

Note: this is the second attempt (prev: r258473). Now, libc++ build is fixed.

Reviewers: aizatsky, kcc

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D16487

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258571 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-22 22:28:27 +00:00
Ivan Krasin
55b3567cb1 Revert r258473 as it's breaking the build with libc++
Reviewers: kcc

Differential Revision: http://reviews.llvm.org/D16441

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258479 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-22 03:21:52 +00:00
Ivan Krasin
3e0fdb8944 Use std::piecewise_constant_distribution instead of ad-hoc binary search.
Summary:
Fix the issue with the most recently discovered unit receiving much less attention.

Note: I had to change the seed for one test to make it pass. Alternatively,
the number of runs could be increased. I believe that the average time of
'foo' discovery is not increased, just seed=1 was particularly convenient
for the previous PRNG scheme used.

Reviewers: aizatsky, kcc

Subscribers: llvm-commits, kcc

Differential Revision: http://reviews.llvm.org/D16419

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258473 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-22 01:32:34 +00:00
Kostya Serebryany
75da488104 [libFuzzer] don't do expensive memmem if the result will not be used
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258462 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-22 01:04:58 +00:00
Kostya Serebryany
7e85cfa5be [libFuzzer] don't use std::vector in one more hot path
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258380 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-21 01:52:14 +00:00
Mike Aizatsky
b1020e3809 [libfuzzer] use %p for printing addresses
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258370 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-21 00:02:09 +00:00
Kostya Serebryany
429a7edf9b [libFuzzer] use std::mt19937 for generating random numbers by default. Fix MyStoll to handle negative values. Use std::any_of instead of std::find_if
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258178 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-19 20:33:57 +00:00
Kostya Serebryany
baa00e52a5 [libFuzzer] replace vector with a simpler data structure in the Dictionaries to avoid memory allocations on hot path
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257985 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-16 03:53:32 +00:00
Kostya Serebryany
1e595319e3 [libFuzzer] introduce LLVMFuzzerInitialize
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257980 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-16 01:23:12 +00:00
Kostya Serebryany
f7dd1d2c0c [libFuzzer] move some code from public interface header to a non-public header. NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257963 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-16 00:04:36 +00:00
Kostya Serebryany
f1af856009 [libFuzzer] do mutations based on memcmp/strcmp interceptors under a separate flag (-use_memcmp, default=1)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257873 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-15 06:24:05 +00:00
Kostya Serebryany
9039b1ff4e [libFuzzer] use custom stol; also introduce __libfuzzer_is_present so that users can check for its presence.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257848 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-15 00:17:37 +00:00
Kostya Serebryany
a416b73b12 [libFuzzer] suggest a dictionary to the user of some of the trace-based dictionary entries were successful
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257736 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-14 02:36:44 +00:00
Kostya Serebryany
92e8dcd607 [libFuzzer] make CurrentUnit a POD object instead of vector to avoid extra allocations
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257713 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-13 23:46:01 +00:00
Kostya Serebryany
3888fb079a [libFuzzer] make sure we find buffer overflow in the input buffer. Previously, re-using the same vector object was hiding buffer overflows (unless we used annotated vector)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257701 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-13 23:02:30 +00:00
Kostya Serebryany
43a24b5d93 [libFuzzer] make sure to update CurrentUnit when drilling
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257560 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-13 01:58:27 +00:00
Kostya Serebryany
53ff84bf11 [libFuzzer] add a macro LLVM_FUZZER_DEFINES_SANITIZER_WEAK_HOOOKS
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257482 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-12 16:50:18 +00:00
Kostya Serebryany
7b0624d17e [libFuzzer] when a new unit is discovered using a dictionary, print all used dictionary entries
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257435 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-12 02:36:59 +00:00
Kostya Serebryany
71afbb96dc [libFuzzer] add various debug prints. Also don't mutate based on a cmp trace like (a eq a) or (a neq a)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257434 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-12 02:08:37 +00:00
Kostya Serebryany
d89bfb65a8 [libFuzzer] extend the weak memcmp/strcmp/strncmp interceptors to receive the result of the computations. With that, don't do any mutations if memcmp/etc returned 0
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257423 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-12 00:43:42 +00:00
Kostya Serebryany
38c36e3216 [libFuzzer] debug prints in tracing
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257249 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-09 03:46:08 +00:00