Commit Graph

754 Commits

Author SHA1 Message Date
Vitaly Buka
6c78bdab49 [libFuzzer] Delete llvm/lib/Fuzzer
Summary: Code is already in compiler-rt

Reviewers: kcc

Subscribers: krytarowski, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D38912

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315937 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-16 20:48:19 +00:00
Tim Shen
811e9302eb [FuzzerUtil] Partially revert D38481 on FuzzerUtil
This is because lib/Fuzzer doesn't really depend on llvm infrastucture.
It's not easy to access the llvm hardware_concurrency here.

Differential Reivision: https://reviews.llvm.org/D38481

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@314870 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-04 01:05:34 +00:00
Rafael Espindola
70deadb171 Use sched_getaffinity instead of std:🧵:hardware_concurrency.
The issue with std:🧵:hardware_concurrency is that it forwards
to libc and some implementations (like glibc) don't take thread
affinity into consideration.

With this change a llvm program that can execute in only 2 cores will
use 2 threads, even if the machine has 32 cores.

This makes benchmarking a lot easier, but should also help if someone
doesn't want to use all cores for compilation for example.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@314809 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-03 16:25:15 +00:00
George Karpenkov
ec925a2578 Moving libFuzzer from LLVM to compiler-rt.
This change only removes libFuzzer tests and CMake machinery,
the source copy temporarily remains at the old location.

Differential Revision: https://reviews.llvm.org/D36980

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311405 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-21 23:25:12 +00:00
George Karpenkov
5078f592b0 Always compile libFuzzer with no coverage
Do not compile libFuzzer itself with coverage, regardless of LLVM variables

Differential Revision: https://reviews.llvm.org/D36887

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311374 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-21 20:12:58 +00:00
Kuba Mracek
a7c3f3d69c Get rid of even more "%T" expansions, see <https://reviews.llvm.org/D35396>.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311294 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-20 17:05:22 +00:00
Matt Morehouse
6dcfafe8ab [SanitizerCoverage] Add stack depth tracing instrumentation.
Summary:
Augment SanitizerCoverage to insert maximum stack depth tracing for
use by libFuzzer.  The new instrumentation is enabled by the flag
-fsanitize-coverage=stack-depth and is compatible with the existing
trace-pc-guard coverage.  The user must also declare the following
global variable in their code:
  thread_local uintptr_t __sancov_lowest_stack

https://bugs.llvm.org/show_bug.cgi?id=33857

Reviewers: vitalybuka, kcc

Reviewed By: vitalybuka

Subscribers: kubamracek, hiraditya, cfe-commits, llvm-commits

Differential Revision: https://reviews.llvm.org/D36839

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311186 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-18 18:43:30 +00:00
Petr Hosek
7fb0921b73 [CMake] Add install target for LLVMFuzzer
This allows including LLVMFuzzer as distribution component.

Differential Revision: https://reviews.llvm.org/D36540

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310897 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-14 23:37:31 +00:00
Kostya Serebryany
70284788c6 [libFuzzer] try to use less RAM while processing the initial corpus
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310881 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-14 20:34:35 +00:00
Kostya Serebryany
29a5193a62 [libFuzzer] explicitly use -fsanitize-coverage=trace-pc-guard in test/dump_coverage.test; mark print_coverage/dump_coverage as To-be-deprecated
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310877 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-14 19:55:23 +00:00
Kostya Serebryany
07fb566d4b [libFuzzer] experimental support for Clang's coverage (fprofile-instr-generate), Linux-only
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310771 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-11 23:03:22 +00:00
George Karpenkov
65b6f2d174 [libFuzzer] Re-enable coverage.test on Darwin.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310750 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-11 20:30:52 +00:00
Alexander Potapenko
d78be4451d [libFuzzer] Update LibFuzzer w.r.t. the new comparisons instrumentation API
Added the _sanitizer_cov_trace_const_cmp[1248] callbacks.
For now they are implemented the same way as _sanitizer_cov_trace_cmp[1248].
For more details, please see https://reviews.llvm.org/D36465.

Patch by Victor Chibotaru.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310592 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-10 14:01:45 +00:00
Kostya Serebryany
1aea640366 [libFuzzer] simplify code, NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310326 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-08 00:17:20 +00:00
Kostya Serebryany
7550cbd415 [libFuzzer] remove stale code
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310325 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-08 00:14:49 +00:00
Kostya Serebryany
468107aa76 [libFuzzer] simplify the implementation of -print_coverage=1
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310324 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-08 00:12:09 +00:00
George Karpenkov
79302ff8ea Do not instrument libFuzzer itself when built with -DLLVM_USE_SANITIZE_COVERAGE
Fixes regression from https://reviews.llvm.org/D36295

Differential Revision: https://reviews.llvm.org/D36428

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310305 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-07 20:56:11 +00:00
Kostya Serebryany
dea6df7765 [libFuzzer] use the in-binary pc table (instead of PCs captured at run-time) to implement -exit_on_src_pos
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310151 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-04 23:49:53 +00:00
Kostya Serebryany
0b67c739d8 [libFuzzer] print PCs using the in-binary PC-table instead of relying on PCs captured at run-time
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310148 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-04 23:13:58 +00:00
Kostya Serebryany
d7f757cc82 [libFuzzer] re-enable fuzzer-printcovpcs.test
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310126 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-04 20:47:22 +00:00
Kostya Serebryany
369072e544 [libFuzzer] make a test more robust
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310113 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-04 20:09:15 +00:00
Kostya Serebryany
6a3fa85472 [libFuzzer] remove the now redundant 'LLVMFuzzer-' prefix from libFuzzer tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310110 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-04 20:05:25 +00:00
Kostya Serebryany
c4d2f87680 [libFuzzer] split one test into several
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310106 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-04 20:01:04 +00:00
George Karpenkov
ba96863e68 [libFuzzer tests] Only enable libFuzzer tests if
-DLIBFUZZER_ENABLE_TESTS=ON is set.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310100 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-04 19:29:16 +00:00
Kostya Serebryany
abc4b62a7d [libFuzzer] make trace-pc.test more reliable
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310091 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-04 18:43:39 +00:00
George Karpenkov
e22da6aaf1 Fixing buildbots: do not register check-fuzzer if clang or asan are not
present.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310077 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-04 17:43:29 +00:00
George Karpenkov
dd5a253273 Drop Windows support from libFuzzer tests.
Differential Revision: https://reviews.llvm.org/D36205

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310076 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-04 17:43:28 +00:00
George Karpenkov
32fab3343b Port libFuzzer tests to LIT. Do not require two-stage build for check-fuzzer.
This revision ports all libFuzzer tests apart from the unittest to LIT.
The advantages of doing so include:

 - Tests being self-contained
 - Much easier debugging of a single test
 - No need for using a two-stage compilation

The unit-test is still compiled using CMake, but it does not need a
freshly built compiler.

NOTE: The previous two-stage bot configuration will NOT work, as in the
second stage build LLVM_USE_SANITIZER is set, which disables ASAN from
being built.
Thus bots will be reconfigured in the next few commits.

Differential Revision: https://reviews.llvm.org/D36295

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310075 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-04 17:19:45 +00:00
George Karpenkov
ca0f9bbbd8 Disable libFuzzer tests on Windows
Differential Revision: https://reviews.llvm.org/D36297

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310009 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-04 00:26:12 +00:00
George Karpenkov
48c84d8de6 [libFuzzer] Un-reverting change in tests after fixing the failure on Linux.
Differential Revision: https://reviews.llvm.org/D36242

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309982 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-03 20:28:16 +00:00
George Karpenkov
51c4734828 Revert "[libFuzzer tests] Use substring comparison in libFuzzer tests"
This reverts commit 3592d8049660dcdd07f7c2e797f2de9790f93111.

Breaks the bots, reverting for now.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309899 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-02 23:09:57 +00:00
George Karpenkov
6795f26af5 [libFuzzer tests] Use substring comparison in libFuzzer tests
LIT launches executables with absolute, and not relative, path.
strncmp would try to do exact comparison and fail.

Differential Revision: https://reviews.llvm.org/D36242

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309889 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-02 21:38:50 +00:00
Kostya Serebryany
9e6b2e7f9d [libFuzzer] temporarty remove pc-tables and disable test/fuzzer-printcovpcs.test until this can be fixed on Windows
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309716 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-01 18:02:19 +00:00
Kostya Serebryany
c19eec3262 [libFuzzer] implement more correct way of computing feature index for Inline8bitCounters
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309647 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-01 01:16:26 +00:00
Kostya Serebryany
c6d173bbf6 [libFuzzer] enable -fsanitize-coverage=pc-table for all tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309646 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-01 00:48:44 +00:00
Kostya Serebryany
f15adebe75 [libFuzzer] implement __sanitizer_cov_pcs_init and add pc-table to build flags for one test (for now)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309615 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-31 20:20:59 +00:00
Kostya Serebryany
dfa17126d8 [libFuzzer] improve support for inline-8bit-counters (make it more correct and faster)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309443 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-28 22:00:56 +00:00
Kostya Serebryany
8330752796 [libFuzzer] don't disable msan for TracePC::CollectFeatures: this started to cause false positives in msan. No tests for libFuzzer+msan yet -- tests will need to wait until we move libFuzzer to compiler-rt
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309038 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-25 22:05:31 +00:00
Kostya Serebryany
74b6501bc4 [libFuzzer] make one test faster, fix compiler warnings in tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@308945 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-25 02:09:46 +00:00
George Karpenkov
9bf22f2451 Revert "Revert "[libFuzzer] Add a dependency on symbolizer from libFuzzer tests""
This reverts commit 15425f2bc6eac6249ee957a2a280511306c07547.

Should work now that atos is a default symbolizer on Darwin.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@308910 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-24 18:38:14 +00:00
Kostya Serebryany
5ae0dfe6b2 [libFuzzer] reimplement experimental_len_control=1: bump the temporary max_len every time we failed to find new coverage during the last 1000 runs and 1 second. Also fix FileToVector to not load unfinished files
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@308811 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-22 00:10:29 +00:00
Matt Morehouse
5187ebbe87 Generate error reports when a fuzz target exits.
Summary:
Implements https://github.com/google/sanitizers/issues/835.

Flush stdout before exiting in test cases.

Since the atexit hook is used for exit reports, pending prints to
stdout can be lost if they aren't flushed before calling exit().

Expect tests to have non-zero exit code if exit() is called.

Reviewers: vitalybuka, kcc

Reviewed By: kcc

Subscribers: eraman, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D35602

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@308669 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-20 20:43:39 +00:00
Kostya Serebryany
29fa98aa84 [libFuzzer] delete stale code
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@308663 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-20 20:15:13 +00:00
Kostya Serebryany
9dbc448fd9 [libFuzzer] make sure CheckExitOnSrcPosOrItem is called after the new input is saved to the corpus
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@308653 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-20 18:53:25 +00:00
Kostya Serebryany
e531917082 [libFuzzer] prototype implementation of recursion-depth coverage features (commented out; real implementation needs to use inlined instrumentation)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@308577 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-20 01:35:17 +00:00
Kostya Serebryany
6c825738bb [libFuzzer] add DeepRecursionTest, inspired by https://guidovranken.wordpress.com/2017/07/08/libfuzzer-gv-new-techniques-for-dramatically-faster-fuzzing/ (Stack-depth-guided fuzzing). libFuzzer does not solve it yet.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@308571 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-20 00:37:08 +00:00
Reid Kleckner
77c923809c Try to deflake fuzzer-oom.test on Windows
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@308568 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-20 00:11:39 +00:00
Kostya Serebryany
b905fdae0c [libFuzzer] simplify two more tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@308560 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-19 23:52:54 +00:00
Kostya Serebryany
0199c64787 [libFuzzer] change several tests to not limit the max len: with reduce_inputs=1 they are now fast enough even w/o this
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@308553 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-19 23:45:46 +00:00
Reid Kleckner
9ed8bad9d6 Fix fuzzer-flags.test on Windows
The optional external function callbacks have to be exported in order
for them to be called. The test was failing because libFuzzer wasn't
calling LLVMFuzzerInitialize.

We can reconsider if this is the best way to mark these optional
callbacks exported later.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@308548 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-19 23:22:06 +00:00