46 Commits

Author SHA1 Message Date
Kostya Serebryany
300f834d9e [libFuzzer] increase kFeatureSetSize to 2^21 and make InputCorpus scale to that size. This will potentially make libFuzzer more sensitive on targets with lots of signals
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298671 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-24 00:45:15 +00:00
Kostya Serebryany
550e23d356 [libFuzzer] create experimental support for user-provided coverage signal
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298654 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-23 22:43:12 +00:00
Kostya Serebryany
5db3fb7fb6 [libFuzzer] add two experimental flags to make corpus merging more scalable: -save_coverage_summary/-load_coverage_summary. This is still WIP, the documentation will come later if these flags survive
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298548 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-22 20:32:44 +00:00
Kostya Serebryany
cc82ac2f3b [libFuzzer] use less memory for merge
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290039 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-17 08:20:24 +00:00
Kostya Serebryany
a3def13005 [libFuzzer] implement crash-resistant merge (https://github.com/google/sanitizers/issues/722). This is a first experimental variant that needs some more testing, thus not yet adding a lit test (but there are unit tests).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289166 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-09 01:17:24 +00:00
Kostya Serebryany
9b71256a5b [libFuzzer] mutation: insert the size of the input in bytes as one of the ways to mutate a binary integer
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284909 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-22 03:48:53 +00:00
Kostya Serebryany
1a60ba886d [libFuzzer] when shrinking the corpus, delete evicted files previously created by the current process
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283682 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-08 23:24:45 +00:00
Kostya Serebryany
cc6cbfdebc [libFuzzer] refactoring to make -shrink=1 work for value profile, added a test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283409 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-05 22:56:21 +00:00
Kostya Serebryany
07016d4bd5 [libFuzzer] fix merging with trace-pc-guard
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282224 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-23 01:58:51 +00:00
Kostya Serebryany
e9b850b5c5 [libFuzzer] add 'features' to the corpus elements, allow mutations with Size > MaxSize, fix sha1 in corpus stats; various refactorings
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282129 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-22 01:34:58 +00:00
Kostya Serebryany
e4977b2b7e [libFuzzer] add stats to the corpus; more refactoring
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282121 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-21 22:42:17 +00:00
Kostya Serebryany
a6b3c30ea2 [libFuzzer] more refactoring; don't compute sha1sum every time we mutate a unit from the corpus, use the stored one.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282115 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-21 21:41:48 +00:00
Kostya Serebryany
2a546868cb [libFuzzer] refactoring: split the large header into many; NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282044 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-21 01:50:50 +00:00
Kostya Serebryany
9744bb48e4 [libFuzzer] one more mutation: ChangeBinaryInteger; also fix the breakage from r278970
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278982 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-17 21:30:30 +00:00
Kostya Serebryany
79d100891f [libFuzzer] more mutations
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278950 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-17 18:10:42 +00:00
Kostya Serebryany
137c3957ed [libFuzzer] add InsertRepeatedBytes and EraseBytes.
New mutation: InsertRepeatedBytes.
Updated mutation: EraseByte => EraseBytes.

This helps https://github.com/google/sanitizers/issues/710
where libFuzzer was not able to find a known bug.
Now it finds it in minutes.

Hopefully, the change is general enough to help other targets.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278687 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-15 17:48:28 +00:00
Kostya Serebryany
730992dc3b [libFuzzer] add hooks for strstr, strcasestr, strcasecmp, strncasecmp
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275648 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-15 23:27:19 +00:00
Mike Aizatsky
8bbe924637 [libfuzzer] moving is_ascii handler inside mutation dispatcher.
Summary: It also fixes a bug, when first random might not be ascii.

Differential Revision: http://reviews.llvm.org/D21573

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@273611 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-23 20:44:48 +00:00
Dan Liew
7029af9787 [LibFuzzer] Fix `FuzzerMutate.ShuffleBytes2` unit test on OSX.
The ``FuzzerMutate.ShuffleBytes2`` unit test was failing on
OSX due to the implementation of ``std::random_shuffle()``
being different between libcxx and libstdc++.

@kcc has decided (see http://reviews.llvm.org/D21218) it is acceptable
for there to be different mutation behavior on different platforms so
this commit just adjusts the test to perform the minimum number of
iterations (that is a power of 2) to see all the mutations the unit test
is looking for.

Differential Revision: http://reviews.llvm.org/D21359

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272743 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-15 01:40:02 +00:00
Dan Liew
bb05836e33 [LibFuzzer] Fix some unit test crashes on OSX.
This fixes the following unit tests:

FuzzerDictionary.ParseOneDictionaryEntry
FuzzerDictionary.ParseDictionaryFile

The issue appears to be mixing non-ASan-ified code (LibFuzzer) and
ASan-ified code (the unittest) as the tests would pass fine if
everything was built with ASan enabled.

I believe the issue is that different implementations of std::vector<>
are being used in LibFuzzer and outside LibFuzzer (in the unittests).
For Libcxx (I've not seen the issue manifest for libstdc++) we can disable
the ASanified std::vector<> by definining the ``_LIBCPP_HAS_NO_ASAN`` macro.
Doing this fixes the tests on OSX.

Differential Revision: http://reviews.llvm.org/D21049

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272374 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-10 05:33:07 +00:00
Dan Liew
1a1c8bea2d [LibFuzzer] Declare and use sanitizer functions in `fuzzer::ExternalFunctions`
This fixes linking problems on OSX.

Unfortunately it turns out we need to use an instance of the
``fuzzer::ExternalFunctions`` object in several places so this
commit also replaces all instances with a single global instance.

It also turns out initializing a global ``fuzzer::ExternalFunctions``
before main is entered (i.e. letting the object be initialised by the
global initializers) is not safe (on OSX the call to ``Printf()`` in the
CTOR crashes if it is called from a global initializer) so we instead
have a global ``fuzzer::ExternalFunctions*`` and initialize it inside
``FuzzerDriver()``.

Multiple unit tests depend also depend on the
``fuzzer::ExternalFunctions*`` global so a ``main()`` function has been
added that initializes it before running any tests.

Differential Revision: http://reviews.llvm.org/D20943

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272072 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-07 23:32:50 +00:00
Mike Aizatsky
215450cd0e [libfuzzer] replacing unittest for truncate_units with functional test.
Differential Revision: http://reviews.llvm.org/D20641

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270755 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 21:00:17 +00:00
Mike Aizatsky
b0d0618635 [libfuzzer] Trying random unit prefixes during corpus load.
Differential Revision: http://reviews.llvm.org/D20301

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270632 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-24 23:14:29 +00:00
Mike Aizatsky
f4643f6392 [libfuzzer] adding license headers to cpp files
Differential Revision: http://reviews.llvm.org/D18705

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265174 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-01 18:38:58 +00:00
Kostya Serebryany
efb0cc7640 [libFuzzer] get rid of UserSuppliedFuzzer; NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260798 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-13 03:25:16 +00:00
Kostya Serebryany
728ca1266f [libFuzzer] simplify the code around Random. NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@260797 91177308-0d34-0410-b5e6-96231b3b80d8
2016-02-13 03:00:53 +00:00
Ivan Krasin
da57df2854 Use std::piecewise_constant_distribution instead of ad-hoc binary search.
Summary:
Fix the issue with the most recently discovered unit receiving much less attention.

Note: this is the second attempt (prev: r258473). Now, libc++ build is fixed.

Reviewers: aizatsky, kcc

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D16487

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258571 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-22 22:28:27 +00:00
Ivan Krasin
55b3567cb1 Revert r258473 as it's breaking the build with libc++
Reviewers: kcc

Differential Revision: http://reviews.llvm.org/D16441

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258479 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-22 03:21:52 +00:00
Ivan Krasin
3e0fdb8944 Use std::piecewise_constant_distribution instead of ad-hoc binary search.
Summary:
Fix the issue with the most recently discovered unit receiving much less attention.

Note: I had to change the seed for one test to make it pass. Alternatively,
the number of runs could be increased. I believe that the average time of
'foo' discovery is not increased, just seed=1 was particularly convenient
for the previous PRNG scheme used.

Reviewers: aizatsky, kcc

Subscribers: llvm-commits, kcc

Differential Revision: http://reviews.llvm.org/D16419

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@258473 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-22 01:32:34 +00:00
Kostya Serebryany
baa00e52a5 [libFuzzer] replace vector with a simpler data structure in the Dictionaries to avoid memory allocations on hot path
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257985 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-16 03:53:32 +00:00
Kostya Serebryany
a416b73b12 [libFuzzer] suggest a dictionary to the user of some of the trace-based dictionary entries were successful
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257736 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-14 02:36:44 +00:00
Kostya Serebryany
ab240104f0 [libFuzzer] add a position hint to the dictionary-based mutator
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@257013 91177308-0d34-0410-b5e6-96231b3b80d8
2016-01-07 01:49:35 +00:00
Kostya Serebryany
a1e5f35b02 [libFuzzer] make CrossOver just one of the other mutations
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@256081 91177308-0d34-0410-b5e6-96231b3b80d8
2015-12-19 02:49:09 +00:00
Kostya Serebryany
3f8065b694 [libFuzzer] compute base64 in-process instead of using an external lib. Since libFuzzer should not depend on anything, just re-implement base64 encoder. PR25746
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@254784 91177308-0d34-0410-b5e6-96231b3b80d8
2015-12-04 22:29:39 +00:00
Kostya Serebryany
872c556fa8 [libFuzzer] add one more mutator: Mutate_ChangeASCIIInteger
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@247027 91177308-0d34-0410-b5e6-96231b3b80d8
2015-09-08 17:19:31 +00:00
Kostya Serebryany
6cc3ed7cdd [libFuzzer] actually make the dictionaries work (+docs)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246825 91177308-0d34-0410-b5e6-96231b3b80d8
2015-09-04 00:12:11 +00:00
Kostya Serebryany
bf6bd487d5 [libFuzzer] refactor the mutation functions so that they are now methods of a class. NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246808 91177308-0d34-0410-b5e6-96231b3b80d8
2015-09-03 21:24:19 +00:00
Kostya Serebryany
446d906a52 [libFuzzer] adding a parser for AFL-style dictionaries + tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@246800 91177308-0d34-0410-b5e6-96231b3b80d8
2015-09-03 20:23:46 +00:00
Kostya Serebryany
596885b016 [libFuzzer] add one more mutation strategy: byte shuffling
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@244188 91177308-0d34-0410-b5e6-96231b3b80d8
2015-08-06 01:29:13 +00:00
Kostya Serebryany
0dd9ec77d1 [libFuzzer] more refactoring of the Mutator and adding tests to it
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@243818 91177308-0d34-0410-b5e6-96231b3b80d8
2015-08-01 02:23:06 +00:00
Kostya Serebryany
8d4c8061c9 [libFuzzer] start refactoring the Mutator and adding tests to it
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@243817 91177308-0d34-0410-b5e6-96231b3b80d8
2015-08-01 01:42:51 +00:00
Kostya Serebryany
33a9a09cd7 [libFuzzer] allow users to supply their own implementation of rand
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@243078 91177308-0d34-0410-b5e6-96231b3b80d8
2015-07-24 01:06:40 +00:00
Kostya Serebryany
ebada2c2bc [lib/Fuzzer] extend the fuzzer interface to allow user-supplied mutators
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238059 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-22 22:35:31 +00:00
Kostya Serebryany
6f1b9541ba [lib/Fuzzer] Add SHA1 implementation from public domain.
Summary:
This adds a SHA1 implementation taken from public domain code.
The change is trivial, but as it involves third-party code I'd like
a second pair of eyes before commit.

LibFuzzer can not use SHA1 from openssl because openssl may not be available
and because we may be fuzzing openssl itself.
Using sha1sum via a pipe is too slow.

Test Plan: n/a

Reviewers: chandlerc

Reviewed By: chandlerc

Subscribers: majnemer, llvm-commits

Differential Revision: http://reviews.llvm.org/D9733

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@237400 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-14 22:41:49 +00:00
Kostya Serebryany
b6ca45c39f [lib/Fuzzer] rename TestOneInput to LLVMFuzzerTestOneInput to make it more unique
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236652 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-06 22:19:00 +00:00
Kostya Serebryany
05efde62f5 [fuzzer] Add a gtest-style test
Summary: Add one gtest-style test.

Test Plan: run on bot

Reviewers: samsonov

Reviewed By: samsonov

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D7287

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227639 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-30 23:26:57 +00:00