135 Commits

Author SHA1 Message Date
Kostya Serebryany
93bacfd838 [libFuzzer] improve error handling during the merge (handle various IO failures)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291182 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-05 22:05:47 +00:00
Kostya Serebryany
d31ce373a9 [libFuzzer] add an experimental flag -experimental_len_control=1 that sets max_len to 1M and tries to increases the actual max sizes of mutations very gradually (second attempt)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290637 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-27 23:24:55 +00:00
Kostya Serebryany
5249221f24 [libFuzzer] fix UB and simplify the computation of the RNG seed (https://llvm.org/bugs/show_bug.cgi?id=31456)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290622 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-27 19:51:34 +00:00
Mike Aizatsky
d576930959 [libfuzzer] dump_coverage command line flag
Reviewers: kcc, vitalybuka

Differential Revision: https://reviews.llvm.org/D27942

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290138 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-19 22:18:08 +00:00
Daniel Jasper
a21e8a0091 Revert "[libFuzzer] add an experimental flag -experimental_len_control=1 that sets max_len to 1M and tries to increases the actual max sizes of mutations very gradually. Also remove a bit of dead code"
This reverts commit r289998.

See comment:
https://reviews.llvm.org/rL289998

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290043 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-17 12:27:49 +00:00
Kostya Serebryany
eb30028464 [libFuzzer] add an experimental flag -experimental_len_control=1 that sets max_len to 1M and tries to increases the actual max sizes of mutations very gradually. Also remove a bit of dead code
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289998 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-16 22:42:05 +00:00
Kostya Serebryany
abe2ee53b6 [libFuzzer] enable the failure-resistant merge by default (with trace-pc-guard only)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289772 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-15 06:21:21 +00:00
Marcos Pividori
f51c50474c [libFuzzer] Clean up headers and file formatting of LibFuzzer files.
Reorganize #includes to follow LLVM Coding Standards.
Include some missing headers. Required to use `Printf()`.

Aside from that, this patch contains no functional change.
It is purely a re-organization.

Differential Revision: https://reviews.llvm.org/D27363

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289560 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-13 17:46:11 +00:00
Marcos Pividori
611506b936 [libFuzzer] Properly use unsigned for workers, jobs and NumberOfCpuCores.
std:🧵:hardware_concurrency() returns an unsigned, so I modify
NumberOfCpuCores() to return unsigned too.
The number of cpus is used to define the number of workers, so I decided
to update the worker and jobs flags to be declared as unsigned too.

Differential Revision: https://reviews.llvm.org/D27685

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289559 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-13 17:45:53 +00:00
Marcos Pividori
9fb21ea4f5 [libFuzzer] Improve Signal Handler interface.
Add new flags to FuzzingOptions to represent the different conditions
on the signal handling. These options are passed when calling
SetSignalHandler().
This changes simplify the implementation of Windows's exception
handling. Now we can define a unique handler for all the exceptions.

Differential Revision: https://reviews.llvm.org/D27238

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289557 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-13 17:45:20 +00:00
Kostya Serebryany
d7b8181022 [libFuzzer] don't require extra flags with -minimize_crash=1 (default to -max_total_time=600). Also respect exact_artifact_path when outputting the end result
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289506 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-13 00:40:47 +00:00
Kostya Serebryany
a3def13005 [libFuzzer] implement crash-resistant merge (https://github.com/google/sanitizers/issues/722). This is a first experimental variant that needs some more testing, thus not yet adding a lit test (but there are unit tests).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@289166 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-09 01:17:24 +00:00
Zachary Turner
1a45ade2ce Resubmit "[LibFuzzer] Split FuzzerUtil for Posix and Windows."
This resubmits r288529, which was resubmitted because it broke a
fuzzer bot.  According to kcc@ the test that broke was flakey
and it is unlikely to be a result of this patch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288549 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-02 23:02:01 +00:00
Zachary Turner
609477eea7 Revert "[LibFuzzer] Split FuzzerUtil for Posix and Windows."
This reverts commit r288529, as it seems to introduce some
problems on the Linux bots.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288533 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-02 20:54:56 +00:00
Zachary Turner
fd186677a2 [LibFuzzer] Split FuzzerUtil for Posix and Windows.
Pave the way for separating out platform specific
utility functions into separate files.

Patch by Marcos Pividori
Differential Revision: https://reviews.llvm.org/D27234

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288529 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-02 19:38:19 +00:00
Zachary Turner
1a3900e81e [LibFuzzer] Split up some functions among different headers.
In an effort to get libfuzzer working on Windows, we need to make
a distinction between what functions require platform specific
code (e.g. different code on Windows vs Linux) and what code
doesn't.  IO functions, for example, tend to be platform
specific.

This patch separates out some of the functions which will need
to have platform specific implementations into different headers,
so that we can then provide different implementations for each
platform.

Aside from that, this patch contains no functional change.  It
is purely a re-organization.

Patch by Marcos Pividori
Differential Revision: https://reviews.llvm.org/D27230

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288264 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-30 19:06:14 +00:00
Kostya Serebryany
43aeb78cb8 [libFuzzer] replace 'auto' with 'auto *' to better follow the LLVM style
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286870 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-14 19:21:38 +00:00
Kostya Serebryany
8f3d5a342b [libFuzzer] use a valid ASCII string for a dummy seed corpus
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286702 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-12 02:27:21 +00:00
Kostya Serebryany
22480ffc37 [libFuzzer] use less stack
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286689 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-12 00:24:35 +00:00
Kostya Serebryany
2d1f4f5e9e [libFuzzer] fix -error_exitcode=N, now with a test
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@285958 91177308-0d34-0410-b5e6-96231b3b80d8
2016-11-03 19:31:18 +00:00
Kostya Serebryany
761c1ffc64 [libFuzzer] detect leaks after every run when executing fixed inputs (./fuzzer -runs=1000000 my-file)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284514 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-18 18:38:08 +00:00
Kostya Serebryany
75281e6436 [libFuzzer] better algorithm for -minimize_crash
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284299 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-15 01:00:24 +00:00
Kostya Serebryany
3af68729f8 [libFuzzer] add -trace_cmp=1 (guiding mutations based on the observed CMP instructions). This is a reincarnation of the previously deleted -use_traces, but using a different approach for collecting traces. Still a toy, but at least it scales well. Also fix -merge in trace-pc-guard mode
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284273 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-14 20:20:33 +00:00
Kostya Serebryany
f980fc0b37 [libFuzzer] add -trace_malloc= flag
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284149 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-13 19:06:46 +00:00
Kostya Serebryany
1a60ba886d [libFuzzer] when shrinking the corpus, delete evicted files previously created by the current process
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283682 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-08 23:24:45 +00:00
Kostya Serebryany
19e25ecdf5 [libFuzzer] control the reload interval by a flag, make it 10 seconds by default
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283676 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-08 22:12:14 +00:00
Kostya Serebryany
89268017c2 [libFuzzer] be more careful with memory usage, print peak rss in status lines
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283418 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-06 05:14:00 +00:00
Kostya Serebryany
cc6cbfdebc [libFuzzer] refactoring to make -shrink=1 work for value profile, added a test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283409 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-05 22:56:21 +00:00
Kostya Serebryany
d277734b71 [libFuzzer] clear the corpus elements if they are evicted (i.e. smaller elements with proper coverage are found). Make sure we never try to mutate empty element. Print the corpus size in bytes in the status lines
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283279 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-05 00:25:17 +00:00
Kostya Serebryany
f46303af79 [libFuzzer] implement the -shrink=1 option that tires to make elements of the corpus smaller, off by default
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282995 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-01 01:04:29 +00:00
Kostya Serebryany
5d78fc63a4 [libFuzzer] remove unused option
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282971 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-30 22:29:57 +00:00
Kostya Serebryany
65f502816c [libFuzzer] add -exit_on_src_pos to test libFuzzer itself, add a test script for RE2 that uses this flag
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282458 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-27 00:10:20 +00:00
Kostya Serebryany
07016d4bd5 [libFuzzer] fix merging with trace-pc-guard
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282224 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-23 01:58:51 +00:00
Kostya Serebryany
5a965a68f3 [libFuzzer] move value profiling logic into TracePC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282219 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-23 00:46:18 +00:00
Kostya Serebryany
43896df9dd [libFuzzer] simplify the crash minimizer; split MaxLen into two: MaxInputLen and MaxMutationLen, allow MaxMutationLen to be less than MaxInputLen
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282211 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-22 23:16:36 +00:00
Kostya Serebryany
e9b850b5c5 [libFuzzer] add 'features' to the corpus elements, allow mutations with Size > MaxSize, fix sha1 in corpus stats; various refactorings
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282129 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-22 01:34:58 +00:00
Kostya Serebryany
e4977b2b7e [libFuzzer] add stats to the corpus; more refactoring
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282121 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-21 22:42:17 +00:00
Kostya Serebryany
2a546868cb [libFuzzer] refactoring: split the large header into many; NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282044 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-21 01:50:50 +00:00
Kostya Serebryany
60dd435850 [libFuzzer] refactoring: move the Corpus into a separate class; delete two unused experimental features
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282042 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-21 01:04:43 +00:00
Kostya Serebryany
304a0b44f3 [libFuzzer] add -print_coverage=1 flag to print coverage directly from libFuzzer
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281866 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-18 21:47:08 +00:00
Kostya Serebryany
74ad0cfa1d [libFuzzer] don't print help for internal flags
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281124 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-10 00:35:30 +00:00
Kostya Serebryany
05e1dea79e [libFuzzer] remove use_traces=1 since use_value_profile seems to be strictly better
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281007 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-09 01:17:03 +00:00
Kostya Serebryany
501485f6e4 [libFuzzer] add -minimize_crash flag (to minimize crashers). also add two tests that I failed to commit last time
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280332 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-01 01:22:27 +00:00
Kostya Serebryany
72187d4993 [libFuzzer] fix a bug when running a single unit of N bytes with -max_len=M, M<N, caused a buffer overflow
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280098 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-30 14:52:05 +00:00
Kostya Serebryany
7750b380af [libFizzer] rename -print_new_cov_pcs=1 into -print_pcs=1 and make it more useful: print PCs only after the initial corpus has been read and symbolize them
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279787 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-25 22:35:08 +00:00
Kostya Serebryany
2e1aa9cd1c [libFuzzer] new experimental feature: value profiling. Profiles values that affect control flow and treats new values as new coverage.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278839 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-16 19:33:51 +00:00
Kostya Serebryany
e286c13fc8 [libFuzzer] print a verbose message after executing inputs in non-fuzzing mode
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278724 91177308-0d34-0410-b5e6-96231b3b80d8
2016-08-15 19:44:04 +00:00
Kostya Serebryany
730992dc3b [libFuzzer] add hooks for strstr, strcasestr, strcasecmp, strncasecmp
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275648 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-15 23:27:19 +00:00
Mike Aizatsky
8bbe924637 [libfuzzer] moving is_ascii handler inside mutation dispatcher.
Summary: It also fixes a bug, when first random might not be ascii.

Differential Revision: http://reviews.llvm.org/D21573

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@273611 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-23 20:44:48 +00:00
Kostya Serebryany
ee71b6abb0 [libFuzzer] make the single-run output more reliable
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272998 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-17 13:07:06 +00:00