Commit Graph

37 Commits

Author SHA1 Message Date
Marcos Pividori
0e996298b5 [libFuzzer] Properly use Handle instead of FD on Windows.
For Windows, sanitizers work with Handles, not with posix file descriptors,
because they use the windows-specific API. So we need to convert the fds to
handles before passing them to the sanitizer library.
After this change, close_fd_mask is fixed for Windows (this fix some tests too).

Differential Revision: https://reviews.llvm.org/D29548

llvm-svn: 294388
2017-02-08 00:03:18 +00:00
Marcos Pividori
af86b663cc [libFuzzer] Avoid undefined behavior, properly discard output to stdout/stderr.
Fix libFuzzer when setting -close_fd_mask to a non-zero value.
In previous implementation, libFuzzer closes the file descriptors for
stdout/stderr. This has some disavantages:

For `fuzzer-fdmask.test`, we write directly to stdout and stderr using the
file streams stdout and stderr, after the file descriptors are closed, which is
undefined behavior. In Windows, in particular, this was making the test fail.

Also, if we close stdout and we open a new file in libFuzzer, we get the file
descriptor 1, which could generate problem if some code assumes file descriptors
refers to stdout and works directly writing to the file descriptor 1, but it
will be writing to the opened file (for example using std::cout).

Instead of closing the file descriptors, I redirect the output to /dev/null on
linux and nul on Windows.

Differential Revision: https://reviews.llvm.org/D28718

llvm-svn: 292743
2017-01-22 01:58:45 +00:00
Marcos Pividori
750e7046bb [libFuzzer] Clean up headers and file formatting of LibFuzzer files.
Reorganize #includes to follow LLVM Coding Standards.
Include some missing headers. Required to use `Printf()`.

Aside from that, this patch contains no functional change.
It is purely a re-organization.

Differential Revision: https://reviews.llvm.org/D27363

llvm-svn: 289560
2016-12-13 17:46:11 +00:00
Zachary Turner
f7c5b2eef9 [LibFuzzer] Add Windows implementations of some IO functions.
This patch moves some posix specific file i/o code into a new
file, FuzzerIOPosix.cpp, and provides implementations for these
functions on Windows in FuzzerIOWindows.cpp.  This is another
incremental step towards getting libfuzzer working on Windows,
although it still should not be expected to be fully working.

Patch by Marcos Pividori
Differential Revision: https://reviews.llvm.org/D27233

llvm-svn: 288275
2016-11-30 21:44:26 +00:00
Zachary Turner
03329c2c8b [LibFuzzer] Split up some functions among different headers.
In an effort to get libfuzzer working on Windows, we need to make
a distinction between what functions require platform specific
code (e.g. different code on Windows vs Linux) and what code
doesn't.  IO functions, for example, tend to be platform
specific.

This patch separates out some of the functions which will need
to have platform specific implementations into different headers,
so that we can then provide different implementations for each
platform.

Aside from that, this patch contains no functional change.  It
is purely a re-organization.

Patch by Marcos Pividori
Differential Revision: https://reviews.llvm.org/D27230

llvm-svn: 288264
2016-11-30 19:06:14 +00:00
Kostya Serebryany
441ec03873 [libFuzzer] when shrinking the corpus, delete evicted files previously created by the current process
llvm-svn: 283682
2016-10-08 23:24:45 +00:00
Kostya Serebryany
4a57719450 [libFuzzer] more refactoring; NFC
llvm-svn: 282047
2016-09-21 02:05:39 +00:00
Kostya Serebryany
5e6a145744 [libFuzzer] refactoring: move the Corpus into a separate class; delete two unused experimental features
llvm-svn: 282042
2016-09-21 01:04:43 +00:00
Dan Liew
c8948855b6 [LibFuzzer] Declare and use sanitizer functions in `fuzzer::ExternalFunctions`
This fixes linking problems on OSX.

Unfortunately it turns out we need to use an instance of the
``fuzzer::ExternalFunctions`` object in several places so this
commit also replaces all instances with a single global instance.

It also turns out initializing a global ``fuzzer::ExternalFunctions``
before main is entered (i.e. letting the object be initialised by the
global initializers) is not safe (on OSX the call to ``Printf()`` in the
CTOR crashes if it is called from a global initializer) so we instead
have a global ``fuzzer::ExternalFunctions*`` and initialize it inside
``FuzzerDriver()``.

Multiple unit tests depend also depend on the
``fuzzer::ExternalFunctions*`` global so a ``main()`` function has been
added that initializes it before running any tests.

Differential Revision: http://reviews.llvm.org/D20943

llvm-svn: 272072
2016-06-07 23:32:50 +00:00
Kostya Serebryany
7dfd360c85 [libFuzzer] use __sanitizer_set_report_fd with -close_fd_mask. This allows us to keep asan reports when closing target's stderr
llvm-svn: 271053
2016-05-27 21:46:22 +00:00
Kostya Serebryany
cd9a640c0e [libFuzzer] use fflush after every Printf
llvm-svn: 264459
2016-03-25 20:31:26 +00:00
Kostya Serebryany
da046b76fb [libFuzzer] use fdopen+vfprintf instead of fsnprintf+write
llvm-svn: 264230
2016-03-24 00:57:32 +00:00
Kostya Serebryany
01af064f8d [libFuzzer] add a flag close_fd_mask so that we can silence spammy targets by closing stderr/stdout
llvm-svn: 263831
2016-03-18 20:58:29 +00:00
Kostya Serebryany
59ba2beb86 [libFuzzer] read corpus dirs recursively
llvm-svn: 263773
2016-03-18 01:36:00 +00:00
Mike Aizatsky
34cbd6a60e [libfuzzer] speeding up corpus load
llvm-svn: 263591
2016-03-15 21:47:21 +00:00
Kostya Serebryany
c6f46ed530 [libFuzzer] log less when re-loading files; fix a silly bug: when running single files actually run all of them, not just the first one
llvm-svn: 262754
2016-03-04 22:35:40 +00:00
Kostya Serebryany
7ef7f142ff [libFuzzer] only read MaxLen bytes from every file in the corpus to speedup loading the corpus
llvm-svn: 261267
2016-02-18 21:49:10 +00:00
Kostya Serebryany
a45a009f24 [libFuzzer] don't timeout when loading the corpus. Be a bit more verbose when loading large corpus.
llvm-svn: 261143
2016-02-17 19:42:34 +00:00
Kostya Serebryany
4b2ab57d9f [libFuzzer] allow passing 1 or more files as individual inputs
llvm-svn: 259459
2016-02-02 03:03:47 +00:00
Kostya Serebryany
dd149f22ae [libFuzzer] fail if the corpus dir does not exist
llvm-svn: 259454
2016-02-02 02:07:26 +00:00
Kostya Serebryany
a69b412cc5 [libFuzzer] compute base64 in-process instead of using an external lib. Since libFuzzer should not depend on anything, just re-implement base64 encoder. PR25746
llvm-svn: 254784
2015-12-04 22:29:39 +00:00
Kostya Serebryany
2f218614e9 [libFuzzer] make libFuzzer build even with a compiler that does not have sanitizer headers
llvm-svn: 253003
2015-11-13 01:54:40 +00:00
Kostya Serebryany
47e5e62e71 [libFuzzer] When -test_single_input crashes the test it is not necessary to write crash-file because input is already known to the user. Patch by Mike Aizatsky
llvm-svn: 250564
2015-10-16 22:41:47 +00:00
Kostya Serebryany
04cc0059e2 [libFuzzer] be more robust when dealing with files on disk (e.g. don't crash if a file was there but disappeared)
llvm-svn: 247066
2015-09-08 20:36:33 +00:00
Kostya Serebryany
0e83baec1a [libFuzzer] fix minor inefficiency, PR24584
llvm-svn: 246087
2015-08-26 21:55:19 +00:00
Kostya Serebryany
5a4f36556e [libFuzzer] use raw C IO to reduce the risk of a deadlock in a signal handler.
llvm-svn: 244707
2015-08-12 00:55:09 +00:00
Kostya Serebryany
7c2874be12 [libFuzzer] require the files and directories passed to the fuzzer to exist
llvm-svn: 242596
2015-07-18 00:03:37 +00:00
Kostya Serebryany
c28d1607f2 [lib/Fuzzer] start getting rid of std::cerr. Sadly, these parts of C++ library used in libFuzzer badly interract with the same code used in the target function and also with dfsan. It's easier to just not use std::cerr than to defeat these issues.
llvm-svn: 238078
2015-05-23 01:07:46 +00:00
Kostya Serebryany
a98902fdfc [lib/Fuzzer] when -sync_command=<CMD> is given, periodically execute 'CMD CORPUS' to synchronize with other processes
llvm-svn: 237617
2015-05-18 21:34:20 +00:00
Kostya Serebryany
9387837867 [lib/Fuzzer] use -fsanitize-coverage=trace-cmp when building LLVM with LLVM_USE_SANITIZE_COVERAGE; in lib/Fuzzer try to reload the corpus to pick up new units from other processes
llvm-svn: 236906
2015-05-08 21:30:55 +00:00
Kostya Serebryany
177467fad4 [lib/Fuzzer] on crash print the contents of the crashy input as base64
llvm-svn: 236548
2015-05-05 21:59:51 +00:00
Kostya Serebryany
c69c3a7d4e [fuzzer] Add support for token-based fuzzing (e.g. for C++). Allow string flags.
llvm-svn: 233745
2015-03-31 20:13:20 +00:00
Kostya Serebryany
31b6858756 [fuzzer] make multi-process execution more verbose; fix mutation to actually respect mutation depth and to never produce empty units
llvm-svn: 228170
2015-02-04 19:10:20 +00:00
Aaron Ballman
ea7bb26fdf Reverting r227452, which adds back the fuzzer library. Now excluding the fuzzer library based on LLVM_USE_SANITIZE_COVERAGE being set or unset.
llvm-svn: 227464
2015-01-29 16:58:29 +00:00
Aaron Ballman
28eea44386 Temporarily reverting the fuzzer library as it causes too many build issues for MSVC users. This reverts: 227445, 227395, 227389, 227357, 227254, 227252
llvm-svn: 227452
2015-01-29 15:49:22 +00:00
Kostya Serebryany
940c7a3149 [fuzzer] add option -save_minimized_corpus
llvm-svn: 227395
2015-01-28 23:48:39 +00:00
Kostya Serebryany
87931c3355 Add a Fuzzer library
Summary:
A simple genetic in-process coverage-guided fuzz testing library.

I've used this fuzzer to test clang-format
(it found 12+ bugs, thanks djasper@ for the fixes!)
and it may also help us test other parts of LLVM.
So why not keep it in the LLVM repository?

I plan to add the cmake build rules later (in a separate patch, if that's ok)
and also add a clang-format-fuzzer target.

See README.txt for details.

Test Plan: Tests will follow separately.

Reviewers: djasper, chandlerc, rnk

Reviewed By: rnk

Subscribers: majnemer, ygribov, dblaikie, llvm-commits

Differential Revision: http://reviews.llvm.org/D7184

llvm-svn: 227252
2015-01-27 22:08:41 +00:00