Commit Graph

1865 Commits

Author SHA1 Message Date
Dmitry Vyukov
1d3e907710 tools/syz-symbolize: assume that kernel is in the current dir 2017-12-27 15:02:46 +01:00
Dmitry Vyukov
9dd89763a1 syz-ci: don't mark build errors as corrupted
Otherwise they get merged with other corrupted reports.
2017-12-27 14:28:08 +01:00
Dmitry Vyukov
a113b8e4d7 dashboard/app: handle no bug ID in emails better
We see these warnings when people CC syzbot on unrelated threads.
If there is no command in the email, don't produce warning at all.
On the other hand if there is a command, warn and send sender
reply that syzbot can't find the corresponding bug.
2017-12-27 13:14:53 +01:00
Dmitry Vyukov
fd3e9f2b97 executor: introduce uint64/32/16/8 types
The "define uint64_t unsigned long long" were too good to work.
With a different toolchain I am getting:

cstdint:69:11: error: expected unqualified-id
  using ::uint64_t;
          ^
executor/common.h:34:18: note: expanded from macro 'uint64_t'

Do it the proper way: introduce uint64/32/16/8 types and use them.

pkg/csource then does s/uint64/uint64_t/ to not clutter code with
additional typedefs.
2017-12-27 11:15:04 +01:00
Dmitry Vyukov
34c18f5f43 executor: fix another format bug
Detected only by clang.
2017-12-27 10:21:12 +01:00
Dmitry Vyukov
09c8f4c00a pkg/report: add more cases where we fail to parse reports 2017-12-27 09:18:26 +01:00
Dmitry Vyukov
0c5edf4180 executor: fix reply status on loop kills
We use exitf on loop failures, anbd exitf is retry-able.
However, we use different status when replying to ipc,
and that different status is what ipc actually uses.
Use kRetryStatus status in case on unexpected loop failures.
2017-12-27 09:18:26 +01:00
Dmitry Vyukov
17f5c9e05f pkg/csource: add top-level repeat loop
Even if all 3 levels of processes in executor exit,
execprog will still recreate them.
Model the same in csource.
This matters when the inner process kills loop
and then everything stops.
2017-12-27 09:18:26 +01:00
Dmitry Vyukov
c1d8c1b352 pkg/csource: simplify generated code
We already have procid variable, no need to introduce i.
2017-12-27 09:18:26 +01:00
Dmitry Vyukov
376f424e08 vm/gce: connect to instances by ip
Don't connect by hostname, this seems to be broken on GCE.
Episodically connecting by hostname gives:

Could not resolve hostname: Name or service not known
2017-12-27 09:18:26 +01:00
Dmitry Vyukov
b7b7ac19fd executor: check format strings
I see a crash which says:

	#0: too much cover 0 (errno 0)

while the code is:

	uint64_t n = ...;
	if (n >= kCoverSize)
		fail("#%d: too much cover %u", th->id, n);

It seems that the high part of n is set, but we don't see it.

Add printf format attribute to fail and friends and fix all similar cases.
Caught a bunch of similar cases and a missing argument in:

exitf("opendir(%s) failed due to NOFILE, exiting");
2017-12-27 09:18:26 +01:00
Dmitry Vyukov
6f03c35620 dashboard/app: extract fixing tags from commits
Support the new scheme of associating fixing commits with bugs.
Now we provide a tag along the lines of:

Reported-by: <syzbot+a4a91f6fc35e102@syzkaller.appspotmail.com>

The tag is supposed to be added to the commit.
Then we parse commit logs and extract these tags.

The final part on the dashboard is not ready yet,
but syz-ci should already parse and send the tags.
2017-12-27 09:09:18 +01:00
Dmitry Vyukov
73aba437a7 dashboard/app: bump max repros per bug to 10
We badly need repros. Developers complain.
2017-12-22 19:59:56 +01:00
Dmitry Vyukov
8e40909025 pkg/csource: mimic the way syscalls are scheduled in executor
Currently csource uses completely different, simpler way of scheduling
syscalls onto threads (thread per call with random sleeps).
Mimic the way calls are scheduled in executor.

Fixes #312
2017-12-22 18:55:38 +01:00
Dmitry Vyukov
26cd53f078 executor: remove dead code
doexit already contains an infinite loop.
2017-12-22 13:42:27 +01:00
Dmitry Vyukov
3645389673 pkg/csource: fix handling of proc types
Generated program always uses pid=0 even when there are multiple processes.
Make each process use own pid.

Unfortunately required to do quite significant changes to prog,
because the current format only supported fixed pid.

Fixes #490
2017-12-22 11:59:46 +01:00
Dmitry Vyukov
6f298a18e5 pkg/csource: limit thread stacks
We always set RLIMIT_AS to 128MB. I've debugged a program with 21 syscalls.
With collide it creates 42 threads. With default stack size of 8MB this
requires: 42*8 = 336MB. Thread creation fails and nothing works.
Limit thread stacks the same way executor does.

Fixes #488
2017-12-22 11:59:46 +01:00
Dmitry Vyukov
7b62abdb0a pkg/csource: fix string escaping bug 2017-12-22 11:59:46 +01:00
Ed Maste
81fe66b42a Makefile: fix git "modified tree" test
Makefile falsely detected a modified git tree when the first ifeq
argument was unquoted. Also switch to the $(shell ... style for
consistency.

As requested, add myself to AUTHORS and CONTRIBUTORS.
2017-12-21 13:53:39 +01:00
Tim Tianyang Chen
eaadba986d syz-manager: enable sending group emails
Email_Addr variable has been changed to Email_Addrs that contains
a list of recipient.

Signed-off-by: Tim Tianyang Chen <soapcn@gmail.com>
2017-12-21 10:27:12 +01:00
Tim Tianyang Chen
49bed8cf4d syz-manager: remove duplicated emails on restart
With commit: syz-manager: add simple email support, it will send
emails when a bug is hit for the first time during that particular
run of syz-manager. In other words, if you restart syz-manager and
the same bug is hit, a new email will be sent again. This is due to
the fact that mgr.crashTypes[crash.Title] doesn't keep track of logs
already written to the disk.

Fixed by moving emailCrash() to logic handling log writing.

Fixes #484

Signed-off-by: Tim Tianyang Chen <soapcn@gmail.com>
2017-12-21 10:27:12 +01:00
Dmitry Vyukov
90a46995a8 Makefile: fix Android builds
There are 2 known problems with current Android support:
1. It does not work with newer NDK
(happens on every NDK update).
2. Dynamic Go binaries do not start on Android emulator.

Drop special Android support and just build static linux binaries.

For context see:
https://groups.google.com/forum/#!msg/syzkaller/etg1ZJmTMzg/NYE-yjxxAQAJ
https://groups.google.com/d/msg/syzkaller/8KjCYWslTFY/1oTXn5tTAgAJ

Fixes #478
2017-12-20 10:52:19 +01:00
Dmitry Vyukov
2d836b1d35 dashboard/app: add default maintainers to email config
Crashes without maintainers are nasty. There is no way to do
anything with them without altering the datastore (they are not mailed).

Add DefaultMaintainers to email config.
These addresses are added to all reported bugs as maintainers (e.g. LKML).
One the report is mailed it's possible to CC more people on it.
2017-12-19 15:03:33 +01:00
Dmitry Vyukov
af9163c763 pkg/report: add test where we fail to detect guilty function 2017-12-19 13:44:03 +01:00
Dmitry Vyukov
779a655b43 pkg/report: add test where we fail to detect corrupted report 2017-12-19 13:39:14 +01:00
Dmitry Vyukov
a87e30dc1b pkg/email: improve parsing of splitted lines
Allow:

full-commit-title-on-next-line

This allows commit titles between 70 and 80 cols with gmail.

Also be more permissive wrt spaces and tabs.
2017-12-19 13:36:40 +01:00
Dmitry Vyukov
f2909d097e dashboard/app: add API for polling for closed bugs
External reporting may need to know when dashboard
is not interested in bugs anymore.
Add API that returns list of bugs dashboard considers closed.
2017-12-19 11:05:42 +01:00
Dmitry Vyukov
25793abb59 syz-fuzzer: wipe all global state 2017-12-19 10:53:39 +01:00
Dmitry Vyukov
8d5ba3821f syz-fuzzer: move program logging into a separate function 2017-12-18 15:51:00 +01:00
Dmitry Vyukov
d30c3e81eb pkg/report: add test for a bug in report parsing 2017-12-18 15:34:08 +01:00
Dmitry Vyukov
1c4160efd5 syz-fuzzer: encapsulate signal 2017-12-18 14:52:50 +01:00
Dmitry Vyukov
a20097eafe syz-manager, syz-fuzzer: allow re-minimizing/re-smashing inputs
By default we don't re-minimize/re-smash programs from corpus,
it takes lots of time on start and is unnecessary.
However, when we improve/fix minimization/smashing,
we may want to.

Introduce corpus database versions and allow to re-minimize/re-smash
on version bumps.
2017-12-18 14:10:56 +01:00
Dmitry Vyukov
465b0b7833 syz-fuzzer: improve deflaking during minimization
Currently we run an input 3 times to get minimal new coverage,
and then during minimization trying only 1 time to get the same coverage.
This plays poorly with flaky kernel coverage.
Require at least 1 out of 3 runs during minimization to get the same new coverage.
Experimental results suggest that this leads to higher quality corpus
(though, systematic tuning proved to be very hard due to flakes
and hard to explain effects on corpus size, program size, coverage and signal).
2017-12-18 14:10:23 +01:00
Dmitry Vyukov
9f48e03d80 syz-fuzzer: encapsulate corpus in fuzzer
Make corpus a fuzzer member rather than global var.
This resolves existing races on corpus.
2017-12-18 11:40:51 +01:00
Dmitry Vyukov
0d231ceb73 syz-fuzzer: refactor
syz-fuzzer organically grew from a small nice main function
into a huge single-file monster with tons of global state.

Start refactoring it into something more managable.
This change separates 2 things:
1. Proc: a single fuzzing process (ipc.Env wrapper).
2. WorkQueue: holds global non-fuzzing work items.
More work needed, but this is good first step.
2017-12-18 09:50:17 +01:00
Dmitry Vyukov
d5beb42ace pkg/csource: make strings more readable
If string contains a file name or a crypto alg name,
don't escape it all to hex.
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
1db7a350a9 vm/qemu: respect cpu config parameter
Fixes #42
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
a33677f8bf prog: use dense indexes for copyout instructions
Fixes #174
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
fea5478f46 prog: add DeserializeExec
Factor out program parsing from pkg/csource.
csource code that parses program and at the same time
formats output is very messy and complex.
New aproach also allows to understand e.g.
when a call has copyout instructions which is
useful for better C source output.
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
431d3c90b1 pkg/csource: refactor
csource.go is too large and messy.
Move Build/Format into buid.go.
Move generation of common header into common.go.
Split generation of common header into smaller managable functions.
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
9004acd9cc prog: remove unused writeArg argument 2017-12-17 11:39:14 +01:00
Dmitry Vyukov
dcfdc02b77 prog: minor refactoring around arguments
Introduce isUsed(arg) helper, use it in several places.
Move method definitions closer to their types.
Simplify presence check for ArgUsed.Used() in several places.
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
9c21f3116f executor: remove unneeded NONFAILING
copyin is already NONFAILING.
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
26eab5e866 syz-manager: speed up main page rendering
Remove unused calculation of total coverage.
Reduce critical section duration.
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
535474e22e syz-manager: add favicon handler 2017-12-17 11:39:14 +01:00
Dmitry Vyukov
35ec89f46f syz-fuzzer: turn off collide during triage/minimization 2017-12-17 11:39:14 +01:00
Dmitry Vyukov
a80750a095 syz-fuzzer: remove unused execute flag 2017-12-17 11:39:14 +01:00
Dmitry Vyukov
c5826ff7aa pkg/ipc: make threaded/collide per-program options
Currently threaded/collide are global environment flags.
It can be useful to turn off collider during some executions
(minimization, triage, etc).
Make them per-program options.
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
8ef0050706 prog: don't serialize output data args
Fixes #188

We now will write just ""/1000 to denote a 1000-byte output buffer.
Also we now don't store 1000-byte buffer in memory just to denote size.
Old format is still parsed.
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
eaeccee1d7 pkg/report: add another test 2017-12-17 11:39:14 +01:00