223 Commits

Author SHA1 Message Date
Dmitry Vyukov
66f22a7fdc syz-manager: don't abort on disabled syscalls
It turns out to be too difficult to specify a precise set
of syscalls when, say, all setsockopt's for some sockets
need to be enabled, but not enabled for other sockets.
Just warn user about disabled syscalls, but don't abort.
2018-04-07 14:02:03 +02:00
Dmitry Vyukov
cd44cce1df syz-manager/mgrconfig: restore old enable/disable_syscalls behavior
The previous change in behavior break a bunch of existing configs
("bpf" does not match anything). Restore old behavior.
To get only write syscall, one can do:
enable_syscalls: "write",
disable_syscalls: "write$*"
2018-04-07 13:48:55 +02:00
Dmitry Vyukov
53196ce262 syz-manager: warn about disabled syscalls in enable_syscalls
syz-manager used to silently transitively disable syscalls
for which input resources can't be created.
This caused lots of confusion, or worse, users did not notice
that syzkaller does not actually test what they want.
Fail loudly with a readable explanation when a syscall
explicitly enabled in enable_syscalls is actually disabled.

Note: this requires to slightly change enable/disable_syscalls
matching logic. Previously "foo" would match "foo" and all "foo$BAR",
now it matches only "foo". But "foo*" can be used to match all
disciminations.
2018-04-06 21:08:54 +02:00
Dmitry Vyukov
48a846e42b syz-manager/mgrconfig: check that ssh key is properly protected 2018-04-06 17:49:50 +02:00
Dmitry Vyukov
0d845719c3 syz-manager: move syscall info to a separate page
It's rarely needed. So move from main page.
2018-04-02 20:09:30 +02:00
Dmitry Vyukov
188daeb272 syz-manager: show what crashes are currently being reproduced 2018-04-02 20:09:26 +02:00
Dmitry Vyukov
185ac3525e prog: support big-endian during hints matching
Use big-endian match/replace for both blobs and ints.
Sometimes we have unmarked blobs (no little/big-endian info);
for ANYBLOBs we intentionally lose all marking;
but even for marked ints we may need this too.
Consider that kernel code does not convert the data
(i.e. not ntohs(pkt->proto) == ETH_P_BATMAN),
but instead converts the constant (i.e. pkt->proto == htons(ETH_P_BATMAN)).
In such case we will see dynamic operand that does not
match what we have in the program.
2018-04-01 15:28:01 +02:00
Dmitry Vyukov
63f4cf6fff syz-manager: log when a crash report is corrupted 2018-03-31 17:27:58 +02:00
Dmitry Vyukov
fef07bd04c syz-manager: add special type=none
Type "none" is a special case for debugging/development when manager
does not start any VMs, but instead you start them manually
and start syz-fuzzer there.
2018-03-31 16:25:15 +02:00
Dmitry Vyukov
36d1c4540a all: fix gometalinter warnings
Fix typos, non-canonical code, remove dead code, etc.
2018-03-08 18:48:26 +01:00
Dmitry Vyukov
87044370b8 syz-manager: ask reproducers from hub only if connected to dashboard
All reproducers from hub produce too much load on local managers.
2018-02-22 20:21:07 +01:00
Dmitry Vyukov
04cbdbd1ae syz-fuzzer: prioritize signal from successful syscalls
Signal on successful syscalls is more valuable than
signal on unsuccessful syscalls.y
2018-02-20 20:51:41 +01:00
Dmitry Vyukov
75a7c5e2d1 prog: rework address allocation
1. mmap all memory always, without explicit mmap calls in the program.
This makes lots of things much easier and removes lots of code.
Makes mmap not a special syscall and allows to fuzz without mmap enabled.

2. Change address assignment algorithm.
Current algorithm allocates unmapped addresses too frequently
and allows collisions between arguments of a single syscall.
The new algorithm analyzes actual allocations in the program
and places new arguments at unused locations.
2018-02-19 21:48:20 +01:00
Dmitry Vyukov
1f693e0219 tools/syz-mutate: allow limiting set of syscalls 2018-02-17 19:02:12 +01:00
Dmitry Vyukov
ad2c9da9fb syz-manager: log number of VMs that are currently fuzzing 2018-02-01 17:30:46 +01:00
Dmitry Vyukov
e5b101ddff syz-manager: fix coverage page refresh 2018-01-24 11:25:14 +01:00
Dmitry Vyukov
90c33e5d0c syz-manager: check new inputs from fuzzers
Fuzzer must not send broken programs, but we see such cases episodically.
Reason unknown. But at least prevent manager from crashing later.
2018-01-14 16:23:39 +01:00
Dmitry Vyukov
02a19b646c syz-manager: add comment explaining why we don't set corrupted for repros 2018-01-10 09:41:28 +01:00
Dmitry Vyukov
7a2f0c7927 syz-manager/mgrconfig: add builtin suppression for crashing init 2018-01-06 17:38:44 +01:00
Tim Tianyang Chen
eaadba986d syz-manager: enable sending group emails
Email_Addr variable has been changed to Email_Addrs that contains
a list of recipient.

Signed-off-by: Tim Tianyang Chen <soapcn@gmail.com>
2017-12-21 10:27:12 +01:00
Tim Tianyang Chen
49bed8cf4d syz-manager: remove duplicated emails on restart
With commit: syz-manager: add simple email support, it will send
emails when a bug is hit for the first time during that particular
run of syz-manager. In other words, if you restart syz-manager and
the same bug is hit, a new email will be sent again. This is due to
the fact that mgr.crashTypes[crash.Title] doesn't keep track of logs
already written to the disk.

Fixed by moving emailCrash() to logic handling log writing.

Fixes #484

Signed-off-by: Tim Tianyang Chen <soapcn@gmail.com>
2017-12-21 10:27:12 +01:00
Dmitry Vyukov
a20097eafe syz-manager, syz-fuzzer: allow re-minimizing/re-smashing inputs
By default we don't re-minimize/re-smash programs from corpus,
it takes lots of time on start and is unnecessary.
However, when we improve/fix minimization/smashing,
we may want to.

Introduce corpus database versions and allow to re-minimize/re-smash
on version bumps.
2017-12-18 14:10:56 +01:00
Dmitry Vyukov
26eab5e866 syz-manager: speed up main page rendering
Remove unused calculation of total coverage.
Reduce critical section duration.
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
535474e22e syz-manager: add favicon handler 2017-12-17 11:39:14 +01:00
Tim Tianyang Chen
ac20b98c1b syz-manager: add simple email support
Users can specify an email address to reveive notifications when a
bug is discovered for the first time, without setting up a full fledged
dashboard. The supported mailer is mailx.

Signed-off-by: Tim Tianyang Chen <soapcn@gmail.com>
2017-12-14 08:57:27 +01:00
Dmitry Vyukov
8b77d41235 syz-manager: show the same file after cover page reload
Handy when you are looking at a single file
and periodically reload page to get new coverage.
2017-12-12 09:13:35 +01:00
Dmitry Vyukov
1aecf4ed31 syz-manager/mgrconfig: add another flavour of OOM messages 2017-12-05 20:08:06 +01:00
Dmitry Vyukov
09582d5306 syz-manager: send public web addr to dashboard 2017-12-04 11:58:55 +01:00
Dmitry Vyukov
48359b9777 dashboard/app: improve needRepro
Don't reproduce corrupted reports.
Don't reproduce if canonical bug already has repro.
2017-12-01 17:27:08 +01:00
Dmitry Vyukov
2fa91450df dashboard/app: add manager monitoring
Make it possible to monitor health and operation
of all managers from dashboard.
1. Notify dashboard about internal syz-ci errors
   (currently we don't know when/if they happen).
2. Send statistics from managers to dashboard.
2017-12-01 13:58:11 +01:00
Dmitry Vyukov
29b0fd90e6 pkg/report: include Maintainers into report
Currently getting a complete report requires a complex,
multi-step dance (including getting information that
external users are not interested in -- guilty file).

Simplify interface down to 2 functions: Parse and Symbolize.
Parse does what it did before, Symbolize symbolizes report
and fills in maintainers. This simplifies both implementations
of Reporter interface and all users of the interface.

Potentially we could get this down to 1 function Parse
that does everything. However, (1) Symbolize can fail,
while Parse cannot, (2) usually we want to ignore (log)
Symbolize errors, but otherwise proceed with the report,
(3) repro does not need symbolization for all but the
last report.
2017-11-29 18:24:30 +01:00
Dmitry Vyukov
34f2c2332b pkg/report: add Output to Report
Whole raw output is indivisble part of Report,
currently we always pass Output separately along with Report.
Make Output a Report field.

Then, put whole Report into manager Crash and repro context and Result.
There is little point in passing Report as aa bunch of separate fields.
2017-11-29 14:36:51 +01:00
Dmitry Vyukov
d19770f1b9 syz-manager: fix up osutil.RunCmd call
Fixes #435
2017-11-24 13:41:28 +01:00
Dmitry Vyukov
ee01a72bf4 syz-manager: don't reproduce corrupted reports
Corrupted reports are usually associated with frequently happenning races.
Since they are frequently happenning, we should get a repro for them
without corrupted reports. Reproducing is expensive, so doing it
when we will the repro anyway is harmful.
2017-11-22 12:47:00 +01:00
Dmitry Vyukov
6440d6940f syz-manager: send corrupted flag to dashboard 2017-11-21 19:11:54 +01:00
Dmitry Vyukov
ad0af9fff5 vm: return Report from MonitorExecution
This allows callers to get access to Report.Corrupted.
Better than adding 6-th return value and will allow
to pipe other report properties if necessary.
2017-11-21 19:02:35 +01:00
Dmitry Vyukov
fba338cd51 pkg/csource: add function to parse serialized options
Also move options and options tests into a separate file,
add serialization function.
2017-11-17 17:57:51 +01:00
Dmitry Vyukov
c4d43f4773 pkg/osutil: don't leace runaway processes
When manager is stopped there are sometimes runaway qemu
processes still running. Set PDEATHSIG for all subprocesses.
We never need child processes outliving parents.
2017-11-16 12:48:02 +01:00
Dmitry Vyukov
4bd78cef05 pkg/report, pkg/repro, syz-manager: name crash attributes consistently
We currently have several names for crash attributes, which is disturbing.
E.g. crash title is called "Title" or "Desc". Name them consistently.

Title - single line bug identity.
Report - whole crash text.
Log - whole fuzzer/kernel output.
2017-11-14 10:04:22 +01:00
Dmitry Vyukov
10112655d7 vm: remove needOutput arg for MonitorExecution
Always wait 10 secs for output.
If anything this can only lead to missed crashes during repro.
Let's unify manager and repro behavior.
2017-11-14 09:45:34 +01:00
Dmitry Vyukov
3b7ad24d6c syz-manager/mgrconfig: add another OOM suppression
If sshd was killed, we will get "lost connection".
2017-11-08 18:01:43 +01:00
Dmitry Vyukov
6755f62592 syz-fuzzer: fix manager polling
We need to always poll manager to send stats/maxsignal,
we just need not request candidates if we have plenty of work.
2017-10-23 09:59:39 +02:00
Dmitry Vyukov
fe864987bd syz-manager: pass kernel object dir to report.NewReporter 2017-10-18 18:19:01 +02:00
Dmitry Vyukov
441d64d9a5 syz-manager/mgrconfig: fix selection of executor binary
We currently select executor for vm arch, need to select target arch.
This breaks linux/amd64/386 configuration where we select amd64 instead of 386.
2017-10-18 16:49:25 +02:00
Dmitry Vyukov
85c802e4cf pkg/report: support multiple OSes
Introduce report.Reporter interface.
Add an implementation per-OS.
Make users be explicit about OS they are testing.
2017-10-18 12:01:24 +02:00
Dmitry Vyukov
a1bdb604cc syz-manager: extend periodic messages
Add coverage and number of reproducing programs to the periodic messages.
When all machines are busy reproducing crashes, it appears that
syz-manager hanged as number of executed programs does not increase.
Coverage is just a nice characteristic.
Also print machine check message, it appears once and contains useful info.
2017-10-17 15:00:01 +02:00
Dmitry Vyukov
974ad2210c syz-manager: check git revision on start
If we have no git revision, we will fail later anyway.
Better to fail earlier.
2017-10-16 20:21:47 +02:00
Dmitry Vyukov
128c09f19d syz-manager: expose raw cover in http handler
This adds /rawcover handler which returns a file with all covered so far PCs, e.g.:

0xffffffff8100763e
0xffffffff81007667
...
0xffffffff8100767d

Useful for offline coverage processing, diffing coverage, etc.
In particular allows to do:

curl http://localhost:1234/rawcover | addr2line -e vmlinux
2017-10-11 22:47:40 +02:00
Dmitry Vyukov
817b677396 syz-manager/mgrconfig: add gce examples 2017-10-10 15:47:27 +02:00
Dmitry Vyukov
9fc15c7ea3 vm/gce: windows support
Support custom pre-created images.
Support non-root user.
Use dir instead of pwd on windows.
Don't use sudo on windows.
2017-09-27 18:59:50 +02:00