Currently getting a complete report requires a complex,
multi-step dance (including getting information that
external users are not interested in -- guilty file).
Simplify interface down to 2 functions: Parse and Symbolize.
Parse does what it did before, Symbolize symbolizes report
and fills in maintainers. This simplifies both implementations
of Reporter interface and all users of the interface.
Potentially we could get this down to 1 function Parse
that does everything. However, (1) Symbolize can fail,
while Parse cannot, (2) usually we want to ignore (log)
Symbolize errors, but otherwise proceed with the report,
(3) repro does not need symbolization for all but the
last report.
This adds /rawcover handler which returns a file with all covered so far PCs, e.g.:
0xffffffff8100763e
0xffffffff81007667
...
0xffffffff8100767d
Useful for offline coverage processing, diffing coverage, etc.
In particular allows to do:
curl http://localhost:1234/rawcover | addr2line -e vmlinux
It's currently both optional and non optional.
We require it to be non-empty, but at the same time allow fake "-"
which effectively means "no vmlinux". Make it optional.
Now each prog function accepts the desired target explicitly.
No global, implicit state involved.
This is much cleaner and allows cross-OS/arch testing, etc.
Large overhaul moves syscalls and arg types from sys to prog.
Sys package now depends on prog and contains only generated
descriptions of syscalls.
Introduce prog.Target type that encapsulates all targer properties,
like syscall list, ptr/page size, etc. Also moves OS-dependent pieces
like mmap call generation from prog to sys.
Update #191
Currently hub allows managers to exchange programs from corpus.
But reproducers are not exchanged and we don't know if a crash
happens on other managers as well or not.
Allow hub to exchange reproducers.
Reproducers are stored in a separate db file with own sequence numbers.
This allows to throttle distribution of reproducers to managers,
so that they are not overloaded with reproducers and don't lose them on restarts.
Based on patch by Andrey Konovalov:
https://github.com/google/syzkaller/pull/325Fixes#282
We usually store reports as []byte, not as string. They can be large.
So change arg type to []byte.
Also rename it from log to report. In our terminology log is
not symblized/processed crash output. What this function wants
is called report in manager.
Currently syzkaller uses per-call basic block (BB) coverage.
This change implements edge (not-per-call) coverage.
Edge coverage is more detailed than BB coverage as it captures
not-taken branches, looping, etc. So it provides better feedback signal.
This coverage is now called "signal" throughout the code.
BB code coverage is also collected as it is required for visualisation.
Not doing per-call coverage reduces corpus ~6-7x (from ~35K to ~5K),
this has profound effect on fuzzing efficiency.
Fuzzing time is amount of time we spent actually fuzzing.
It excludes VM creation time, crash reproducing time, etc.
On the other hand it is multipled by number of currently
fuzzing VMs, so it can be larger than uptime time.
Currently we read lots of unnecessary files. This is slow on GCE.
Read only necessary info.
For summary report use on readdirnames (which does not do stat on every file).
For detailed crash report read additional info, but only for this crash.
With this change manager will run reproduction on crashes
until reproducer is discovered, but at most 3 times.
If reproducer is discovered it is saved with crashes and shown on the web UI.
Log is a simple wrapper around std log package.
It is meant to solve 2 main problems:
1. Logging from non-main packages (mainly, vm/* packages).
Currently they can either always log or not log at all.
But they can't respect program verbosity setting.
Log package allows all packages to use the same verbosity setting.
2. Exposing recent logs in html UI.
Namely we want to tee logs to console and html UI.
Add an option to view unique coverage per syscall (i.e. not covered
by any other calls) and unique coverage per-program (not covered by
any other program).
Remove master process entirely, it is not useful in its current form.
We first need to understand what we want from it, and them re-implement it.
Prefix all binaries with syz- to avoid name clashes.
