Currently hub sends all inputs on first manager connect.
This can be 100K+ inputs and can take long time
and consume tons of memory. Send inputs in 1K parts.
Also increase rpc timeouts as hub still has global mutex.
This commit adds Odroid C2 support to syzkaller.
It's now possible to specify "type": "odroid" in manager config.
Documentation on how to setup fuzzing with Odroid C2 board is here:
https://github.com/google/syzkaller/wiki/Setup:-Odroid-C2
Note, that after this change libusb-1.0-0-dev package should be
installed to build syzkaller.
Recalculating dynamic priorities requires deserializing all programs,
and that is slow. So do it at most once per 30 mins and don't hold
the mutex during prio calculation.
If hub hangs, it causes all managers to hang as well as they call
hub under the global mutex. So move common rpc code into rpctype
and make it more careful about timeouts (tcp keepalives, call timeouts).
Also don't call hub under the mutex, the call can be slow.
Currently syzkaller uses per-call basic block (BB) coverage.
This change implements edge (not-per-call) coverage.
Edge coverage is more detailed than BB coverage as it captures
not-taken branches, looping, etc. So it provides better feedback signal.
This coverage is now called "signal" throughout the code.
BB code coverage is also collected as it is required for visualisation.
Not doing per-call coverage reduces corpus ~6-7x (from ~35K to ~5K),
this has profound effect on fuzzing efficiency.
In benchmarking mode (if the new -bench flag is specified)
syz-manager writes execution statistics into the specified file.
This allows later comparison of different runs (baseline vs some experiment).
For example, verify that some fuzzing modification indeed leads to larger coverage.
Fuzzing time is amount of time we spent actually fuzzing.
It excludes VM creation time, crash reproducing time, etc.
On the other hand it is multipled by number of currently
fuzzing VMs, so it can be larger than uptime time.
Minimization takes considerable time on start, but the programs were already minimized.
There are some chances that we could minimize it better this time,
but still it does not worth very slow start (which is especially painful for development).
Uncovered PCs were handled very badly:
we added PCs from the same function multiple times
and did not remove covered PCs. As the result total
number of uncovered PCs was terrific.
Fix that.
Currently we read lots of unnecessary files. This is slow on GCE.
Read only necessary info.
For summary report use on readdirnames (which does not do stat on every file).
For detailed crash report read additional info, but only for this crash.
Package db implements a simple key-value database.
The database is cached in memory and mirrored on disk.
It is used to store corpus in syz-manager and syz-hub.
The database strives to minimize number of disk accesses
as they can be slow in virtualized environments (GCE).
Use db in syz-manager instead of the old PersistentSet.
Add new config parameter "ignores" which contains list of regexp expressions.
If one of the expressions is matched against oops line,
crash report is not saved and VM is not restarted.
With this change manager will run reproduction on crashes
until reproducer is discovered, but at most 3 times.
If reproducer is discovered it is saved with crashes and shown on the web UI.
Log is a simple wrapper around std log package.
It is meant to solve 2 main problems:
1. Logging from non-main packages (mainly, vm/* packages).
Currently they can either always log or not log at all.
But they can't respect program verbosity setting.
Log package allows all packages to use the same verbosity setting.
2. Exposing recent logs in html UI.
Namely we want to tee logs to console and html UI.
It is not useful to pass manager verbosity flag to fuzzer,
as fuzzer output is not visible. But it increases amount of fuzzer
output that needs to be parsed by manager. Also increased fuzzer
verbosity reduces effective crash log size (less programs fit).
Enable fuzzer verbosity only if debug flag is given.
Save up to 100 reports. If we already have 100, overwrite the oldest one.
Newer reports are generally more useful. Overwriting is also needed
to be able to understand if a particular bug still happens or already fixed.
If config contains "tag" parameter, save it along with crash reports.
The tag is meant to contain kernel branch/commit hash.
If workdir contains crashes from different kernel versions,
it is useful to be able to find out on what kernel revision a crash happened.
