A bunch of spot optmizations after cpu/memory profiling:
1. Optimize hot-path coverage comparison in fuzzer.
2. Don't allocate and copy serialized program, serialize directly into shmem.
3. Reduce allocations during parsing of output shmem (encoding/binary sucks).
4. Don't allocate and copy coverage arrays, refer directly to the shmem region
(we are not going to mutate them).
5. Don't validate programs outside of tests, validation allocates tons of memory.
6. Replace the choose primitive with simpler switches.
Choose allocates fullload of memory (for int, func, and everything the func refers).
7. Other minor optimizations.
Currently we generate 1 program per 9 mutations.
Do it per 99 mutations.
Benchmark shows both coverage increase and corpus reduction:
baseline generate100 diff
coverage 65467 65569 102
corpus 35423 35363 -60
exec total 5474879 5030990 -443889
It was never proved to be useful.
We don't have evidence that a partiuclar generated program is interesting,
so don't spend time mutating it. If it is in fact interesting,
then we will add it to corpus and mutate later.
There is no reason to not check for new coverage during minimization.
We execute new programs, and can well get new interesting programs with new coverage.
So do that. Since execute updates maxCover synchronously, we should not get any
kind of positive feedback loop due to that.
Minimization takes considerable time on start, but the programs were already minimized.
There are some chances that we could minimize it better this time,
but still it does not worth very slow start (which is especially painful for development).
1. Drop non-reproducible programs.
Currently we can fail to execute the target syscall 3 times,
and decided that the program gives new stable coverage.
That's not true. Permit only one failure to execute target syscall.
2. If we see only flaky coverage, bail out of the triage loop sooner.
1. Logical executor failures were not handled in waitServing.
2. Fuzzer did not distinguish temporal errors and logical errors
and always waited for 10 errors before crashing.
Fix that.
Log is a simple wrapper around std log package.
It is meant to solve 2 main problems:
1. Logging from non-main packages (mainly, vm/* packages).
Currently they can either always log or not log at all.
But they can't respect program verbosity setting.
Log package allows all packages to use the same verbosity setting.
2. Exposing recent logs in html UI.
Namely we want to tee logs to console and html UI.
The new namespace-based sanboxing is good,
but it's not always what one wants
(and also requires special kernel configs).
Change dropprivs config value to sandbox,
which can have different values (currently: none, setuid, namespace).
Setuid mode uses setuid(nobody) before fuzzing as before.
In future we can add more sandboxing modes or, say,
extend -sandbox=setuid to -sandbox=setuid:johndoe
to impersonolate into given user.
Kmemleak has false positives. To mitigate most of them, it checksums
potentially leaked objects, and reports them only on the next scan
iff the checksum does not change. Because of that we do the following
intricate dance:
Scan, sleep, scan again. At this point we can get some leaks.
If there are leaks, we sleep and scan again, this can remove
false leaks. Then, read kmemleak again. If we get leaks now, then
hopefully these are true positives during the previous testing cycle.
Add timeout to adb invocations and do more reliable reboot.
Clean up temporary files from previous runs.
Also pass enabled syscalls via rpc, as adb barks at too long command line.
Abd is still unreliable, though. Devices hang.
Current interface is suitable only for running syz-fuzzer.
Make the interface more generic (boot, copy file, run an arbitrary command).
This allows to build other tools on top of vm package
(e.g. reproducer creation).
Remove master process entirely, it is not useful in its current form.
We first need to understand what we want from it, and them re-implement it.
Prefix all binaries with syz- to avoid name clashes.
