Large overhaul moves syscalls and arg types from sys to prog.
Sys package now depends on prog and contains only generated
descriptions of syscalls.
Introduce prog.Target type that encapsulates all targer properties,
like syscall list, ptr/page size, etc. Also moves OS-dependent pieces
like mmap call generation from prog to sys.
Update #191
We have implemented a new version of KCOV, which is able to dump
comparison operands' data, obtained from Clang's instrumentation hooks
__sanitizer_cov_trace_cmp[1248], __sanitizer_cov_trace_const_cmp[1248]
and __sanitizer_cov_trace_switch.
Current KCOV implementation can work in two modes: "Dump only the PCs"
or "Dump only comparisons' data". Mode selection is done by the
following series of calls:
fd = open(KCOV_PATH, ...); // works as previous
ioctl(fd, KCOV_INIT_TRACE, ...); // works as previous
mmap(fd, ...); // works as previous
ioctl(fd, KCOV_ENABLE, mode);
// mode = KCOV_MODE_TRACE_CMP or mode = KCOV_MODE_TRACE_PC
Note that this new interface is backwards compatible, as old KCOV
devices will just return -EINVAL for the last ioctl. This way we can
distinguish if the KCOV device is able to dump the comparisons.
Main changes in this commit:
1. Fuzzer now checks at startup which type (new/old) of KCOV device
is running.
2. Executor now receives an additional flag, which indicates if
executor should read the comparisons data from KCOV. The flag works on
per-call basis, so executor can collect PCs or Comps for each
individual syscall.
Currently hub allows managers to exchange programs from corpus.
But reproducers are not exchanged and we don't know if a crash
happens on other managers as well or not.
Allow hub to exchange reproducers.
Reproducers are stored in a separate db file with own sequence numbers.
This allows to throttle distribution of reproducers to managers,
so that they are not overloaded with reproducers and don't lose them on restarts.
Based on patch by Andrey Konovalov:
https://github.com/google/syzkaller/pull/325Fixes#282
If manager is connected to dashboard it now does not save crashes.
Which means that when we save a repro the crash dir may not exist yet.
Create the dir when saving repros.
We can start reproducing one crash, but end up reproducing another.
Currently we still attribute the resulting repro to the original crash.
This is wrong.
Save the resulting desc/report for reproducers and use that in manager.
Currently we have unix permissions for new files/dirs
hardcoded throughout the code base. Some places use 0644,
some - 0640, some - 0600 and a variety of other constants.
Introduce osutil.MkdirAll/WriteFile that use the default
permissions and use them throughout the code base.
This makes permissions consistent and also allows to easily
change the permissions later if we change our minds.
Also merge pkg/fileutil into pkg/osutil as they become
dependent on each other. The line between them was poorly
defined anyway as both operate on files.
Sshkey is a property of image, which is in manager config.
Move sshkey to the same location as image.
The motivation for the move is as follows.
Continuous build produces an image and the key,
both need to be passed manager instance.
Continuous build system should not distinguish
different VM types and mess with their configs.
NOTE FOR USERS: this breaks manager configs again.
Hopefully the last time for now. Docs are updated.
It was useful only for vm/local which was removed.
The param wasn't documented and if one tries to change it,
it will break manager in obscure way (i.e. spurious
"test machine is not executing programs" crashes).
We have 2 packages with the same name: pkg/config and syz-manager/config.
This leads to constant clashes. We either rename one to pkgconfig or
another to mgrconfig. This is not good and will become worse when/if
we have another program-specific config in a separate package.
Rename manager config to mgrconfig.
Other program-specific configs can use the same convention
in future -- fooconfig.
1. Don't start reproducing crashes until we triage
all inputs from corpus and hub. This minimizes
chances of losing inputs from hub. Also allows
to faster get idea of total coverage.
2. Fix bug when vmCount%instacesPerRepro != 0.
Currently we stop the remainder of instances
and it stays idle.
VM infrastructure currently has several problems:
- Config struct is complete mess with a superset of params for all VM types
- verification of Config is mess spread across several places
- there is no place where VM code could do global initialization
like creating GCE connection, uploading GCE image to GCS,
matching adb devices with consoles, etc
- it hard to add private VM implementations
such impl would need to add code to config package
which would lead to constant merge conflicts
- interface for VM implementation is mixed with interface for VM users
this does not allow to provide best interface for both of them
- there is no way to add common code for all VM implementations
This change solves these problems by:
- splitting VM interface for users (vm package) and VM interface
for VM implementations (vmimpl pacakge), this in turn allows
to add common code
- adding Pool concept that allows to do global initialization
and config checking at the right time
- decoupling manager config from VM-specific config
each VM type now defines own config
Note: manager configs need to be changed after this change:
VM-specific parts are moved to own "vm" subobject.
Note: this change also drops "local" VM type.
Its story was long unclear and there is now syz-stress which solves the same problem.
Introduce generic config.Load function that can be
reused across multiple programs (syz-manager, syz-gce, etc).
Move the generic config functionality to pkg/config package.
The idea is to move all helper (non-main) packages to pkg/ dir,
because we have more and more of them and they pollute the top dir.
Move the syz-manager config parts into syz-manager/config package.
Currently hub sends all inputs on first manager connect.
This can be 100K+ inputs and can take long time
and consume tons of memory. Send inputs in 1K parts.
Also increase rpc timeouts as hub still has global mutex.
This commit adds Odroid C2 support to syzkaller.
It's now possible to specify "type": "odroid" in manager config.
Documentation on how to setup fuzzing with Odroid C2 board is here:
https://github.com/google/syzkaller/wiki/Setup:-Odroid-C2
Note, that after this change libusb-1.0-0-dev package should be
installed to build syzkaller.
Recalculating dynamic priorities requires deserializing all programs,
and that is slow. So do it at most once per 30 mins and don't hold
the mutex during prio calculation.
If hub hangs, it causes all managers to hang as well as they call
hub under the global mutex. So move common rpc code into rpctype
and make it more careful about timeouts (tcp keepalives, call timeouts).
Also don't call hub under the mutex, the call can be slow.
