365 Commits

Author SHA1 Message Date
Greg Steuck
370797126e tools/create-openbsd-gce-ci.sh mount 10G ramdisk for worker images 2018-09-20 20:46:45 +02:00
Greg Steuck
b117b6ed3d tools/*openbsd*: smaller VMM images and tweaks. 2018-09-20 11:03:43 +02:00
Greg Steuck
7f125108ae More packages and fewer daemons. 2018-09-17 09:44:01 +02:00
Greg Steuck
68def56dce tools/create-openbsd-gce-ci.sh: Add VMM configuration.
Added verification of successful package install.
2018-09-17 09:44:01 +02:00
Greg Steuck
8c88323f94 tools/create-openbsd-gce-ci.sh: image for build machine
Mostly derived from Go buildlet generator with blessing from bradfitz@.

Update #712
2018-09-10 18:28:00 +02:00
Dmitry Vyukov
58f18e3f60 tools/create-gce-image.sh: disable ftrace_dump_on_oops
We don't have anything useful there.
But sometimes fuzzer somehow poppulates it with some nonsense,
and then it all dumped in crashes.
Disable it.
2018-09-10 16:19:40 +02:00
Dmitry Vyukov
6b5120a464 tools/create-gce-image.sh: auto-detect block device type
Currently we choose block device to use (nbd/loop) based on SYZ_VM_TYPE.
Strictly saying these things are orthogonal.
losetup is broken on Ubuntu. qemu-nbd is broken on Debian.
Try to auto-detect what will work based on uname.
2018-09-07 19:04:47 +02:00
Kees Cook
873745f2ff tools: update to Debian stretch
This updates the image creation tool to use Debian stretch (current stable)
instead of wheezy, which is very out of date. The only change needed here
was a hint to systemd to make the root filesystem read-write after booting.
Documentation has also been updated.
2018-09-05 19:26:06 +02:00
Dmitry Vyukov
196410e4f5 dashboard/config: re-enable selinux
Upstream "selinux: fix mounting of cgroup2 under older policies"
commit fixes mounting of cgroup2 under wheezy selinux policy.
So don't disable selinux on start.
Create separate cmdline arguments that enable selinux and apparmor.
2018-09-05 12:50:53 +02:00
Dmitry Vyukov
98bfd6d34c tools/create-gce-image.sh: add default ext4 options
Set some realistic modern ext4 options when creating the image.
2018-09-03 16:42:59 +02:00
Dmitry Vyukov
3653592507 pkg/runtest: assorted improvements
Support checking "blocked"/"unfinished" flags for calls.
Support test constanints, e.g. "requires: threaded" or "requires: -sandbox=setuid".
Some improvements in tools/syz-runtest.

Update #603
2018-08-08 15:05:01 +02:00
Dmitry Vyukov
4207dbaa99 tools/syz-execprog: print blocked/unfinished/faulted flags for calls 2018-08-08 15:03:31 +02:00
Dmitry Vyukov
2763e04c22 tools/syz-runtest: add tool for program unit testing
The tool is run as:

$ syz-runtest -config manager.config

This runs all programs from sys/*/test/* in different modes
on actual VMs and checks results.

Fixes #603
2018-08-03 21:08:02 +02:00
Dmitry Vyukov
6bfd4f09db pkg/ipc: move flags into subpackage
Move all ipc flags into pkg/ipc/ipcconfig package
so that importing pkg/ipc does pull in the flags.
2018-08-03 18:12:24 +02:00
Dmitry Vyukov
fbedd425b5 pkg/mgrconfig: move from syz-manager/mgrconfig
mgrconfig was used only by syz-manager initially,
but now it's used by a dozen of packages and it's
weird to import from under a binary dir.
pkg/ is much more reasonable dir for a widely used
helper package.
2018-08-02 16:57:32 +02:00
Dmitry Vyukov
531d157044 tools/syz-execprog: refactor
Reduce cyclomatic complexity of the main function.
It's too huge.

Update #538
2018-07-31 16:05:03 +02:00
Dmitry Vyukov
f5d67fbd9c .gometalinter.json: enable gofmt
The part that we want from gofmt is simplify (-s).
Fix all code that needs fixing.

Update #538
2018-07-31 12:16:54 +02:00
Dmitry Vyukov
b25fc7b831 pkg/csource: add option to trace syscall results
This will be needed for testing of generated programs.
2018-07-27 10:22:23 +02:00
Dmitry Vyukov
c7725f52b1 syz-execprog: initialize net devices 2018-07-27 10:22:23 +02:00
Dmitry Vyukov
9fe4bdc5f1 executor: overhaul
Make as much code as possible shared between all OSes.
In particular main is now common across all OSes.
Make more code shared between executor and csource
(in particular, loop function and threaded execution logic).
Also make loop and threaded logic shared across all OSes.
Make more posix/unix code shared across OSes
(e.g. signal handling, pthread creation, etc).
Plus other changes along similar lines.
Also support test OS in executor (based on portable posix)
and add 4 arches that cover all execution modes
(fork server/no fork server, shmem/no shmem).

This change paves way for testing of executor code
and allows to preserve consistency across OSes and executor/csource.
2018-07-24 12:04:27 +02:00
Dmitry Vyukov
bad4246bf2 pkg/report: improve akaros reporter and implement symbolization 2018-07-16 17:03:14 +02:00
Dmitry Vyukov
92a4950507 pkg/host: add "network devices" feature
Linux executor sets up some network devices for testing,
detect when that's supported on the machine and don't
do it if it's not supported.
2018-07-13 12:46:32 +02:00
Dmitry Vyukov
0b95b8ec49 pkg/host: disable for akaros
akaros can't have own host version
because fuzzer does not run on akaros,
so just disable it all.
2018-07-06 20:18:05 +02:00
Dmitry Vyukov
04bd6c3d9e pkg/instance: pass -os to execprog/fuzzer only for akaros
Only akaros needs OS, because the rest assume host OS.
But speciying OS for all OSes breaks patch testing on syzbot
because old execprog does not have os flag.
2018-07-06 14:43:24 +02:00
Dmitry Vyukov
538df42ec7 pkg/repro: provide stats even for failed repro
Provide stats and logs for failed repro and save it in manager.
In particular log is useful for failed repros,
currently there is no visibility into why bugs
failed to reproduce.
2018-07-05 13:14:00 +02:00
Daniel Borkmann
3a35170a24 bpf: disable hardening in favor of unwinding
I had missed that once hardening is enabled, it automatically disables
any exposure of JITed addresses, therefore when crashes or warnings are
thrown we don't unwind beyond a helper function. For now disable hardening.

After merge window I'll see if it's possible to detangle the case where
kernel queries kallsyms internally to find function names whenever a WARN
or BUG is thrown. If that's not possible easily, we can potentially add a
harden mode 3 which does hardening but does not disable kallsyms exposure
and then set this here for tools like syzkaller.

Fixes: ac9b19d2e435 ("bpf: enable hardening mode 1 for jited images")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>

[dvyukov: also updated dashboard/config/upstream.sysctl]
2018-07-05 10:48:20 +02:00
Dmitry Vyukov
bc1210b614 pkg/ipc: create default config based on target
Pass actual target to DefaultConfig, that's what we really want,
not the stray ipc flag.
2018-07-05 10:44:34 +02:00
Dmitry Vyukov
2c6c896b36 tools/syz-repro: add debug mode 2018-07-05 10:44:34 +02:00
Dmitry Vyukov
9dd8cf63e0 executor, pkg/ipc: support output over pipes 2018-06-29 16:33:07 +02:00
Dmitry Vyukov
a736f2e8e6 tools/syz-prog2c: allow to test build of the resulting program 2018-06-29 09:22:42 +02:00
Dmitry Vyukov
2a075d57ab pkg/report: allow to specify suppressions per OS
Currently all (linux-specific) suppressions are hardcoded in mgrconfig.
This is very wrong. Move them to pkg/report and allow to specify per OS.
Add gvisor-specific suppressions.
This required a bit of refactoring. Introduce mgrconfig.KernelObj finally.
Make report.NewReporter and vm.Create accept mgrconfig directly
instead of passing it as multiple scattered args.
Remove tools/syz-parse and it always did the same as tools/syz-symbolize.
Simplify global vars in syz-manager/cover.go.
Create reporter eagerly in manager. Use sort.Slice more.
Overall -90 lines removed.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
9a7d0a5412 pkg/report: pass vm type to NewReporter
For the case when VM type affects output.
Will be needed for gvisor. It is kinda linux, but kinda not.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
87bfb99cfe vm: pass instance to MonitorExecution
It may need it later to try to obtain additional
diagnostic from hanged instances.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
06ece2ca66 pkg/host: rework host feature detection/setup
Currently host feature detection/setup code is spread
across platform-independent fuzzer code, pkg/host, pkg/ipc
and executor.
Move this all into pkg/host and show readable info
about features on manager start.

Fixes #46
2018-06-12 14:53:22 +02:00
Daniel Borkmann
ac9b19d2e4 bpf: enable hardening mode 1 for jited images
This will harden non-root programs from kernel side, but not
root-only ones. Helps also to increase coverage a bit since
syzkaller generates programs for both cases.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2018-06-07 12:39:12 +03:00
Daniel Borkmann
b17ae9398c bpf: enable full unwind and kallsym export support for jited images
Helps syzkaller in particular for unwinding full stack in case
of warnings or crashes.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2018-06-07 12:39:12 +03:00
Dmitry Vyukov
6479ab2a75 Makefile, sys/targets: move all native compilation logic to sys/targets
We currently have native cross-compilation logic duplicated
in Makefile and in sys/targets. Some pieces are missed in one
place, some are in another. Only pkg/csource knows how to check
for -static support.

Move all CC/CFLAGS logic to sys/targets and pull results in Makefile.

This should make Makefile work on distros that have broken x86_64-linux-gnu-gcc,
now we will use just gcc. And this removes the need to define NOSTATIC,
as it's always auto-detected.

This also paves the way for making pkg/csource work on OSes other than Linux.
2018-06-06 10:02:03 +02:00
Dmitry Vyukov
f48c20b8f9
tools/create-image.sh: fix watchdog_thresh value
watchdog_thresh is capped at 60, so 120 causes EINVAL.
2018-05-19 10:48:34 +02:00
Dmitry Vyukov
70c84d118a tools/create-image.sh: add few useful sysctl's 2018-05-18 19:29:06 +02:00
Dmitry Vyukov
61ef28da7a tools/syz-execprog: restore the previous tun disabling logic
This probably can break some things.
I feel that tun setup can affect other things even if
syz_emit_ethernet/syz_extract_tcp_res are not used.
So it can make sense to setup tun even if they are not used.
But let's be more careful.
2018-05-18 10:01:22 +02:00
Dmitry Vyukov
738d58ade0 pkg/csource: minimize netdevices and net reset
Add separate options to minimize netdevices setup and net namespace reset.

Fixes #581
2018-05-17 19:57:54 +02:00
Dmitry Vyukov
9753d3be5e tools/syz-prog2c: add missing csource option 2018-05-17 19:07:33 +02:00
Dmitry Vyukov
4e1c0dbaea pkg/bisect: add bisection logic
Add first version of bisection package
(supports both bug and fix bisection).
And tools/syz-bisect wrapper for testing.

Update #501
2018-05-17 14:52:39 +02:00
Dmitry Vyukov
6595937c34 tools/create-gce-image.sh: support both nbd and loop
Pass target OS/arch and VM type to kernel.CreateImage.
Use nbd for gce and loop for qemu VM type.
2018-05-17 14:50:18 +02:00
Dmitry Vyukov
a367c1d7a8 tools/create-gce-image.sh: support MKE2FS_CONFIG 2018-05-16 17:21:29 +02:00
Dmitry Vyukov
86ea19e4c2 tools/create-gce-image.sh: revert loop to nbd
loop devices are nice, but unfortunately this creates images
that does not boot on GCE. Reason is unclear.
Revert back to nbd for now.
2018-05-14 19:32:39 +02:00
Dmitry Vyukov
d5dc4006db tools/kcovtrace: add missing include 2018-05-14 11:17:41 +02:00
Dmitry Vyukov
ea9496cdbd tools/create-gce-image.sh: use loop instead of nbd
loop is much more standard than nbd and does not require additional modules.
nbd broke on Debian rolling.
loop also allows parallel execution thanks to losetup -f.

Use loop instead of nbd.
Also improve cleanup logic and add one missing sudo.

Update #501
2018-05-14 11:16:36 +02:00
Dmitry Vyukov
9e0846e8a4 all: get rid of underscores in identifiers
Underscores are against Go coding style.

Update #538
2018-05-07 17:59:06 +02:00
Dmitry Vyukov
3f70522c7e tools/syz-execprog: split overly large function
Update #538
2018-05-07 16:23:18 +02:00