Commit Graph

710 Commits

Author SHA1 Message Date
Dmitry Vyukov
48613af61c pkg/vcs: use garnet instead of topaz layer for fuchsia
Topaz contains end user applications that are not interesting for us
and only increase build time. The applications also consume lots of memory,
so switching to garnet should leave more memory for the fuzzer
(maybe eliminate some OOMs).
2018-08-18 13:55:01 -07:00
Julia Hansbrough
738da825b2 [fuchsia] Change QEMU args to use multiboot shim. 2018-08-17 07:02:12 -07:00
Dmitry Vyukov
5084a6252d pkg/build: build asan-ified fuchsia
Building with asan is useful to detect any bugs in C/C++ services.
2018-08-14 08:50:56 -07:00
Dmitry Vyukov
565e2ad078 pkg/report: ignore another mutex lock guts function 2018-08-13 17:14:08 -07:00
Dmitry Vyukov
7a88b14122 pkg/compiler: prohibit subargs 2018-08-10 18:50:03 +02:00
Dmitry Vyukov
1fb62d5815 executor: fix FS_XFLAG_IMMUTABLE for 32-bit mode
FS_IOC_FSSETXATTR is not supported in compat mode, use FS_IOC_SETFLAGS instead.
2018-08-09 16:05:46 +02:00
Dmitry Vyukov
fa9be14164 pkg/csource: fix 32-bit syscall calls
syscall accepts args as ellipsis, resources are uint64
and take 2 slots without the cast, which is wrong.
Cast resources to long when passing to syscall.
2018-08-09 16:05:46 +02:00
Dmitry Vyukov
3a2fe60529 executor: fix cgroups
1. Create per-proc cgroup in executor.
2. Setup cgroups after chdir into test dir
   (it assumes relative paths against test dir).
3. Add test.
2018-08-09 16:05:46 +02:00
Dmitry Vyukov
922e8e2734 executor: mount binfmt_mist
We forgot to mount binfmt_misc. Mount it. Add a test.
Increase per-call timeout, otherwise last execve timesout.
Fix csource waiting for call completion at the end of program.
2018-08-09 16:05:46 +02:00
Dmitry Vyukov
acf31536e9 executor: clean up immutable dirs
Turns out dirs can be immutable too.

Update #650
2018-08-09 16:05:46 +02:00
Dmitry Vyukov
23b45a7525 executor: remove unnecessary hooks on fuchsia
These are not needed now.
2018-08-09 16:05:46 +02:00
Dokyung Song
2eeda842c8 sys/fuchsia: add syscall description for binding channels to Launcher 2018-08-08 19:31:56 +02:00
Dmitry Vyukov
17a0470620 executor: manually define struct fsxattr
It should be in <linux/fs.h> but is not there on some distros/arches as expected.

Travis build fails with:

      		<stdin>: In function ‘remove_dir’:
        		<stdin>:152:13: error: variable ‘attr’ has initializer but incomplete type
        		<stdin>:152:13: error: excess elements in struct initializer [-Werror]
        		<stdin>:152:13: error: (near initialization for ‘attr’) [-Werror]
        		<stdin>:152:21: error: storage size of ‘attr’ isn’t known
        		<stdin>:153:20: error: ‘FS_IOC_FSSETXATTR’ undeclared (first use in this function)
        		<stdin>:153:20: note: each undeclared identifier is reported only once for each function it appears in
        		<stdin>:152:21: error: unused variable ‘attr’ [-Werror=unused-variable]
        		cc1: all warnings being treated as errors

https://travis-ci.org/google/syzkaller/jobs/413574080
2018-08-08 17:04:50 +02:00
Dmitry Vyukov
d46deef51c executor: clean up immutable files
Fixes #650
2018-08-08 15:06:29 +02:00
Dmitry Vyukov
3653592507 pkg/runtest: assorted improvements
Support checking "blocked"/"unfinished" flags for calls.
Support test constanints, e.g. "requires: threaded" or "requires: -sandbox=setuid".
Some improvements in tools/syz-runtest.

Update #603
2018-08-08 15:05:01 +02:00
Dmitry Vyukov
91e4f273d6 pkg/csource: don't do checkpoint/reset net with setuid sandbox
Missing part of
"executor: don't checkpoint/reset net namespace with setuid sandbox"
2018-08-08 15:02:25 +02:00
Dmitry Vyukov
ddeb9f8d88 executor: don't checkpoint/reset net namespace with setuid sandbox
After setuid the process won't have rights to reset net namespace,
but it should not be able to change it during fuzzing too.
So just skip that part.
2018-08-08 11:16:54 +02:00
Dan Austin
14f8a76fe7 executor: checkpoint net namespace in the right namespace
With checkpoint_net_namespace moved to setup_common,
and Android fuzzing session terminates prematurely due to
ipv4_tables not being initialized at this time.
Moving the call back to loop fixes this behavior.
2018-08-08 11:03:33 +02:00
Dmitry Vyukov
1beb81367c executor: check write return value
gcc complains:

error: ignoring return value of ‘write’, declared with attribute warn_unused_result [-Werror=unused-result]

Also fix includes.
2018-08-05 13:05:47 +02:00
Dmitry Vyukov
3a7200e49b executor: abort fuse connection
If the test process is not dying after 100ms,
abort all fuse connections in the system.
This gets rid at least of simple fuse deadlocks,
let's see how well this works in all cases.
2018-08-04 17:50:58 +02:00
Dmitry Vyukov
bf45aa5363 pkg/report: skip more spinlock frames
Currently this report is named:
"general protection fault in perf_trace_lock"
which is bad.
2018-08-04 17:39:21 +02:00
Dmitry Vyukov
3476a2dfb9 pkg/osutil: replace UmountAll with RemoveAll
In pkg/ipc we don't just want to UmountAll,
we want to remove all handling as many cases as possible
(mounts, read-only files, etc, similar to executor's remove_dir).
So unmounting and removing needs to be a single function,
so that it can handle all these cases.
2018-08-04 15:50:40 +02:00
Dmitry Vyukov
df7f6947ba pkg/runtest: skip 32-bit tests in short mode
Fails on travis...
2018-08-03 21:24:35 +02:00
Dmitry Vyukov
2763e04c22 tools/syz-runtest: add tool for program unit testing
The tool is run as:

$ syz-runtest -config manager.config

This runs all programs from sys/*/test/* in different modes
on actual VMs and checks results.

Fixes #603
2018-08-03 21:08:02 +02:00
Dmitry Vyukov
78e3ad98f6 sys/test: add more tests
Add syz_errno syscall which sets errno to the argument,
and add a test with different errno values.
This mostly tests the testing infrastructure itself.

Add syz_compare syscall which compare two blobs,
this can be used for testing of argument memory layout.

Implement syz_mmap and fix Makefile to allow building syz-execprog for test OS.
Useful for debugging.

Update #603
2018-08-03 19:53:14 +02:00
Dmitry Vyukov
5ba57bfe16 pkg/runtest: add package for syzkaller program unit-testing
Package runtest is a driver for end-to-end testing of syzkaller programs.
It tests program execution via both executor and csource,
with different sandboxes and execution modes (threaded, repeated, etc).
It can run test OS programs locally via run_test.go
and all other real OS programs via tools/syz-runtest
which uses manager config to wind up VMs.
Test programs are located in sys/*/test/* files.

Update #603
2018-08-03 18:47:42 +02:00
Dmitry Vyukov
649477b6a5 pkg/ipc: remove abort signal and buffer size
They were needed for intermediate gvisor support.
Now that we have end-to-end support for gvisor,
they are not needed anymore. Remove.
2018-08-03 18:29:20 +02:00
Dmitry Vyukov
6bfd4f09db pkg/ipc: move flags into subpackage
Move all ipc flags into pkg/ipc/ipcconfig package
so that importing pkg/ipc does pull in the flags.
2018-08-03 18:12:24 +02:00
Dmitry Vyukov
5ff1f9faec pkg/csource: minor fixes
1. Print errno with %u instead of %d
2. Avoid unused var warning for syz_emit_ethernet
   when tracing is enabled.
2018-08-03 16:57:08 +02:00
Dmitry Vyukov
534859fe5a pkg/osutil: minor tweaks
1. Make it clear when a command times out.
2. Don't add trailing newline for VerboseError if output is empty.
3. Fix WriteExecFile for the case when the file already exists.
2018-08-03 16:51:48 +02:00
Dmitry Vyukov
0a2c27237b pkg/rpctype: ignore SetDeadline error
SetDeadline is not implemented on fuchsia and always fails.
2018-08-03 12:22:36 +02:00
Dmitry Vyukov
41beddb6da executor: extend logging in netfilter code
We see some failures there, extend logging so that it's least
possible to understand what exactly call has failed.
2018-08-02 19:45:31 +02:00
Dmitry Vyukov
5cbdd9f444 gometalinter: strengthen gocyclo limit
Strengthen gocycle limit 35->24! Yay! No more jumbo functions!

Fixes #538
2018-08-02 16:57:32 +02:00
Dmitry Vyukov
394e6910d0 pkg/compiler: refactor checkType
checkType is too long and complex.
Move basic type checks and args checks into separate functions.

Update #538
2018-08-02 16:57:32 +02:00
Dmitry Vyukov
1baf121c2f pkg/csource: refactor generateCalls
Move call generation into a separate function.

Update #538
2018-08-02 16:57:32 +02:00
Dmitry Vyukov
3c5c9177b5 pkg/csource: refactor defineList
Make it simpler and shorter.

Update #538
2018-08-02 16:57:32 +02:00
Dmitry Vyukov
fbedd425b5 pkg/mgrconfig: move from syz-manager/mgrconfig
mgrconfig was used only by syz-manager initially,
but now it's used by a dozen of packages and it's
weird to import from under a binary dir.
pkg/ is much more reasonable dir for a widely used
helper package.
2018-08-02 16:57:32 +02:00
Dmitry Vyukov
72ad4a3c4c pkg/report: refactor test
Split a long function.

Update #538
2018-08-02 16:57:31 +02:00
Dmitry Vyukov
1637002cde pkg/ipc: refactor output parsing
readOutCoverage was complete mess.
Split it into several functions.
Use callReply struct directly.
Make error handling more idiomatic.

Update #538
2018-08-02 16:57:31 +02:00
Dmitry Vyukov
bdc8436c58 pkg/serializer: refactor
Factor out serialization of pointers, slices and
structs into separate functions.

Update #538
2018-08-02 16:57:31 +02:00
Dmitry Vyukov
af15ef43a8 pkg/compiler: refactor checkNames
Too complex. Split into 2 functions.

Update #538
2018-08-02 16:57:31 +02:00
Dmitry Vyukov
3a4641d90c pkg/compiler: refactor structGen
Still too complex. Split more.

Update #538
2018-08-02 16:57:31 +02:00
Dmitry Vyukov
c67a9331a4 gometalinter: clean up some errcheck warnings
Check some errors where relevant.
Unfortunately enabling errcheck does not look feasible, too many warnings.

Update #538
2018-08-02 16:57:31 +02:00
Dmitry Vyukov
1477993e1b pkg/compiler: work around vet crash
vetshadow not works, so enable it.

Update https://github.com/golang/go/issues/26725
Update #538
2018-07-31 20:47:36 +02:00
Dmitry Vyukov
8ef497b652 gometalinter: clean up vetshadow
This just cleans up existing warnings.
vetshadow is not enabled yet because it crashes.

Update #538
2018-07-31 20:38:33 +02:00
Dmitry Vyukov
efc2683657 pkg/csource: rafactor option checking
Update #538
2018-07-31 18:43:45 +02:00
Dmitry Vyukov
46bad78542 pkg/report: refactor linux report parsing
Split a very long function into 2.

Update #538
2018-07-31 18:38:54 +02:00
Dmitry Vyukov
8d7727990b pkg/csource: refactor call generation
Slightly reduce cyclomatic complexity.

Update #538
2018-07-31 17:05:09 +02:00
Dmitry Vyukov
69eaab186d pkg/compiler: refactor genStructDescs
Reduce cyclomatic complexity of genStructDescs.

Update #538
2018-07-31 17:04:29 +02:00
Dmitry Vyukov
50c3709eb0 .gometalinter: reduce dupl threshold
Reduce dupl threshold from 63 to 60 and fix violations.

Update #538
2018-07-31 12:42:52 +02:00
Dmitry Vyukov
f5d67fbd9c .gometalinter.json: enable gofmt
The part that we want from gofmt is simplify (-s).
Fix all code that needs fixing.

Update #538
2018-07-31 12:16:54 +02:00
Dmitry Vyukov
b8bd991ee2 executor: make current_time_ms/sleep_ms
Fix conditions for when current_time_ms/sleep_ms are used
and make them static.
2018-07-30 11:17:44 +02:00
Dmitry Vyukov
1a381291a3 executor: collect coverage from unfinished syscalls
Write coverage from unfinished syscalls.
Also detect when a syscall was blocked during execution,
even if it finished. Helpful for fallback coverage.

Fixes #580
2018-07-29 19:46:05 +02:00
Dmitry Vyukov
a43c5ed638 pkg/csource: turn off 32-bit tests
Test OS fails the same was as linux on travis.
2018-07-27 13:48:44 +02:00
Dmitry Vyukov
9faf980ad5 executor: disable clang-format for some parts
clang-format badly mishandles this part,
moreover different versions mishandle it differently.
2018-07-27 13:48:44 +02:00
Dmitry Vyukov
788351b1b0 pkg/csource: rewrite gen.sh in Go
Shell files cause portability problems.
On Linux it's hard to install /bin/sh,
/bin/bash is not present on *BSD.
Any solution is hard to test on Darwin.
Don't even want to mention Windows.

Just do it in Go.
2018-07-27 10:22:23 +02:00
Dmitry Vyukov
b25fc7b831 pkg/csource: add option to trace syscall results
This will be needed for testing of generated programs.
2018-07-27 10:22:23 +02:00
Dmitry Vyukov
9d92841b4e pkg/csource: tidy generated code
1. Remove unnecessary includes.
2. Remove thunk function in threaded mode.
3. Inline syscalls into main for the simplest case.
4. Define main in common.h rather than form with printfs.
5. Fix generation for repeat mode
   (we had 2 infinite loops: in main and in loop).
6. Remove unused functions (setup/reset_loop, setup/reset_test,
   sandbox_namespace, etc).
2018-07-27 10:22:23 +02:00
Dmitry Vyukov
c3da5dc5e0 executor: simplify event_timedwait
We always have current_time_ms in event_timedwait
so use it instead of manual clock_gettime calls
which tend to be bulkier.
2018-07-27 10:22:23 +02:00
Dmitry Vyukov
4dcfea28eb pkg/host: support test OS
test OS parasitizeson other OSes just as akaros,
and there is nothing to detect as well.
So use the same stub for test as for akaros.
2018-07-27 10:22:23 +02:00
Dmitry Vyukov
22105302cc executor: ignore unshare errors in setuid sandbox
We already do this in sandbox=none, but forgot about setuid.
2018-07-27 10:22:23 +02:00
Dmitry Vyukov
ad155aa428 pkg/csource: say what compiler is missing 2018-07-25 10:44:45 +02:00
Dmitry Vyukov
e0c1bcbabc prog: move MaxPids const to prog
The const may be needed by other packages.
E.g. when checking manager config.
2018-07-25 10:39:29 +02:00
Dmitry Vyukov
3cbc0ac516 pkg/csource: tun/cgroups are linux-specific 2018-07-25 10:39:29 +02:00
Anton Lindqvist
9182049979 pkg/csource: remove /bin/bash assumption
OpenBSD and probably other BSDs does not ship with /bin/bash. This particular
script runs fine with regular /bin/sh.
2018-07-25 08:30:07 +02:00
Dmitry Vyukov
375a3e31e1 pkg/build: save kernel config early
Currently we don't capture kernel config for broken builds
because the config is saved as the last step.
Save it as early as possible.
2018-07-24 14:30:05 +02:00
Dmitry Vyukov
9fe4bdc5f1 executor: overhaul
Make as much code as possible shared between all OSes.
In particular main is now common across all OSes.
Make more code shared between executor and csource
(in particular, loop function and threaded execution logic).
Also make loop and threaded logic shared across all OSes.
Make more posix/unix code shared across OSes
(e.g. signal handling, pthread creation, etc).
Plus other changes along similar lines.
Also support test OS in executor (based on portable posix)
and add 4 arches that cover all execution modes
(fork server/no fork server, shmem/no shmem).

This change paves way for testing of executor code
and allows to preserve consistency across OSes and executor/csource.
2018-07-24 12:04:27 +02:00
Dmitry Vyukov
db7957bc09 pkg/report: add format for akaros warnings 2018-07-24 10:10:45 +02:00
Dmitry Vyukov
af255b09a7 pkg/vcs: poll fixing commits on akaros and fuchsia 2018-07-20 20:26:59 +02:00
Dmitry Vyukov
49f3583951 pkg/report: add another akaros backtrace message 2018-07-18 17:34:19 +02:00
Dmitry Vyukov
6d5bd5b5eb syz-ci: fix arch passed to build.Clean 2018-07-17 17:55:52 +02:00
Dmitry Vyukov
bad4246bf2 pkg/report: improve akaros reporter and implement symbolization 2018-07-16 17:03:14 +02:00
Dmitry Vyukov
58293c9eb3 pkg/report: fix kmalloc bug in krealloc 2018-07-16 10:16:21 +02:00
Dmitry Vyukov
92a4950507 pkg/host: add "network devices" feature
Linux executor sets up some network devices for testing,
detect when that's supported on the machine and don't
do it if it's not supported.
2018-07-13 12:46:32 +02:00
Dmitry Vyukov
d1dc5f516d pkg/compiler: fix error format string 2018-07-13 12:35:16 +02:00
Dmitry Vyukov
3d741171f2 pkg/report: add "lock held when returning to user space" format for older kernels 2018-07-13 10:31:26 +02:00
Dmitry Vyukov
3986ce95fd pkg/report: add report format for "lock held when returning to user space"
This report does not have stack trace, so we always detected it as corrupted.
2018-07-12 12:38:52 +02:00
Dmitry Vyukov
95e7a88b97 sys/linux: implement fuse as normal syscalls
Remove syz_fuse* and implement them as normal syscalls.
We not have enough expressive power to form mount options.
2018-07-10 16:18:45 +02:00
Dmitry Vyukov
01e3571841 pkg/host: check that we can open files rather than that they exist
See issue #640 where /dev/net/tun is present, but open fails with ENODEV.
Check that we can actually open all these files.

Fixes #640
2018-07-10 13:44:59 +02:00
Dmitry Vyukov
2e0e3130f9 pkg/build: update initrd name
Seems to have been renamed.
2018-07-10 11:14:36 +02:00
Dmitry Vyukov
7c31f7a78d pkg/build: always do clean build for fuchsia
We frequently see boot errors like:

[00010.201] 02991.03067> pkgsvr: 2018/06/30 23:39:41 system: failed to set system root from blob "ccbadb3901372b1e0fc5275f627f708bf3e5f3acfb0d4268638db0ff75fc7fd4": file does not exist

or:

[00003.691] 01126.01153> devmgr: launchpad /fs/blob/e66739acdd3d8efa3b7c9021e2107cf8431765c0b8eb0a1ec7f7dc7fd305f2f7 (pkgfs) failed: launchpad_vmo_from_file failure: -40

Presumably clean build may help.
2018-07-10 11:05:04 +02:00
Dmitry Vyukov
710eefe85a pkg/compiler: support negative integers
Currently we have to use 0xffffffffffffffff to represent -1,
and we can't express e.g. -20:20 int range.
Support negative consts to fix both problems.
2018-07-09 20:47:07 +02:00
Dmitry Vyukov
7004c9c8a8 executor: tweaks for better 9p testing
See the added comments.
2018-07-08 22:52:24 +02:00
Dmitry Vyukov
306ca0571c prog, pkg/compiler: support fmt type
fmt type allows to convert intergers and resources
to string representation.
2018-07-08 22:52:24 +02:00
Dmitry Vyukov
c9a7a4dccd executor: executor fix fuchsia build 2018-07-07 19:08:38 +02:00
Dmitry Vyukov
3abb98207d pkg/report: detect partial result errors 2018-07-06 20:25:02 +02:00
Dmitry Vyukov
0b95b8ec49 pkg/host: disable for akaros
akaros can't have own host version
because fuzzer does not run on akaros,
so just disable it all.
2018-07-06 20:18:05 +02:00
Dmitry Vyukov
0fb38ef496 pkg/report: detect fuchsia double faults 2018-07-06 16:35:21 +02:00
Dmitry Vyukov
04bd6c3d9e pkg/instance: pass -os to execprog/fuzzer only for akaros
Only akaros needs OS, because the rest assume host OS.
But speciying OS for all OSes breaks patch testing on syzbot
because old execprog does not have os flag.
2018-07-06 14:43:24 +02:00
Dmitry Vyukov
8c2335a205 vm/vmimpl: factor out common code for ssh args and waiting for ssh
Move common code from 4 vm implementations to vmimpl.
2018-07-06 14:02:06 +02:00
Dmitry Vyukov
d02d039529 pkg/report: fix lambda demangling
demangle package does not minimize lambda symbols properly
(does not strip parameters and template paratemeter).
Do it manually.

See: https://github.com/ianlancetaylor/demangle/issues/5
2018-07-05 13:21:01 +02:00
Dmitry Vyukov
538df42ec7 pkg/repro: provide stats even for failed repro
Provide stats and logs for failed repro and save it in manager.
In particular log is useful for failed repros,
currently there is no visibility into why bugs
failed to reproduce.
2018-07-05 13:14:00 +02:00
Dmitry Vyukov
7e2e7aa3e4 pkg/instance: pass os flag to execprog/fuzzer 2018-07-05 13:09:40 +02:00
Dmitry Vyukov
63226a5844 pkg/ipc: add rate limiting for akaros 2018-07-05 13:03:38 +02:00
Dmitry Vyukov
1c9d305894 pkg/build: add akaros support 2018-07-05 13:03:10 +02:00
Dmitry Vyukov
8ebdf5923d pkg/vcs: add akaros support 2018-07-05 13:02:56 +02:00
Dmitry Vyukov
bc1210b614 pkg/ipc: create default config based on target
Pass actual target to DefaultConfig, that's what we really want,
not the stray ipc flag.
2018-07-05 10:44:34 +02:00
Dmitry Vyukov
2290cea0a3 pkg/repro: pass target OS to execprog
Pass target OS to execprog and pass executor the same way
manager passes it to fuzzer.
2018-07-05 10:44:34 +02:00
Dmitry Vyukov
3e6e034413 executor: repair akaros support 2018-07-05 10:44:34 +02:00
Dmitry Vyukov
c32749f1e8 pkg/csource: disable most features on akaros
Akaros does not support tun, fault injections, sandboxes, etc.
2018-07-05 10:44:34 +02:00