reactos/syzkaller
1
0
Fork 0
mirror of https://github.com/reactos/syzkaller.git synced 2024-11-23 19:39:40 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
3,168 Commits 4 Branches 0 Tags 180 MiB
7ed11ab916
Commit Graph

763 Commits

Author SHA1 Message Date
Dmitry Vyukov
8d7727990b pkg/csource: refactor call generation 
Slightly reduce cyclomatic complexity.

Update #538
 2018-07-31 17:05:09 +02:00
Dmitry Vyukov
69eaab186d pkg/compiler: refactor genStructDescs 
Reduce cyclomatic complexity of genStructDescs.

Update #538
 2018-07-31 17:04:29 +02:00
Dmitry Vyukov
50c3709eb0 .gometalinter: reduce dupl threshold 
Reduce dupl threshold from 63 to 60 and fix violations.

Update #538
 2018-07-31 12:42:52 +02:00
Dmitry Vyukov
f5d67fbd9c .gometalinter.json: enable gofmt 
The part that we want from gofmt is simplify (-s).
Fix all code that needs fixing.

Update #538
 2018-07-31 12:16:54 +02:00
Dmitry Vyukov
b8bd991ee2 executor: make current_time_ms/sleep_ms 
Fix conditions for when current_time_ms/sleep_ms are used
and make them static.
 2018-07-30 11:17:44 +02:00
Dmitry Vyukov
1a381291a3 executor: collect coverage from unfinished syscalls 
Write coverage from unfinished syscalls.
Also detect when a syscall was blocked during execution,
even if it finished. Helpful for fallback coverage.

Fixes #580
 2018-07-29 19:46:05 +02:00
Dmitry Vyukov
a43c5ed638 pkg/csource: turn off 32-bit tests 
Test OS fails the same was as linux on travis.
 2018-07-27 13:48:44 +02:00
Dmitry Vyukov
9faf980ad5 executor: disable clang-format for some parts 
clang-format badly mishandles this part,
moreover different versions mishandle it differently.
 2018-07-27 13:48:44 +02:00
Dmitry Vyukov
788351b1b0 pkg/csource: rewrite gen.sh in Go 
Shell files cause portability problems.
On Linux it's hard to install /bin/sh,
/bin/bash is not present on *BSD.
Any solution is hard to test on Darwin.
Don't even want to mention Windows.

Just do it in Go.
 2018-07-27 10:22:23 +02:00
Dmitry Vyukov
b25fc7b831 pkg/csource: add option to trace syscall results 
This will be needed for testing of generated programs.
 2018-07-27 10:22:23 +02:00
Dmitry Vyukov
9d92841b4e pkg/csource: tidy generated code 
1. Remove unnecessary includes.
2. Remove thunk function in threaded mode.
3. Inline syscalls into main for the simplest case.
4. Define main in common.h rather than form with printfs.
5. Fix generation for repeat mode
   (we had 2 infinite loops: in main and in loop).
6. Remove unused functions (setup/reset_loop, setup/reset_test,
   sandbox_namespace, etc).
 2018-07-27 10:22:23 +02:00
Dmitry Vyukov
c3da5dc5e0 executor: simplify event_timedwait 
We always have current_time_ms in event_timedwait
so use it instead of manual clock_gettime calls
which tend to be bulkier.
 2018-07-27 10:22:23 +02:00
Dmitry Vyukov
4dcfea28eb pkg/host: support test OS 
test OS parasitizeson other OSes just as akaros,
and there is nothing to detect as well.
So use the same stub for test as for akaros.
 2018-07-27 10:22:23 +02:00
Dmitry Vyukov
22105302cc executor: ignore unshare errors in setuid sandbox 
We already do this in sandbox=none, but forgot about setuid.
 2018-07-27 10:22:23 +02:00
Dmitry Vyukov
ad155aa428 pkg/csource: say what compiler is missing 2018-07-25 10:44:45 +02:00
Dmitry Vyukov
e0c1bcbabc prog: move MaxPids const to prog 
The const may be needed by other packages.
E.g. when checking manager config.
 2018-07-25 10:39:29 +02:00
Dmitry Vyukov
3cbc0ac516 pkg/csource: tun/cgroups are linux-specific 2018-07-25 10:39:29 +02:00
Anton Lindqvist
9182049979 pkg/csource: remove /bin/bash assumption 
OpenBSD and probably other BSDs does not ship with /bin/bash. This particular
script runs fine with regular /bin/sh.
 2018-07-25 08:30:07 +02:00
Dmitry Vyukov
375a3e31e1 pkg/build: save kernel config early 
Currently we don't capture kernel config for broken builds
because the config is saved as the last step.
Save it as early as possible.
 2018-07-24 14:30:05 +02:00
Dmitry Vyukov
9fe4bdc5f1 executor: overhaul 
Make as much code as possible shared between all OSes.
In particular main is now common across all OSes.
Make more code shared between executor and csource
(in particular, loop function and threaded execution logic).
Also make loop and threaded logic shared across all OSes.
Make more posix/unix code shared across OSes
(e.g. signal handling, pthread creation, etc).
Plus other changes along similar lines.
Also support test OS in executor (based on portable posix)
and add 4 arches that cover all execution modes
(fork server/no fork server, shmem/no shmem).

This change paves way for testing of executor code
and allows to preserve consistency across OSes and executor/csource.
 2018-07-24 12:04:27 +02:00
Dmitry Vyukov
db7957bc09 pkg/report: add format for akaros warnings 2018-07-24 10:10:45 +02:00
Dmitry Vyukov
af255b09a7 pkg/vcs: poll fixing commits on akaros and fuchsia 2018-07-20 20:26:59 +02:00
Dmitry Vyukov
49f3583951 pkg/report: add another akaros backtrace message 2018-07-18 17:34:19 +02:00
Dmitry Vyukov
6d5bd5b5eb syz-ci: fix arch passed to build.Clean 2018-07-17 17:55:52 +02:00
Dmitry Vyukov
bad4246bf2 pkg/report: improve akaros reporter and implement symbolization 2018-07-16 17:03:14 +02:00
Dmitry Vyukov
58293c9eb3 pkg/report: fix kmalloc bug in krealloc 2018-07-16 10:16:21 +02:00
Dmitry Vyukov
92a4950507 pkg/host: add "network devices" feature 
Linux executor sets up some network devices for testing,
detect when that's supported on the machine and don't
do it if it's not supported.
 2018-07-13 12:46:32 +02:00
Dmitry Vyukov
d1dc5f516d pkg/compiler: fix error format string 2018-07-13 12:35:16 +02:00
Dmitry Vyukov
3d741171f2 pkg/report: add "lock held when returning to user space" format for older kernels 2018-07-13 10:31:26 +02:00
Dmitry Vyukov
3986ce95fd pkg/report: add report format for "lock held when returning to user space" 
This report does not have stack trace, so we always detected it as corrupted.
 2018-07-12 12:38:52 +02:00
Dmitry Vyukov
95e7a88b97 sys/linux: implement fuse as normal syscalls 
Remove syz_fuse* and implement them as normal syscalls.
We not have enough expressive power to form mount options.
 2018-07-10 16:18:45 +02:00
Dmitry Vyukov
01e3571841 pkg/host: check that we can open files rather than that they exist 
See issue #640 where /dev/net/tun is present, but open fails with ENODEV.
Check that we can actually open all these files.

Fixes #640
 2018-07-10 13:44:59 +02:00
Dmitry Vyukov
2e0e3130f9 pkg/build: update initrd name 
Seems to have been renamed.
 2018-07-10 11:14:36 +02:00
Dmitry Vyukov
7c31f7a78d pkg/build: always do clean build for fuchsia 
We frequently see boot errors like:

[00010.201] 02991.03067> pkgsvr: 2018/06/30 23:39:41 system: failed to set system root from blob "ccbadb3901372b1e0fc5275f627f708bf3e5f3acfb0d4268638db0ff75fc7fd4": file does not exist

or:

[00003.691] 01126.01153> devmgr: launchpad /fs/blob/e66739acdd3d8efa3b7c9021e2107cf8431765c0b8eb0a1ec7f7dc7fd305f2f7 (pkgfs) failed: launchpad_vmo_from_file failure: -40

Presumably clean build may help.
 2018-07-10 11:05:04 +02:00
Dmitry Vyukov
710eefe85a pkg/compiler: support negative integers 
Currently we have to use 0xffffffffffffffff to represent -1,
and we can't express e.g. -20:20 int range.
Support negative consts to fix both problems.
 2018-07-09 20:47:07 +02:00
Dmitry Vyukov
7004c9c8a8 executor: tweaks for better 9p testing 
See the added comments.
 2018-07-08 22:52:24 +02:00
Dmitry Vyukov
306ca0571c prog, pkg/compiler: support fmt type 
fmt type allows to convert intergers and resources
to string representation.
 2018-07-08 22:52:24 +02:00
Dmitry Vyukov
c9a7a4dccd executor: executor fix fuchsia build 2018-07-07 19:08:38 +02:00
Dmitry Vyukov
3abb98207d pkg/report: detect partial result errors 2018-07-06 20:25:02 +02:00
Dmitry Vyukov
0b95b8ec49 pkg/host: disable for akaros 
akaros can't have own host version
because fuzzer does not run on akaros,
so just disable it all.
 2018-07-06 20:18:05 +02:00
Dmitry Vyukov
0fb38ef496 pkg/report: detect fuchsia double faults 2018-07-06 16:35:21 +02:00
Dmitry Vyukov
04bd6c3d9e pkg/instance: pass -os to execprog/fuzzer only for akaros 
Only akaros needs OS, because the rest assume host OS.
But speciying OS for all OSes breaks patch testing on syzbot
because old execprog does not have os flag.
 2018-07-06 14:43:24 +02:00
Dmitry Vyukov
8c2335a205 vm/vmimpl: factor out common code for ssh args and waiting for ssh 
Move common code from 4 vm implementations to vmimpl.
 2018-07-06 14:02:06 +02:00
Dmitry Vyukov
d02d039529 pkg/report: fix lambda demangling 
demangle package does not minimize lambda symbols properly
(does not strip parameters and template paratemeter).
Do it manually.

See: https://github.com/ianlancetaylor/demangle/issues/5
 2018-07-05 13:21:01 +02:00
Dmitry Vyukov
538df42ec7 pkg/repro: provide stats even for failed repro 
Provide stats and logs for failed repro and save it in manager.
In particular log is useful for failed repros,
currently there is no visibility into why bugs
failed to reproduce.
 2018-07-05 13:14:00 +02:00
Dmitry Vyukov
7e2e7aa3e4 pkg/instance: pass os flag to execprog/fuzzer 2018-07-05 13:09:40 +02:00
Dmitry Vyukov
63226a5844 pkg/ipc: add rate limiting for akaros 2018-07-05 13:03:38 +02:00
Dmitry Vyukov
1c9d305894 pkg/build: add akaros support 2018-07-05 13:03:10 +02:00
Dmitry Vyukov
8ebdf5923d pkg/vcs: add akaros support 2018-07-05 13:02:56 +02:00
Dmitry Vyukov
bc1210b614 pkg/ipc: create default config based on target 
Pass actual target to DefaultConfig, that's what we really want,
not the stray ipc flag.
 2018-07-05 10:44:34 +02:00
Dmitry Vyukov
2290cea0a3 pkg/repro: pass target OS to execprog 
Pass target OS to execprog and pass executor the same way
manager passes it to fuzzer.
 2018-07-05 10:44:34 +02:00
Dmitry Vyukov
3e6e034413 executor: repair akaros support 2018-07-05 10:44:34 +02:00
Dmitry Vyukov
c32749f1e8 pkg/csource: disable most features on akaros 
Akaros does not support tun, fault injections, sandboxes, etc.
 2018-07-05 10:44:34 +02:00
Dmitry Vyukov
850c0a5cf3 pkg/osutil: allow to provide own stdout/stderr during cmd execution 
Useful if caller wants to collect only stdout or stderr.
 2018-07-05 10:44:34 +02:00
Dmitry Vyukov
d9ed95ab6f pkg/report: add simpler reporter for akaros 2018-07-05 10:44:34 +02:00
Dmitry Vyukov
0dee9249b7 executor: remove unnecessary parens 2018-07-05 10:44:34 +02:00
Dmitry Vyukov
f525fd7250 pkg/report: suppress more gvisor OOMs 2018-07-05 07:02:48 +02:00
Dmitry Vyukov
58924941cc pkg/report: fix race on symbolizer in test 
Symbolizer is not meant to be invoked with an empty binary.
 2018-07-02 17:00:00 +02:00
Dmitry Vyukov
574780b002 pkg/csource: prevent unused-result warnings 
Warnings for write unused-result fire on travis,
somehow I don't get them locally.
Use the result in a fake way to prevent the warning.
 2018-07-02 16:24:52 +02:00
Dmitry Vyukov
664ef9a3e1 pkg/compiler: check for unused declarations 
Error on unused structs/unions/resources/flags.
Finds tons of bugs.
 2018-06-30 19:34:41 +02:00
Dmitry Vyukov
b7d8ccf74f executor: include more headers on fuchsia 
Since we are taking address of functions in syscall table,
we need all headers even if we don't use them directly.
 2018-06-30 17:28:48 +02:00
Dmitry Vyukov
3e16f33c65 vm: suppress "no output" and "lost connection" reports 2018-06-30 14:51:07 +02:00
Dmitry Vyukov
3c690e2e53 pkg/repro: ignore suppressed crashes 
This looks like the simplest way to deal with suppressed crashes reported out of repro.

Fixes #645
 2018-06-30 14:37:59 +02:00
Dmitry Vyukov
44594e6362 pkg/report: suppress fuzzer crashes on fuchsia 2018-06-30 13:59:52 +02:00
Dmitry Vyukov
2960589335 pkg/csource: account for different types of syscalls on fuchsia 2018-06-30 13:40:00 +02:00
Dmitry Vyukov
65c456e595 pkg/csource: don't use pthread_cond_timedwait for fuchsia 
We removed it in executor, do the same in csource.
 2018-06-30 13:32:05 +02:00
Dmitry Vyukov
5012ddc8eb prog: detect when flags are a bitmask 2018-06-30 13:27:24 +02:00
Dmitry Vyukov
906b9e38f1 pkg/ipc: don't consider hang as failure 2018-06-30 13:21:47 +02:00
Dmitry Vyukov
1d788bb883 pkg/report: better detect hangs on fuchsia 2018-06-30 13:21:17 +02:00
Dmitry Vyukov
f7498af7af prog: add stronger fallback signal 
Also mixin resource constructors and some signature
of flags values for successful calls into fallback coverage.
 2018-06-29 20:34:43 +02:00
Dmitry Vyukov
0c4b1960d0 pkg/ipc: suppress gometalinter warnings 2018-06-29 18:18:50 +02:00
Dmitry Vyukov
0a971ab1d0 syz-manager: implement fallback coverage report 2018-06-29 18:15:41 +02:00
Dmitry Vyukov
9dd8cf63e0 executor, pkg/ipc: support output over pipes 2018-06-29 16:33:07 +02:00
Dmitry Vyukov
c45c8c2a07 pkg/instance: fix Accept error check 2018-06-29 12:58:02 +02:00
Dmitry Vyukov
5c7ca877eb pkg/vcs: fix fuchsia sandboxing 2018-06-29 12:11:46 +02:00
Dmitry Vyukov
bdac5e17f1 pkg/build: don't fail without compiler 
Some OSes don't need a compiler.
Just return an empty string in such case.
 2018-06-29 12:03:04 +02:00
Dmitry Vyukov
ca1f32d15a pkg/ipc: cleanup binary copy 2018-06-29 11:58:22 +02:00
Dmitry Vyukov
ac5f183dc0 pkg/ipc: fix cleanup in test 
Currently we first send on errs and then close env.
As the result process can exit before env.Close finishes,
which will leave garbage behind.
Close env before sending on errs.
 2018-06-29 11:46:16 +02:00
Dmitry Vyukov
346edcb763 executor: fix pid check after fork 2018-06-29 11:45:59 +02:00
Dmitry Vyukov
7b45fa115b pkg/csource: support fuchsia 
Lots of assorted heavylifting to support csource on fuchsia.
 2018-06-29 10:47:42 +02:00
Dmitry Vyukov
df42529884 pkg/ipc: use absolute path to start subprocess 
fuchsia mishandles relative paths.
 2018-06-29 10:46:25 +02:00
Dmitry Vyukov
68ce63c468 pkg/build: support fuchsia builds 2018-06-28 13:42:07 +02:00
Dmitry Vyukov
a2c1dcb323 pkg/report: implement fuchsia reporter 2018-06-27 19:44:00 +02:00
Dmitry Vyukov
919e77001d pkg/vcs: add fuchsia support 
For now only checking out and polling.
 2018-06-27 14:56:46 +02:00
Dmitry Vyukov
58e8587f64 pkg/vcs: pave way for multi-vcs support 
Wrap current git interface in abstract interface.
Provide constructor that create repo interface for the given os/vm.
 2018-06-27 13:57:21 +02:00
Dmitry Vyukov
43da5e3a1b pkg/vcs: move from pkg/git 
Rename pkg/git to pkg/vcs because we need to support not only git.
 2018-06-27 13:07:03 +02:00
Dmitry Vyukov
43e60f7e09 pkg/report: supporess another OOM for gvisor 2018-06-27 09:35:10 +02:00
Dmitry Vyukov
b0294c53d0 syz-fuzzer: partially revert "syz-fuzzer: fix gvisor testing" 
This breaks patch testing on syz-ci as it can use older version of syz-fuzzer.
 2018-06-26 16:42:22 +02:00
Dmitry Vyukov
be3706ff1e pkg/build: better support for gvisor race build 
Explicitly pass --features=race.
Expect linux_amd64_static_stripped as a potential output.
 2018-06-26 15:36:46 +02:00
Dmitry Vyukov
089f11817e syz-fuzzer: fix gvisor testing 
Testing code wasn't ready to dial stdin.
Make it use the same logic rpc package uses
to connecto to host.
 2018-06-26 14:12:43 +02:00
Dmitry Vyukov
e726bdf922 syz-manager: make rpc communication finer grained 
RPC package does excessive caching per connection,
so if a larger object is ever sent in any direction,
rpc connection consumes large amount of memory persistently.
This makes manager consume gigs of memory with large
number of VMs and larger corpus/coverage.

Make all communication done in very limited batches.
 2018-06-26 13:59:47 +02:00
Dmitry Vyukov
826b5aabc4 pkg/rpctype: compress rpc traffic 
Both manager and fuzzer consume huge amount of memory
(lots of gigs for manager) due to excessive caching
in rpc connections. Compress traffic to reduce memory
consumption.
 2018-06-26 13:52:36 +02:00
Dmitry Vyukov
3f9c293199 executor: don't crash during tun setup if ipv6 is not enabled 
Some options that tun initialization sets up are optional.
Don't fail if they are missing in kernel.
 2018-06-26 13:47:21 +02:00
Dmitry Vyukov
21e16efbf0 pkg/report: add suppressions for Go race OOMs 2018-06-23 10:13:12 +02:00
Dmitry Vyukov
802897bc33 pkg/instance: say that gvisor can create instances out of thin air 2018-06-23 10:12:35 +02:00
Dmitry Vyukov
665947c70a pkg/build: support gvisor race binaries 2018-06-23 09:46:28 +02:00
Dmitry Vyukov
8e0b1c8525 pkg/ipc: add fallback signal only if normal signal is not enabled 
It's possible to get no signal from normal coverage due to dedup,
in that case we don't want to add fallback coverage
because it can lead to corpus bloat.
 2018-06-23 08:38:50 +02:00
Dmitry Vyukov
cb93a59bbf pkg/report: fix gvisor segv reports 2018-06-22 20:01:10 +02:00
Dmitry Vyukov
67ce863533 pkg/build: save kernel config for gvisor 
gvisor does not use it, but save it verbatim.
This can be useful to keep some additional description of the build.
 2018-06-22 19:13:38 +02:00
Dmitry Vyukov
b5323c280f pkg/report: add more gvisor formats 2018-06-22 17:38:37 +02:00
Dmitry Vyukov
c97f0d7a86 pkg/build: add gvisor support 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
ea804a7120 pkg/build: pave way for multi-OS support 
Unify kernel and image build, that distinction is really uninteresting.
Define interface that each OS needs to implement.
Add gvisor stub.
 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
8c9738f9c7 pkg/build: support bazel in CompilerIdentity 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
94ef62054c pkg/build: move from pkg/kernel 
Rename pkg/kernel to pkg/build and prepare for multi-OS support.
 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
2a075d57ab pkg/report: allow to specify suppressions per OS 
Currently all (linux-specific) suppressions are hardcoded in mgrconfig.
This is very wrong. Move them to pkg/report and allow to specify per OS.
Add gvisor-specific suppressions.
This required a bit of refactoring. Introduce mgrconfig.KernelObj finally.
Make report.NewReporter and vm.Create accept mgrconfig directly
instead of passing it as multiple scattered args.
Remove tools/syz-parse and it always did the same as tools/syz-symbolize.
Simplify global vars in syz-manager/cover.go.
Create reporter eagerly in manager. Use sort.Slice more.
Overall -90 lines removed.
 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
c31f96a8c6 executor: rework fallback coverage 
We have fallback coverage implmentation for freebsd.
1. It's broken after some recent changes.
2. We need it for fuchsia, windows, akaros, linux too.
3. It's painful to work with C code.

Move fallback coverage to ipc package,
fix it and provide for all OSes.
 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
14e6c472f5 vm/gvisor: add package 
gvisor package provides support for gVisor, user-space kernel, testing.
See https://github.com/google/gvisor
 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
c71647f2cc pkg/report: add gvisor support 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
3cf0de82e9 pkg/report: move title sanitization from linux to common code 
Stripping dynamic data (addresses, numbers) is required for all OSes.
Move this code from linux to common code.
 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
9a7d0a5412 pkg/report: pass vm type to NewReporter 
For the case when VM type affects output.
Will be needed for gvisor. It is kinda linux, but kinda not.
 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
87bfb99cfe vm: pass instance to MonitorExecution 
It may need it later to try to obtain additional
diagnostic from hanged instances.
 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
ef9ddfbe36 executor: handle case when AF_INET is not enabled 
No AF_INET is somewhat crazy, but why not.
 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
095ef80678 pkg/host: fix race in test 2018-06-19 14:33:56 +02:00
Dmitry Vyukov
ed73f9c3aa pkg/host: more precise detection for syz_mount_image support 2018-06-18 19:45:50 +02:00
Dmitry Vyukov
af9f337ea6 pkg/host: support trial supported syscall detection 
Detect supported syscall by directly executing them
if kallsyms is not present. This is required for gvisor testing.
 2018-06-18 19:45:45 +02:00
Dmitry Vyukov
7bd97c6ff6 pkg/host: better detection of supported sockets 
Check socketpair.
Check non-constant socket types.
 2018-06-18 19:45:45 +02:00
Dmitry Vyukov
b7d00d1e14 pkg/email: support quoted-printable emails 
We did not handle quoted-printable because mime package handles it.
But we can have a non-mime email in quoted-printable.
Simply handle it always, it's not hard.
 2018-06-18 19:45:44 +02:00
Dmitry Vyukov
27c5f59f50 all: fix gometalinter warnings 2018-06-12 20:10:58 +02:00
Dmitry Vyukov
06ece2ca66 pkg/host: rework host feature detection/setup 
Currently host feature detection/setup code is spread
across platform-independent fuzzer code, pkg/host, pkg/ipc
and executor.
Move this all into pkg/host and show readable info
about features on manager start.

Fixes #46
 2018-06-12 14:53:22 +02:00
Dmitry Vyukov
62d1af2467 pkg/instance: more robust instance testing 
Strictly saying, we may not get the connection when
the fuzzer process exits. The accepting goroutine
may have not been scheduled yet.
For the connection for up to 10 seconds.
 2018-06-11 16:47:12 +02:00
Dmitry Vyukov
ae8bdb5058 syz-fuzzer: don't crash on commit mismatch 
We now check for manager-fuzzer-executor commit mismatch (see Manager.Check).
But in some cases commit mismatch is not detected gracefully, and instead
leads to panics in fuzzer. Namely, when -enabled_syscalls fuzzer flag includes
large syscalls numbers, so large that they are no present at all in the an old
revision that fuzzer uses, in such case fuzzer panics.

Notify manager about invalid calls instead.

Fixes #464
 2018-06-11 12:12:01 +03:00
Dmitry Vyukov
deb0e69e10 executor: always use errors=continue when mounting ext2/3/4 
For ext2/3/4 we have to have errors=continue because the image
can contain errors=panic flag and can legally crash kernel.

Fixes #599
 2018-06-11 12:12:01 +03:00
Dmitry Vyukov
6db75fefbe pkg/report: add another format for null RIP 
Kernel keeps inventing new formats...
 2018-06-08 09:34:02 +02:00
Dmitry Vyukov
645e75f803 executor: make syscall table and number constant 
We see some crashes that suggest corruption of the syscall number:

invalid command number 1296 (errno 11)
invalid command number 107 (errno 110)

Make the table and the number constant to prevent corruption.
 2018-06-07 15:32:30 +02:00
Dmitry Vyukov
7fb9023fd7 pkg/report: ignore down functions for task hung bugs 2018-06-07 14:01:22 +02:00
Dmitry Vyukov
d2ba33fac0 pkg/kernel: pass CC to oldconfig 
Passing in compiler is important since 4.17
(at the very least it's noted in the config).
 2018-06-07 10:18:14 +02:00
Dmitry Vyukov
a6bf43aeeb pkg/report: add simplistic fuchsia reporter 2018-06-06 16:29:59 +02:00
Dmitry Vyukov
606f9a96d8 pkg/csource: allow to build source fom memory 
In most cases we have source in memory,
so provide a function to build right from memory
without creating temp files.
 2018-06-06 10:30:54 +02:00
Dmitry Vyukov
6479ab2a75 Makefile, sys/targets: move all native compilation logic to sys/targets 
We currently have native cross-compilation logic duplicated
in Makefile and in sys/targets. Some pieces are missed in one
place, some are in another. Only pkg/csource knows how to check
for -static support.

Move all CC/CFLAGS logic to sys/targets and pull results in Makefile.

This should make Makefile work on distros that have broken x86_64-linux-gnu-gcc,
now we will use just gcc. And this removes the need to define NOSTATIC,
as it's always auto-detected.

This also paves the way for making pkg/csource work on OSes other than Linux.
 2018-06-06 10:02:03 +02:00
Dmitry Vyukov
9820e1cae6 pkg/report: more "NMI handler took too long" ignores 2018-06-06 10:02:03 +02:00
Dmitry Vyukov
3f4f3d5ec9 pkg/report: ignore INFO messages about long interrupts 2018-06-04 12:07:28 +02:00
Dmitry Vyukov
738d58ade0 pkg/csource: minimize netdevices and net reset 
Add separate options to minimize netdevices setup and net namespace reset.

Fixes #581
 2018-05-17 19:57:54 +02:00
Dmitry Vyukov
3717901c10 pkg/csource: serialize options into json 
Current format is painful to parse after changes.
Switch to json.
 2018-05-17 19:07:46 +02:00
Dmitry Vyukov
be47ebb0eb pkg/kernel: allow to split full make output 
Currently kernel build failures are insanely verbose
(contain full kernel build output) and there is no
way to separate short descriptions from full output.
Make it possible.
Also try to extract failure root cause froom build log.
Use this in pkg/bisect to not pollute log on build failures.

Update #501
 2018-05-17 17:18:24 +02:00
Dmitry Vyukov
28cbff8c7d pkg/bisect: stop at Linux v3.8 
Update #501
 2018-05-17 16:40:00 +02:00
Dmitry Vyukov
8dd3159f51 pkg/bisect: increase number of tries to 8 
With 5 tries sometimes only 1 fails,
and sometimes we probably have false negatives.
Increase number of tries to 8 and compress
results if they all are the same.

Update #501
 2018-05-17 16:30:20 +02:00
Dmitry Vyukov
edbe39a26d pkg/git: extract relevant email addresses from commits 
Update #501
 2018-05-17 15:58:58 +02:00
Dmitry Vyukov
4e1c0dbaea pkg/bisect: add bisection logic 
Add first version of bisection package
(supports both bug and fix bisection).
And tools/syz-bisect wrapper for testing.

Update #501
 2018-05-17 14:52:39 +02:00
Dmitry Vyukov
6595937c34 tools/create-gce-image.sh: support both nbd and loop 
Pass target OS/arch and VM type to kernel.CreateImage.
Use nbd for gce and loop for qemu VM type.
 2018-05-17 14:50:18 +02:00
Dmitry Vyukov
9c84f7c395 pkg/osutil: allow disabling sandboxing with env var 
If SYZ_DISABLE_SANDBOXING=yes is set, don't do user sandboxing.
Will be usefule for bisection tool which runs locally,
but needs to build kernel.

Update #501
 2018-05-17 14:03:46 +02:00
Dmitry Vyukov
031c7941d7 pkg/compiler: add optional builtin template 
type optional[T] [
	val	T
	void	void
] [varlen]
 2018-05-17 12:02:30 +02:00
Dmitry Vyukov
90c54c496b pkg/compiler: detect duplicate fields in template structs 2018-05-17 11:46:45 +02:00
Dmitry Vyukov
a367c1d7a8 tools/create-gce-image.sh: support MKE2FS_CONFIG 2018-05-16 17:21:29 +02:00
Dmitry Vyukov
86ea19e4c2 tools/create-gce-image.sh: revert loop to nbd 
loop devices are nice, but unfortunately this creates images
that does not boot on GCE. Reason is unclear.
Revert back to nbd for now.
 2018-05-14 19:32:39 +02:00
Dmitry Vyukov
d131f1b9fc pkg/instance: relax syzkaller path check 
Only check that syzkaller path is in GOPATH if we are going to build it.
syz-ci image testing does not have syzkaller path in GOPATH,
but it also does not build syzkaller.
 2018-05-14 13:47:44 +02:00
Dmitry Vyukov
2407e7407a pkg/instance: add package for testing of images/patches/bisection 
Move helper image/patch testing code from syz-ci/testing.go
to a separate package so that it can be reused during bisection.

Update #501
 2018-05-14 11:32:56 +02:00
Dmitry Vyukov
9fb3ae4dfd pkg/osutil: introduce TempFile helper 
Introduce TempFile helper and use it in several packages.
 2018-05-14 11:17:23 +02:00
Dmitry Vyukov
acfd774f46 pkg/git: add bisection functionality 
Bisect bisects good..bad commit range against the provided predicate (wrapper around git bisect).
The predicate should return an error only if there is no way to proceed
(it will abort the process), if possible it should prefer to return BisectSkip.
Progress of the process is streamed to the provided trace.
Returns the first commit on which the predicate returns BisectBad.

Update #501
 2018-05-14 11:17:23 +02:00
Dmitry Vyukov
b4df103f78 pkg/git: add PreviousReleaseTags 
PreviousReleaseTags returns list of preceding release tags
that are reachable from the given commit.

Update #501
 2018-05-14 11:17:23 +02:00
Dmitry Vyukov
5060be1a90 pkg/git: add SwitchCommit and GetCommit 
Will be useful for bisection logic.

Update #501
 2018-05-14 11:17:23 +02:00
Dmitry Vyukov
d39e52252c pkg/kernel: use oldconfig instead of olddefconfig 
See the added comment for motivation.

Update #501
 2018-05-14 11:17:23 +02:00
Dmitry Vyukov
2a287a4dfc pkg/kernel: accept kernel config as byte slice 
We usually have the config in memory,
so saving it to disk is an unnecessary step.
Accept byte slice directly.

Update #501
 2018-05-14 11:17:23 +02:00
Dmitry Vyukov
1944eeb95d pkg/kernel: run distclean with -jNCPU 2018-05-14 11:17:18 +02:00
Dmitry Vyukov
ea9496cdbd tools/create-gce-image.sh: use loop instead of nbd 
loop is much more standard than nbd and does not require additional modules.
nbd broke on Debian rolling.
loop also allows parallel execution thanks to losetup -f.

Use loop instead of nbd.
Also improve cleanup logic and add one missing sudo.

Update #501
 2018-05-14 11:16:36 +02:00
Ioana Ciornei
481f030ccd executor: fix strncpy compile error 
gcc8 is stricter when dealing with strings and strncpy and demands that
the size of the actual string to be copied to be explicitly smaller than
the size of the destination, just to make sure the NULL terminator is
taken into considerantion. This patch fixes the issue.

Signed-off-by: Ioana Ciornei <ciorneiioana@gmail.com>
 2018-05-13 16:27:22 +02:00
Dmitry Vyukov
045bbd4afb pkg/csource: fix test exit code 2018-05-07 20:40:44 +02:00
Dmitry Vyukov
02e3401638 pkg/csource: skip tests under race detector 
csource tests consume too much memory under race detector (>1GB),
and periodically timeout on Travis. So we skip them.
 2018-05-07 20:27:06 +02:00
Dmitry Vyukov
bdff58b1a1 pkg/csource: reduce test size 
The test times out on travis:

https://travis-ci.org/google/syzkaller/jobs/375965899
https://travis-ci.org/google/syzkaller/jobs/375965900
 2018-05-07 19:24:46 +02:00
Dmitry Vyukov
9e0846e8a4 all: get rid of underscores in identifiers 
Underscores are against Go coding style.

Update #538
 2018-05-07 17:59:06 +02:00
Dmitry Vyukov
99c1f48659 sys/syz-extract: split overly long function 
Update #538
 2018-05-07 16:50:19 +02:00
Dmitry Vyukov
55a5e98a0a pkg/ifuzz: move generated code to a separate package 
This helps to ignore generated code with gometalinter.
There is currently no other way:
https://github.com/alecthomas/gometalinter/issues/270

Update #538
 2018-05-07 15:42:28 +02:00
Dmitry Vyukov
5f9dcfdadb pkg/git: remove duplicated code in tests 
Update #538
 2018-05-07 13:16:29 +02:00
Dmitry Vyukov
b9fea20df7 pkg/report: remove duplicated stub code 
Update #538
 2018-05-07 13:05:41 +02:00
Dmitry Vyukov
a211da1a40 pkg/db: remove code duplication in test 
Update #538
 2018-05-06 19:47:40 +02:00
Dmitry Vyukov
2c7e14a847 gometalinter: enable cyclomatic complexity checking 
Refactor some functions to be simpler.

Update #538
 2018-05-04 18:03:46 +02:00
Dmitry Vyukov
08141db61a gometalinter: enable line length checking 
120 columns looks like a reasonable limit
and requires few changes to existing code.

Update #538
 2018-05-04 14:24:51 +02:00
Dmitry Vyukov
a630fd8b41 gometalinter: some fixes for unparam 
But we still can't enable it as there are more [uninteresting] warnings.

Update #538
 2018-05-03 15:48:26 +02:00
Dmitry Vyukov
9fe5658a1b gometalinter: check dot imports 
Update #538
 2018-05-03 14:41:34 +02:00
Dmitry Vyukov
39302300d9 gometalinter: check unkeyed composite literals 
Update #538
 2018-05-03 14:01:48 +02:00
Dmitry Vyukov
4c24e4a467 gometalinter: enable package comment checking 
Update #538
 2018-05-03 13:53:01 +02:00
Dmitry Vyukov
6e2ce6613b pkg/host: fix syz_mount_image detection 
Only init admin can mount filesystems with images.
 2018-04-29 18:45:45 +02:00
Dmitry Vyukov
1e85f7b9af pkg/ast: support char constants 
Frequently it's useful to do something like:

int8['a':'z']
punctuation = ',', '-', ':'
 2018-04-29 12:04:22 +02:00
Dmitry Vyukov
c7f6891ca7 pkg/report: reformat code 
There is some regression in Go formatting on tip.
I am constantly getting diffs after formatting.
Filed: https://github.com/golang/go/issues/25161
 2018-04-29 12:04:22 +02:00
Dmitry Vyukov
190d92e056 sys/linux: extend namespace desciptions 
Slightly extend namespace descriptions and move
them to a separate file.
 2018-04-27 18:47:58 +02:00
Dmitry Vyukov
0008f52678 pkg/report: fix remaining lockdep report formats 
We fixed only 1, but new format affects all lockdep reports.
Fix all of them.
 2018-04-27 15:44:47 +02:00
Dmitry Vyukov
3c7caf1d6e pkg/host: enable more fault injection 2018-04-27 15:43:29 +02:00
Dmitry Vyukov
d0b7645387 pkg/report: few report parsing fixes for linux 2018-04-26 14:46:16 +02:00
Dmitry Vyukov
37e76fe20b pkg/scource: rename compiled binary to syz-executor 
We call the binary syz-executor because it sometimes shows in bug titles,
and we don't want 2 different bugs for when a crash is triggered during
fuzzing and during repro.
 2018-04-24 19:17:07 +02:00
Dmitry Vyukov
fd5157bebf pkg/report: sanitize titles better 
Strip non-printable characters from titles.
Trim whitespaces on both ends.
Replace tabs with spaces.
Remove duplicate spaces.
Reduce max title length.
 2018-04-24 19:17:07 +02:00
Dmitry Vyukov
98e363b852 pkg/git: don't create shallow repos 
Shallow repos created by CheckoutBranch conflict with
what CheckoutCommit tries to do.
Fetch of a shallow repo does not unshallow it.
And then checkout of a non-head commit fails.
 2018-04-24 19:17:07 +02:00
Dmitry Vyukov
9366d03f00 dashboard/app: allow testing fixes on exact commit and without patch 
This implements 2 features:
1. It's now possible to specify exact commit when testing as:

2. It's possible to test without patch attached
assuming the patch is already committed to the tested tree.

Fixes #558
 2018-04-24 13:23:01 +02:00
Dmitry Vyukov
e2f4bf8f38 pkg/gce: improve API rate limit logic 
Sometimes we see rate limiting errors.
This is especially bad for e.g. patch testing requests.
Increase default API delay and add backoff logic.
 2018-04-24 13:23:01 +02:00
Dmitry Vyukov
33424b08c3 pkg/report: more corrupted report detection 2018-04-24 13:23:01 +02:00
Dmitry Vyukov
e7e85d3609 pkg/kernel: fix gometalinter warning 2018-04-23 16:50:25 +02:00
Dmitry Vyukov
8b5dcf17b0 syz-ci: clean kernel dir before patch test build 
This leads to false errors when we are switching between gcc and clang:

kernel build failed: failed to run /usr/bin/make [make bzImage -j 32 CC=/syzkaller/clang-kmsan/bin/clang]: exit status 2
arch/x86/Makefile:184: *** Compiler lacks asm-goto support..  Stop.

Fixes #568
 2018-04-23 16:26:26 +02:00
Dmitry Vyukov
ff12bea91c pkg/ipc: fix data race on config.Timeout 2018-04-23 16:26:26 +02:00
Dmitry Vyukov
8b9a3f2c76 pkg/report: add some example arm crashes 2018-04-20 20:14:10 +02:00
Dmitry Vyukov
bf2018dd07 executor: don't fail when ipv6 is not enabled 
Check in checkpoint_iptables is not enough as
reset_iptables will fail even if no tables enabled.
Add similar check to reset_iptables.
 2018-04-20 20:14:10 +02:00
Dmitry Vyukov
e5453dd396 executor: increase max image size to 128MB 
Images of some filesystems need to be that large (xfs, btrfs, f2fs).
 2018-04-19 16:59:49 +02:00
Dmitry Vyukov
81daaca980 pkg/host: fix kallsyms names again 
Some x86 syscalls now seem to be __x64_ and __ia32_ instead of sys_ or ksys_.
 2018-04-19 16:58:24 +02:00
Dmitry Vyukov
19af2a1b35 pkg/report: coarsen "unregister_netdevice" title 
This leads to unbounded number of bug reports as syzkaller
can generate new device names dynamically.
Exclude actual device name from title.
 2018-04-19 13:58:34 +02:00
Dmitry Vyukov
829f023456 pkg/host: fix kallsyms check 
Recent kernel changes to not call syscalls directly from kernel code
renamed syscall symbols from sys_foo to ksys_foo.
Support both formats.
 2018-04-18 16:30:13 +02:00
Dmitry Vyukov
2199644490 pkg/report: reformat code 2018-04-18 09:23:00 +02:00
Dmitry Vyukov
85d38101ef pkg/report: ignore pkg/host debug output 2018-04-16 21:23:32 +02:00
Dmitry Vyukov
f24e540fe5 executor: don't fail if IPv6 is not enabled 2018-04-16 21:11:10 +02:00
Dmitry Vyukov
61155cf882 pkg/report: filter out more corrupted reports due to fault injection 2018-04-16 14:01:05 +02:00
Dmitry Vyukov
7a67784ca8 pkg/report: detect hangs in unregister_netdevice 
Add special report format for hangs in unregister_netdevice.
And support new format of lockdep reports (otherwise they all
are marked as corrupted).
 2018-04-13 17:44:45 +02:00
Hangbin Liu
0a0c5db668 executor: add two slave interfaces for bridge,bond,team 
Bridge device is used for forwarding. Bond/team device is used for
load balance and fail over. So it would make more sense to add two
slave interfaces for these devices.

Signed-off-by: Hangbin Liu <liuhangbin@gmail.com>
 2018-04-13 09:38:31 +02:00
Dmitry Vyukov
eb2295de3e pkg/repro: clean up temp files 2018-04-12 16:06:06 +02:00
Dmitry Vyukov
9cd56d71a8 sys/linux: add veth devices to devnames 
Also comment new veth code for future me.
 2018-04-11 11:36:25 +02:00