Commit Graph

2705 Commits

Author SHA1 Message Date
Dmitry Vyukov
58e8587f64 pkg/vcs: pave way for multi-vcs support
Wrap current git interface in abstract interface.
Provide constructor that create repo interface for the given os/vm.
2018-06-27 13:57:21 +02:00
Dmitry Vyukov
43da5e3a1b pkg/vcs: move from pkg/git
Rename pkg/git to pkg/vcs because we need to support not only git.
2018-06-27 13:07:03 +02:00
Dmitry Vyukov
43e60f7e09 pkg/report: supporess another OOM for gvisor 2018-06-27 09:35:10 +02:00
Dmitry Vyukov
21caae5df6 syz-ci: don't report transient errors during patch testing
If no instances crashed and at least one booted and completed
testing successfully, report success.

Fixes #641
2018-06-26 19:17:45 +02:00
Dmitry Vyukov
4dd045dff9 syz-manager: remove unused field 2018-06-26 19:02:01 +02:00
Dmitry Vyukov
ecbe6d99d3 syz-fuzzer: fix testing when shmem is not used
We get program output info only if shmem is used.
Output info over pipes is not implemented.
So don't fail when we got nothing.
2018-06-26 18:59:39 +02:00
Dmitry Vyukov
eecefab32c Makefile: fix fuchsia build 2018-06-26 18:59:07 +02:00
Dmitry Vyukov
b5f361149e sys/fuchsia: remove zx_job_set_relative_importance
It seems to be removed from zircon.
2018-06-26 18:48:45 +02:00
Dmitry Vyukov
b0294c53d0 syz-fuzzer: partially revert "syz-fuzzer: fix gvisor testing"
This breaks patch testing on syz-ci as it can use older version of syz-fuzzer.
2018-06-26 16:42:22 +02:00
Dmitry Vyukov
be3706ff1e pkg/build: better support for gvisor race build
Explicitly pass --features=race.
Expect linux_amd64_static_stripped as a potential output.
2018-06-26 15:36:46 +02:00
Dmitry Vyukov
089f11817e syz-fuzzer: fix gvisor testing
Testing code wasn't ready to dial stdin.
Make it use the same logic rpc package uses
to connecto to host.
2018-06-26 14:12:43 +02:00
Dmitry Vyukov
e726bdf922 syz-manager: make rpc communication finer grained
RPC package does excessive caching per connection,
so if a larger object is ever sent in any direction,
rpc connection consumes large amount of memory persistently.
This makes manager consume gigs of memory with large
number of VMs and larger corpus/coverage.

Make all communication done in very limited batches.
2018-06-26 13:59:47 +02:00
Dmitry Vyukov
826b5aabc4 pkg/rpctype: compress rpc traffic
Both manager and fuzzer consume huge amount of memory
(lots of gigs for manager) due to excessive caching
in rpc connections. Compress traffic to reduce memory
consumption.
2018-06-26 13:52:36 +02:00
Dmitry Vyukov
3f9c293199 executor: don't crash during tun setup if ipv6 is not enabled
Some options that tun initialization sets up are optional.
Don't fail if they are missing in kernel.
2018-06-26 13:47:21 +02:00
Dmitry Vyukov
5e7849c5d1 docs: add gvisor page 2018-06-25 11:58:28 +02:00
Dmitry Vyukov
2064fc5c91 sys/linux: add 2 custom uids
Give fuzzer 2 custom uids to mess with.
Ideally these should be proc, we can't do this for resources.
2018-06-23 11:36:55 +02:00
Dmitry Vyukov
f199f3a27d sys/linux: fix /dev/urandom descriptions
Don't use pseudo syscall without a reason.
Remove duplicate syscall.
2018-06-23 11:25:20 +02:00
Dmitry Vyukov
21e16efbf0 pkg/report: add suppressions for Go race OOMs 2018-06-23 10:13:12 +02:00
Dmitry Vyukov
802897bc33 pkg/instance: say that gvisor can create instances out of thin air 2018-06-23 10:12:35 +02:00
Dmitry Vyukov
665947c70a pkg/build: support gvisor race binaries 2018-06-23 09:46:28 +02:00
Dmitry Vyukov
8e0b1c8525 pkg/ipc: add fallback signal only if normal signal is not enabled
It's possible to get no signal from normal coverage due to dedup,
in that case we don't want to add fallback coverage
because it can lead to corpus bloat.
2018-06-23 08:38:50 +02:00
Dmitry Vyukov
89d2e60023 vm/gvisor: fix empty runsc_args 2018-06-22 20:50:19 +02:00
Dmitry Vyukov
b9c7c31970 syz-fuzzer: prevent not output crashes during machine checking
Machine checking can be very slow on some machines
(qemu without kvm, KMEMLEAK linux, etc),
so print periodic heartbeats for vm.MonitorExecution
so that it does not decide that we are dead.
2018-06-22 20:32:54 +02:00
Dmitry Vyukov
cb93a59bbf pkg/report: fix gvisor segv reports 2018-06-22 20:01:10 +02:00
Dmitry Vyukov
bf0a1af0cb vm/gvisor: always give vm all caps
runsc can crash on nil deref without any caps.
So give all of them all the time.
2018-06-22 19:56:39 +02:00
Dmitry Vyukov
67ce863533 pkg/build: save kernel config for gvisor
gvisor does not use it, but save it verbatim.
This can be useful to keep some additional description of the build.
2018-06-22 19:13:38 +02:00
Dmitry Vyukov
39a5713d72 vm/gvisor: allow specifying arbitrary runsc flags 2018-06-22 19:00:12 +02:00
Dmitry Vyukov
412bfd3c58 vm/gvisor: use timeout for all runsc invocations
runsc likes to hang dead. Add timeouts to all invocations.
2018-06-22 18:37:02 +02:00
Dmitry Vyukov
b5323c280f pkg/report: add more gvisor formats 2018-06-22 17:38:37 +02:00
Dmitry Vyukov
c97f0d7a86 pkg/build: add gvisor support 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
91f52697d1 syz-fuzzer: panic less
Use log.Fatalf with readable messages instead of panic's.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
2b949be2e1 executor: extend error on output overflow 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
ea804a7120 pkg/build: pave way for multi-OS support
Unify kernel and image build, that distinction is really uninteresting.
Define interface that each OS needs to implement.
Add gvisor stub.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
8c9738f9c7 pkg/build: support bazel in CompilerIdentity 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
94ef62054c pkg/build: move from pkg/kernel
Rename pkg/kernel to pkg/build and prepare for multi-OS support.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
2a075d57ab pkg/report: allow to specify suppressions per OS
Currently all (linux-specific) suppressions are hardcoded in mgrconfig.
This is very wrong. Move them to pkg/report and allow to specify per OS.
Add gvisor-specific suppressions.
This required a bit of refactoring. Introduce mgrconfig.KernelObj finally.
Make report.NewReporter and vm.Create accept mgrconfig directly
instead of passing it as multiple scattered args.
Remove tools/syz-parse and it always did the same as tools/syz-symbolize.
Simplify global vars in syz-manager/cover.go.
Create reporter eagerly in manager. Use sort.Slice more.
Overall -90 lines removed.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
c31f96a8c6 executor: rework fallback coverage
We have fallback coverage implmentation for freebsd.
1. It's broken after some recent changes.
2. We need it for fuchsia, windows, akaros, linux too.
3. It's painful to work with C code.

Move fallback coverage to ipc package,
fix it and provide for all OSes.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
14e6c472f5 vm/gvisor: add package
gvisor package provides support for gVisor, user-space kernel, testing.
See https://github.com/google/gvisor
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
c71647f2cc pkg/report: add gvisor support 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
3cf0de82e9 pkg/report: move title sanitization from linux to common code
Stripping dynamic data (addresses, numbers) is required for all OSes.
Move this code from linux to common code.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
87dda8591f syz-fuzzer: use single connection to manager
This will simplify gvisor support as it's somewhat problematic
to proxy multiple connections between host and guest.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
d3bbcc35ee vm/vmimpl: add vm.Diagnose method
Diagnose is called on machine hang to try to get
some additional diagnostic information from it.
For now it's all stubs.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
9a7d0a5412 pkg/report: pass vm type to NewReporter
For the case when VM type affects output.
Will be needed for gvisor. It is kinda linux, but kinda not.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
87bfb99cfe vm: pass instance to MonitorExecution
It may need it later to try to obtain additional
diagnostic from hanged instances.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
ef9ddfbe36 executor: handle case when AF_INET is not enabled
No AF_INET is somewhat crazy, but why not.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
095ef80678 pkg/host: fix race in test 2018-06-19 14:33:56 +02:00
Dmitry Vyukov
7bdf6e025f dashboard/app: fix table sorting on firefox
Under Firefox 60 browser, sort operation does not work and

  ReferenceError: event is not defined

error message is printed in the Web Console window.
Let's explicitly pass an object reference to the sortTable function.

Credit goes to Tetsuo Handa.
2018-06-19 13:07:54 +02:00
Dmitry Vyukov
732e4256fb sys/linux: regenerate consts
@evdenis
2018-06-19 12:41:12 +02:00
Denis Efremov
14e96aad73 sys/linux: additional scsi_ioctls of sg device
Add previously skipped SCSI_IOCTL_* ioctls in
the sg rule.

Signed-off-by: Denis Efremov <efremov@linux.com>
2018-06-19 12:35:31 +02:00
Dmitry Vyukov
45c54f755c syz-manager: fix race during reporter init 2018-06-18 19:50:50 +02:00