Commit Graph

2696 Commits

Author SHA1 Message Date
Dmitry Vyukov
be3706ff1e pkg/build: better support for gvisor race build
Explicitly pass --features=race.
Expect linux_amd64_static_stripped as a potential output.
2018-06-26 15:36:46 +02:00
Dmitry Vyukov
089f11817e syz-fuzzer: fix gvisor testing
Testing code wasn't ready to dial stdin.
Make it use the same logic rpc package uses
to connecto to host.
2018-06-26 14:12:43 +02:00
Dmitry Vyukov
e726bdf922 syz-manager: make rpc communication finer grained
RPC package does excessive caching per connection,
so if a larger object is ever sent in any direction,
rpc connection consumes large amount of memory persistently.
This makes manager consume gigs of memory with large
number of VMs and larger corpus/coverage.

Make all communication done in very limited batches.
2018-06-26 13:59:47 +02:00
Dmitry Vyukov
826b5aabc4 pkg/rpctype: compress rpc traffic
Both manager and fuzzer consume huge amount of memory
(lots of gigs for manager) due to excessive caching
in rpc connections. Compress traffic to reduce memory
consumption.
2018-06-26 13:52:36 +02:00
Dmitry Vyukov
3f9c293199 executor: don't crash during tun setup if ipv6 is not enabled
Some options that tun initialization sets up are optional.
Don't fail if they are missing in kernel.
2018-06-26 13:47:21 +02:00
Dmitry Vyukov
5e7849c5d1 docs: add gvisor page 2018-06-25 11:58:28 +02:00
Dmitry Vyukov
2064fc5c91 sys/linux: add 2 custom uids
Give fuzzer 2 custom uids to mess with.
Ideally these should be proc, we can't do this for resources.
2018-06-23 11:36:55 +02:00
Dmitry Vyukov
f199f3a27d sys/linux: fix /dev/urandom descriptions
Don't use pseudo syscall without a reason.
Remove duplicate syscall.
2018-06-23 11:25:20 +02:00
Dmitry Vyukov
21e16efbf0 pkg/report: add suppressions for Go race OOMs 2018-06-23 10:13:12 +02:00
Dmitry Vyukov
802897bc33 pkg/instance: say that gvisor can create instances out of thin air 2018-06-23 10:12:35 +02:00
Dmitry Vyukov
665947c70a pkg/build: support gvisor race binaries 2018-06-23 09:46:28 +02:00
Dmitry Vyukov
8e0b1c8525 pkg/ipc: add fallback signal only if normal signal is not enabled
It's possible to get no signal from normal coverage due to dedup,
in that case we don't want to add fallback coverage
because it can lead to corpus bloat.
2018-06-23 08:38:50 +02:00
Dmitry Vyukov
89d2e60023 vm/gvisor: fix empty runsc_args 2018-06-22 20:50:19 +02:00
Dmitry Vyukov
b9c7c31970 syz-fuzzer: prevent not output crashes during machine checking
Machine checking can be very slow on some machines
(qemu without kvm, KMEMLEAK linux, etc),
so print periodic heartbeats for vm.MonitorExecution
so that it does not decide that we are dead.
2018-06-22 20:32:54 +02:00
Dmitry Vyukov
cb93a59bbf pkg/report: fix gvisor segv reports 2018-06-22 20:01:10 +02:00
Dmitry Vyukov
bf0a1af0cb vm/gvisor: always give vm all caps
runsc can crash on nil deref without any caps.
So give all of them all the time.
2018-06-22 19:56:39 +02:00
Dmitry Vyukov
67ce863533 pkg/build: save kernel config for gvisor
gvisor does not use it, but save it verbatim.
This can be useful to keep some additional description of the build.
2018-06-22 19:13:38 +02:00
Dmitry Vyukov
39a5713d72 vm/gvisor: allow specifying arbitrary runsc flags 2018-06-22 19:00:12 +02:00
Dmitry Vyukov
412bfd3c58 vm/gvisor: use timeout for all runsc invocations
runsc likes to hang dead. Add timeouts to all invocations.
2018-06-22 18:37:02 +02:00
Dmitry Vyukov
b5323c280f pkg/report: add more gvisor formats 2018-06-22 17:38:37 +02:00
Dmitry Vyukov
c97f0d7a86 pkg/build: add gvisor support 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
91f52697d1 syz-fuzzer: panic less
Use log.Fatalf with readable messages instead of panic's.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
2b949be2e1 executor: extend error on output overflow 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
ea804a7120 pkg/build: pave way for multi-OS support
Unify kernel and image build, that distinction is really uninteresting.
Define interface that each OS needs to implement.
Add gvisor stub.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
8c9738f9c7 pkg/build: support bazel in CompilerIdentity 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
94ef62054c pkg/build: move from pkg/kernel
Rename pkg/kernel to pkg/build and prepare for multi-OS support.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
2a075d57ab pkg/report: allow to specify suppressions per OS
Currently all (linux-specific) suppressions are hardcoded in mgrconfig.
This is very wrong. Move them to pkg/report and allow to specify per OS.
Add gvisor-specific suppressions.
This required a bit of refactoring. Introduce mgrconfig.KernelObj finally.
Make report.NewReporter and vm.Create accept mgrconfig directly
instead of passing it as multiple scattered args.
Remove tools/syz-parse and it always did the same as tools/syz-symbolize.
Simplify global vars in syz-manager/cover.go.
Create reporter eagerly in manager. Use sort.Slice more.
Overall -90 lines removed.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
c31f96a8c6 executor: rework fallback coverage
We have fallback coverage implmentation for freebsd.
1. It's broken after some recent changes.
2. We need it for fuchsia, windows, akaros, linux too.
3. It's painful to work with C code.

Move fallback coverage to ipc package,
fix it and provide for all OSes.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
14e6c472f5 vm/gvisor: add package
gvisor package provides support for gVisor, user-space kernel, testing.
See https://github.com/google/gvisor
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
c71647f2cc pkg/report: add gvisor support 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
3cf0de82e9 pkg/report: move title sanitization from linux to common code
Stripping dynamic data (addresses, numbers) is required for all OSes.
Move this code from linux to common code.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
87dda8591f syz-fuzzer: use single connection to manager
This will simplify gvisor support as it's somewhat problematic
to proxy multiple connections between host and guest.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
d3bbcc35ee vm/vmimpl: add vm.Diagnose method
Diagnose is called on machine hang to try to get
some additional diagnostic information from it.
For now it's all stubs.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
9a7d0a5412 pkg/report: pass vm type to NewReporter
For the case when VM type affects output.
Will be needed for gvisor. It is kinda linux, but kinda not.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
87bfb99cfe vm: pass instance to MonitorExecution
It may need it later to try to obtain additional
diagnostic from hanged instances.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
ef9ddfbe36 executor: handle case when AF_INET is not enabled
No AF_INET is somewhat crazy, but why not.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
095ef80678 pkg/host: fix race in test 2018-06-19 14:33:56 +02:00
Dmitry Vyukov
7bdf6e025f dashboard/app: fix table sorting on firefox
Under Firefox 60 browser, sort operation does not work and

  ReferenceError: event is not defined

error message is printed in the Web Console window.
Let's explicitly pass an object reference to the sortTable function.

Credit goes to Tetsuo Handa.
2018-06-19 13:07:54 +02:00
Dmitry Vyukov
732e4256fb sys/linux: regenerate consts
@evdenis
2018-06-19 12:41:12 +02:00
Denis Efremov
14e96aad73 sys/linux: additional scsi_ioctls of sg device
Add previously skipped SCSI_IOCTL_* ioctls in
the sg rule.

Signed-off-by: Denis Efremov <efremov@linux.com>
2018-06-19 12:35:31 +02:00
Dmitry Vyukov
45c54f755c syz-manager: fix race during reporter init 2018-06-18 19:50:50 +02:00
Dmitry Vyukov
ee54fec6da syz-manager: ignore inputs that are not enabled on target
Currently we only ignore programs that contain syscalls
that are not statically enabled in config. This does not
account for syscalls that are not supported on target
machine. Load corpus after we got machine check with
actual list of supported syscalls.
2018-06-18 19:45:52 +02:00
Dmitry Vyukov
ed73f9c3aa pkg/host: more precise detection for syz_mount_image support 2018-06-18 19:45:50 +02:00
Dmitry Vyukov
dde158e0af sys/linux: refine xdp description
Split sockaddr_xdp for bind.
Bind accepts another sock_xdp in addr.
Without the split getsockaddr "can" create sock_xdp's
because it returns generic sockaddr which contains
all addresses, including sockaddr_xdp, which in turn
contains sock_xdp.
2018-06-18 19:45:49 +02:00
Dmitry Vyukov
dcb1eebb7f prog: more precise constructor calculation
Currently a call that both accepts and creates a resource
self-justifies itself and thus is always enabled.
A good example is accept call. Accepts are always self-enable
and thus enable all other syscalls that work with the socket.

Calculate TransitivelyEnabledCalls in the opposite direction
to resolve this. Start with empty set of enable syscalls,
then enable syscalls that don't accept any resources,
then enable syscalls that accept resources created by the
previous batch of syscalls, and so on.

This prevents self-enablement of accept.
2018-06-18 19:45:48 +02:00
Dmitry Vyukov
920b18be87 sys: mark output resources as opt
Mark output resources as opt in preparation for more
precise constructor calculation.
2018-06-18 19:45:47 +02:00
Dmitry Vyukov
2dcee3b338 prog: use all resource special values for opt args 2018-06-18 19:45:46 +02:00
Dmitry Vyukov
af9f337ea6 pkg/host: support trial supported syscall detection
Detect supported syscall by directly executing them
if kallsyms is not present. This is required for gvisor testing.
2018-06-18 19:45:45 +02:00
Dmitry Vyukov
7bd97c6ff6 pkg/host: better detection of supported sockets
Check socketpair.
Check non-constant socket types.
2018-06-18 19:45:45 +02:00
Dmitry Vyukov
b7d00d1e14 pkg/email: support quoted-printable emails
We did not handle quoted-printable because mime package handles it.
But we can have a non-mime email in quoted-printable.
Simply handle it always, it's not hard.
2018-06-18 19:45:44 +02:00