Commit Graph

183 Commits

Author SHA1 Message Date
Dmitry Vyukov
ad0af9fff5 vm: return Report from MonitorExecution
This allows callers to get access to Report.Corrupted.
Better than adding 6-th return value and will allow
to pipe other report properties if necessary.
2017-11-21 19:02:35 +01:00
Dmitry Vyukov
4bd78cef05 pkg/report, pkg/repro, syz-manager: name crash attributes consistently
We currently have several names for crash attributes, which is disturbing.
E.g. crash title is called "Title" or "Desc". Name them consistently.

Title - single line bug identity.
Report - whole crash text.
Log - whole fuzzer/kernel output.
2017-11-14 10:04:22 +01:00
Dmitry Vyukov
10112655d7 vm: remove needOutput arg for MonitorExecution
Always wait 10 secs for output.
If anything this can only lead to missed crashes during repro.
Let's unify manager and repro behavior.
2017-11-14 09:45:34 +01:00
Dmitry Vyukov
7a53e7e35d pkg/report: combine report data into a struct
Parse returns 5 variables now. Later we may want to add crash "priority".
Introduce Report struct that holds all report data.
2017-11-14 09:41:55 +01:00
Andrey Konovalov
f9a8d567eb pkg/report: add corrupted report detection
This change makes pkg/report try to detect corrupted reports by
using some heuristics.
2017-11-13 17:18:16 +03:00
Andrey Konovalov
a2c64463a2 execprog: correctly handle fault injections
syz-execprog doesn't utilize info about fault injections from a prog log.
Since syz-execprog is used by the repro package to reproduce crashes,
crashes caused by fault injections might not reproduce.
2017-11-07 16:17:18 +01:00
Dmitry Vyukov
d5a1adcc06 tools/syz-execprog: allow to override target OS
This is currently useful for akaros, which is tested remotely.
2017-11-06 15:01:28 +01:00
Andrey Konovalov
26d265c811 docs, tools: add local link checker
This commit adds tools/check_links.py script, that checks that all local
links from documentation files are valid; fixes some of the invalid links
that we had; and makes travis buildbot check them as well.
2017-10-27 10:04:34 +02:00
Dmitry Vyukov
8fa0c867d4 syz-fuzzer: generates hints only for the call that gave new coverage
During smashing we know what call gave new coverage,
so we can concentrate just on it.
This helps to reduce amount of hints generated (we have too many of them).
2017-10-23 09:59:39 +02:00
Dmitry Vyukov
54ae9c6db3 tools/syz-execprog: print total number of comps/hints 2017-10-23 09:59:39 +02:00
Dmitry Vyukov
85c802e4cf pkg/report: support multiple OSes
Introduce report.Reporter interface.
Add an implementation per-OS.
Make users be explicit about OS they are testing.
2017-10-18 12:01:24 +02:00
Dmitry Vyukov
a8a0b01a8b tools/syz-mutate: allow to specify target 2017-10-17 10:54:19 +02:00
Dmitry Vyukov
aa2533b98d tools/syz-prog2c: allow to specify target OS 2017-10-16 14:21:54 +02:00
Dmitry Vyukov
9444f97045 tools/syz-stress: allow to specify target OS 2017-10-16 14:21:54 +02:00
Dmitry Vyukov
d49f04b345 tools/syz-prog2c: import targets
Currently syz-prog2c is broken as it does not import any targets.
Import sys package.
2017-10-02 13:57:04 +02:00
Dmitry Vyukov
64b6c0724d sys/windows: add more descriptions 2017-09-27 20:17:09 +02:00
Dmitry Vyukov
913d592f97 all: more assorted fuchsia support 2017-09-22 13:10:55 +02:00
Dmitry Vyukov
8cb7d3dcfc all: initial support for fuchsia
Nothing works, but builds.

Update #191
2017-09-20 21:19:29 +02:00
Dmitry Vyukov
52a33fd516 prog: remove default target and all global state
Now each prog function accepts the desired target explicitly.
No global, implicit state involved.
This is much cleaner and allows cross-OS/arch testing, etc.
2017-09-15 16:02:37 +02:00
Dmitry Vyukov
c0cabacda7 syz-fuzzer, syz-execprog: add -arch flag
arch flag specifies target arch, which can be different from GOARCH.
For example, 386 executor with amd64 fuzzer.
2017-09-15 16:02:37 +02:00
Dmitry Vyukov
f7b1163afb syz-manager/mgrconfig: explicitly specify target in config
Add target config parameter (e.g. linux/amd64) which controls target OS/arch.
No more explicit assumptions about target.
2017-09-15 16:02:37 +02:00
Dmitry Vyukov
ffe7e17368 prog, sys: move types to prog
Large overhaul moves syscalls and arg types from sys to prog.
Sys package now depends on prog and contains only generated
descriptions of syscalls.
Introduce prog.Target type that encapsulates all targer properties,
like syscall list, ptr/page size, etc. Also moves OS-dependent pieces
like mmap call generation from prog to sys.

Update #191
2017-09-05 15:52:42 +02:00
Dmitry Vyukov
5db39ab953 sys: rename Call to Syscall
In preparation for moving sys types to prog
to avoid confusion between sys.Call and prog.Call.
2017-09-05 10:38:22 +02:00
Victor Chibotaru
49c11eb514 ipc, prog, fuzzer, execprog: add hints generation code
A hint is basically a tuple consisting of a pointer to an argument
in one of the syscalls of a program and a value, which should be
assigned to that argument.

A simplified version of hints workflow looks like this:
    1. Fuzzer launches a program and collects all the comparisons' data
for every syscall in the program.
    2. Next it tries to match the obtained comparison operands' values
vs. the input arguments' values.
    3. For every such match the fuzzer mutates the program by
replacing the pointed argument with the saved value.
    4. If a valid program is obtained, then fuzzer launches it and
checks if new coverage is obtained.

This commit includes:
    1. All the code related to hints generation, parsing and mutations.
    2. Fuzzer functions to launch the process.
    3. Some new stats gathered by fuzzer and manager, related to hints.
    4. An updated version of execprog to test the hints process.
2017-08-30 18:40:14 +02:00
Dmitry Vyukov
4074aed7c0 pkg/compiler: more static error checking
Update #217
2017-08-27 20:19:41 +02:00
Dmitry Vyukov
a3857c4e90 pkg/compiler, sys/syz-sysgen: move const handling to pkg/compiler
Now pkg/compiler deals with consts.
2017-08-27 11:51:40 +02:00
Zubin Mithra
c3631fc789 tools: add headerparser as a tool to assist in writing system call descriptions
The tool can be found inside tools/syz-headerparser. Details on how to use
headerparser can be found inside docs/headerparser_usage.md.
2017-08-25 17:23:52 +02:00
Dmitry Vyukov
41bbf437e1 Makefile: enforce formatting of sys files in presubmit 2017-08-18 11:26:50 +02:00
Dmitry Vyukov
127a9c2b65 pkg/ast: new parser for sys descriptions
The old parser in sys/sysparser is too hacky, difficult to extend
and drops debug info too early, so that we can't produce proper error messages.

Add a new parser that is build like a proper language parser
and preserves full debug info for every token.
2017-08-18 11:26:50 +02:00
Dmitry Vyukov
a0330c0f5e tools/create-gce-image.sh: explicitly specify target for grub
On most distributions default grub target is i386-pc, which works.
However, on some default is x86_64-efi, which fails with:

grub-install: error: cannot find EFI directory.

Explicitly specify i386-pc target.
2017-08-11 20:04:33 +02:00
Dmitry Vyukov
ab0ced187d tools/syz-benchcmp: add execution speed
Exec total is affected by initial triage/minimize phase,
so two experiments can have the same execution speed
in the stable mode, but have constant diff due to the initial phase.
The one that is higher looks better, but that's not very important.
Provide execution speed characteristic that is not affected
by initial phase. It is not displayed by default.
2017-08-10 19:31:15 +02:00
Dmitry Vyukov
5766dbe796 syz-dash, syz-gce: remove
This is now superseded by dashboard/app and syz-ci.
2017-08-02 19:50:48 +02:00
Dmitry Vyukov
7c1ee0634b tools/create-gce-image.sh: ignore SIGINT
If the script is aborted at an unfortunate point, it leaves the whole system broken.
E.g. we've seen that fdisk cannot update partition table until the next reboot.
If you really need to kill it, use a different signal. But better wait.
2017-07-17 14:45:41 +02:00
Dmitry Vyukov
8382eb648f pkg/kernel: actually pass cmdline/sysctl files to the build script 2017-07-17 14:45:41 +02:00
Dmitry Vyukov
cc1c342923 syz-ci: allow to specify cmdline/sysctls
Allow to specify per-kernel command line and sysctl values
to more closely mimic the target kernel.
2017-07-17 12:39:11 +02:00
Dmitry Vyukov
558280a686 tools/create-image.sh: create 2G images
1G does not seem to be enough.
Create 2G images as we do in create-gce-image.sh.
2017-07-17 09:56:08 +02:00
Dmitry Vyukov
a7b199253f all: use consistent file permissions
Currently we have unix permissions for new files/dirs
hardcoded throughout the code base. Some places use 0644,
some - 0640, some - 0600 and a variety of other constants.

Introduce osutil.MkdirAll/WriteFile that use the default
permissions and use them throughout the code base.

This makes permissions consistent and also allows to easily
change the permissions later if we change our minds.

Also merge pkg/fileutil into pkg/osutil as they become
dependent on each other. The line between them was poorly
defined anyway as both operate on files.
2017-07-03 14:00:47 +02:00
Dmitry Vyukov
51a013e7b9 tools/syz-symbolize: add report mode
Currently syz-symbolize symbolizes whole input file.
Add a new mode (controlled with -report flag) when
it prints report as would be extracted by syz-manager.
2017-06-30 15:17:42 +02:00
Zach Riggle
c30c1ddc7b Fix incorrect argument in syz-tty (#260) 2017-06-27 16:55:11 +02:00
Dmitry Vyukov
f6267bfc36 tools/syz-tty: add utility for testing of usb console reading code 2017-06-26 16:22:29 +02:00
Dmitry Vyukov
9af3153420 syz-dash/dashboard: move from dashboard
Move the old dashboard API into syz-dash
to get it out of the way of the new dashboard.
2017-06-23 15:22:34 +02:00
raminfp
90d10743fc Added GCC and glibc 2017-06-22 17:22:32 +04:30
Dmitry Vyukov
e130d95518 vm/gce: accept un-tar-ed image
vm/gce differs from other VM types in that it accepts image
in a weird, GCE-specific format (namely, image named disk.raw
is put into .tar.gz file). This makes it impossible to write
generic code that creates images for any VM types.

Make vm/gce accept just image like e.g. vm/qemu
and handle own specifics internally.
2017-06-17 17:06:29 +02:00
Dmitry Vyukov
de258e6d7e tools/create-gce-image.sh: don't use verbose output for tar invocation 2017-06-17 15:25:29 +02:00
Dmitry Vyukov
97b58e7eae syz-manager/mgrconfig: move sshkey from vm config to manager config
Sshkey is a property of image, which is in manager config.
Move sshkey to the same location as image.

The motivation for the move is as follows.
Continuous build produces an image and the key,
both need to be passed manager instance.
Continuous build system should not distinguish
different VM types and mess with their configs.

NOTE FOR USERS: this breaks manager configs again.
Hopefully the last time for now. Docs are updated.
2017-06-17 15:02:58 +02:00
Dmitry Vyukov
68621900a3 pkg/report: move from report 2017-06-17 14:41:15 +02:00
Dmitry Vyukov
e8e63830a6 pkg/ipc: move from ipc 2017-06-17 14:41:15 +02:00
Dmitry Vyukov
baad3b4b02 pkg/csource: move from csource 2017-06-17 14:41:15 +02:00
Dmitry Vyukov
ec95d17c32 pkg/cover: move from cover 2017-06-17 14:41:15 +02:00
Dmitry Vyukov
6fd7a75a91 pkg/repro: move from pkg 2017-06-17 14:41:15 +02:00