Commit Graph

156 Commits

Author SHA1 Message Date
Dmitry Vyukov
41bbf437e1 Makefile: enforce formatting of sys files in presubmit 2017-08-18 11:26:50 +02:00
Dmitry Vyukov
127a9c2b65 pkg/ast: new parser for sys descriptions
The old parser in sys/sysparser is too hacky, difficult to extend
and drops debug info too early, so that we can't produce proper error messages.

Add a new parser that is build like a proper language parser
and preserves full debug info for every token.
2017-08-18 11:26:50 +02:00
Dmitry Vyukov
a0330c0f5e tools/create-gce-image.sh: explicitly specify target for grub
On most distributions default grub target is i386-pc, which works.
However, on some default is x86_64-efi, which fails with:

grub-install: error: cannot find EFI directory.

Explicitly specify i386-pc target.
2017-08-11 20:04:33 +02:00
Dmitry Vyukov
ab0ced187d tools/syz-benchcmp: add execution speed
Exec total is affected by initial triage/minimize phase,
so two experiments can have the same execution speed
in the stable mode, but have constant diff due to the initial phase.
The one that is higher looks better, but that's not very important.
Provide execution speed characteristic that is not affected
by initial phase. It is not displayed by default.
2017-08-10 19:31:15 +02:00
Dmitry Vyukov
5766dbe796 syz-dash, syz-gce: remove
This is now superseded by dashboard/app and syz-ci.
2017-08-02 19:50:48 +02:00
Dmitry Vyukov
7c1ee0634b tools/create-gce-image.sh: ignore SIGINT
If the script is aborted at an unfortunate point, it leaves the whole system broken.
E.g. we've seen that fdisk cannot update partition table until the next reboot.
If you really need to kill it, use a different signal. But better wait.
2017-07-17 14:45:41 +02:00
Dmitry Vyukov
8382eb648f pkg/kernel: actually pass cmdline/sysctl files to the build script 2017-07-17 14:45:41 +02:00
Dmitry Vyukov
cc1c342923 syz-ci: allow to specify cmdline/sysctls
Allow to specify per-kernel command line and sysctl values
to more closely mimic the target kernel.
2017-07-17 12:39:11 +02:00
Dmitry Vyukov
558280a686 tools/create-image.sh: create 2G images
1G does not seem to be enough.
Create 2G images as we do in create-gce-image.sh.
2017-07-17 09:56:08 +02:00
Dmitry Vyukov
a7b199253f all: use consistent file permissions
Currently we have unix permissions for new files/dirs
hardcoded throughout the code base. Some places use 0644,
some - 0640, some - 0600 and a variety of other constants.

Introduce osutil.MkdirAll/WriteFile that use the default
permissions and use them throughout the code base.

This makes permissions consistent and also allows to easily
change the permissions later if we change our minds.

Also merge pkg/fileutil into pkg/osutil as they become
dependent on each other. The line between them was poorly
defined anyway as both operate on files.
2017-07-03 14:00:47 +02:00
Dmitry Vyukov
51a013e7b9 tools/syz-symbolize: add report mode
Currently syz-symbolize symbolizes whole input file.
Add a new mode (controlled with -report flag) when
it prints report as would be extracted by syz-manager.
2017-06-30 15:17:42 +02:00
Zach Riggle
c30c1ddc7b Fix incorrect argument in syz-tty (#260) 2017-06-27 16:55:11 +02:00
Dmitry Vyukov
f6267bfc36 tools/syz-tty: add utility for testing of usb console reading code 2017-06-26 16:22:29 +02:00
Dmitry Vyukov
9af3153420 syz-dash/dashboard: move from dashboard
Move the old dashboard API into syz-dash
to get it out of the way of the new dashboard.
2017-06-23 15:22:34 +02:00
raminfp
90d10743fc Added GCC and glibc 2017-06-22 17:22:32 +04:30
Dmitry Vyukov
e130d95518 vm/gce: accept un-tar-ed image
vm/gce differs from other VM types in that it accepts image
in a weird, GCE-specific format (namely, image named disk.raw
is put into .tar.gz file). This makes it impossible to write
generic code that creates images for any VM types.

Make vm/gce accept just image like e.g. vm/qemu
and handle own specifics internally.
2017-06-17 17:06:29 +02:00
Dmitry Vyukov
de258e6d7e tools/create-gce-image.sh: don't use verbose output for tar invocation 2017-06-17 15:25:29 +02:00
Dmitry Vyukov
97b58e7eae syz-manager/mgrconfig: move sshkey from vm config to manager config
Sshkey is a property of image, which is in manager config.
Move sshkey to the same location as image.

The motivation for the move is as follows.
Continuous build produces an image and the key,
both need to be passed manager instance.
Continuous build system should not distinguish
different VM types and mess with their configs.

NOTE FOR USERS: this breaks manager configs again.
Hopefully the last time for now. Docs are updated.
2017-06-17 15:02:58 +02:00
Dmitry Vyukov
68621900a3 pkg/report: move from report 2017-06-17 14:41:15 +02:00
Dmitry Vyukov
e8e63830a6 pkg/ipc: move from ipc 2017-06-17 14:41:15 +02:00
Dmitry Vyukov
baad3b4b02 pkg/csource: move from csource 2017-06-17 14:41:15 +02:00
Dmitry Vyukov
ec95d17c32 pkg/cover: move from cover 2017-06-17 14:41:15 +02:00
Dmitry Vyukov
6fd7a75a91 pkg/repro: move from pkg 2017-06-17 14:41:15 +02:00
Dmitry Vyukov
4b2a9e225c pkg/host: move from host 2017-06-17 14:41:15 +02:00
Dmitry Vyukov
a853b91c58 syz-manager/mgrconfig: allow loading from memory
Allow loading manager config from memory.
Follow pkg/config naming convention,
i.e. LoadFile/LoadData instead of Parse.
2017-06-17 14:41:15 +02:00
Dmitry Vyukov
75c73461d8 syz-manager/config: rename to syz-manager/mgrconfig
We have 2 packages with the same name: pkg/config and syz-manager/config.
This leads to constant clashes. We either rename one to pkgconfig or
another to mgrconfig. This is not good and will become worse when/if
we have another program-specific config in a separate package.

Rename manager config to mgrconfig.
Other program-specific configs can use the same convention
in future -- fooconfig.
2017-06-17 14:41:10 +02:00
Dmitry Vyukov
4cc7e08628 all: use osutil.IsExist instead of os.Stat 2017-06-17 12:23:52 +02:00
Dmitry Vyukov
10d9c52cf0 tools/create-gce-image.sh: add nmi_watchdog=panic to cmd line 2017-06-15 18:29:47 +02:00
Andrey Konovalov
07335203da prog2c: use 1 prog by default 2017-06-12 19:48:23 +02:00
Andrey Konovalov
120e26c2fe csourse: don't generate debug printfs 2017-06-12 19:48:23 +02:00
Andrey Konovalov
c99b02d224 csource: try to simplify repeat loop 2017-06-12 19:48:23 +02:00
Andrey Konovalov
73a895df61 csource: use sandbox only when required 2017-06-12 19:48:23 +02:00
Andrey Konovalov
10c9064bfc csource: only handle SIGSEGV when necessary 2017-06-12 19:48:23 +02:00
Andrey Konovalov
5597911fbf csource: use tmp dir only when necessary 2017-06-12 19:48:23 +02:00
Andrey Konovalov
7d7c9c550f csource: add EnableTun option 2017-06-12 19:48:23 +02:00
Andrey Konovalov
4ca73f9c87 tools: repro: fix vm count calculation 2017-06-12 18:01:38 +02:00
Andrey Konovalov
d8ff00bacc tools: repro: fix vm count calculation 2017-06-07 16:20:53 +02:00
Dmitry Vyukov
af643baa32 vm: overhaul
VM infrastructure currently has several problems:
 - Config struct is complete mess with a superset of params for all VM types
 - verification of Config is mess spread across several places
 - there is no place where VM code could do global initialization
   like creating GCE connection, uploading GCE image to GCS,
   matching adb devices with consoles, etc
 - it hard to add private VM implementations
   such impl would need to add code to config package
   which would lead to constant merge conflicts
 - interface for VM implementation is mixed with interface for VM users
   this does not allow to provide best interface for both of them
 - there is no way to add common code for all VM implementations

This change solves these problems by:
 - splitting VM interface for users (vm package) and VM interface
   for VM implementations (vmimpl pacakge), this in turn allows
   to add common code
 - adding Pool concept that allows to do global initialization
   and config checking at the right time
 - decoupling manager config from VM-specific config
   each VM type now defines own config

Note: manager configs need to be changed after this change:
VM-specific parts are moved to own "vm" subobject.

Note: this change also drops "local" VM type.
Its story was long unclear and there is now syz-stress which solves the same problem.
2017-06-03 11:31:42 +02:00
Dmitry Vyukov
ea2295f3e2 pkg/db: move from db 2017-06-03 10:41:09 +02:00
Dmitry Vyukov
23b94422d3 pkg/log: move from log 2017-06-03 10:41:09 +02:00
Dmitry Vyukov
d1032c21a0 pkg/hash: move from hash 2017-06-03 10:41:09 +02:00
Dmitry Vyukov
61a577885b tools: more reliable network config in create-gce-image.sh
Currently we append to /etc/network/interfaces,
which can lead to duplicate lo entry.
Write out the whole interfaces file instead.
2017-06-03 10:41:09 +02:00
Dmitry Vyukov
ac2962a1c5 tools: sync create-image.sh and create-gce-image.sh
Move few additional settings from create-gce-image.sh to create-image.sh.
2017-06-03 10:41:09 +02:00
Dmitry Vyukov
84eb5fd389 config: split and refactor
Introduce generic config.Load function that can be
reused across multiple programs (syz-manager, syz-gce, etc).
Move the generic config functionality to pkg/config package.
The idea is to move all helper (non-main) packages to pkg/ dir,
because we have more and more of them and they pollute the top dir.
Move the syz-manager config parts into syz-manager/config package.
2017-06-01 11:17:54 +02:00
Michael Pratt
29fc5b76cd all: cleanup executor/ipc status checking
This is mostly a cleanup change with little functional change.

In ipc.command.exec, remove the status fallback from the pipe to the
exit status. Once the executor is serving, it always writes the status
over the pipe; anything else is an error.

Remove the panic check in syz-stress, which is no longer needed.
2017-05-30 09:33:51 -07:00
Dmitry Vyukov
220dc49106 csource: reproduce crashes with fault injection 2017-05-26 17:22:57 +02:00
Dmitry Vyukov
8f58526cb8 all: add fault injection capability
Systematically inject faults during smashing.
Requires kernel patch:
"fault-inject: support systematic fault injection"
(currently in linux-next).
2017-05-26 17:22:57 +02:00
Andrey Konovalov
f919224c44 sys, executor: extract tcp sequence numbers from /dev/net/tun
This commit adds a new pseudo syscall syz_extract_tcp_res, that reads
a packet from /dev/net/tun and extracts tcp sequence numbers to be used
in subsequent packets.

As a result this syzkaller program:

mmap(&(0x7f0000000000/0x10000)=nil, (0x10000), 0x3, 0x32, 0xffffffffffffffff, 0x0)
r0 = socket$inet_tcp(0x2, 0x1, 0x0)
bind$inet(r0, &(0x7f0000001000)={0x2, 0x0, @empty=0x0, [0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0]}, 0x10)
listen(r0, 0x5)
syz_emit_ethernet(0x36, &(0x7f0000002000)={@local={[0xaa, 0xaa, 0xaa, 0xaa, 0xaa], 0x0}, @random="4c6112cc15d8", [], {{0x800, @ipv4={{0x5, 0x4, 0x0, 0x0, 0x28, 0x0, 0x0, 0x0, 0x6, 0x0, @remote={0xac, 0x14, 0x0, 0xbb}, @local={0xac, 0x14, 0x0, 0xaa}, {[]}}, @tcp={{0x1, 0x0, 0x42424242, 0x42424242, 0x0, 0x0, 0x5, 0x2, 0x0, 0x0, 0x0, {[]}}, {""}}}}}})
syz_extract_tcp_res(&(0x7f0000003000)={<r1=>0x42424242, <r2=>0x42424242}, 0x1, 0x0)
syz_emit_ethernet(0x38, &(0x7f0000004000)={@local={[0xaa, 0xaa, 0xaa, 0xaa, 0xaa], 0x0}, @remote={[0xbb, 0xbb, 0xbb, 0xbb, 0xbb], 0x0}, [], {{0x800, @ipv4={{0x5, 0x4, 0x0, 0x0, 0x2a, 0x0, 0x0, 0x0, 0x6, 0x0, @remote={0xac, 0x14, 0x0, 0xbb}, @local={0xac, 0x14, 0x0, 0xaa}, {[]}}, @tcp={{0x1, 0x0, r2, r1, 0x0, 0x0, 0x5, 0x10, 0x0, 0x0, 0x0, {[]}}, {"0c10"}}}}}})
r3 = accept$inet(r0, &(0x7f0000005000)={0x0, 0x0, @multicast1=0x0, [0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0]}, &(0x7f0000006000)=0x10)

established a TCP connection:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:20000           0.0.0.0:*               LISTEN      5477/a.out
tcp        2      0 172.20.0.170:20000      172.20.0.187:20001      ESTABLISHED 5477/a.out

Similar program for IPv6:

mmap(&(0x7f0000000000/0x10000)=nil, (0x10000), 0x3, 0x32, 0xffffffffffffffff, 0x0)
r0 = socket$inet6_tcp(0xa, 0x1, 0x0)
bind$inet6(r0, &(0x7f0000000000)={0xa, 0x1, 0x0, @empty={[0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0]}, 0x0}, 0x1c)
listen(r0, 0x5)
syz_emit_ethernet(0x4a, &(0x7f0000001000)={@local={[0xaa, 0xaa, 0xaa, 0xaa, 0xaa], 0x0}, @random="de895db1468d", [], {{0x86dd, @ipv6={0x0, 0x6, "a228af", 0x14, 0x6, 0x0, @remote={0xfe, 0x80, [0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0], 0x0, 0xbb}, @local={0xfe, 0x80, [0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0], 0x0, 0xaa}, {[], @tcp={{0x0, 0x1, 0x42424242, 0x42424242, 0x0, 0x0, 0x5, 0x2, 0x0, 0x0, 0x0, {[]}}, {""}}}}}}})
syz_extract_tcp_res(&(0x7f0000002000)={<r1=>0x42424242, <r2=>0x42424242}, 0x1, 0x0)
syz_emit_ethernet(0x4a, &(0x7f0000003000)={@local={[0xaa, 0xaa, 0xaa, 0xaa, 0xaa], 0x0}, @random="de895db1468d", [], {{0x86dd, @ipv6={0x0, 0x6, "a228af", 0x14, 0x6, 0x0, @remote={0xfe, 0x80, [0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0], 0x0, 0xbb}, @local={0xfe, 0x80, [0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0], 0x0, 0xaa}, {[], @tcp={{0x0, 0x1, r2, r1, 0x0, 0x0, 0x5, 0x10, 0x0, 0x0, 0x0, {[]}}, {""}}}}}}})
r3 = accept$inet6(r0, &(0x7f0000004000)={0x0, 0x0, 0x0, @empty={[0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0]}, 0x0}, &(0x7f0000005000)=0x1c)

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp6       0      0 :::20001                :::*                    LISTEN      5527/a.out
tcp6       0      0 fe80::aa:20001          fe80::bb:20000          ESTABLISHED 5527/a.out
2017-05-26 14:28:09 +02:00
baishuai
6e254bdd13 manager: save and reuse allSymbols, vmOffset 2017-05-25 17:30:27 +08:00
Dmitry Vyukov
38b947b94f syz-symbolize: symbolize all console output
Currently syz-symbolize uses report.Parse function
that extracts crash messages from console output.
Symbolize all console output instead.
E.g. there can be something on the console that is not crash.
2017-05-23 14:44:07 +02:00