Commit Graph

2995 Commits

Author SHA1 Message Date
Dmitry Vyukov
23454ce6df dashboard/app: make build upload more robust
Currently it triggers datastore timeouts episodically.
Don't fail build upload if bug commit update fails.
Split the large query into two smaller queries.
2018-10-10 14:03:14 +02:00
Dmitry Vyukov
7eaec1b83e dashboard/app: fetch bugs in batches
We are seeing lots of datastore timeout errors during bug fetch.
It's a bad idea to fetch thousands of bugs in one query,
so fetch them in batches of 1000 bugs.
Hopefully will reduce rate of errors.
2018-10-10 13:40:22 +02:00
Julia Hansbrough
5b11ac2c91 Update fidlgen to keep current with upstream.
The Syzkaller IR upstream has been updated, so we can properly update
consts/fidl descriptions now.
2018-10-10 12:00:14 +02:00
Zach Riggle
f37861df63 targets: Use OS=linux when GOOS=android
This avoids the issue of "android" not having any registered configurations
or syscalls / ioctls / etc, when built with GOOS=android.

This occurs when building in Google3, since --config=android_arm64 selects
the Android toolchain.
2018-10-10 11:54:53 +02:00
Kees Cook
8b311eafa7 docs: Update package names for modern Ubuntu
The package names for perf and qemu are out of date. This updates them.
2018-10-03 06:05:00 +02:00
Julia Hansbrough
0f3e026191 Include generate_fidl as part of the extract step.
Anytime we're running `make extract/generate` with Fuchsia as a target, we
probably want to update FIDL descriptions too, so we don't fall behind
what upstream is up to.

This includes generate_fidl as part of the `make extract` workflow.

Also, corrects fidlgenPath to properly find the path for ARM targets (we
only have host_x64 right now).
2018-10-02 20:47:12 +02:00
Alexander Potapenko
a316a2afee kmsan: add kmsan_from_kasan_config.sh to generate KMSAN config 2018-10-02 15:41:28 +02:00
Paul Chaignon
e06f77131b docs: fix typo in reproduction documentation 2018-10-01 16:13:01 +02:00
Dmitry Vyukov
c90dde97ba
Update syzbot.md 2018-10-01 12:07:00 +02:00
Anton Lindqvist
530c6bc32b docs/openbsd: improve setup instructions
The majority of the changes are prodded by Klemens Nanni, another fellow OpenBSD
hacker.
2018-10-01 11:37:27 +02:00
Anton Lindqvist
bb3e4ee68c docs: mention OpenBSD 2018-10-01 11:37:27 +02:00
Greg Steuck
48a50c6b36 doc: Not -> Note 2018-10-01 09:58:57 +02:00
Julia Hansbrough
d7101e2c60 Fix executor build for Fuchsia on ARM, too.
Whups, pissed this in pull request #746.

As before, location of sysroot includes changed, so Fuchsia
cross-compile flags need an update.
2018-10-01 09:57:57 +02:00
Dmitry Vyukov
41e4b32952 pkg/report: ignore printk as guilty file 2018-09-28 16:46:41 +02:00
Dmitry Vyukov
137d7c662e syz-manager: don't send memory leaks without repros to dashboard
KMEMLEAK has lots of false positives and bugs without repros
may be unactionable. It's not completely clear how to handle
such cases in automatic systematic testing.
But let's try this and see how it works.
2018-09-28 15:18:02 +02:00
Dmitry Vyukov
6939672737 dashboard/config: add KMEMLEAK config
Based on KASAN config, but with KASAN and most other debug
configs disabled and KMEMLEAK enabled.
2018-09-28 14:57:20 +02:00
Dmitry Vyukov
7296c0747f pkg/host: improve KMEMLEAK support
Rewind kmemleak fd before reading it second time,
otherwise we will read truncated reports.

Auto-learn what leak reports we've already seen
and ignore them in future. This is required because
there are some false positives and some fire too frequently.
So now we will hit each leak only once per manager run,
but we still will try to reproduce them.
2018-09-28 14:57:20 +02:00
Dmitry Vyukov
a6143bc982 pkg/report: improve KMEMLEAK report parsing
Skip few more common allocation functions.
2018-09-28 14:57:20 +02:00
Dmitry Vyukov
746df66f2c dashboard/config: disable CONFIG_DYNAMIC_DEBUG and CONFIG_BPFILTER
syzkaller manages to turn on dynamic debug messages somehow,
which then produces tons of output on console. Disable dynamic
debug on kasan.
Disable bpfilter on kmsan as we already did on kasan.
2018-09-28 14:57:20 +02:00
Julia Hansbrough
8bb1f0fce1 Fix executor build for Fuchsia.
Location of some sysroot includes changed; ergo, Fuchsia cross-compile
flags need an update.
2018-09-28 11:15:49 +02:00
Greg Steuck
0b624c8069 tools/create-openbsd-*: fully functional images from scratch
* tools/create-openbsd-gce-ci: use config from /syzkaller

* Use syzkaller copy of src instead of a separate one.

* Using /dev/null disk in vm.conf

* Use KVM, enable doas, no longer symlink.

* Use a dummmy.img

* Revert "Use a dummmy.img"

This reverts commit 656b24d5e4573dde5e95c6158852001c7241e65a.
2018-09-28 11:13:51 +02:00
Dmitry Vyukov
0c2fa87bc6 executor: fix build on newer linux distros
This sucks a lot, but ebtables.h is now broken too on Debian 4.17:

ebtables.h: In function ‘ebt_entry_target* ebt_get_target(ebt_entry*)’:
ebtables.h:197:19: error: invalid conversion from ‘void*’ to ‘ebt_entry_target*’
2018-09-27 15:34:52 +02:00
Dmitry Vyukov
98b28ead6c pkg/report: fix few fuzz bugs
Improve go-fuzz fuzzer function and fix few new bugs it finds:
1. Panic in linux parser (bad).
2. Akaros can report empty rep.Report.
3. Fuchsia can return empty rep.Report.
2018-09-27 12:50:25 +02:00
Dmitry Vyukov
58fca249ef sys/linux: add encrypted key instantiation payloads 2018-09-26 15:37:33 +02:00
Dmitry Vyukov
8899d58437 tools/create-gce-image.sh: mount securityfs and configfs 2018-09-26 14:06:11 +02:00
Dmitry Vyukov
881bc4f956 sys/linux: add IMA mount options 2018-09-26 13:50:35 +02:00
Dmitry Vyukov
db716d6653 pkg/report: fix guilty file extraction
Account for the case that some file names can appear _before_ crash report starts.
Start extracting guilty file starting from StartPos.
2018-09-26 12:02:43 +02:00
Greg Steuck
455b6354e8 tools/create-openbsd-gce-ci auto-start syz-ci, redirect 8080 to 80 2018-09-26 09:41:21 +02:00
Greg Steuck
b7e1128951 docs/syz-ci fixed link 2018-09-25 20:02:06 +02:00
Dmitry Vyukov
134f5f14f4 sys/linux: add ima/evm/capability xattrs 2018-09-25 19:56:45 +02:00
Dmitry Vyukov
b1b7b3a864 dashboard/config: enable ima on command line
Does it actually enable ima?
2018-09-25 19:56:04 +02:00
Dmitry Vyukov
30f618d19d sys/linux: add security mount options 2018-09-25 18:47:08 +02:00
Dmitry Vyukov
90fc4b7a89 sys/linux: add erofs descriptions 2018-09-25 17:20:32 +02:00
Dmitry Vyukov
19a403430d dashboard/config: disable BPFILTER
BPFILTER leads to frequent failures:
https://groups.google.com/d/msg/syzkaller/SxiyMLyIAVA/JU0tLqqkCQAJ
2018-09-25 17:04:08 +02:00
Alexander Popov
0e7547d77c executor: make the debug output useful for race condition debugging
Sometimes race conditions are reproduced by syz-execprog and are not
reproduced by the programs generated with syz-prog2c. In such cases
it's very helpful to know when exactly the fuzzing syscalls are executed.

Unfortunately, adding timestamps to the output of the original 'debug'
mode doesn't work. This mode provides very verbose output, which slows
down executor and breaks the repro.

So let's make the executor debug output less verbose and add
the timestamps.

Signed-off-by: Alexander Popov <alex.popov@linux.com>
2018-09-24 14:55:35 +02:00
Dmitry Vyukov
2f485cdfa9 docs: add research work based on syzkaller 2018-09-24 12:11:57 +02:00
Dmitry Vyukov
7ffd14ab85 dashboard/app: add similarity domains for namespaces
It does not make sense to show similar bugs across linux/openbsd/akaros.
Allow restricting similarity domains for namespaces.

Fixes #742
2018-09-24 11:12:44 +02:00
Dmitry Vyukov
684e7e3250 pkg/report: ignore more str* functions 2018-09-24 10:59:44 +02:00
Anton Lindqvist
e029c3e058 pkg/symbolizer: favor portable nm(1) format
On OpenBSD, nm(1) does not support the `-S' option.

The order of symbols from nm(1) varies between platforms, therefore rework the
test case to cope with that.

Fixes #723
2018-09-24 10:12:38 +02:00
Dmitry Vyukov
28d9ac763d sys/linux: add lsm attrs and xattrs
Add descriptions for /proc/self/attr/* and known lsm file xattrs.
2018-09-23 20:30:20 +02:00
Dmitry Vyukov
07a35107a4 syz-ci: split Manager.loop function
gometalinter says the function is too complex:

syz-ci/manager.go:155:⚠️ cyclomatic complexity 30 of function (*Manager).loop() is high (> 24) (gocyclo)

Split into 2 functions.
2018-09-21 10:44:49 +02:00
Greg Steuck
370797126e tools/create-openbsd-gce-ci.sh mount 10G ramdisk for worker images 2018-09-20 20:46:45 +02:00
Dmitry Vyukov
6cee973cb5 syz-ci: fix manager http addr
Manager http addr is usually just ":port".
Prepend "127.0.0.1" in such case.
2018-09-20 19:58:12 +02:00
Dmitry Vyukov
2d41b5d964 vm/vmm: recover from vmctl errors faster
vmctl start periodically fails with:
	vmctl: start vm command failed: Operation already in progress

So try to sleep for a bit after vmctl stop.
And detect when vmctl start terminates prematurely
to avoid 10 minute timeout for ip extraction.
2018-09-20 19:44:50 +02:00
Dmitry Vyukov
3c8de4d941 dashboard/app: show coverage report per-manager
Now that syz-ci can upload coverage reports to GCS,
we can show up-to-date coverage for each manager
rather than a single static outdated file per namespace.
2018-09-20 19:32:54 +02:00
Dmitry Vyukov
4fb3cca18f dashboard/app: tidy up UI
Don't show maintainers for crashes if we don't have them (only linux has).
Show short git hashes for kernel/syzkaller, 40 chars looks excessive (full hash is in hint).
Show current build kernel/syzkaller git hash in managers table.
Remove fuzzing time from managers table (does not seem too useful).
2018-09-20 18:29:38 +02:00
Dmitry Vyukov
9a6f6af645 syz-ci: upload coverage reports to GCS
Upload coverage reports from all managers to GCS
after 6 hours of runtime (to make it more apples-to-apples).
2018-09-20 18:29:38 +02:00
Dmitry Vyukov
0e88373b1c dashboard/app: change backup bucket
Backup datastore into a separate bucket as it creates lots of garbage.
2018-09-20 18:29:38 +02:00
Tobias Klauser
9ba5d84192 sys/linux: fix RTA_EXPIRES in rtm_ipv6_policy 2018-09-20 16:07:15 +02:00
Dmitry Vyukov
565a5452a1 vm/vmm: don't start separate process for console
vmctl console fails from time to time with:

vmctl: console not found

Probably there is some race (most of these things assume
that there is a human typing commands with delays).

Also, vmctl start can connect to console itself with -c flag.
So use that because it both solves the console race and
also makes code much more similar to other VM implementations (qemu, gvisor).
This also eliminates 3 additional goroutines per VM.
2018-09-20 14:54:07 +02:00