1290 Commits

Author SHA1 Message Date
Andrey Konovalov
16668351f6 pkg/report: improve task hung report headers 2017-11-30 22:35:35 +01:00
Dmitry Vyukov
5153aeaffd syz-ci: test images before using them
Boot and minimally test images before declaring them as good
and switching to using them.

If image build/boot/test fails, upload report about this to dashboard.
2017-11-30 14:50:50 +01:00
Andrey Konovalov
d5bd1f79fd pkg/report: improve trying to register non-static key header 2017-11-30 11:45:21 +01:00
Dmitry Vyukov
29b0fd90e6 pkg/report: include Maintainers into report
Currently getting a complete report requires a complex,
multi-step dance (including getting information that
external users are not interested in -- guilty file).

Simplify interface down to 2 functions: Parse and Symbolize.
Parse does what it did before, Symbolize symbolizes report
and fills in maintainers. This simplifies both implementations
of Reporter interface and all users of the interface.

Potentially we could get this down to 1 function Parse
that does everything. However, (1) Symbolize can fail,
while Parse cannot, (2) usually we want to ignore (log)
Symbolize errors, but otherwise proceed with the report,
(3) repro does not need symbolization for all but the
last report.
2017-11-29 18:24:30 +01:00
Andrey Konovalov
46c864566a pkg/report: improve bad unlock balance corruption detection 2017-11-29 15:18:44 +01:00
Dmitry Vyukov
34f2c2332b pkg/report: add Output to Report
Whole raw output is indivisble part of Report,
currently we always pass Output separately along with Report.
Make Output a Report field.

Then, put whole Report into manager Crash and repro context and Result.
There is little point in passing Report as aa bunch of separate fields.
2017-11-29 14:36:51 +01:00
Andrey Konovalov
19d272a98f pkg/report: detect suspicious RCU usage report corruption 2017-11-29 14:26:38 +01:00
Andrey Konovalov
4263b2afbe pkg/report: add sl*b.c to guilty blacklist 2017-11-29 13:45:46 +01:00
Andrey Konovalov
c2a67000b6 pkg/repro: rerun repro when report is corrupted 2017-11-29 13:23:31 +01:00
Andrey Konovalov
7bd85f2aea pkg/report: fix corrupted stack trace detection
Look for stack frames in the next 10 lines after 'Call Trace' instead of 3.
2017-11-28 15:54:13 +01:00
Dmitry Vyukov
ac93d7e1d8 pkg/compiler: add check that len does not refer to array with varlen elements
This [almost] always means a bug in descriptions.
Fix all bugs identified by the check.
2017-11-28 14:55:21 +01:00
Dmitry Vyukov
afba0b55e6 sys/linux: add binder descriptions 2017-11-27 15:09:30 +01:00
Dmitry Vyukov
8257de047b sys/linux: open files from /proc 2017-11-27 09:08:59 +01:00
Andrey Konovalov
6834199b8c pkg/report: various corrupted report detection improvements 2017-11-23 16:17:40 +01:00
Dmitry Vyukov
7061d1973b sys/syz-extract: fix mmap on arm
__NR_mmap is missing on arm entirely,
so we disable mmap during generate.
Patch mmap to mmap2 right in syz-extract,
so that mmap is never missing.
2017-11-23 08:51:04 +01:00
Dmitry Vyukov
31af2ce022 vm/gce: fix boot output capture
Turns out GetSerialPortOutput API does not work if instance has
serial port connections enabled (which we always have).
Get output from serial port relay service instead.
2017-11-22 17:56:48 +01:00
Andrey Konovalov
7bd6e42d35 pkg/report: fix corrupted reports detection
Add another regexp to oopses that should match the whole report.
Report is considered corrupted when it doesn't.
2017-11-22 16:12:55 +01:00
Andrey Konovalov
c8b87c9cf7 pkg/report: fix corrupted KASAN reports detection
KASAN report might not have Allocated or Freed stack traces at all.
2017-11-22 13:41:47 +01:00
Andrey Konovalov
53a23f2a37 pkg/report: corrupted report detection fixes 2017-11-22 13:17:21 +01:00
Dmitry Vyukov
657a01b951 pkg/email: ignore missing To: header
We've got such email.
2017-11-22 12:03:31 +01:00
Andrey Konovalov
e06afd3259 pkg/repro: don't bisect single entry 2017-11-22 11:45:38 +01:00
Dmitry Vyukov
ad0af9fff5 vm: return Report from MonitorExecution
This allows callers to get access to Report.Corrupted.
Better than adding 6-th return value and will allow
to pipe other report properties if necessary.
2017-11-21 19:02:35 +01:00
Dmitry Vyukov
f07ea3fc22 sys/linux: remove get_kernel_syms, add quotactl syscall
get_kernel_syms does not seem to be present upstream.
Describe an old quotactl syscall.
Also fix umount/umount2 names in kallsyms.
2017-11-20 17:23:24 +01:00
Dmitry Vyukov
9badd05327 vm/gce: provide VM console output on boot failures
"can't ssh into the instance" is not a very useful error.
2017-11-19 12:29:00 +01:00
Dmitry Vyukov
a1469efbdd pkg/email: unsplit arguments for test command 2017-11-17 20:43:47 +01:00
Dmitry Vyukov
00f6ff581c pkg/email: truncate garbage after patches 2017-11-17 18:54:19 +01:00
Dmitry Vyukov
fba338cd51 pkg/csource: add function to parse serialized options
Also move options and options tests into a separate file,
add serialization function.
2017-11-17 17:57:51 +01:00
Dmitry Vyukov
3fb087023a pkg/osutil: properly set gid for sandboxing 2017-11-17 14:28:45 +01:00
Dmitry Vyukov
e3d7179396 pkg/git: fix branch during fetch 2017-11-17 14:17:47 +01:00
Dmitry Vyukov
2f7fc0ff65 pkg/kernel: sandbox make invocation 2017-11-17 14:56:34 +03:00
Dmitry Vyukov
c4d43f4773 pkg/osutil: don't leace runaway processes
When manager is stopped there are sometimes runaway qemu
processes still running. Set PDEATHSIG for all subprocesses.
We never need child processes outliving parents.
2017-11-16 12:48:02 +01:00
Dmitry Vyukov
9a98ae3fb6 pkg/git: provide more helper functions
Add Patch, Checkout, CheckRepoAddress and CheckBranch.
Will be needed for patch testing.
2017-11-16 10:12:17 +01:00
Dmitry Vyukov
95cf3e7247 pkg/email: fix base64-encoded body parsing
We currently handle base64 only for attachments,
but text/plain body can also be base64-encoded.
2017-11-16 10:10:12 +01:00
Dmitry Vyukov
447a290a8c pkg/config: provide SaveData function
Parallel to LoadFile/LoadData.
2017-11-16 10:10:12 +01:00
Dmitry Vyukov
cf38de0018 pkg/report: avoid compiling a bunch of regexps on every crash 2017-11-14 10:16:27 +01:00
Dmitry Vyukov
bbbea5a373 pkg/report: pass report as []byte to isCorrupted
We always pass report/log as []byte.
Pass it here the same way for consistency and to avoid
unnecessary large memory allocation/copy.
2017-11-14 10:13:38 +01:00
Dmitry Vyukov
4bd78cef05 pkg/report, pkg/repro, syz-manager: name crash attributes consistently
We currently have several names for crash attributes, which is disturbing.
E.g. crash title is called "Title" or "Desc". Name them consistently.

Title - single line bug identity.
Report - whole crash text.
Log - whole fuzzer/kernel output.
2017-11-14 10:04:22 +01:00
Dmitry Vyukov
82b3b903a0 pkg/report: make isCorrupted linux-specific
isCorrupted is linux-specific, but is a global function.
Name can collide with other OSes. Make is linux method.
2017-11-14 09:47:44 +01:00
Dmitry Vyukov
10112655d7 vm: remove needOutput arg for MonitorExecution
Always wait 10 secs for output.
If anything this can only lead to missed crashes during repro.
Let's unify manager and repro behavior.
2017-11-14 09:45:34 +01:00
Dmitry Vyukov
7a53e7e35d pkg/report: combine report data into a struct
Parse returns 5 variables now. Later we may want to add crash "priority".
Introduce Report struct that holds all report data.
2017-11-14 09:41:55 +01:00
Andrey Konovalov
f9a8d567eb pkg/report: add corrupted report detection
This change makes pkg/report try to detect corrupted reports by
using some heuristics.
2017-11-13 17:18:16 +03:00
Andrey Konovalov
dd1609f876 executor: proceed even if /dev/net/tun is not available
For some racy bugs syzkaller can generate a C reproducer with tun
enabled, when it's not actuallly required to trigger the bug.
Some kernel developers (that don't have CONFIG_TUN=y on their setups)
complain about such C repros.
When tun is not available, instead of exiting, print a message that tun
initialization failed and proceed.
2017-11-08 17:43:40 +01:00
Dmitry Vyukov
6c412fa20c pkg/git: add another commit prefix 2017-11-08 14:16:36 +01:00
Andrey Konovalov
a2c64463a2 execprog: correctly handle fault injections
syz-execprog doesn't utilize info about fault injections from a prog log.
Since syz-execprog is used by the repro package to reproduce crashes,
crashes caused by fault injections might not reproduce.
2017-11-07 16:17:18 +01:00
Dmitry Vyukov
8f3e774b76 syz-ci: improve commit matching
1. Fetch last 200K commits instead of commits for past year.
For merged commits both author date and commit date can be
arbitrary long in past (e.g. we got a commit dated by 2014).

2. Strip some commit prefixes from commits.
We have some trees where backports are prefixed with "BACKPORT:".
Previously we could no match such commits.
2017-11-07 13:31:00 +01:00
zoulasc
0460a8dda0 csource: Fix sed(1) invocation
There is no need to specify '-' as the filename for sed(1):
- The default behavior is to read stdin
- It was not done in all places
- It breaks on NetBSD sed(1) (although I am tempted to fix it now :-)
  and it does not work
2017-11-06 11:27:45 +03:00
Dmitry Vyukov
6ddaf205ad dashboard/app: email fixes
1. Allows sending emails upstream.
2. Filter out duplicate emails coming from our mailing lists.
3. Increase retry attempts for email commands
   (don't want them to fail due to concurrent crash reports from managers).
2017-10-31 10:06:02 +01:00
Dmitry Vyukov
636a4dbf38 pkg/csource: add freebsd/netbsd support 2017-10-26 15:31:23 +02:00
Dmitry Vyukov
0038451914 pkg/report: add netbsd stub 2017-10-25 18:36:49 +02:00
Dmitry Vyukov
e729550825 all: basic building on netbsd
This just makes make TARGETOS=netbsd succeed.
We don't yet have prog target for netbsd.
2017-10-23 10:10:05 +02:00