783 Commits

Author SHA1 Message Date
Dmitry Vyukov
2b4970017d pkg/report: fix Start/EndPos calculation for fuchsia
We computed Start/EndPos after trimming line prefix,
this resulted in offsetted values which are not correct.
Fix that. Add more tests and checks for Start/EndPos.
2018-12-20 13:22:46 +01:00
Dmitry Vyukov
dbe573e3b8 pkg/osutil: always remove old file in Rename 2018-12-20 13:22:46 +01:00
Greg Steuck
a8b6d1af46 pkg/build/openbsd: always make clean to manage config in tree
clean build barely takes 3 minutes end-to-end on our CI machine.

Undo debug options which caused prevented kernels from booting on GCE.
2018-12-19 17:15:31 +01:00
Dmitry Vyukov
4edaba9329 pkg/vcs: disable sandboxing in tests
syz-ci disabling does not work across syz-ci updates.
2018-12-18 12:48:59 +01:00
Dmitry Vyukov
a9cc88c19f pkg/osutil: provide better Rename
os.Rename fails for cross-device renaming (e.g. to/from tmpfs).
This is quite unpleasant. Provide own version that falls back to copying.
2018-12-17 19:09:10 +01:00
Dmitry Vyukov
b38da77e00 pkg/bisect: check existence of input files
It's better to fail early then to wait for kernel build.

Update #501
2018-12-17 19:09:03 +01:00
Dmitry Vyukov
61adbb167e pkg/vcs: don't use --no-tags when fetching git
pkg/bisect uses Poll and then relies on tags for compiler detection.
Don't use --no-tags when fetching.

Update #501
2018-12-17 19:08:52 +01:00
Dmitry Vyukov
1749e412ca pkg/instance: make kernel reboot detection portable
The current code is linux specific, generalize it to other OSes.
2018-12-16 16:48:05 +01:00
Dmitry Vyukov
ce6744512e pkg/vcs: fix fetching of commits on non master branch
Fixes #728
2018-12-16 16:18:06 +01:00
Dmitry Vyukov
def91db3fe prog, pkg/csource: more readable serialization for strings
Always serialize strings in readable format (non-hex).
Serialize binary data in readable format in more cases.

Fixes #792
2018-12-15 15:17:13 +01:00
Dmitry Vyukov
7a944a0a66 pkg/report: fix RIP line symbolization
Don't subtract 1 for RIP lines.

Fixes #361
2018-12-15 13:36:01 +01:00
Dmitry Vyukov
dd2fb82fa0 pkg/report: another gvisor OOM suppression 2018-12-13 16:45:56 +01:00
Dmitry Vyukov
61b76f0097 pkg/report: another gvisor OOM suppression 2018-12-13 16:40:18 +01:00
Dmitry Vyukov
e7cae6a7f2 pkg/report: another gvisor OOM suppression 2018-12-13 16:36:15 +01:00
Dmitry Vyukov
ecf8ceeef2 pkg/report: pkg/report: relaxed gvisor OOM suppressions more
There are more variations of this panic format.
2018-12-13 12:33:17 +01:00
Dmitry Vyukov
a553162c82 pkg/report: replace more moving parts in gvisor crash titles
"container" seems to have been renamed to "sandbox".
Also exact pid numbers are harmful.
2018-12-13 12:29:52 +01:00
Dmitry Vyukov
3f9ec66365 pkg/report: relaxed gvisor OOM suppressions
Existing ones don't match actual gvisor output after address mangling.
Not matching exact context in parens should be good enough re false positives.
2018-12-13 12:22:49 +01:00
Greg Steuck
f3d9d5948c pkg/csource: support tun and setuid repros on {free,open}bsd
* expose procid on BSD for tun, always declare loop()
* deal with terrible bsd includes
* replicate loop() declaration
2018-12-13 09:44:24 +01:00
Michael Pratt
ec0147d47f Merge pull request #874 from prattmic/bazel_version
pkg/build: fix bazel version parsing
2018-12-13 09:38:59 +01:00
Michael Pratt
1789e8d66c pkg/report: update gvisor ptrace regs suppressions
99d5958693
changed the format of these to include the registers.
2018-12-13 09:38:16 +01:00
Dmitry Vyukov
0f9424b449 pkg/report: skip strncpy in linux reports 2018-12-12 13:05:51 +01:00
Greg Steuck
d90f8af59c executor: reapply setuid sandbox for bsd
* Revert "Revert "executor: add setuid sandbox for openbsd""

The problem is the low file descriptor limit.

This reverts commit 4093e33b1338f274ae0062f555de9d6af8640d61.

* executor/executor make sure the file descriptor limit is sufficient
2018-12-11 07:39:44 +01:00
Greg Steuck
4093e33b13 Revert "executor: add setuid sandbox for openbsd"
This reverts commit 6565f24da9f4eb36702339ba290213995fcc902f.
2018-12-10 20:09:24 +01:00
Dmitry Vyukov
c7ba317e9b tools/syz-runtest: test program parsing before booting VMs
It sucks to wait for VMs to boot just to discover that programs don't parse.
2018-12-10 16:37:02 +01:00
Dmitry Vyukov
593b260b02 pkg/ipc: move sandbox helpers from ipcconfig
Currently syz-runtest fails to start because -debug flag is defined
both in syz-runtest and ipcconfig.
But moving sandbox functions we prevent ipcconfig from being imported into syz-runtest.
2018-12-10 16:37:02 +01:00
Dmitry Vyukov
95fe19c19e prog: introduce strict parsing mode
Over time we relaxed parsing to handle all kinds of invalid programs
(excessive/missing args, wrong types, etc).
This is useful when reading old programs from corpus.
But this is harmful for e.g. reading test inputs as they can become arbitrary outdated.
For runtests which creates additional problem of executing not
what is actually written in the test (or at least what author meant).
Add strict parsing mode that does not tolerate any errors.
For now it just checks excessive syscall arguments.
2018-12-10 16:37:01 +01:00
Dmitry Vyukov
5dd495422e syz-manager: move coverage report code to pkg/cover
This will allow better testing and make it possible to reuse this code.
2018-12-10 16:37:01 +01:00
Greg Steuck
6565f24da9 executor: add setuid sandbox for openbsd
* executor/common_bsd: add setuid sandbox

Fixes #833

cc @mptre

* Reduced duplications, resolved TODO.
2018-12-10 11:20:16 +01:00
Dmitry Vyukov
c791837863 executor: fix handling of big-endian bitfields
Currently we apply big-endian-ness and bitfield-ness in the wrong order in copyin.
This leads to totally bogus result. Fix this.
2018-12-08 19:08:08 +01:00
Anton Lindqvist
3350826625 pkg/report: add initial symbolize suppport to OpenBSD 2018-12-08 18:59:10 +01:00
Anton Lindqvist
4e1aade2fe pkg/report: pass the target to each OS report constructor
In order to use the already defined kernel name in sys/targets to reduce
duplications.
2018-12-08 18:59:10 +01:00
Greg Steuck
610352d5d7 pkg/build/openbsd: require kernel config files as configuration data
Previously the config was generated directly, but testing multiple
configurations makes this cumbersome going forward. This makes
kernel_config a mandatory parameter.
2018-12-07 09:50:12 +01:00
Dmitry Vyukov
c1641491e4 pkg/db: provide helper function for database creation
This is needed for both tools/syz-db and tools/syz-trace2syz.
Also, remove code to resolve SHA1 collisions.
Also, don't set db version as we actually want to minimize
and smash these programs like anything else
(not minimizing nor smashing them is only useful during tool testing).
2018-12-06 16:49:37 +01:00
Dmitry Vyukov
764b42c46b pkg/csource: exclude linux/arm64 tests
I think I misinterpreted the error that episodically happens on ci:

collect2: error: ld terminated with signal 11 [Segmentation fault], core dumped
compiler invocation: aarch64-linux-gnu-gcc [-Wall -Werror -O1 -g -o /tmp/syz-executor570589071 -pthread -DGOOS_linux=1 -DGOARCH_arm64=1 -x c - -static]

as OOM, but they all involve aarch64-linux-gnu-gcc:

https://travis-ci.org/google/syzkaller/jobs/461827347
https://travis-ci.org/google/syzkaller/jobs/460226110
https://travis-ci.org/google/syzkaller/jobs/463564291

So I guess the problem can be with the arm64 toolchain that just crashes randomly.
2018-12-05 13:59:01 +01:00
Dmitry Vyukov
2192790481 pkg/csource: reduce short tests
pkg/csource test gets OOM-killed on travis:
https://travis-ci.org/google/syzkaller/jobs/461827347
https://travis-ci.org/google/syzkaller/jobs/460226110

Add several measures:
 - set GOMAXPROCS=1 to restrict parallel processes
 - remove -g from compiler invocation
 - reduce set of tests run in short mode to compensate for GOMAXPROCS=1
 - also reduce set of tests in full mode as they timeout now
2018-12-03 13:29:57 +01:00
Greg Steuck
88746fdf89 pkg/csource: use defines from sys/syscall.h on *bsd
Fixes #841
2018-12-02 13:18:07 +00:00
Michael Tüxen
5a58167323 executor: add support for tap interfaces on FreeBSD 2018-12-01 10:22:39 +00:00
Greg Steuck
d8988561c0 openbsd: repair pkg/csource_test 2018-11-30 19:55:51 +00:00
Dmitry Vyukov
f0d4c650a3 pkg/report: suppress another gvisor OOM 2018-11-30 19:17:05 +00:00
Michael Tuexen
8b13c56200 executor: Use correct macros.
Suggested by Dmitry Vyukov.
2018-11-30 13:48:23 +00:00
Michael Tuexen
9656dc4c83 executor: Fix compilation on FreeBSD.
This is an autogenerated file, but it seem that changes to it
resulting from changes in executor/common_bsd.h must also be
committed.
e
2018-11-30 13:48:23 +00:00
Greg Steuck
6419afbb77 openbsd: run on gce
* build/openbsd: minor cleanup (use tuples instead of maps)

* Grammar nits in comments.

* Simplify openbsd.Create, will defer when there's more than one error exit.

* pkg/build: Support copying kernel into GCE image

* Simple test for openbsd image copy build.

* Cleanup in case something failed before.

* Support multi-processor VMs on GCE.

* More debug

* Reformat

* OpenBSD gce image needs to be raw.

* GC

* Force format to GNU directly on Go 1.10 or newer.

* Use vmType passed as a parameter inside openbsd.go

* gofmt

* more fmt

* Can't use GENERIC.mp just yet.

* capitalize

* Copyright
2018-11-27 13:14:06 +01:00
Dmitry Vyukov
646afdf841 dashboard/app: enlarge stats value column for manager
When numbers become large, they may not fit into 100px.
2018-11-23 14:44:58 +01:00
msvoelker
0ab8db6467 executor: FreeBSD: fixed endian.h path
The include statement for endian.h failed on FreeBSD. Changed include statement to syz/endian.h for FreeBSD.
2018-11-23 11:50:41 +01:00
Dmitry Vyukov
13ab4beeef syz-manager: modernize web UI
1. Use dashboard style.
2. Allow sorting of tables.
3. Show old crashes in grey.
4. Use tables instead of text output for more pages.
5. Show corpus inputs on a separate page to allow copy-pasting.
6. Use standard JS sorting instead of custom bubble sort (much faster).
7. Fix off-by one in table sorting.

Fixes #694
2018-11-22 14:36:32 +01:00
Andrey Konovalov
582e1f0d1d ipc: add ProgInfo struct
This patch add a new struct ProgInfo that for now holds info about each
call in a program []CallInfo, but in the future will be expanded with remote
coverage info. Update all the callers to use the new interface as well.
2018-11-22 13:49:50 +01:00
Dmitry Vyukov
2ee778023a pkg/report: refine rcu stall detection
Add another anchor frame.
2018-11-22 08:22:30 +01:00
Dmitry Vyukov
ccd67a7611 pkg/report: detect Go service panics for fuchsia 2018-11-22 08:21:45 +01:00
Dmitry Vyukov
6839de7050 pkg/report: fix corrupted stack trace checking
We started detecting all kernel reboots as corrupted,
because we considered that after any "Allocated" line
a stack trace should follow.
Kernel boot output now contains:
ima: Allocated hash algorithm: sha256
and there is no stack trace after that.

1. Refine stack trace regexps (we actually want to look for
"Allocated by task PID:" lines).
2. Don't check stacks if report format says that it
does not contain stacks.
2018-11-22 07:40:50 +01:00
Dmitry Vyukov
76b59936da pkg/report: add "PANIC: double fault" report format
Amusing that's another kernel failure mode that we are discovering after 3 years.
One can't even reliably understand when kernel has crashed.
I wonder if syzkaller never hit these, or just never recognized and reported them. We will see.
Don't even want to think about arm kernel output parsing.
2018-11-22 07:10:16 +01:00