Commit Graph

5523 Commits

Author SHA1 Message Date
Dmitry Vyukov
a2267789b8 .github: reorder CODEOWNERS owner entries
I mis-interpreted the order of entries:
"Order is important; the last matching pattern takes the most precedence."
https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners#codeowners-syntax

The idea is to not pull in all OS maintainers for vendor/ updates.
2020-09-15 19:34:30 +02:00
Dmitry Vyukov
c478111a98 pkg/build, pkg/html: move generation logic into separate scripts
Mentions of "// Code generated ... DO NOT EDIT." in these Go files
somehow prevents golangci-lint from checking them.
And github treats them as auto-generated as well.
I guess some code somewhere has some imprecise heuristics.
Move generation into separate shell scripts. Also cleaner.
2020-09-15 19:34:30 +02:00
Dmitry Vyukov
2d37b9667e sys/openbsd: fix linter warnings
sys/openbsd/init.go:58:36: Top level comment should end in a period
	// kCoverFd in executor/executor.cc
	                                  ^
sys/openbsd/init.go:60:38: Top level comment should end in a period
	// kOutPipeFd in executor/executor.cc
	                                    ^
2020-09-15 19:34:30 +02:00
Dmitry Vyukov
712de1c63d vendor/github.com/golangci/golangci-lint: update to v1.31 2020-09-15 19:34:30 +02:00
Dmitry Vyukov
298a69c38d docs/linux: fix whitespaces in coverage doc 2020-09-15 15:50:24 +02:00
m5imunovic
0f06037d76 docs/linux/coverage.md: expand readelf section and reformat 2020-09-15 15:50:24 +02:00
m5imunovic
ffd13eb166 docs/linux/coverage.md: fix doc format 2020-09-15 15:50:24 +02:00
m5imunovic
a7abe2602c CONTRIBUTORS: add Marijo Simunovic 2020-09-15 15:50:24 +02:00
m5imunovic
ca3158ee9a docs/linux/coverage.md: expand coverage reporting doc 2020-09-15 15:50:24 +02:00
Alexander Egorenkov
3b5eb29afb pkg/host: use 'machine' field as cpu model on IBM/Z
Use the field 'machine' in /proc/cpuinfo on IBM/Z
to find out the "CPU model".

Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-09-15 15:06:24 +02:00
Heyuan Shi
6989d6f61d docs: clarify coverage point in different compliers 2020-09-15 09:39:09 +02:00
Dmitry Vyukov
9e681632f5 pkg/osutil: fix LinkFiles/FilesExist for the new pattern format 2020-09-15 09:37:22 +02:00
Dmitry Vyukov
a2360d0742 executor: fix definition of __NR_io_uring_setup
Sone syzbot instances broke with:

<stdin>: In function ‘syz_io_uring_setup’:
<stdin>:476:33: error: ‘__NR_io_uring_setup’ undeclared (first use in this function)
<stdin>:476:33: note: each undeclared identifier is reported only once for each function it appears in

pkg/csource resolves #ifdef's at generation time.
2020-09-15 08:40:10 +02:00
Dmitry Vyukov
6c16e36a7b pkg/host: make machine info tests linux-specific
TestScanCPUInfo does not build on !linux.
TestMachineInfoLinux builds, but does not do anything useful.
2020-09-14 15:15:06 +02:00
Dmitry Vyukov
3e84253bf4 dashboard/app: fix up machine info handling
Assorted local fixes fixes, like dedup machine info in the database,
fix up HTML table markup, enforce and check access to the machine info
entities, etc.

Follow up to #2085
Fixes #466
2020-09-14 15:00:02 +02:00
Cheng-Min Chiang
9eff3337ee dashboard/app: send machine information to dashboard
- Change syz-manager so that it will send machine info the first time a
  crash occurs.
- Add a field in entities.Crash to store machine info.
- Add a field in dashapi.BugReport to store machine info.
- Change the HTML template and struct uiCrash to display machine info.
- Add a test to make sure that the link to machine info appears on the
  webpage.

Update #466
2020-09-14 15:00:02 +02:00
Alexander Egorenkov
df4f5a9c4a tools/create-gce-image.sh: support s390x
Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-09-14 13:09:25 +02:00
Dmitry Vyukov
dfb4c36914 tools/create_btrfs_image.sh: fix up and re-run
The previous commit did not get all intended changes.

Follow up to 2066
2020-09-14 11:08:16 +02:00
Dmitry Vyukov
fcf44ed305 tools/create_btrfs_image.sh: fix script
1. Use set -eu
2. Use consistent name of the generated files
3. Add "requires: manual"
4. Fix some option combinations that lead to errors

Follow up to 2066
2020-09-14 10:56:04 +02:00
Jiaheng Hu
bf0df4cba7 sys/linux: mount btrfs tests
Add several unit tests that mount different versions of btrfs images, and
the .sh file that generates these tests.
2020-09-14 10:56:04 +02:00
Dmitry Vyukov
dd93b2327c syz-manager: load test programs as candidates
1. Load test programs directly from sys/OS/test.
   Since we have sykaller dir, we don't need separate workdir/seeds.

2. Load test programs into candidates avoiding pulling them into corpus.
   This unbreaks mgr.fresh detection and does not pollute corpus with
   programs that don't give coverage/contain unsupported syscalls, etc.

Follow up to #2053
2020-09-14 10:38:36 +02:00
Dmitry Vyukov
605d25b511 syz-ci: fix seeds copying
1. Copy seeds from syzkaller checkout into syzkaller build dir.
   They need to be stable.

2. Make the code generic (current is linux-specific).

3. Don't copy seeds to workdir/seeds.
   We can load them directly from sys/OS/test.
   There are some unresolved comments for LinkDir on #2053 anyway.

Follow up to #2053
2020-09-14 10:38:36 +02:00
Dmitry Vyukov
bf14d79b03 pkg/osutil: support glob patterns in CopyFiles
Follow up to #2053
2020-09-14 10:38:36 +02:00
Dmitry Vyukov
fab7609913 tools/check-whitespace.sh: check for trailing whitespaces
File types that we don't format automatically can end up
with such basic untidiness as trailing whitespaces.
Check for these. Remove all existing precedents.
2020-09-14 09:55:45 +02:00
Jiaheng Hu
3f1d02b23f syz-manager: add test file as corpus
This commit enables the syz-manager to add unit test files as corpus to
accelerate fuzzing. The syz-ci would copy unit tests into the
worker/seeds folder for each manager process, and the manager would add
those tests as seed into the corpus.
2020-09-14 09:52:09 +02:00
Seonghyun Park
75cbe1c0ce tools/create-image.sh: resolve mismatch between help message and opt parsing 2020-09-14 06:47:09 +02:00
Dmitry Vyukov
2d3cdd63ad tools/create_f2fs_image.sh: name tests after the syscall
Make things consistent.

Follow up to #2032
2020-09-13 19:28:47 +02:00
Dmitry Vyukov
55b6cd4333 pkg/runtest: test parsing of all tests
Currently we only test parsing in tools/syz-runtest
and for test OS in pkg/runtest tests.
This means errors in tests for other OSes won't be
noticed until somebody runs tests manually.
Test parsing of all tests in pkg/runtest tests.
Fix up 2 broken tests.
2020-09-13 19:28:47 +02:00
Dmitry Vyukov
c52d06e410 sys/linux/test: mark f2fs tests as manual
Introduce "manual" requirement for tests (only run if explicitly selected)
and mark f2fs tests as manual. There are too many of them.

Follow up to #2032
2020-09-13 19:28:47 +02:00
Jiaheng Hu
84b0c3d8f6 sys/linux: add mount f2fs test
Add several unit tests that mount different versions of  f2fs image, and
the .sh file that generates these tests.
2020-09-13 18:42:14 +02:00
Mark Johnston
8a9bdb4a7e sys/freebsd: add definitions for large pages
Also add SHM_GROW_ON_WRITE, missed previously.
2020-09-13 17:15:57 +02:00
Mark Johnston
d0a4af89ac sys/freebsd: switch to the new .const format 2020-09-13 17:15:57 +02:00
Alexander Egorenkov
d56d957c29 docs/linux: describe how to setup s390x test environment
Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-09-13 16:06:07 +02:00
Alexander Egorenkov
86bd8f9d48 tools/create-image.sh: support for foreign architectures
Also update sy-env to be able to build the root image inside.

Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-09-13 16:05:25 +02:00
Dmitry Vyukov
e5f38ffe77 syz-manager: fix nil defer
Don't leave nil fuzzers in the map.
2020-09-13 13:42:03 +02:00
Dmitry Vyukov
455eff3ca1 pkg/report: skip rb tree functions in linux reports
RB tree is just a container (like list we already skip),
the bug is usually in the caller. Skip RB frames.
The new titles are much more informative and have lower chances of collisions.
2020-09-13 09:26:44 +02:00
Dmitry Vyukov
ad54dc7a6d syz-manager: cosmetic refactoring
1. Don't duplicate VM name generation.
2. Always "shutdown" fuzzers to not keep stale machine info.
2020-09-12 17:38:54 +02:00
Dmitry Vyukov
01622de2d0 pkg/host: move machine info functionality from syz-fuzzer
It's better to keep functionality in packages rather than in main.
It makes it reusable and better organized.
Move machine info functionality to pkg/host and do some cosmetic refactoring.
2020-09-12 17:38:54 +02:00
bobogei81123
7aa6bd6859
syz-manager: collect machine information
* syz-manager: finish a prototype

Extract machine info from /proc/cpuinfo and /sys/kvm*/parameters/* and
send it from syz-fuzzer to syz-manager. Append the machine info after
crash reports.

* syz-manager: refactor the code

- Add kvm parameters machine info.
- Store the machine info in the RPCServer instead of the manager.
- Store the machine info in another field instead of appending it after
  the original report
- Save the machine info locally in machineInfo*.

* syz-manager: fix coding-style problems

* syz-fuzzer: improve the output from /proc/cpuinfo

Improve the machine info extracted from /proc/cpuinfo by grouping lines
with the same key.

* syz-manager: fix race condition in runInstance

* syz-fuzzer: add tests for collecting machine info

- Add some tests to test collecting machine information.
- Split readCPUInfo into scanCPUInfo so that we can test it.

* syz-fuzzer: refactor scanCPUInfo

Refactor scanCPUInfo so that no sorting is needed.

* syz-fuzzer: refactor some code

Fix some issue that was pointed out on Github.
2020-09-12 17:17:22 +02:00
Dmitry Vyukov
9296c80bbc tools/docker: mirror images on github
Some users don't have access to the gcr.io registry.
Mirror images on github as well.
2020-09-12 16:26:58 +02:00
Anton Lindqvist
ce441f065b executor: improve opendir(3) error handling
While investigating an OpenBSD reproducer[1][2] I discovered the
following:

* All threads are stuck on the last `sleep(1000000)` syscall in main(),
  hence no output for the test machine.

* Each executor process created in loop() performs one iteration but
  exits abnormally during the call to remove_dir().

* Calling remove_dir() will eventually invoke itself recursively since
  one of the executed syscall is `mkdir("./file0", 0)` meaning that it
  will try to remove the directory created by execute_one(). However,
  `opendir(3)` fails with `EACCES` due to the permissions passed to
  `mkdir(2)` is zero.

Instead of exiting, trying to remove the problematic directory in a best
effort manner makes the reproducer continue executing the generated
syscalls. This work around might be considered to narrow. Another option
would be to replace the `sleep(1000000)` with `waitpid(-1, NULL, 0)`
until ECHILD is hit.

[1] https://syzkaller.appspot.com/bug?id=6f7ce2a0536580a94f65f44e478732ec505e88af
[2] https://syzkaller.appspot.com/text?tag=ReproC&x=10fd1a71900000
2020-09-12 13:21:21 +02:00
Dmitry Vyukov
7b0683780a executor: detect gvisor
gvisor coverage is not in the range of linux kernel coverage.
So the coverage filter does not work. Detect if running under gvisor
and skip the coverage filter.
2020-09-12 13:03:35 +02:00
Dmitry Vyukov
c38fcca50d pkg/repro: fix execution of non-repeatig C programs
If we have a non-repeating C reproducer with timeout > vm.NoOutputTimeout and it hangs
(the reproducer itself does not terminate on its own, note: it does not have builtin timeout),
then we will falsely detect "not output from test machine" kernel bug.
We could fix it by adding a builtin timeout to such reproducers (like we have in all other cases).
However, then it will exit within few seconds and we will finish the test without actually waiting
for full vm.NoOutputTimeout, which breaks the whole reason of using vm.NoOutputTimeout in the first
place. So we would need something more elaborate: let the program exist after few seconds, but
continue waiting for kernel hang errors for minutes, but at the same time somehow ignore "no output"
error because it will be false in this case.
Instead we simply prohibit !Repeat with long timeouts.
It makes sense on its own to some degree: if we are chasing an elusive bug, repeating the test
will increase chances of reproducing it and can make the reproducer less flaky.
Syz repros does not have this problem because they always have internal timeout, however
(1) it makes sense on its own, (2) we will either not use the whole timeout or waste the remaining
time as mentioned above, (3) if we remove repeat for syz repro, we won't be able to handle it
when/if we switch to C repro (we can simplify options, but we can't "complicate" them back).
2020-09-12 13:03:27 +02:00
Dmitry Vyukov
306464056c sys/test/test: add a hanging test
Ensure that we can handle hanging syscalls in all modes.
2020-09-12 13:03:27 +02:00
Dmitry Vyukov
cc8045ff1f tools/syz-prog2c: add -repro flag
For manual testing of what pkg/repro uses.
2020-09-12 13:03:27 +02:00
Dmitry Vyukov
21d289c28f pkg/report: add gvisor "stuck task" pattern 2020-09-12 12:04:32 +02:00
Dmitry Vyukov
79fb24e23d
docs: clarify what is coverage 2020-09-11 11:59:39 +02:00
Dmitry Vyukov
adfb8b4e39
docs: clarify that Go 1.14 is required for contributors 2020-09-11 08:32:44 +02:00
bobogei81123
409809d8a7
sys/linux: improve descriptions of bpf tracing (#2076)
- Rename bpf_lsm to bpf_trace and put all bpf program types
  that use BPF_RAW_TRACEPOINT_OPEN here.
- Add descriptions for types RAW_TRACEPOINT(_WRITABLE), BPF_TRACING and
  BPF_EXT.
- Add the hook names for RAW_TRACEPOINT(_WRITABLE).
2020-09-09 14:02:04 -07:00
Albert van der Linde
ac7ca78e73 dashboard/config: enable CONFIG_FAULT_INJECTION_USERCOPY
As CONFIG_FAULT_INJECTION_USERCOPY is already included in linux-next,
enable the config.

Fixes #1979
2020-09-09 14:39:22 +02:00