Commit Graph

1603 Commits

Author SHA1 Message Date
Dmitry Vyukov
2111afe851 pkg/cover: test that CSV profile contains main function
The test also passed if there is no main at all.

Follow up to #2074
2020-09-28 09:41:25 +02:00
Dmitry Vyukov
f9bcf3095b pkg/cover: fix error message capitalization in tests
Follow up to #2074
2020-09-28 09:41:25 +02:00
Dmitry Vyukov
3bbd2f185f pkg/cover: unexport CSVHeader
No point for it to be a part of the package public interface.

Follow up to #2074
2020-09-28 09:41:25 +02:00
Jouni Hogander
769c9b7a94 pkg/cover: add new view for function coverage
Add new view where function coverage is shown when clicking file coverage
percent
2020-09-26 17:47:37 +02:00
Jouni Hogander
6f0ea384b1 pkg/cover: implement function coverage calculation 2020-09-26 17:47:37 +02:00
Radoslav Gerganov
54289b0835 tools/create-gce-image.sh: mount the root filesystem read-write
Add fstab entry to mount the root filesystem read-write after boot. This
has been done with commit 873745 for create-image.sh. Maybe this hint is
not necessary for GCE but it is required when running on other platforms
(e.g. VMware).
2020-09-23 17:21:11 +02:00
Dmitry Vyukov
287cd75a6d pkg/report: fix a test 2020-09-23 08:42:25 +02:00
Dmitry Vyukov
d983f3059e pkg/report: detect hrtimer_run as stall anchor frame 2020-09-23 08:27:07 +02:00
Aleksandr Nogikh
9133037195 all: integrate with mac80211_hwsim
Two virtual wireless devices are instantiated during network devices
initialization.

A new flag (-wifi) is added that controls whether these virtual wifi
devices are instantiated and configured during proc initialization.

Also, two new pseudo syscalls are added:
1. syz_80211_inject_frame(mac_addr, packet, packet_len) -- injects an
arbitrary packet into the wireless stack. It is injected as if it
originated from the device identitied by mac_addr.
2. syz_80211_join_ibss(interface_name, ssid, ssid_len, mode) --
puts a specific network interface into IBSS state and joins an IBSS
network.

Arguments of syz_80211_join_ibss:
1) interface_name -- null-terminated string that identifies
a wireless interface
2) ssid, ssid_len -- SSID of an IBSS network to join to
3) mode -- mode of syz_80211_join_ibss operation (see below)

Modes of operation:
JOIN_IBSS_NO_SCAN (0x0) -- channel scan is not performed and
syz_80211_join_ibss waits until the interface reaches IF_OPER_UP.
JOIN_IBSS_BG_SCAN (0x1) -- channel scan is performed (takes ~ 9
seconds), syz_80211_join_ibss does not await IF_OPER_UP.
JOIN_IBSS_BG_NO_SCAN (0x2) -- channel scan is not performed,
syz_80211_join_ibss does not await IF_OPER_UP.

Local testing ensured that these syscalls are indeed able to set up an
operating network and inject packets into mac80211.
2020-09-22 10:12:23 +02:00
Aleksandr Nogikh
1125444eb8 executor/common_linux.h: refactor __NR_syz_genetlink_get_family_id
As netlink helpers now include a function to query generic netlink
familty id, it makes no sense to duplicate implementation of
essentially the same function.
2020-09-22 10:12:23 +02:00
Aleksandr Nogikh
a44e0f15f3 executor/common_linux.h: remove assumption that netlink always returns nonpositive error codes
The code in common_linux.h assumes that nlmsgerr can either be 0 or a
negative value in case of an error. However, this is not always the
case. For example, some commands of mac80211_hwsim use nonnegative
values to indicate success (e.g. HWSIM_CMD_NEW_RADIO returns either a
negative error or a nonnegative radio index). Therefore, negation of
error code inside netlink_send_ext is not correct.

This patch changes this behavior. Now netlink_send_ext returns the
exact value it received via netlink.
2020-09-22 10:12:23 +02:00
Aleksandr Nogikh
6304883e1b sys/common_linux.h: replace netlink_devlink_id_get and netlink_wireguard_id_get by a more generic function
These two functions are almost entirely idential. In order to avoid
adding similar functions later on, a generic one is created.
2020-09-22 10:12:23 +02:00
Aleksandr Nogikh
56f696d1d0 sys/common_linux.h: move the definition of a global nlmsg variable
This global variable cannot be used for pseudo syscalls as they can
run concurrently (in threaded mode). It can only be used during
initialization, and if initialization routines are not enabled, nlmsg
will become an unused variable.
2020-09-22 10:12:23 +02:00
Aleksandr Nogikh
aff59df257 executor/common_linux.h: extend netlink helpers to support zero-payload attributes (flags)
This is required for 802.11 pseudo syscalls (that will be added
later).
2020-09-22 10:12:23 +02:00
Aleksandr Nogikh
f7907acd76 executor/common_linux.h: increase default nlmsg buffer size
1024 bytes are not enough to contain an arbirary 802.11 MAC frame
together with netlink protocol overhead.
2020-09-22 10:12:23 +02:00
Jouni Hogander
bbb921d248 pkg/bisect: add ccache option
Add option to use ccache in kernel builds.

Signed-off-by: Jouni Hogander <jouni.hoegander@partner.bmw.de>
2020-09-21 11:23:04 +02:00
Dmitry Vyukov
a5e37d4a8e pkg/runtest: split parsing tests into OS/arch subtests 2020-09-20 15:12:12 +02:00
Dmitry Vyukov
7b84681e92 pkg/csource: don't test auto-generated files
syz_mount_image tests are very large and TestSysTests takes too long.
syz-imagegen that generates them does some of this testing
(Deserialize/SerializeForExec).
2020-09-20 15:12:12 +02:00
Alexander Egorenkov
49ad2baf8c executor: forbid inlining of 'remove_dir'
Fixes the issue with gcc 10 on Fedora 32 s390x:

In file included from ../../executor/executor.cc:147:
../../executor/common.h: In function ‘void remove_dir(const char*)’:
../../executor/common.h:229:44: error: ‘%s’ directive output may be
truncated writing up to 255 bytes into a region of size between 0 and 4095 [-Werror=format-truncation=]
229 | snprintf(filename, sizeof(filename), "%s/%s", dir, ep->d_name);
| ^~
../../executor/common.h:229:11: note: ‘snprintf’ output between 2 and 4352 bytes into a destination of size 4096
229 | snprintf(filename, sizeof(filename), "%s/%s", dir, ep->d_name);

../../executor/common.h:243:1: error: the frame size of 21200 bytes is larger than 16384 bytes
[-Werror=frame-larger-than=]
243 | }
| ^
cc1plus: all warnings being treated as errors

compiler invocation: gcc [-o /tmp/syz-executor383272105 -DGOOS_test=1 -DGOARCH_64_fork=1 -DHOSTGOOS_linux=1
../../executor/executor.cc -m64 -no-pie -O2 -pthread -Wall -Werror -Wparentheses
-Wunused-const-variable -Wframe-larger-than=16384]
FAIL
FAIL github.com/google/syzkaller/pkg/runtest 0.998s
FAIL

Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-09-18 09:58:50 +02:00
Alexander Egorenkov
8247808b71 pkg/host: use 'processor 0' as cpu model on s390x inside container
There is no field 'machine' in /proc/cpuinfo when we run inside
a Docker container but there is a 'processor 0' field which
provides the same information.

Example:

processor 0: version = 00,  identification = 310C57,  machine = 3906

Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-09-16 21:15:40 +02:00
Dmitry Vyukov
da5350eb9a pkg/report: support new linux UBSAN format
Update #1523
2020-09-16 15:22:52 +02:00
Dmitry Vyukov
c478111a98 pkg/build, pkg/html: move generation logic into separate scripts
Mentions of "// Code generated ... DO NOT EDIT." in these Go files
somehow prevents golangci-lint from checking them.
And github treats them as auto-generated as well.
I guess some code somewhere has some imprecise heuristics.
Move generation into separate shell scripts. Also cleaner.
2020-09-15 19:34:30 +02:00
Alexander Egorenkov
3b5eb29afb pkg/host: use 'machine' field as cpu model on IBM/Z
Use the field 'machine' in /proc/cpuinfo on IBM/Z
to find out the "CPU model".

Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-09-15 15:06:24 +02:00
Dmitry Vyukov
9e681632f5 pkg/osutil: fix LinkFiles/FilesExist for the new pattern format 2020-09-15 09:37:22 +02:00
Dmitry Vyukov
a2360d0742 executor: fix definition of __NR_io_uring_setup
Sone syzbot instances broke with:

<stdin>: In function ‘syz_io_uring_setup’:
<stdin>:476:33: error: ‘__NR_io_uring_setup’ undeclared (first use in this function)
<stdin>:476:33: note: each undeclared identifier is reported only once for each function it appears in

pkg/csource resolves #ifdef's at generation time.
2020-09-15 08:40:10 +02:00
Dmitry Vyukov
6c16e36a7b pkg/host: make machine info tests linux-specific
TestScanCPUInfo does not build on !linux.
TestMachineInfoLinux builds, but does not do anything useful.
2020-09-14 15:15:06 +02:00
Alexander Egorenkov
df4f5a9c4a tools/create-gce-image.sh: support s390x
Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-09-14 13:09:25 +02:00
Dmitry Vyukov
dd93b2327c syz-manager: load test programs as candidates
1. Load test programs directly from sys/OS/test.
   Since we have sykaller dir, we don't need separate workdir/seeds.

2. Load test programs into candidates avoiding pulling them into corpus.
   This unbreaks mgr.fresh detection and does not pollute corpus with
   programs that don't give coverage/contain unsupported syscalls, etc.

Follow up to #2053
2020-09-14 10:38:36 +02:00
Dmitry Vyukov
605d25b511 syz-ci: fix seeds copying
1. Copy seeds from syzkaller checkout into syzkaller build dir.
   They need to be stable.

2. Make the code generic (current is linux-specific).

3. Don't copy seeds to workdir/seeds.
   We can load them directly from sys/OS/test.
   There are some unresolved comments for LinkDir on #2053 anyway.

Follow up to #2053
2020-09-14 10:38:36 +02:00
Dmitry Vyukov
bf14d79b03 pkg/osutil: support glob patterns in CopyFiles
Follow up to #2053
2020-09-14 10:38:36 +02:00
Jiaheng Hu
3f1d02b23f syz-manager: add test file as corpus
This commit enables the syz-manager to add unit test files as corpus to
accelerate fuzzing. The syz-ci would copy unit tests into the
worker/seeds folder for each manager process, and the manager would add
those tests as seed into the corpus.
2020-09-14 09:52:09 +02:00
Dmitry Vyukov
55b6cd4333 pkg/runtest: test parsing of all tests
Currently we only test parsing in tools/syz-runtest
and for test OS in pkg/runtest tests.
This means errors in tests for other OSes won't be
noticed until somebody runs tests manually.
Test parsing of all tests in pkg/runtest tests.
Fix up 2 broken tests.
2020-09-13 19:28:47 +02:00
Dmitry Vyukov
c52d06e410 sys/linux/test: mark f2fs tests as manual
Introduce "manual" requirement for tests (only run if explicitly selected)
and mark f2fs tests as manual. There are too many of them.

Follow up to #2032
2020-09-13 19:28:47 +02:00
Dmitry Vyukov
455eff3ca1 pkg/report: skip rb tree functions in linux reports
RB tree is just a container (like list we already skip),
the bug is usually in the caller. Skip RB frames.
The new titles are much more informative and have lower chances of collisions.
2020-09-13 09:26:44 +02:00
Dmitry Vyukov
01622de2d0 pkg/host: move machine info functionality from syz-fuzzer
It's better to keep functionality in packages rather than in main.
It makes it reusable and better organized.
Move machine info functionality to pkg/host and do some cosmetic refactoring.
2020-09-12 17:38:54 +02:00
bobogei81123
7aa6bd6859
syz-manager: collect machine information
* syz-manager: finish a prototype

Extract machine info from /proc/cpuinfo and /sys/kvm*/parameters/* and
send it from syz-fuzzer to syz-manager. Append the machine info after
crash reports.

* syz-manager: refactor the code

- Add kvm parameters machine info.
- Store the machine info in the RPCServer instead of the manager.
- Store the machine info in another field instead of appending it after
  the original report
- Save the machine info locally in machineInfo*.

* syz-manager: fix coding-style problems

* syz-fuzzer: improve the output from /proc/cpuinfo

Improve the machine info extracted from /proc/cpuinfo by grouping lines
with the same key.

* syz-manager: fix race condition in runInstance

* syz-fuzzer: add tests for collecting machine info

- Add some tests to test collecting machine information.
- Split readCPUInfo into scanCPUInfo so that we can test it.

* syz-fuzzer: refactor scanCPUInfo

Refactor scanCPUInfo so that no sorting is needed.

* syz-fuzzer: refactor some code

Fix some issue that was pointed out on Github.
2020-09-12 17:17:22 +02:00
Anton Lindqvist
ce441f065b executor: improve opendir(3) error handling
While investigating an OpenBSD reproducer[1][2] I discovered the
following:

* All threads are stuck on the last `sleep(1000000)` syscall in main(),
  hence no output for the test machine.

* Each executor process created in loop() performs one iteration but
  exits abnormally during the call to remove_dir().

* Calling remove_dir() will eventually invoke itself recursively since
  one of the executed syscall is `mkdir("./file0", 0)` meaning that it
  will try to remove the directory created by execute_one(). However,
  `opendir(3)` fails with `EACCES` due to the permissions passed to
  `mkdir(2)` is zero.

Instead of exiting, trying to remove the problematic directory in a best
effort manner makes the reproducer continue executing the generated
syscalls. This work around might be considered to narrow. Another option
would be to replace the `sleep(1000000)` with `waitpid(-1, NULL, 0)`
until ECHILD is hit.

[1] https://syzkaller.appspot.com/bug?id=6f7ce2a0536580a94f65f44e478732ec505e88af
[2] https://syzkaller.appspot.com/text?tag=ReproC&x=10fd1a71900000
2020-09-12 13:21:21 +02:00
Dmitry Vyukov
c38fcca50d pkg/repro: fix execution of non-repeatig C programs
If we have a non-repeating C reproducer with timeout > vm.NoOutputTimeout and it hangs
(the reproducer itself does not terminate on its own, note: it does not have builtin timeout),
then we will falsely detect "not output from test machine" kernel bug.
We could fix it by adding a builtin timeout to such reproducers (like we have in all other cases).
However, then it will exit within few seconds and we will finish the test without actually waiting
for full vm.NoOutputTimeout, which breaks the whole reason of using vm.NoOutputTimeout in the first
place. So we would need something more elaborate: let the program exist after few seconds, but
continue waiting for kernel hang errors for minutes, but at the same time somehow ignore "no output"
error because it will be false in this case.
Instead we simply prohibit !Repeat with long timeouts.
It makes sense on its own to some degree: if we are chasing an elusive bug, repeating the test
will increase chances of reproducing it and can make the reproducer less flaky.
Syz repros does not have this problem because they always have internal timeout, however
(1) it makes sense on its own, (2) we will either not use the whole timeout or waste the remaining
time as mentioned above, (3) if we remove repeat for syz repro, we won't be able to handle it
when/if we switch to C repro (we can simplify options, but we can't "complicate" them back).
2020-09-12 13:03:27 +02:00
Dmitry Vyukov
306464056c sys/test/test: add a hanging test
Ensure that we can handle hanging syscalls in all modes.
2020-09-12 13:03:27 +02:00
Dmitry Vyukov
21d289c28f pkg/report: add gvisor "stuck task" pattern 2020-09-12 12:04:32 +02:00
Alexander Egorenkov
a86a864df5 pkg/build: support Linux s390x
Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-09-09 07:13:52 +02:00
Stefano Duo
816e0689d7 executor/common_linux.h: add missing FUSE opcodes
Add the following missing FUSE opcodes to the syz_fuse_handle_req
pseudo-syscall: FUSE_COPY_FILE_RANGE, FUSE_UNLINK, FUSE_DESTROY and
FUSE_BATCH_FORGET.
2020-08-27 07:47:59 +02:00
Tetsuo Handa
318430cbb3 executor/linux: change mount propagation type to private
unshare(CLONE_NEWNS) might not be sufficient for making all test processes run in
separate mount namespace, for "mount --make-rshared /" request issued by systemd
causes mount operations issued by test processes visible from outside of test
processes. Issue "mount --make-rprivate /" request after unshare(CLONE_NEWNS).
2020-08-26 12:25:46 +02:00
Andy Nguyen
cef5ae6814 executor/common_linux.h: make rfkill_unblock_all static 2020-08-23 12:31:52 +02:00
Andy Nguyen
5bde7c3b04 executor/common_linux.h: unblock rfkill and retry HCIDEVUP on ERFKILL 2020-08-23 12:31:52 +02:00
Dmitry Vyukov
1da71ab086 pkg/ipc: fix cross-arch build in test
BrokenCompiler is only initialized after Get.
2020-08-22 21:46:43 +02:00
Dmitry Vyukov
df5c8fa25f tools/docker: add old-env
old-env is based on Ubuntu 16.04 and allows to test
executor build on older distributions.

Fixes #2055
2020-08-22 21:42:13 +02:00
Dmitry Vyukov
49a5a1ab2f pkg/cover: fix calculation of prev PC for i386
Make the report generation test more realistic to use PCs
we will use in real life. This shows that PreviousInstructionPC
for 386 is broken. Fix it.

Reported-by: Alexander Lochmann <flipreverse>
See #2067
2020-08-22 21:26:42 +02:00
Dmitry Vyukov
7288521c2a pkg/ipc: run cross-arch executor tests
We may run some cross-arch tests using qemu-user.
2020-08-22 21:26:42 +02:00
Stefano Duo
6fe61c6bc7 executor/common_linux.go: fix build breakage for Linux 4.14
Locally define structures and constants required by
syz_fuse_handle_req() and remove linux/fuse.h. dependency.

Reported-by: syzbot+f5f98e1608b272285aa5@syzkaller.appspotmail.com
2020-08-18 14:18:38 +02:00