Implement several performance improvements:
1. Don't run addr2line for the whole binary.
Frequently only a small part of the vmlinux is covered,
running addr2line over whole binary ahead of time takes insane amount of time.
Instread run addr2line incrementally only for symbols that have any coverage.
2. Run addr2line in parallel.
3. Instead of running objdump -d on the whole object file to find
coverage points, look for call instructions in the .text section directly.
Currently this is implemented only for amd64.
Also this Go change cuts another 7 seconds:
f92c64045f
(faster interation over DWARF compile units, should speed up syz-check as well).
Update #2006
The HTML code assumes that files and functions match one-to-one
as they are identified by indices (file_N should match function_N).
Since we only add non-empty functions, this relation is broken
and a the report shows function coverage for a random file.
Moreover, since the order is based on map iteration (random each time),
function coverage shown for a file also randomly changes each time.
Follow up to #2074
Add fstab entry to mount the root filesystem read-write after boot. This
has been done with commit 873745 for create-image.sh. Maybe this hint is
not necessary for GCE but it is required when running on other platforms
(e.g. VMware).
Two virtual wireless devices are instantiated during network devices
initialization.
A new flag (-wifi) is added that controls whether these virtual wifi
devices are instantiated and configured during proc initialization.
Also, two new pseudo syscalls are added:
1. syz_80211_inject_frame(mac_addr, packet, packet_len) -- injects an
arbitrary packet into the wireless stack. It is injected as if it
originated from the device identitied by mac_addr.
2. syz_80211_join_ibss(interface_name, ssid, ssid_len, mode) --
puts a specific network interface into IBSS state and joins an IBSS
network.
Arguments of syz_80211_join_ibss:
1) interface_name -- null-terminated string that identifies
a wireless interface
2) ssid, ssid_len -- SSID of an IBSS network to join to
3) mode -- mode of syz_80211_join_ibss operation (see below)
Modes of operation:
JOIN_IBSS_NO_SCAN (0x0) -- channel scan is not performed and
syz_80211_join_ibss waits until the interface reaches IF_OPER_UP.
JOIN_IBSS_BG_SCAN (0x1) -- channel scan is performed (takes ~ 9
seconds), syz_80211_join_ibss does not await IF_OPER_UP.
JOIN_IBSS_BG_NO_SCAN (0x2) -- channel scan is not performed,
syz_80211_join_ibss does not await IF_OPER_UP.
Local testing ensured that these syscalls are indeed able to set up an
operating network and inject packets into mac80211.
As netlink helpers now include a function to query generic netlink
familty id, it makes no sense to duplicate implementation of
essentially the same function.
The code in common_linux.h assumes that nlmsgerr can either be 0 or a
negative value in case of an error. However, this is not always the
case. For example, some commands of mac80211_hwsim use nonnegative
values to indicate success (e.g. HWSIM_CMD_NEW_RADIO returns either a
negative error or a nonnegative radio index). Therefore, negation of
error code inside netlink_send_ext is not correct.
This patch changes this behavior. Now netlink_send_ext returns the
exact value it received via netlink.
This global variable cannot be used for pseudo syscalls as they can
run concurrently (in threaded mode). It can only be used during
initialization, and if initialization routines are not enabled, nlmsg
will become an unused variable.
syz_mount_image tests are very large and TestSysTests takes too long.
syz-imagegen that generates them does some of this testing
(Deserialize/SerializeForExec).
Fixes the issue with gcc 10 on Fedora 32 s390x:
In file included from ../../executor/executor.cc:147:
../../executor/common.h: In function ‘void remove_dir(const char*)’:
../../executor/common.h:229:44: error: ‘%s’ directive output may be
truncated writing up to 255 bytes into a region of size between 0 and 4095 [-Werror=format-truncation=]
229 | snprintf(filename, sizeof(filename), "%s/%s", dir, ep->d_name);
| ^~
../../executor/common.h:229:11: note: ‘snprintf’ output between 2 and 4352 bytes into a destination of size 4096
229 | snprintf(filename, sizeof(filename), "%s/%s", dir, ep->d_name);
../../executor/common.h:243:1: error: the frame size of 21200 bytes is larger than 16384 bytes
[-Werror=frame-larger-than=]
243 | }
| ^
cc1plus: all warnings being treated as errors
compiler invocation: gcc [-o /tmp/syz-executor383272105 -DGOOS_test=1 -DGOARCH_64_fork=1 -DHOSTGOOS_linux=1
../../executor/executor.cc -m64 -no-pie -O2 -pthread -Wall -Werror -Wparentheses
-Wunused-const-variable -Wframe-larger-than=16384]
FAIL
FAIL github.com/google/syzkaller/pkg/runtest 0.998s
FAIL
Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
There is no field 'machine' in /proc/cpuinfo when we run inside
a Docker container but there is a 'processor 0' field which
provides the same information.
Example:
processor 0: version = 00, identification = 310C57, machine = 3906
Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
Mentions of "// Code generated ... DO NOT EDIT." in these Go files
somehow prevents golangci-lint from checking them.
And github treats them as auto-generated as well.
I guess some code somewhere has some imprecise heuristics.
Move generation into separate shell scripts. Also cleaner.
Sone syzbot instances broke with:
<stdin>: In function ‘syz_io_uring_setup’:
<stdin>:476:33: error: ‘__NR_io_uring_setup’ undeclared (first use in this function)
<stdin>:476:33: note: each undeclared identifier is reported only once for each function it appears in
pkg/csource resolves #ifdef's at generation time.
1. Load test programs directly from sys/OS/test.
Since we have sykaller dir, we don't need separate workdir/seeds.
2. Load test programs into candidates avoiding pulling them into corpus.
This unbreaks mgr.fresh detection and does not pollute corpus with
programs that don't give coverage/contain unsupported syscalls, etc.
Follow up to #2053
1. Copy seeds from syzkaller checkout into syzkaller build dir.
They need to be stable.
2. Make the code generic (current is linux-specific).
3. Don't copy seeds to workdir/seeds.
We can load them directly from sys/OS/test.
There are some unresolved comments for LinkDir on #2053 anyway.
Follow up to #2053
This commit enables the syz-manager to add unit test files as corpus to
accelerate fuzzing. The syz-ci would copy unit tests into the
worker/seeds folder for each manager process, and the manager would add
those tests as seed into the corpus.
Currently we only test parsing in tools/syz-runtest
and for test OS in pkg/runtest tests.
This means errors in tests for other OSes won't be
noticed until somebody runs tests manually.
Test parsing of all tests in pkg/runtest tests.
Fix up 2 broken tests.
Introduce "manual" requirement for tests (only run if explicitly selected)
and mark f2fs tests as manual. There are too many of them.
Follow up to #2032
RB tree is just a container (like list we already skip),
the bug is usually in the caller. Skip RB frames.
The new titles are much more informative and have lower chances of collisions.
It's better to keep functionality in packages rather than in main.
It makes it reusable and better organized.
Move machine info functionality to pkg/host and do some cosmetic refactoring.
* syz-manager: finish a prototype
Extract machine info from /proc/cpuinfo and /sys/kvm*/parameters/* and
send it from syz-fuzzer to syz-manager. Append the machine info after
crash reports.
* syz-manager: refactor the code
- Add kvm parameters machine info.
- Store the machine info in the RPCServer instead of the manager.
- Store the machine info in another field instead of appending it after
the original report
- Save the machine info locally in machineInfo*.
* syz-manager: fix coding-style problems
* syz-fuzzer: improve the output from /proc/cpuinfo
Improve the machine info extracted from /proc/cpuinfo by grouping lines
with the same key.
* syz-manager: fix race condition in runInstance
* syz-fuzzer: add tests for collecting machine info
- Add some tests to test collecting machine information.
- Split readCPUInfo into scanCPUInfo so that we can test it.
* syz-fuzzer: refactor scanCPUInfo
Refactor scanCPUInfo so that no sorting is needed.
* syz-fuzzer: refactor some code
Fix some issue that was pointed out on Github.
While investigating an OpenBSD reproducer[1][2] I discovered the
following:
* All threads are stuck on the last `sleep(1000000)` syscall in main(),
hence no output for the test machine.
* Each executor process created in loop() performs one iteration but
exits abnormally during the call to remove_dir().
* Calling remove_dir() will eventually invoke itself recursively since
one of the executed syscall is `mkdir("./file0", 0)` meaning that it
will try to remove the directory created by execute_one(). However,
`opendir(3)` fails with `EACCES` due to the permissions passed to
`mkdir(2)` is zero.
Instead of exiting, trying to remove the problematic directory in a best
effort manner makes the reproducer continue executing the generated
syscalls. This work around might be considered to narrow. Another option
would be to replace the `sleep(1000000)` with `waitpid(-1, NULL, 0)`
until ECHILD is hit.
[1] https://syzkaller.appspot.com/bug?id=6f7ce2a0536580a94f65f44e478732ec505e88af
[2] https://syzkaller.appspot.com/text?tag=ReproC&x=10fd1a71900000
If we have a non-repeating C reproducer with timeout > vm.NoOutputTimeout and it hangs
(the reproducer itself does not terminate on its own, note: it does not have builtin timeout),
then we will falsely detect "not output from test machine" kernel bug.
We could fix it by adding a builtin timeout to such reproducers (like we have in all other cases).
However, then it will exit within few seconds and we will finish the test without actually waiting
for full vm.NoOutputTimeout, which breaks the whole reason of using vm.NoOutputTimeout in the first
place. So we would need something more elaborate: let the program exist after few seconds, but
continue waiting for kernel hang errors for minutes, but at the same time somehow ignore "no output"
error because it will be false in this case.
Instead we simply prohibit !Repeat with long timeouts.
It makes sense on its own to some degree: if we are chasing an elusive bug, repeating the test
will increase chances of reproducing it and can make the reproducer less flaky.
Syz repros does not have this problem because they always have internal timeout, however
(1) it makes sense on its own, (2) we will either not use the whole timeout or waste the remaining
time as mentioned above, (3) if we remove repeat for syz repro, we won't be able to handle it
when/if we switch to C repro (we can simplify options, but we can't "complicate" them back).
Add the following missing FUSE opcodes to the syz_fuse_handle_req
pseudo-syscall: FUSE_COPY_FILE_RANGE, FUSE_UNLINK, FUSE_DESTROY and
FUSE_BATCH_FORGET.
unshare(CLONE_NEWNS) might not be sufficient for making all test processes run in
separate mount namespace, for "mount --make-rshared /" request issued by systemd
causes mount operations issued by test processes visible from outside of test
processes. Issue "mount --make-rprivate /" request after unshare(CLONE_NEWNS).
