With commit 7ba05d2dd6 we always write a
fresh loader.conf on each build, but this clobbers any pre-existing
settings that may be required for a given setup. This went unnoticed by
me for a while since bhyve requires no additional preconfiguration, but
clearly syzbot is affected. On the other hand, before that commit we
were appending the same lines upon each build. Use
/boot/loader.conf.local instead.
Refactor syz_mount_image() to support filesystems not requiring a
backing device and filesystem image (e.g. FUSE). To do that, we check for
the presence of the pointer to the array of struct fs_image_segment: if
missingi, there is no need to setup the loop device and we can proceed
directly with the mount() syscall.
Add syz_mount_image$fuse() (specialization for FUSE) inside
sys/linux/fs_fuse.txt.
At the moment syzkaller is able to respond to FUSE with a syntactically
correct response using the specific write$FUSE_*() syscalls, but most of
the times these responses are not related to the type of request that
was received.
With this pseudo-syscall we are able to provide the correct response
type while still allowing the fuzzer to fuzz its content. This is done
by requiring each type of response as an input parameter and then
choosing the correct one based on the request opcode.
Notice that the fuzzer is still free to mix write$FUSE_*() and
syz_fuse_handle_req() syscalls, so it is not losing any degree of
freedom.
syz_fuse_handle_req() retrieves the FUSE request and resource
fuse_unique internally (by performing a read() on the /dev/fuse file
descriptor provided as input). For this reason, a new template argument has
been added to fuse_out (renamed to _fuse_out) so that the unique field
can be both an int64 (used by syz_fuse_handle_req()) and a fuse_unique
resource (used by the write$FUSE_*() syscalls) without any code
duplication.
"#if not" does not seem to be a thing in C:
$ cpp -undef -fdirectives-only -dDI -E -P -DSYZ_REPEAT -DSYZ_USE_TMP_DIR executor/common_linux.h 1>/dev/null
executor/common_linux.h:3776:9: error: missing binary operator before token "SYZ_SANDBOX_ANDROID"
3776 | #if not SYZ_SANDBOX_ANDROID
| ^~~~~~~~~~~~~~~~~~~
executor/common_linux.h:3801:9: error: missing binary operator before token "SYZ_SANDBOX_ANDROID"
3801 | #if not SYZ_SANDBOX_ANDROID
| ^~~~~~~~~~~~~~~~~~~
executor/common_linux.h:3837:9: error: missing binary operator before token "SYZ_SANDBOX_ANDROID"
3837 | #if not SYZ_SANDBOX_ANDROID
| ^~~~~~~~~~~~~~~~~~~
executor/common_linux.h:3868:9: error: missing binary operator before token "SYZ_SANDBOX_ANDROID"
3868 | #if not SYZ_SANDBOX_ANDROID
| ^~~~~~~~~~~~~~~~~~~
Currently parts under "#if not SYZ_SANDBOX_ANDROID" are always stripped from
reproducers under all sandboxes. Use the standard !SYZ_SANDBOX_ANDROID.
We also need SYZ_EXECUTOR part because sandbox is not statically known
when we are building syz-executor.
And we also need to remove the use of flag_sandbox_android for C reproducers
because for these sandbox is statically known and we don't have flag_sandbox_*.
We generally use the newer C99 var declarations combined with initialization because:
- declarations are more local, reduced scope
- fewer lines of code
- less potential for using uninit vars and other bugs
However, we have some relic code from times when we did not understand
if we need to stick with C89 or not. Also some external contributions
that don't follow style around.
Add a static check for C89-style declarations and fix existing precedents.
Akaros toolchain uses -std=gnu89 (or something) and does not allow
variable declarations inside of for init statement. And we can't switch
it to -std=c99 because Akaros headers are C89 themselves.
So in common.h we need to declare loop counters outside of for.
We now have 8 arches for Linux and .const files
produce lots of noise in PRs and lots of diffs.
If 3 .txt files are touched, the PR will have 24 .const files,
which will be intermixed with .txt files.
Frequently const values are equal across arches,
and even if they don't spreading a single value
across 8 files is inconvinient.
Merge all 8 *_arch.const files into a single .const file.
See the test for details of the new format.
The old format is still parsed for now,
we can't update all OSes at once.
For Linux this reduces number of const files/lines
from 1288/96599 to 158/11603.
Fixes#1983
We added initialize_vhci to all sandboxes so that we don't have
unused function warnings. We assumed it will fail silently,
but it fails loudly and crashes the whole machine on init,
so no fuzzing can happen with sandboxes other than none.
Initialize vhci earlier while we still have CAP_ADMIN.
As a nice side effect we now don't need to use syz_init_net_socket.
syz-executor uses a heuristic to help fail closed if an invalid access
might corrupt the output region. This heuristic fails on FreeBSD, where
SIGBUS is delievered with si_addr equal to address of the faulting
instruction, rather than 0 when the fault address cannot be determined
(e.g., an amd64 protection fault). Always handle SIGBUS quietly on
FreeBSD.
This fixes pkg/runtest tests for sys/test/test/nonfailing.
This commit includes the following changes:
* executor: add a new syz_btf_id_by_name psuedo-syscall
* sys/linux: add descriptions for BPF LSM subsystem
* sys/linux: add instructions on how to dump vmlinux and install
bpftool
* sys/linux/test: add tests for the new psuedo-syscall
* pkg/host: add support detection for the new psuedo-syscall
* pkg/runtest: skip the coverage test when invoking the new
psuedo-syscall
Update #533.
Move the test from pkg/csource to executor/
in order to be able to (1) run it on *.cc files,
(2) run on unprocessed *.h files, (3) produce line numbers.
Add a check for missed space after //.
Regression introduced in commit cb93dc6a ("pkg/report: flag short
uvm_fault reports as corrupted") causing some valid reports to be
flagged as corrupted.
Use a map: (string => func) instead of a switch for pseudo-syscalls
names. This reduces isSupportedSyzkall() cyclomatic complexity and
makes the linter happy.
1. We don't generally use /* */ block comments,
few precedents we have are inconsistent with the rest of the code.
2. pkg/csource does not strip them from the resulting code.
Remove the cases we have and add a test to prevent new ones being added.
If a resource is never used as an input, it is not useful.
It's effectively the same as using an integer.
Detect such cases, they are quite confusing.
Fix all existing errors in descriptions.
This uncovered some interesting bugs as well,
e.g. use of a completely unrelated fd subtype after copy-paste
(while the resource that was supposed to be used there is completely unused).
Forgot that the build machine must be updated with a newer OpenBSD
snapshot first in order to make the new kcov stuff available.
This reverts commit 96dd36234d.
Create a struct on pkg/vcs to store data of syzkaller email recipients
and update its users. The struct contains default name, email, and a
label to divide user into To and Cc when sending the emails.
Regenerate const files on next-20200729.
Change conn handle to 200 because it also seems to be matches
against phy_handle fields which are int8 (current 256 does not fit into int8).
Use 200 for all handle's and all phy_handle's.
Remove hci_evt_le_cis_req, it does not seem to be used in the kernel.
Restrict some event types and statuses.
Add rssi field to hci_ev_le_advertising_info.
Use bytesize for some of the data length fields.
* all: initialize vhci in linux
* executor/common_linux.h: improve vhci initialization
* pkg/repro/repro.go: add missing vhci options
* executor/common_linux.h: fix type and add missing header
* executor, pkg: do it like NetInjection
* pkg/csource/csource.go: do not emit syz_emit_vhci if vhci is not enabled
* executor/common_linux.h: fix format string
* executor/common_linux.h: initialize with memset
For som reason {0} gets complains about missing braces...
* executor/common_linux.h: simplify vhci init
* executor/common_linux.h: try to bring all available hci devices up
* executor/common_linux.h: find which hci device has been registered
* executor/common_linux.h: use HCI_VENDOR_PKT response to retrieve device id
* sys/linux/dev_vhci.txt: fix structs of inquiry and report packets
* executor/common_linux.h: remove unnecessary return statement and check vendor_pkt read size
* executor/common_linux.h: remove unnecessary return statement and check vendor_pkt read size
* sys/linux/dev_vhci.txt: pack extended_inquiry_info_t
* sys/linux/l2cap.txt: add l2cap_conf_opt struct
* executor/common_linux.h: just fill bd addr will 0xaa
* executor/common_linux.h: just fill bd addr will 0xaa
It is hard for the fuzzer to generate correct programs using mmap calls
with fuzzer-provided mmap length. This wrapper ensures correct length
computation.
They don't contain enough context to be useful.
This change makes `testdata/openbsd/report/0` to be treated as
corrupted, which is desirable since it also lacks context.
* sys/linux: enhanced descs for io_uring
Introduced pseudo-call "syz_io_uring_put_sqes_on_ring()" for writing
submission queue entries (sqes) on sq_ring, which was obtained by
mmap'ping the offsets obtained from io_uring_setup().
Added descriptions for io_ring_register operations that were missing
earlier.
Did misc changes to adapt the descriptions for the updates on the
io_uring subsystem.
* pkg/host: add io_uring pseudo-syscall
* executor/common_linux.h: fix issues with io_uring pseudo-syscall
* executor: fixed io_uring offset computation
* executor: fixes and refactorings in syz_io_uring_submit()
* executor: added syz_io_uring_complete() pseudo-syscall for io_uring
* sys/linux: added descriptions for io_uring operations
Each operation requires a different struct io_uring_sqe set up. Those
are described to be submitted to the sq ring.
* executor: use uint32 instead of uint32_t
* executor: remove nonfailing from pseudo-calls
* sys/linux: fix io_uring epoll_ctl sqe
* prog: fix TestTransitivelyEnabledCallsLinux()
The newly introduced syscall, syz_io_uring_submit$IORING_OP_EPOLL_CTL,
uses fd_epoll. Adapt TestTransitivelyEnabledCallsLinux() to account for
this.
* sys/linux: add IORING_OP_PROVIDE_BUFFERS and IORING_OP_REMOVE_BUFFERS
* sys/linux: fix IORING_OP_WRITE_FIXED and IORING_OP_READ_FIXED
addr and len are for the buffer located at buf_index
* sys/linux: io_uring: use reg. bufs for READ, READV, RECV, RECVMSG
As a result, IOSQE_BUFFER_SELECT_BIT is included in the iosqe_flags.
* sys/linux: io_uring: misc fixes
* sys/linux: io_uring: add IORING_SETUP_ATTACH_WQ
* executor: refactorings on io_uring pseudo syscalls
* sys/linux: io_uring: fix desc for params.cq_entries
* executor: fix SQ_ARRAY_OFFSET computation
This is required with the fix in io_uring kernel code.
https://lore.kernel.org/io-uring/CACT4Y+bgTCMXi3eU7xV+W0ZZNceZFUWRTkngojdr0G_yuY8w9w@mail.gmail.com/T/#t
* executor: added pseudosyscall syz_io_uring_cq_eventfd_toggle()
The usage of cq_ring->flags is only for manipulating
IORING_CQ_EVENTFD_DISABLED bit. This is achieved by a pseudo-syscall,
which toggles the bit.
* executor: added pseudocall syz_io_uring_put_ring_metadata
Removed syz_io_uring_cq_eventfd_toggle() and introduced
syz_io_uring_put_ring_metadata() instead. We have many pieces of
metadata for both sq_ring and cq_ring, for which we are given the
offsets, and some of are not supposed to be manipulated by the
application. Among them, both sq and cq flags can be changed. Both valid
and invalid cases might cause interesting outcomes. Use the newly
introduced pseudo syscall to manipulate them randomly while also
manipulating the flags to their special values.
* executor: added pseudo-syscall syz_memcpy_off
Removed syz_io_uring_put_ring_metadata() and instead added a much more
generic pseudo systemcall to achieve the task. This should benefit other
subsystems as well.
* sys/linux: refactored io_uring descriptions
syz_io_uring_submit() is called with a union of sqes to reduce
duplication of other parameters of the function.
io_uring_sqe is templated with io_uring_sqe_t, and this template type is
used to describe sqes for different ops.
The organization of io_uring.txt is changed.
* sys/linux: io_uring: improved descs to utilize registered files
The files are registered using
io_uring_register$IORING_REGISTER_FILES(). When IOSQE_FIXED_FILE_BIT is
enabled in iosqe_flags in sqe, a variety of operations can use those
registered files using the index of the file instead of fd.
Changed the sqe descriptions for the eligible operations to utilize
this.
* sys/linux: io_uring: improved the descs to utilize personality_id in sqes
A personality_id can be registered for a io_uring fd using
io_uring_register$IORING_REGISTER_PERSONALITY(). This id can be utilized
within sqes. This commit improves the descs for io_uring to utilize it.
In addition, the descriptions for the misc field in io_uring_sqe_t is
refactored as most are shared among sqes.
* sys/linux: io_uring: utilized cqe.res
io_uring_cqe.res is used to carry the return value of operations
achieved through io_uring. The only operations with meaningful return
values (in terms of their possible usage) are openat and openat2. The
pseudo-syscall syz_io_uring_complete() is modified to account for this
and return those fds. The description for sqe_user_data is splitted into
two to identify openat and non-openat io_uring ops.
IORING_OP_IOCTL was suggested but never supported in io_uring. Thus, the
note on this is removed in the descriptions.
tee() expects pipefds, thus, IORING_OP_TEE. The descriptions for the
pipe r/w fds are written as ordinary fd. Thus, in the description for
IORING_OP_TEE, which is io_uring_sqe_tee, fd is used in the place where
pipefds are expected. The note on this is removed in the descriptions.
* sys/linux/test: added test for io_uring
This is not tested yet.
* sys/linux/test: fixed the test for io_uring
The changes successfully pass the sys/linux/test/io_uring test.
sys/linux/io_uring.txt: sq_ring_ptr and cq_ring_ptr are really the same.
Thus, they are replaced with ring_ptr.
executor/common_linux.h: thanks to io_uring test, a bug is found in
where the sq_array's address is computed in syz_io_uring_submit().
Fixed. In addition, similar to the descriptions, the naming for the
ring_ptr is changed from {sq,cq}_ring_ptr to ring_ptr.
* sys/linux: io_uring: misc fixes
* sys/linux: io_uring: changed the sqe_user_data enum
Used a smaller range to ease the collisions. Used comperatively unique
and magic numbers for openat user_data to avoid thinking as if the cqe
belongs to openat while the user_data is coming from some random
location.
* pkg/host: added checks for io_uring syscall
* pkg/host: fixed checks for io_uring syscall
* sys/linux: fixed io_uring test
GCC10 fails to build the code with errors:
executor/common_kvm_amd64.h:143:64: error: ‘gate.kvm_segment::type’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
executor/common_kvm_amd64.h:143:56: error: ‘gate.kvm_segment::base’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
Replace 'case 6' with 'case 5' since 'i % 6' results in [0..5].
Signed-off-by: Denis Efremov <efremov@linux.com>
It seems that different gcc's have different defaults for ASAN flags.
Some fail with:
run.go:67: nonfailing none/thr/repeat : FAIL: run 0: call 0 is not executed
ASAN:DEADLYSIGNAL
=================================================================
==67143==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7f7b0befb0a2 bp 0x7f7b0cfdad10 sp 0x7f7b0cfda490 T1)
#0 0x7f7b0befb0a1 (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x8d0a1)
#1 0x404bbb in syz_compare(long, long, long, long) (/tmp/utor355225125.0+0x404bbb)
#2 0x403cdf in execute_call(thread_t*) (/tmp/utor355225125.0+0x403cdf)
#3 0x404006 in worker_thread(void*) (/tmp/utor355225125.0+0x404006)
#4 0x7f7b0bc584a3 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x74a3)
#5 0x7f7b0b99ad0e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xe8d0e)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x8d0a1)
Thread T1 created by T0 here:
#0 0x7f7b0be9ef59 in __interceptor_pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x30f59)
#1 0x406117 in execute_one() (/tmp/utor355225125.0+0x406117)
Currently we sprinkle NONFAILING all over pseudo-syscall code,
around all individual accesses to fuzzer-generated pointers.
This is tedious manual work and subject to errors.
Wrap execute_syscall invocation with NONFAILING in execute_call once instead.
Then we can remove NONFAILING from all pseudo-syscalls and never get back to this.
Potential downsides: (1) this is coarser-grained and we will skip whole syscall
on invalid pointer, but this is how normal syscalls work as well,
so should not be a problem; (2) we will skip any clean up (closing of files, etc)
as well; but this may be fine as well (programs can perfectly leave open file
descriptors as well).
Update #1918
We set HostFuzzer for all test targets b/c in some contexts
they needed the same special behavior as real HostFuzzer targets
(e.g no checking enabled syscalls).
However, in some other contexts they don't the same special
behavior as real HostFuzzer targets. For example, pkg/ipc
does rate limiting for HostFuzzer and pkg/runtest don't run
C tests for HostFuzzer.
Add special case for test targets in pkg/host,
and don't set HostFuzzer for all test targets
(keep it for one target for better coverage).
Add Timeout config param for bisections.
Specify timeout in syz-ci as 8h based on the following data.
Out of 1049 cause bisections that we have now:
- 891 finished under 6h (84.9%)
- 957 finished under 8h (91.2%)
- 980 finished under 10h (93.4%)
- 989 finished under 12h (94.3%)
- 1011 finished under 18h (96.3%)
- 1025 finished under 24h (97.7%)
There is also a significant increase in errors/inconclusive bisections after ~8h.
Out of 4075 fix bisections:
- 4015 finished under 6h (98.5%)
- 4020 finished under 8h (98.7%)
- 4026 finished under 10h (98.8%)
- 4032 finished under 12h (98.9%)
Significant increase in errors starts after ~12h.
The current timeout also take into account that bisection jobs
compete with patch testing jobs (it's bad delaying patch testing).
When/if bisection jobs don't compete with patch testing,
it makes sense to increase this to 12-24h.
Fixes#1923
Currently we sandbox all repos b/c we assumed
that all builds are also sandboxes. But this causes
havoc for bisection/patch testing b/c syzkaller build
is not actually sandboxed anywhere. Build creates
root-owned files and then git can't do anything with them
but don't report errors either:
$ git checkout 8eda0b95 && echo OK
error: unable to unlink old 'sys/linux/gen/386.go': Permission denied
error: unable to unlink old 'sys/linux/gen/ppc64le.go': Permission denied
...
HEAD is now at 8eda0b957e
OK
We trust own sources and we don't test syzkaller patches,
so don't sandbox syzkaller repos.
