The problem is stupid: <endian.h> should be included as <sys/endian.h> on freebsd.
Pass actual host OS to executor build as HOSTGOOS and use it to figure out
how we should include this header.
syz-extract was removing certain prefixes from syscall names, but this
caused some problems:
- freebsd* prefixes are for compatibility syscalls when the syscall ABI
has changed. For instance, we have both fstat() and
freebsd11_fstat(), and it is desirable to fuzz them both.
- Stripping prefixes may leave us with undefined SYS_ constants. This
resulted in some test failures in pkg/csource, which emitted code
referencing SYS_semctl when it should have been SYS___semctl.
Fix the problem by updating syscall descriptions to match the names
given by the FreeBSD kernel. Add some new descriptions for
compatibility syscalls, fix the mknodat() description (dev_t is now 64
bits wide on FreeBSD), and remove mknod$loop, which appears to be
Linux-specific.
Garnet-layer FIDL descriptions are in a new subdirectory.
This CL changes it to the proper directory, and performed a "make
extract" shortly after.
This should fix some of the errors on the Syzkaller hub.
When we run with sandbox=none, test program can do EXT4_IOC_RESIZE_FS which
shrinks rootfs basically to 0. This breaks the machine as all tests then fail with:
failed to create temp dir: mkdir syzkaller-testdir077269498: no space left on device
This is the most common source of "lost connection" crashes overall and
they are not actually kernel bugs.
Prohibit EXT4_IOC_RESIZE_FS for now.
Alternatively we can mount tmpfs with sandbox=none.
Update #971
* sys/fuchsia: fix fidlgen issues.
This change modifies fidlgen removing fuchsia.mediacodec and renaming
zircon-ethernet to fuchsia-hardware-ethernet.
fuchsia.mediacodec has most of its functionality merged into
fuchsia.media and it's not included in the default amd64 builds.
zircon-ethernet has been renamed to fuchsia-hardware-ethernet
recently:
https://fuchsia-review.googlesource.com/c/zircon/+/237178
* docs/fuchsia: Add troubleshooting sections
This commit adds a troubleshooting section in the fuchsia syzkaller docs
that mentions how to solve fidlgen issues when fidl interfaces are
removed or renamed.
* sys/fuchsia: remove stale autogenerated files.
* sys/fucsia run make extract && make generate.
This ability was never used but we maintain a bunch of code for it.
syzkaller also recently learned to spoof this error code
with some ptrace magic (probably intercepted control flow again
and exploited executor binary).
Drop all of it.
C's \xHH hex constants in strings accept any number of hex digits
(not just 2 or 4). So later non-hex escaped chars glue to the \x construct.
Use \OOO instead as it accepts at most 3 octal digits.
1. Generate .const files for all arches.
2. Don't extract mptcp consts on android.
3. Remove mptcp includes from vnet.txt.
4. Enable CONFIG_MPTCP in syz-extract (mptcp is not like all other kernel headers).
Change TODO comments that refer to missing description
language features to NEED comments as that it's easier to grep for them.
Remove some outdated comments. Fix 2 easy fixable TODOs.
Prevent nodes that maps to an already open kcov fd from being created since
they can corrupt the coverage buffer.
Partial revert of commit 04aed72692 with some
tweaks and testing.
See discussion in PR #942.
Extend support for optional flags in sys/targets
as this flag is not supported by gcc 5.
Make flags consistent across Makefile and pkg/csource.
Kernel modules are in different directories in some cases,
so to include the headers in the module dir or other directories
the includedirs flag is added.
ex: -includedirs path1/include,path2/include
rand_addr option is the first option for the ipv6_addr union
and is just an array of 16 bytes. This option makes it easy
for trace2syz to convert ipv6 address as strace decodes
the ipv6 address as a hex string.
- Add arm64 implementations of various algorithms
(these are usually named -arm64, -neon, or -ce)
- Add missing x86 SHA-NI algorithms (sha*-ni)
- Add adiantum (new in v4.20)
- Add streebog256 (new in v4.20)
- Add streebog512 (new in v4.20)
- Add vmac64 (new in v4.19)
- Add xchacha12 (new in v4.20)
- Add xchacha20 (new in v4.20)
- Remove speck128 (was removed soon after being added to kernel)
- Remove speck64 (likewise)
- Move sm4 from hash list to cipher list
We don't specify trailing unused args for some syscalls
(e.g. ioctl that does not use its arg).
Executor always filled tailing unsed args with 0's
but pkg/csource didn't. Some such syscalls actually
check that the unsed arg is 0 and as the result failed with C repro.
We could statically check and eliminate all such cases,
but it turns out the warning fires in 1500+ cases:
a3ace5a63f/gistfile1.txt
So instead fill such args with 0's in pkg/csource too.
This adds 2 ioctls and an argument type:
- FS_IOC_ENABLE_VERITY
- FS_IOC_MEASURE_VERITY
- struct fsverity_digest
Note fs-verity on-disk structs are not yet added.
AUTO arguments can be used for:
- consts
- lens
- pointers
For const's and len's AUTO is replaced with the natural value,
addresses for AUTO pointers are allocated linearly.
This greatly simplifies writing test programs by hand
as most of the time we want these natural values.
Update tests to use AUTO.
Add bulk of checks for strict parsing mode.
Probably not complete, but we can extend then in future as needed.
Turns out we can't easily use it for serialized programs
as they omit default args and during deserialization it looks like missing args.
Over time we relaxed parsing to handle all kinds of invalid programs
(excessive/missing args, wrong types, etc).
This is useful when reading old programs from corpus.
But this is harmful for e.g. reading test inputs as they can become arbitrary outdated.
For runtests which creates additional problem of executing not
what is actually written in the test (or at least what author meant).
Add strict parsing mode that does not tolerate any errors.
For now it just checks excessive syscall arguments.
As suggested by Dmitry us a better description of the msg_flags
field, which is only used to provide information from the
kernel to the application for recvmsg() calls. This means that
the value provided is basically ignored.
Some syscalls on OpenBSD violates the ordinary SYS_ prefix convention. This is
an exhaustive enumeration of the deviations.
Regression introduced in commit 88746fdf ("pkg/csource: use defines from
sys/syscall.h on *bsd").
* OpebBSD: remove socketpair() for AF_INET and AF_INET6.
socketpair() is only supported on AF_UNIX.
* NetBSD: remove socketpair() for AF_INET and AF_INET6.
socketpair() is only supported for AF_UNIX.
* FreeBSD: remove socketpair() for AF_INET and AF_INET6.
socketpair() only supports AF_UNIX.
* Linux: remove socketpair for AF_INET and AF_INET6.
socketpair only supports AF_UNIX.
* Autogenerated files.
These are manually generated for all platforms you are not
running on. FreeBSD in this case.
* executor: rebase.
* sys/freebsd: rebase.
* sys/linux: use AF_UNIX based socketpair for nbd.
This was suggested by Dmitry.
Fixes#845
This commit fixes two issues related to the task syscalls.
The zx_task_resume syscall has been recently removed from zircon[0]. It
has been deprecated for some time already. This commit removes the
syscall.
The `ZX_EXCEPTION_PORT_UNBIND_QUIETLY` option for the syscall
`zx_task_bind_exception_port` has been removed recently as well[1]. This
commit removes that option from tasks.txt.
To test this change, I followed the procedure for building syzkaller for
fuchsia:
```shell
$ make extract TARGETOS=fuchsia SOURCEDIR=${FUCHSIADIR}
$ make generate
$ make TARGETOS=fuchsia TARGETARCH=amd64 SOURCEDIR=${FUCHSIADIR}
```
I excluded the changes from make extract. This commit only has the
generated files from make generate.
[0]: https://fuchsia-review.googlesource.com/c/zircon/+/228712
[1]: https://fuchsia-review.googlesource.com/c/zircon/+/228658
* sys/fuchsia: update vmar syscalls.
In a previous zircon commit[0], the vmar related syscalls (like
`zx_vmar_map`, `zx_vmar_protect` and `zx_vmar_allocate`) changed the
order of their parameters, making putting the flags parameter as the
second parameter, and renaming it to "options".
This commit modifies vmars.txt so that it reflects the latest state of
the syscalls in zircon. I also modified the usage in
`executor/common_fuchsia.h`
I ran make extract, make generate and compiled syzkaller to test this
change.
[0]: https://fuchsia-review.googlesource.com/c/zircon/+/168060
* sys/fuchsia run make generate
This commit is just the result of running make generate after its
parent. This regenerates the definitions for the modified VMAR syscalls.
This commit is just a run of make extract + make generate with an up to
date fuchsia tree. This commit only updates the fidl definitions, there
are some other syscalls that had changed and need to be manually updated
(that will come in a future commit).
* pkg/compiler: Add error handler in `CollectUnused`.
This commit adds an error handler for the `CollectUnused` function. The
error handler just panics on any error, but is useful for debugging.
The error handler is used any time `comp` finds an error, and if it's
missing, it will panic due to a `nil` pointer dereference. At least now
we get a better understanding of the errors.
The only user of `CollectUnused` is `sys/fuchsia/fidlgen`, which is
failing now and will be fixed in a future commit.
The output message looks like this:
```
panic: could not collect unused nodes. fidl_net-stack.txt:110:15:
unknown type zx_chan_zircon_ethernet_Device_client
```
* pkg/compiler Better error handling in CollectUnused
This commit changes the default error handler for compiler to
`ast.LoggingHandler`, meaning that if `nil` is passed as an error
handler, `LoggingHandler` will be used instead.
`CollectUnused` now returns an error if any of the subfunctions produce errors.
`fidlgen` is the only caller of `CollectUnused`, and now checks for errors
as well.
* pkg/compiler Add tests for CollectUnused
This commit adds basic tests for the CollectUnused function. There's one
test that checks that it returns the right nodes, and another one that
makes sure that it returns errors when needed.
To make the test clearer, I had to add the error handler as an explicit
parameter in `CollectUnunsed`, instead of using the default one. This
avoid printing garbage in the logs. The `TestCollectUnusedError` function
uses a nopErrorHandler to avoid printing anything.
* pkg/compiler fix presubmit warnings
This commits adds "zircon-ethernet" to the list of fidl libraries that
fidlgen would look at. This is required because recently, a change in
fuchsia[0] added a new method to the netstack interface, that allows for
adding an EthernetInterface. One of the parameters is a
zircon.ethernet.Device, so we need to import that type for fidlgen to be
able to understand fidl_net-stack.txt.
I tested this by running:
```shell
$ make extract TARGETOS=fuchsia SOURCEDIR=${FUCHSIADIR}
$ make generate
```
[0]: https://fuchsia-review.googlesource.com/c/zircon/+/220580
Some targets (NetBSD and OpenBSD) have a mmap() padding argument between the
file descriptor and offset. Make sure to omit such argument in MakePosixMmap().
Otherwise, reproduce programs will crash at runtime since the expected mapping
at 0x20000000 is not established.
Squash of:
* Doc typo
* Ported some tun related functions.
* Copy vnet.txt from linux to openbsd.
* Simplified syz_emit_ethernet and stubbed out vnet.txt.
* Undo clang-format header sorting: headers are order sensitive.
* Uniquify tap devices by pid.
* clang-format off for includes
* Happier clang-format.
* Partially revert "Uniquify tap devices by pid."
Just rely on procid magic instead of getting it from a flag.
We can't cross-compile native binaries from just any OS to any other.
For most OSes we can do only native compilation.
Some can only be compiled from linux.
To date we avoided this problem completely (mostly assumed linux build OS).
Make this notion of what can build what explicit.
Trusty is a set of software components supporting
a Trusted Execution Environment (TEE) on mobile devices.
https://source.android.com/security/trusty
Add syscall descriptions and some boilerplate.
Kernel tries to vmalloc whatever we pass as size and it's not accounted against memcg.
As the result it can lead to massive OOM kills of everything running on the machine.
Strictly saying, the same applies to finit_module with a sparse file too,
but there is no simple way to handle that.
1. Extract consts on the latest linux tree.
2. Don't manually define O_TMPFILE/O_ACCMODE/_LINUX_CAPABILITY_VERSION_N,
they are defined in kernel headers.
3. Don't use CLOCK_SGI_CYCLE as clock id, it's not implemented.
* sys/openbsd: added minherit, extended/simplified mmap.
Added a script from anton_at_openbsd.org for regenerating syscalls.
Generated by ksh ./sys/openbsd/extract-openbsd.sh
* Undo whitespace change not passing on CI
* No need for magic script.
For floppy fuzzing you need to enable:
1) CONFIG_BLK_DEV_FD in your kernel configuration
2) "cmdline": " -fda FLOPPY.img " in syzkaller configuration
Signed-off-by: Denis Efremov <efremov@linux.com>
Signed-off-by: Alexander Popov <alex.popov@linux.com>
Currently when we get target consts with target.ConstMap["name"]
during target initialization, we just get 0 for missing consts.
This is error-prone as we can mis-type a const, or a const may
be undefined only on some archs (as we have common unix code
shared between several OSes).
Check that all the consts are actually defined.
The check detects several violations, to fix them:
1. move mremap to linux as it's only defined on linux
2. move S_IFMT to openbsd, as it's only defined and used on openbsd
3. define missing MAP_ANONYMOUS for freebsd and netbsd
4. fix extract for netbsd
This avoids the issue of "android" not having any registered configurations
or syscalls / ioctls / etc, when built with GOOS=android.
This occurs when building in Google3, since --config=android_arm64 selects
the Android toolchain.
Anytime we're running `make extract/generate` with Fuchsia as a target, we
probably want to update FIDL descriptions too, so we don't fall behind
what upstream is up to.
This includes generate_fidl as part of the `make extract` workflow.
Also, corrects fidlgenPath to properly find the path for ARM targets (we
only have host_x64 right now).
We used to use gcc for test OS, but this is linux-specific.
Other OSes may use c++ or clang as main compiler, so use whatever
is the default for the host OS.
Update #712
After generating syscall description for fidl files using fidlgen, prune
all unused structs using the exact same mechanism used by the compiler's
check for unused structs. This allows the FIDL compiler to support
modular compilation; it does not need to have global knowledge of
whether each struct is used or not.
Add simple fuchsia program, the one that is run during image testing.
Fix csource errno printing for fuchsia.
Fix creation of executable files (chmod is not implemented on fuchsia).
Check that we get signal/coverage from all syscalls.
On OpenBSD, the vnode type for a device node of type S_IFMT is interpreted as
VBAD. Such vnodes often causes assertion failures inside kernel producing noisy
crashes. The goal of the OpenBSD specific SanitizeCall() is to prevent such
device nodes from being created. The S_IFMT constant is a mask covering all
possible device types, thus covering both character and block devices. Therefore
when the fuzzer generates a mknod{,at}() syscall with the S_IFMT type we cannot
known if the intent was to create a block or character device. Therefore
simplify the code to create a character device under such circumstances.
The observed failure is caused by the interaction between the OpenBSD specific
SanitizeCall() and the Unix one. The Unix sanitizer will not modify the mode
argument if it contains the S_IFMT mask. But on the second invocation when the
mode no longer contains S_IFMT it would modify it, causing a different program
to be produced. Therefore only delegate to the Unix sanitizer if the syscall is
not equal to mknod{,at}().
Regression introduces in commit b771b17e ("Add mandatory OpenBSD bits (#689)").
Currently we only generate either valid user-space pointers or NULL.
Extend NULL to a set of special pointers that we will use in programs.
All targets now contain 3 special values:
- NULL
- 0xfffffffffffffff (invalid kernel pointer)
- 0x999999999999999 (non-canonical address)
Each target can add additional special pointers on top of this.
Also generate NULL/special pointers for non-opt ptr's.
This restriction was always too restrictive. We may want to generate
them with very low probability, but we do want to generate them.
Also change pointers to NULL/special during mutation
(but still not in the opposite direction).
TestSerializeDeserializeRandom fails from time to time
because program is different after we serialize/deserialize it.
Turns out openbsd SanitizeCall is not idempotent.
Add a test for this and disable the logic for now.
* create a response handle structure for each FIDL method
* generate zx_channel_write for requests having no response
* generate zx_channel_read for events (no request but just response)
* translate FIDL union types to syzkaller union types
(see fuchsia_io_ObjectInfoHandles)
We forgot to mount binfmt_misc. Mount it. Add a test.
Increase per-call timeout, otherwise last execve timesout.
Fix csource waiting for call completion at the end of program.
Support checking "blocked"/"unfinished" flags for calls.
Support test constanints, e.g. "requires: threaded" or "requires: -sandbox=setuid".
Some improvements in tools/syz-runtest.
Update #603
If the test process is not dying after 100ms,
abort all fuse connections in the system.
This gets rid at least of simple fuse deadlocks,
let's see how well this works in all cases.
The tool is run as:
$ syz-runtest -config manager.config
This runs all programs from sys/*/test/* in different modes
on actual VMs and checks results.
Fixes#603
Add syz_errno syscall which sets errno to the argument,
and add a test with different errno values.
This mostly tests the testing infrastructure itself.
Add syz_compare syscall which compare two blobs,
this can be used for testing of argument memory layout.
Implement syz_mmap and fix Makefile to allow building syz-execprog for test OS.
Useful for debugging.
Update #603
Package runtest is a driver for end-to-end testing of syzkaller programs.
It tests program execution via both executor and csource,
with different sandboxes and execution modes (threaded, repeated, etc).
It can run test OS programs locally via run_test.go
and all other real OS programs via tools/syz-runtest
which uses manager config to wind up VMs.
Test programs are located in sys/*/test/* files.
Update #603
Currently target binaries contain support for all OS/arch combinations.
However, obviously a fuchsia target binary won't test windows.
For target binaries we need support only for a single target
(with the exception of 386/arm target in amd64/arm64 binaries).
So compile in only _the_ target into target binaries.
This reduces akaros/amd64 fuzzer binary from 33 to 7 MB
and execprog from 28 to 2 MB.
Make as much code as possible shared between all OSes.
In particular main is now common across all OSes.
Make more code shared between executor and csource
(in particular, loop function and threaded execution logic).
Also make loop and threaded logic shared across all OSes.
Make more posix/unix code shared across OSes
(e.g. signal handling, pthread creation, etc).
Plus other changes along similar lines.
Also support test OS in executor (based on portable posix)
and add 4 arches that cover all execution modes
(fork server/no fork server, shmem/no shmem).
This change paves way for testing of executor code
and allows to preserve consistency across OSes and executor/csource.
Add interface specified policies for IFLA_LINKINFO. I only added the
interfaces we create in initial setup. Remain other types of interfaces
in the TODO list.
Signed-off-by: Hangbin Liu <liuhangbin@gmail.com>
RVERSION needs to use 0xffff tag, otherwise parsing fails.
Use only tags 1 and 2. They are densely allocated,
so most of the time it's 1. And it needs to be correct in
multiple replies for mount to succeed. So limit tag space.
