This commit adds a new attribute to syzkaller targets that tells
syzkaller how to invoke the syz-executor command.
Some systems, like Fuchsia, are now building syz-executor as part of the
build, and there is no need to copy it over, or to run it from `/tmp`.
In fact, that might stop working at some time in the future in Fuchsia.
All places that used to copy syz-executor into the target machine will
now check for the SyzExecutorCmd flag, and won't copy it if the flag is
set.
fops_probe utility helps to understand what file_operations callbacks
are attached to a particular file. Requries KCOV and KALLSYMS.
Build with:
g++ tools/fops_probe/fops_probe.cc -Wall -static -o fops_probe
Then copy the binary to target machine and run as:
./fops_probe /dev/fb0
You should see output similar to:
ffffffff81bcccb9 vfs_read
................
ffffffff83af85c3 fb_read
ffffffff83b52af5 cirrusfb_sync
ffffffff81bcd219 vfs_write
................
ffffffff83af7fe2 fb_write
ffffffff83b52af5 cirrusfb_sync
ffffffff81c1b745 do_vfs_ioctl
ffffffff83af7ea9 fb_ioctl
ffffffff81a4ea44 do_mmap
................
ffffffff83af716c fb_mmap
which allows to understand what callbacks are associated with /dev/fb0.
These used to be required to build Fuchsia's Go toolchain to build
syz-fuzzer to run on Fuchsia. However, Fuchsia now uses HostFuzzer
mode, so only syz-executor (written in C) needs to run on Fuchsia; all
Go code runs on the Host OS (currently Linux).
Detect bisection to merge commits and to commits that don't affect
kernel binary (comments, other arches, whitespaces, etc).
Such bisections are not reported in emails (but shown on web).
Update #1271
* Modify Run() to return the commit object when a failure occurs on the
oldest tested release(for BisectCause) and latest tested
release(for BisectFix).
* Modify tests to check the returned commit object.
In case there is a fixed pci devlink handle "pci/pci/0000:00:10.0"
on the system (initial network namespace), it is moved to a working
network namespace.
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
All callers of EnvForCommit need the compiler path,
so move this logic into EnvForCommit to avoid duplication.
Also simplifies tests because test impl can now return
an empty compiler (which should be unused).
(note: incomplete change)
Refactor existing code as follows:
* Move reusable test utility functions from git_repo_test.go to
pkg/vcs/test_util.go and make them exported.
* Split Run() into Run()+runImpl().
* Change type of bisect.go:env.inst to `instance.BuilderTester`.
Change usage inside syz-testbuild/testbuild.go accordingly.
* Move most of linux.PreviousReleaseTags() into vcs/git.go as
git.previousReleaseTags().
* Allow build.CompilerIdentity to be mocked.
Introduce the following changes:
* instance.BuilderTester is an interface with methods
BuildSyzkaller()
BuildKernel()
Test()
NewEnv() now returns this interface.
* type testEnv implements instance.BuilderTester.
* type testBuilder implements builder interface. Add a entry into table
inside pkg/build/build.go:getBuilder() to return testBuilder object.
The usage of current create-image.sh:
```
./create-image.sh -d=stretch -f=minimal --add-perf
The options are in the following:
-d, --distribution Set on which debian distribution to create
-f, --feature Check what packages to install in the image, options are minimal, full
-s, --seek Image size (MB), default 2048 (2G)
-p, --add-perf Add perf support with this option enabled
The default image size may too small (2G) in some debugging scenarios, so change it to configurable.
A new version of golangci-lint started failing trying to build syz-trace2syz
(because we already ignore some files).
Ignore the rest of files to fix the failure.
By default, the current KCSAN .config does not enable KCSAN during boot,
since we encounter races during boot which would prevent syzkaller from
ever executing.
This adds support to detect if KCSAN is available, and enables it on the
fuzzer host.
The syz-expand tools allows to parse a program and print it including all
the default values. This is mainly useful for debugging, like doing manual
program modifications while trying to come up with a reproducer for some
particular kernel behavior.
syz-symbolize did not use the common mgrconfig logic
to calculate kernel dirs. Thus KernelBuildSrc ended up empty,
+dirs were not extended to absolute paths.
Use the common mgrconfig logic to calculate kernel dirs.
Better coverage reports with hierarchical coverage information,
number of programs covering each line,
handling of partially covered lines,
links to programs covering lines.
Fixes#682
This commit removes a dependency on fuchsia's tools/devshell/contrib/go,
which was removed recently[0].
Now to build go binaries for fuchsia we are not relying on the
fuchsia.git repo at all. The code just needs a copy of the fuchsia
//third_party/go repository, the fuchsia sdk and a copy of clang.
Users should download the sdk and clang beforehand (see documentation
for links), and then run (with the correct environment variables):
```
$ make TARGETOS=fuchsia TARGETARCH=amd64 \
SOURCEDIR=$HOME/fuchsia \
FX_SDK_PATH=${HOME}/sdk/fuchsia-sdk \
CLANG_PATH=${HOME}/sdk/clang \
fuchsia_go
```
After that, they will be able to build the syzkaller go binaries by
doing:
```
$ make TARGETOS=fuchsia TARGETARCH=amd64 \
SOURCEDIR=$HOME/fuchsia \
FX_SDK_PATH=${HOME}/sdk/fuchsia-sdk \
CLANG_PATH=${HOME}/sdk/clang
```
This commits adds two scripts to tools/fuchsia, one for building go in
fuchsia (just calls the ./make.all script in fuchsia's go repo), and one
that replaces the devshell/contrib/go script.
Given that go is unsupported in fuchsia, this change might break at any
point.
[0]: https://fuchsia-review.googlesource.com/c/fuchsia/+/291631
* sys/linux: extract USB HID ids
As it turns out the HID kernel subsystem registers only one USB driver that
checks that the interface of the connected device has HID class and then looks
up its own list of vendor/device ids to find a matching driver. This means
that we currently don't generate proper vendor/device ids for USB HID devices.
This patch updates the syz-usbgen tool to also extract USB HID vendor/device
ids from a running kernel and makes the generated descriptions for HID devices
to be patched using the extracted ids.
This patch also contains some minor improvements to USB descriptions
(better HID descriptions and more replies for some USB classes/drivers).
* sys/linux: run make generate
Ensure that we have at least 1GB per Makefile job.
Go compiler/linker can consume significant amount of memory
(observed to consume at least 600MB). See #1276 for context.
Update #1276
Leak checking support was half done and did not really work.
This is heavy-lifting to make it work.
1. Move leak/fault setup into executor.
pkg/host was a wrong place for them because we need then in C repros too.
The pkg/host periodic callback functionality did not work too,
we need it in executor so that we can reuse it in C repros too.
Remove setup/callback functions in pkg/host entirely.
2. Do leak setup/checking in C repros.
The way leak checking is invoked is slightly different from fuzzer,
but much better then no support at all.
At least the checking code is shared.
3. Add Leak option to pkg/csource and -leak flag to syz-prog2c.
4. Don't enalbe leak checking in fuzzer while we are triaging initial corpus.
It's toooo slow.
5. Fix pkg/repro to do something more sane for leak bugs.
Few other minor fixes here and there.
Using a build tag to exclude files for golangci-lint
reduces memory consumption (it does not parse them).
The naive attempt with skip-dirs did not work.
So add codeanalysis build tag and use it in auto-generated files.
Update #977
The previous default must have changed resulting in apparent failure
to install siteXX.tgz. The observable symptom was:
`Package installation failed. Inspect install_log.`
Add support for generating ppc64le pseries disk images. This will create a
disk image with a PowerPC PReP boot partition at the start, and install
the IEEE1275/OpenFirmware version of GRUB.
If we ever support the powernv platform in the future we're going to have
to do something different here.
Add a command line argument to specify architecture, defaulting to amd64.
Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
This commit adds syzkaller descriptions for USB fuzzing. The descriptions in
vusb.txt are written manually and cover different kinds of USB descriptors.
The descriptions in init_vusb_ids.go are generated automanitally by the
syz-usbgen tool and contain the vendor, device and some other IDs that
map to different USB drivers.
Instead of always closing open fds (number 3 to 30) after each program,
add an options called EnableCloseFds. It can be passed to syz-execprog,
syz-prog2c and syz-stress via the -enable and -disable flags. Set the
default value to true. Also minimize C repros over it, except for when
repeat is enabled.
syz-testbuild tests kernel build/boot on releases as it will be done by pkg/bisect.
This allows to ensure that, for example, a change to kernel config won't break
build/boot on older releases and consequently won't break bisection process.
The binary needs to run under root because it creates images.
The kernel checkout given to the tool will be cleaned and used for in-tree builds.
Example invocation:
sudo syz-testbuild -kernel_src $LINUX_CHECKOUT \
-config dashboard/config/upstream-kasan.config \
-sysctl dashboard/config/upstream.sysctl \
-cmdline dashboard/config/upstream-apparmor.cmdline \
-userspace $WHEEZY_USERSPACE \
-bisect_bin $BISECT_BIN
A suitable wheezy userspace can be downloaded from:
https://storage.googleapis.com/syzkaller/wheezy.tar.gz
A set of binaries required for bisection (older compilers) can be downloaded from:
https://storage.googleapis.com/syzkaller/bisect_bin.tar.gz
A bunch of improvements after more wide bisection testing.
Improve logging.
Support returning several commits for inconclusive bisection.
Return Report with the final crash.
Remove code that was moved to pkg/vcs.
Update #501
If the fault injection flags are not provided, but the log contains a
fault injection like this:
2017/08/12 17:16:04 executing program 5 (fault-call:4 fault-nth:5):
we fail to enable fault injection in ipc.Config. Fix it.
This change makes all syz-execprog, syz-prog2c and syz-stress accept
-enable and -disable flags to enable or disable additional features
(tun, net_dev, net_reset, cgroups and binfmt_misc) instead of having
a separate flag for each of them.
The default (without any flags) behavior isn't changed: syz-execprog
and syz-stress enabled all the features (provided the runtime supports
them) and syz-prog2c disables all of them.
The usage of current create-image.sh:
```
./create-image.sh -d=stretch -f=minimal --add-perf
The options are in the following:
-d, --distribution Set on which debian distribution to create
-f, --feature Check what packages to install in the image, options are minimal, full
-p, --add-perf Add perf support with this option enabled
The default image is stretch with minimal feature, without perf.
SKIP/BROKEN distract too much attention from FAIL tests
and are not usually interesting. Add Verbose flag that
controls printing of SKIP/BROKEN tests. Enable it in
pkf/runtest/run_test.go by default and disable in tools/syz-runtest.
A line length of 79 in the ddb output does not necessarily imply that the
following line is a continuation of the current line. Since there's no way
to distinguish between ordinary and continuation lines, it could end up
corrupting the report by joining two lines that are disjoint[1].
Instead, disable line wrapping in ddb. If we want some kind of wrapping in
the future it's easier done by pkg/report.
[1] https://syzkaller.appspot.com/bug?extid=03f7377a9848d7d008c9
This ability was never used but we maintain a bunch of code for it.
syzkaller also recently learned to spoof this error code
with some ptrace magic (probably intercepted control flow again
and exploited executor binary).
Drop all of it.
We started building netbsd executor on linux.
This fails on CI since cross-compiler is missing.
But we don't want to exclude netbsd from presubmit entirely.
So check if cross-compiler is missing and just produce a warning.
This should unbreak CI and humans will see the warning.
trace2syz has trouble generating structs like sockaddr_storage_in6 which are
wrappers around other structs like sockaddr_in6 because strace just decodes the
wrapped struct (sockaddr_in6). This causes trace2syz to generate the addr field
of sockaddr_storage_in6 from the af_family field of sockaddr_in6 which triggers
an error. We now try and detect these cases and recursively generate the struct.
Right now syzkaller only supports coverage collected from the threads that
execute syscalls. However some useful things happen in background threads,
and it would be nice to collect coverage from those threads as well.
This change adds extra coverage support to syzkaller. This coverage is not
associated with a particular syscall, but rather with the whole program.
Executor passes extra coverage over the same ipc mechanism to syz-fuzzer
with syscall number set to -1. syz-fuzzer then passes this coverage to
syz-manager with the call name "extra".
This change requires the following kcov patch:
https://github.com/xairy/linux/pull/2
Add support to accurately select variants for open and openat system calls.
This is needed because in order to accurately select ioctl variants we need
to use device resource types. The device resource types can only be created
by syz_open_dev and openat variants.
genSockaddrStorage() determines the correct sockaddr_storage union option when we
cannot identify the system call variant. We used to have custom logic per system
call which was not tested and is actually unnecessary. This patch deletes that
code and adds a test to make sure there are no regressions.
strace incorrectly printed sin6_addr before sin6_flowinfo. To fix this,
trace2syz added reorderStructFields() which swapped back the order. Incorrect
orderings of structs is a bug in strace or in our descriptions so we should
fix the source rather than work around. We have submitted a patch to fix the
sockaddr_in6 decoding in strace so we can safely delete this function.
strace decodes certain arguments like sockaddr_in.sin_port or sin_addr
as hex strings under -Xraw. This is because the arguments are in network byte
order. This patch supports converting those hex strings to int args if the size
of the string is 1, 2, 4, or 8.
trace2syz used to always add a null byte to strings.
This isn't correct behavior since we may end up writing null bytes to files.
The extra byte can affect system calls like ioctl FS_IOC_ENABLE_VERITY.
We now only add the byte for filenames.
We don't properly parse octal numbers which is an issue because that's how strace decodes file permissions even under -Xraw.
We used to do so but we didn't have a test for it so as when we added support for negative numbers we broke how we parse octals.
This commit fixes how we parse octals and includes a test to catch future regressions.
Strace decodes bitsets like fd_set or sa_mask into the following structure: [a b c d e].
Before we couldn't parse these structures, but now parse them into the Group IR Type
Currently we only support canExit flag.
However there are actually 3 separate conditions:
- program can exit normally
- program can timeout (e.g. fuzzer test or runtest can't)
- program can exit with error (e.g. C test can)
Allow to specify these 3 conditions separately.
Over time we relaxed parsing to handle all kinds of invalid programs
(excessive/missing args, wrong types, etc).
This is useful when reading old programs from corpus.
But this is harmful for e.g. reading test inputs as they can become arbitrary outdated.
For runtests which creates additional problem of executing not
what is actually written in the test (or at least what author meant).
Add strict parsing mode that does not tolerate any errors.
For now it just checks excessive syscall arguments.
syz-cover generates coverage HTML report from raw coverage files.
Raw coverage files are text files with one PC in hex form per line, e.g.:
0xffffffff8398658d
0xffffffff839862fc
0xffffffff8398633f
Raw coverage files can be obtained either from /rawcover manager HTTP handler,
or from syz-execprog with -coverfile flag.
Usage:
syz-cover [-os=OS -arch=ARCH -kernel_src=. -kernel_obj=.] rawcover.file*
golint suggests to comment all blank imports.
But actually we don't need whole sys, we can import only sys/linux.
Also rename target var to prevent name shadowing.
It's used only by 2 functions: genSockaddrNetlink and genIfrIfru.
Majority of functions just accept the straceType as argument,
which looks like a much more appropriate way to pass an argument to a function.
Amusingly, both functions already accept and use the straceType as argument.
This has number of advantages:
1. Tests are readable and writable.
The current checks [1] are neither.
2. Tests are much more compact.
3. Tests verify all aspects rather than just
1 aspect of the resulting program.
4. Tests are much less fragile.
5. Any diffs in the results will be more clearly visible.
[1]
switch a := p.Calls[1].Args[0].(type) {
case *prog.ResultArg:
if a.Res != p.Calls[0].Ret {
switch a := p.Calls[1].Args[0].(type) {
case *prog.ResultArg:
pipeSecondFd := p.Calls[0].Args[0].(*prog.PointerArg).Res.(*prog.GroupArg).Inner[1]
if a.Res != pipeSecondFd {
write := p.Calls[len(p.Calls)-2]
inotifyRmWatch := p.Calls[len(p.Calls)-1]
switch a := write.Args[0].Type().(type) {
case *prog.ResourceType:
if a.TypeName != "fd" {
t.Fatalf("expected first argument of write to have type fd, got: %s", a.TypeName)
}
default:
t.Fatalf("first argument of write is not resource type: %s", a.Name())
}
switch a := inotifyRmWatch.Args[1].(type) {
case *prog.ResultArg:
b := a.Type().(*prog.ResourceType)
if b.TypeName != "inotifydesc" {
t.Fatalf("expected second argument of inotify_rm_watch to have type inoitfydesc, got: %s", b.TypeName)
}
if a.Res != p.Calls[2].Ret {
t.Fatalf("inotify_rm_watch's second argument should match the result of inotify_add_watch.")
}
}
sockaddr, ok := a.(*prog.PointerArg).Res.(*prog.GroupArg)
if !ok {
t.Fatalf("%s", a.Type().Name())
}
ipv4Addr, ok := sockaddr.Inner[2].(*prog.UnionArg)
if !ok {
t.Fatalf("expected 3rd argument to be unionArg, got %s", sockaddr.Inner[2].Type().Name())
}
optName := ipv4Addr.Option.Type().FieldName()
if !strings.Contains(optName, "rand") {
t.Fatalf("expected ip option to be random opt, got: %s", optName)
}
ip, ok := ipv4Addr.Option.(*prog.ConstArg)
if !ok {
t.Fatalf("ipv4Addr option is not IntType")
}
if ip.Val != expectedIp {
t.Fatalf("parsed != expected, %d != %d", ip.Val, expectedIp)
}
There are 2 bugs:
1. We always allocate 1 page, even if use more.
2. VMA addresses are not aligned, so most mmap-like functions fail with EINVAL.
The added test currently panics with "unaligned vma address".
Current code structuring has 2 problems:
1. parsing anything with proggen requires complex multistep dance including
- parsing data with parser
- walking the resulting tree manually and calling proggen on each
- then for each context
- calling FillOutMemory (unclear why it's not part of parsing)
- calling prog.Finalize
- checking is the program is not too large
All of this duplicated across trace2syz and tests.
And any new tests or fuzzers we will write will need to duplicate
all of this logic too.
2. As the result of this structuring, lots of proggen guts
and implementation details are exposed.
While none of the callers are actually intersted in Context details,
they are not interested in Context itself whatsoever.
What every caller wants is "here is data to parse, give me programs".
Add such function.
We already printed file name of the trace in parseTraces,
no need to print it again and again.
Consequently we don't need Filename in TraceTree.
If needed, caller can always log it before parsing,
or pass along with the TraceTree.
Prog.Finalize combines assignSizesCall, SanitizeCall and validate.
Intended for users who build own programs,
so that we don't need to expose all individual methods.
These set_robust_list and set_tid_address are issued by glibc
for every process/thread start.
Normal programs don't use them and it's unlikely we build
something interesting with them (e.g. we won't get real robust list in memory).
Skip them.
This is needed for both tools/syz-db and tools/syz-trace2syz.
Also, remove code to resolve SHA1 collisions.
Also, don't set db version as we actually want to minimize
and smash these programs like anything else
(not minimizing nor smashing them is only useful during tool testing).
* fixing weird merge error
* fixing presubmit
* fixing presubmit
* removing parsing code because of -Xraw option
* fix presubmit
* update
* deleting vma_call_handlers as we are currently skipping most vma calls. This simplifies memory_tracker as we don't need to keep track of vma allocations
* removing custom handling of bpf_instruction union
* removing ifconf parsing
* update
* removed all expression types and replaced them with constant types. removing ipv6_addr parsing while -Xraw is getting fixed. Removing constants.go
* removing ipv6 parsing
* presubmit
* moving direction check from ipv4_addr out to genUnion
* removing code that parses kcov
* removing redundant test
* removing custom code in generate unions to fill ipv4_addr
* proggen: changing order of imports to make external packages import first
fixing presubmit
* changing log messages to lower case to be consistent with other packages.
* removing pointer type and simplifying memory_tracker
removing comment
* moving context and return_cache to seaparate files
* deleting default argument generation when we should probably throw an error
* build/openbsd: minor cleanup (use tuples instead of maps)
* Grammar nits in comments.
* Simplify openbsd.Create, will defer when there's more than one error exit.
* pkg/build: Support copying kernel into GCE image
* Simple test for openbsd image copy build.
* Cleanup in case something failed before.
* Support multi-processor VMs on GCE.
* More debug
* Reformat
* OpenBSD gce image needs to be raw.
* GC
* Force format to GNU directly on Go 1.10 or newer.
* Use vmType passed as a parameter inside openbsd.go
* gofmt
* more fmt
* Can't use GENERIC.mp just yet.
* capitalize
* Copyright
We used to use len([]CallInfo) to check both, whether the slice is nil or
whether its length is zero. Since ProgInfo is not a slice, we need a
separate check for nil.
This patch add a new struct ProgInfo that for now holds info about each
call in a program []CallInfo, but in the future will be expanded with remote
coverage info. Update all the callers to use the new interface as well.
We can't cross-compile native binaries from just any OS to any other.
For most OSes we can do only native compilation.
Some can only be compiled from linux.
To date we avoided this problem completely (mostly assumed linux build OS).
Make this notion of what can build what explicit.
reportPrefixLen can become wrong after symbolization
if we symbolize any lines in the prefix.
Adjust reportPrefixLen during symbolization.
Automatic testing of this is problematic
because we would need to symbolize which requires
the object file with debug info.
Tested manually with syz-symbolize.
* Use cdn.openbsd.org not cloudflare. Install vmm-firmware explicitly.
* Use cdn.openbsd.org not cloudflare. Install vmm-firmware explicitly.
* set hw.smt=1: the underlying hw on VM is unknown so don't waste SMT.
We don't need consistent image after reboot since we always reimage.
We know of some use cases that don't use journalling, but don't know
of any that use journalling.
* tools/create-openbsd-gce-ci: use config from /syzkaller
* Use syzkaller copy of src instead of a separate one.
* Using /dev/null disk in vm.conf
* Use KVM, enable doas, no longer symlink.
* Use a dummmy.img
* Revert "Use a dummmy.img"
This reverts commit 656b24d5e4573dde5e95c6158852001c7241e65a.
