Commit Graph

161 Commits

Author SHA1 Message Date
Dmitry Vyukov
b6ed147834 prog: dump orig prog if Deserialize panics
We are seeing some one-off panics during Deserialization
and it's unclear if it's machine memory corrpution or
an actual bug in prog. I leam towards machine memory corruption
but it's impossible to prove without seeing the orig program.

Move git revision to prog and it's more base package
(sys can import prog, prog can't import sys).
2020-02-21 10:22:07 +01:00
Dmitry Vyukov
135c18aadb tools: add script that checks copyright headers
Fixes #1604
2020-02-18 16:05:10 +01:00
Dmitry Vyukov
f83f92fd5e tools/syz-check: inject description compilation warnings
Currently we print them as part of `make genereate`,
but nobody reads them, too much output each time.
Don't print them in `make generate` and instead
print in syz-check, the warn files are a good mechanism
to handle "known warnings".
2019-12-20 16:45:34 +01:00
Jouni Hogander
07bfd71dac Implement basic support for MIPS64LE
Add basic stuff to enable MIPS64ELR2 target:
 - build
 - make extract
 - make generate
 - qemu execution
 - system call parsing from /proc/kallsyms
2019-12-17 11:10:52 +01:00
Dmitry Vyukov
d20ee9bd98 tools: add fops_probe utility
fops_probe utility helps to understand what file_operations callbacks
are attached to a particular file. Requries KCOV and KALLSYMS.
Build with:
	g++ tools/fops_probe/fops_probe.cc -Wall -static -o fops_probe
Then copy the binary to target machine and run as:
	./fops_probe /dev/fb0
You should see output similar to:

	ffffffff81bcccb9 vfs_read
	................
	ffffffff83af85c3 fb_read
	ffffffff83b52af5 cirrusfb_sync

	ffffffff81bcd219 vfs_write
	................
	ffffffff83af7fe2 fb_write
	ffffffff83b52af5 cirrusfb_sync

	ffffffff81c1b745 do_vfs_ioctl
	ffffffff83af7ea9 fb_ioctl

	ffffffff81a4ea44 do_mmap
	................
	ffffffff83af716c fb_mmap

which allows to understand what callbacks are associated with /dev/fb0.
2019-12-03 18:48:14 +01:00
Matthew Dempsky
f4b7ed0781 tools/fuchsia: remove custom Go toolchain scripts
These used to be required to build Fuchsia's Go toolchain to build
syz-fuzzer to run on Fuchsia. However, Fuchsia now uses HostFuzzer
mode, so only syz-executor (written in C) needs to run on Fuchsia; all
Go code runs on the Host OS (currently Linux).
2019-11-19 17:20:14 -08:00
Dmitry Vyukov
426631ddb4 Makefile: support overriding CGO_ENABLED
We generally want CGO_ENABLED=0, but cgo is required for odroid.
This change makes the following work:

make manager CGO_ENABLED=1 GOTAGS=odroid
2019-10-11 17:28:37 +02:00
Andrey Konovalov
2b854f96b1 tools: add syz-expand
The syz-expand tools allows to parse a program and print it including all
the default values. This is mainly useful for debugging, like doing manual
program modifications while trying to come up with a reproducer for some
particular kernel behavior.
2019-09-23 17:13:23 +02:00
Matthew Dempsky
fd37b39ea8 all: convert Fuchsia to use "host fuzzing" mode
Go support is not a priority for Fuchsia at the moment, so it's
preferable to use host fuzzing mode for Fuchsia like currently done
for Akaros.

This commit basically looks for all the places where there was special
logic for OS=="akaros" and extends the same logic for OS=="fuchsia".
2019-08-27 14:39:18 -07:00
Dmitry Vyukov
efe659d763 travis: add fuzzit.dev support
Based on https://github.com/fuzzitdev/example-go
Let's see if it works...
2019-07-30 14:13:44 +02:00
Marco Vanotti
1b0aeaa4e8 Makefile: build fuchsia go binaries using the sdk
This commit removes a dependency on fuchsia's tools/devshell/contrib/go,
which was removed recently[0].

Now to build go binaries for fuchsia we are not relying on the
fuchsia.git repo at all. The code just needs a copy of the fuchsia
//third_party/go repository, the fuchsia sdk and a copy of clang.

Users should download the sdk and clang beforehand (see documentation
for links), and then run (with the correct environment variables):

```
$ make TARGETOS=fuchsia TARGETARCH=amd64 \
    SOURCEDIR=$HOME/fuchsia \
    FX_SDK_PATH=${HOME}/sdk/fuchsia-sdk \
    CLANG_PATH=${HOME}/sdk/clang \
    fuchsia_go
```

After that, they will be able to build the syzkaller go binaries by
doing:

```
$ make TARGETOS=fuchsia TARGETARCH=amd64 \
    SOURCEDIR=$HOME/fuchsia \
    FX_SDK_PATH=${HOME}/sdk/fuchsia-sdk \
    CLANG_PATH=${HOME}/sdk/clang
```

This commits adds two scripts to tools/fuchsia, one for building go in
fuchsia (just calls the ./make.all script in fuchsia's go repo), and one
that replaces the devshell/contrib/go script.

Given that go is unsupported in fuchsia, this change might break at any
point.

[0]: https://fuchsia-review.googlesource.com/c/fuchsia/+/291631
2019-07-23 08:44:20 +02:00
Denis Efremov
0a8d1a965a travis: use cache
* travis: cache clang-format installation

* travis: indent

* travis: use user paths for caching

* travis: cache go build results

* travis: use system clang-format

* travis: remove custom libstdc++ installation

Fixes #1252
2019-06-25 13:55:46 +02:00
Andrew Turner
5457ef3463 sys/freebsd: Add support for fuzzing FreeBSD/i386
Add support to fuzz 32 bit FreeBSD system calls.
2019-05-29 14:28:59 +02:00
Dmitry Vyukov
22c391e195 Makefile: switch to golangci-lint
Finally!

Fixes #977
Fixes #1108
2019-05-15 11:39:31 +02:00
Dmitry Vyukov
a7383bfac1 Makefile: enable 386 executor build in presubmit
This hacky way to install 386 packages seems to unbreak executor build.
2019-05-07 18:01:24 +02:00
Dmitry Vyukov
04165d8f66 Makefile: enable arm executor build in presubmit
arm build seems to work now on Travis since we switched to xenial.
2019-05-07 18:01:24 +02:00
Dmitry Vyukov
51fc038380 .golangci.yml: update
Enable more linters.
Tune settings for existings.
Fix excludes.

Update #977
2019-04-23 17:58:54 +03:00
Dmitry Vyukov
132d7060e6 Makefile: install gnueabi arm compiler
The arm compiler was changed in e5d1b3ac44
from gnueabihf to gnueabi. Install the corresponding packages.
@Codysseus
2019-04-12 15:15:18 +02:00
Andrey Konovalov
f4a3dc9128 all: add basic USB fuzzing support
This commits implements 4 syzcalls: syz_usb_connect, syz_usb_io_control,
syz_usb_ep_write and syz_usb_disconnect. Those syzcalls are used to emit USB
packets through a custom GadgetFS-like interface (currently exposed at
/sys/kernel/debug/usb-fuzzer), which requires special kernel patches.

USB fuzzing support is quite basic, as it mostly covers only the USB device
enumeration process. Even though the syz_usb_ep_write syzcall does allow to
communicate with USB endpoints after the device has been enumerated, no
coverage is collected from that code yet.
2019-04-11 16:24:45 +02:00
Marco Vanotti
ce4e53939e Makefile: change go binary path for fuchsia. (#1102)
Recently[0][1], fuchsia moved some of the tools out of `//scripts`, into the
`//tools` directory. The go script was moved into
`//tools/devshell/contrib/go`.

This commit modifies the Makefile so that it references to the new go
binary.

[0]: https://fuchsia-review.googlesource.com/c/fuchsia/+/267708/
[1]: https://fuchsia-review.googlesource.com/c/fuchsia/+/267908/
2019-04-03 15:44:56 -07:00
Dmitry Vyukov
52a20ba400 dashboard/app: properly handle build failures
Separate kernel and syzkaller build failures.
Fix logic to understand when a build is fixed:
look if kernel/syzkaller commit changes to understand
if it's a new good build or re-upload of an old build.

Fixes #1014
2019-03-25 17:00:25 +01:00
Dmitry Vyukov
2dadc32780 Makefile: fix tidy rule
Building executor now requires GOOS/GOARCH macros.
2019-03-21 12:30:53 +01:00
Dmitry Vyukov
d23e90a7b4 all: switch to Go 1.12
Differences in code formatting between Go versions cause constant
problems for us (https://github.com/golang/go/issues/25161).
Currently we support 1.9 and 1.10. Switch to newer 1.11 and 1.12.

Fixes #1013
2019-03-14 14:55:59 +01:00
Dmitry Vyukov
4b69c3cbac pkg/runtest: make tests pass on freebsd
The problem is stupid: <endian.h> should be included as <sys/endian.h> on freebsd.
Pass actual host OS to executor build as HOSTGOOS and use it to figure out
how we should include this header.
2019-03-07 20:37:43 +01:00
Dmitry Vyukov
fa3d6b0b21 Makefile: don't cross-compile executor if cross-compiler is missing
We started building netbsd executor on linux.
This fails on CI since cross-compiler is missing.
But we don't want to exclude netbsd from presubmit entirely.
So check if cross-compiler is missing and just produce a warning.
This should unbreak CI and humans will see the warning.
2019-01-24 11:28:55 +01:00
Dmitry Vyukov
e935237c9c Makefile: enable -Wunused-const-variable
See discussion in PR #942.
Extend support for optional flags in sys/targets
as this flag is not supported by gcc 5.
Make flags consistent across Makefile and pkg/csource.
2019-01-19 19:20:39 +01:00
Dmitry Vyukov
ebacf5cb4d Makefile: fix double $(GO) $(GO) 2019-01-14 18:24:22 +01:00
Dmitry Vyukov
ba1af74335 Makefile: fix coverage collection condition 2018-12-31 13:14:11 +01:00
Dmitry Vyukov
70b2a9e855 Makefile: collect coverage report for codecov.io 2018-12-31 12:51:07 +01:00
shankarapailoor
6a60a19530 tools/syz-trace2syz: add tool to convert strace output to programs
* fixing weird merge error

* fixing presubmit

* fixing presubmit

* removing parsing code because of -Xraw option

* fix presubmit

* update

* deleting vma_call_handlers as we are currently skipping most vma calls. This simplifies memory_tracker as we don't need to keep track of vma allocations

* removing custom handling of bpf_instruction union

* removing ifconf parsing

* update

* removed all expression types and replaced them with constant types. removing ipv6_addr parsing while -Xraw is getting fixed. Removing constants.go

* removing ipv6 parsing

* presubmit

* moving direction check from ipv4_addr out to genUnion

* removing code that parses kcov

* removing redundant test

* removing custom code in generate unions to fill ipv4_addr

* proggen: changing order of imports to make external packages import first

fixing presubmit

* changing log messages to lower case to be consistent with other packages.

* removing pointer type and simplifying memory_tracker

removing comment

* moving context and return_cache to seaparate files

* deleting default argument generation when we should probably throw an error
2018-12-06 16:25:37 +01:00
Dmitry Vyukov
c3ff1810cf Makefile: constrain gometalinter memory usage
gometalinter gets OOM-killed on travis:
https://travis-ci.org/google/syzkaller/jobs/462546388

I've run 4 experiments locally and this combination
seems to make memory usage slightly better:

GOGC=50 GOMAXPROCS=1 TIME="%e %M" time gometalinter.v2 ./...
114.35 8274556
122.54 8293580
117.85 8349960

GOGC=50 GOMAXPROCS=1 TIME="%e %M" time gometalinter.v2 ./...
147.45 7512512
150.13 6820408
149.26 7909636

GOGC=100 GOMAXPROCS=2 TIME="%e %M" time gometalinter.v2 ./...
109.73 8332476
114.91 8563776
114.54 8478368

GOGC=50 GOMAXPROCS=2 TIME="%e %M" time gometalinter.v2 ./...
145.43 6726144
147.40 8247952
155.56 7815656
2018-12-03 13:29:57 +01:00
Dmitry Vyukov
13ab4beeef syz-manager: modernize web UI
1. Use dashboard style.
2. Allow sorting of tables.
3. Show old crashes in grey.
4. Use tables instead of text output for more pages.
5. Show corpus inputs on a separate page to allow copy-pasting.
6. Use standard JS sorting instead of custom bubble sort (much faster).
7. Fix off-by one in table sorting.

Fixes #694
2018-11-22 14:36:32 +01:00
Dmitry Vyukov
adf636a83b tools/syz-fmt: allow to reformat all OSes at once
We forgot trusty in Makefile.
Fix this once and for all by formatting all known targets.
2018-11-17 11:46:48 -08:00
Dmitry Vyukov
d1a8851085 sys/targest: introduce target.BuildOS
We can't cross-compile native binaries from just any OS to any other.
For most OSes we can do only native compilation.
Some can only be compiled from linux.
To date we avoided this problem completely (mostly assumed linux build OS).
Make this notion of what can build what explicit.
2018-11-17 11:42:22 -08:00
Dmitry Vyukov
0d872150de sys: initial trusty support
Trusty is a set of software components supporting
a Trusted Execution Environment (TEE) on mobile devices.

https://source.android.com/security/trusty

Add syscall descriptions and some boilerplate.
2018-11-07 17:52:06 -08:00
Greg Steuck
7df9db2eb2 Conditionally enable test -race when supported on the OS. 2018-10-29 12:28:37 +01:00
Julia Hansbrough
0f3e026191 Include generate_fidl as part of the extract step.
Anytime we're running `make extract/generate` with Fuchsia as a target, we
probably want to update FIDL descriptions too, so we don't fall behind
what upstream is up to.

This includes generate_fidl as part of the `make extract` workflow.

Also, corrects fidlgenPath to properly find the path for ARM targets (we
only have host_x64 right now).
2018-10-02 20:47:12 +02:00
Dmitry Vyukov
bc83c27668 Makefile: test openbsd build 2018-09-20 11:21:57 +02:00
Dmitry Vyukov
02d481205b Makefile: format openbsd syscall descriptions 2018-09-20 11:21:51 +02:00
Dokyung Song
e30d3b5240 sys/fuchsia: add Go script that generates fidl descriptions 2018-09-06 21:18:15 +02:00
Dmitry Vyukov
49312e6d5e Revert "Makefile: Enable working with symlinked directories"
This reverts commit 7ce4cd3a7e.

This breaks syzbot as it executes "cd " which gets it into
a completely unrelated dir and then it tries to format
everything there as Go code.
2018-09-05 11:34:56 +02:00
Zach Riggle
7ce4cd3a7e Makefile: Enable working with symlinked directories
Resolves #698
2018-08-30 22:04:51 -07:00
Dmitry Vyukov
2763e04c22 tools/syz-runtest: add tool for program unit testing
The tool is run as:

$ syz-runtest -config manager.config

This runs all programs from sys/*/test/* in different modes
on actual VMs and checks results.

Fixes #603
2018-08-03 21:08:02 +02:00
Dmitry Vyukov
78e3ad98f6 sys/test: add more tests
Add syz_errno syscall which sets errno to the argument,
and add a test with different errno values.
This mostly tests the testing infrastructure itself.

Add syz_compare syscall which compare two blobs,
this can be used for testing of argument memory layout.

Implement syz_mmap and fix Makefile to allow building syz-execprog for test OS.
Useful for debugging.

Update #603
2018-08-03 19:53:14 +02:00
Dmitry Vyukov
d9a893a554 Makefile: don't compile all targets into target binaries
Currently target binaries contain support for all OS/arch combinations.
However, obviously a fuchsia target binary won't test windows.
For target binaries we need support only for a single target
(with the exception of 386/arm target in amd64/arm64 binaries).
So compile in only _the_ target into target binaries.
This reduces akaros/amd64 fuzzer binary from 33 to 7 MB
and execprog from 28 to 2 MB.
2018-08-02 19:07:22 +02:00
Dmitry Vyukov
788351b1b0 pkg/csource: rewrite gen.sh in Go
Shell files cause portability problems.
On Linux it's hard to install /bin/sh,
/bin/bash is not present on *BSD.
Any solution is hard to test on Darwin.
Don't even want to mention Windows.

Just do it in Go.
2018-07-27 10:22:23 +02:00
Dmitry Vyukov
1ca246b8b2 Makefile: fix build 2018-07-25 08:39:05 +02:00
Dmitry Vyukov
da1670a69d Makefile: install /bin/sh
Installs /bin/sh required for pkg/csource/gen.sh
2018-07-25 08:36:42 +02:00
Dmitry Vyukov
d1ed573636 Makefile: disable 32-bit executor build
Fails on travis...
2018-07-24 14:04:43 +02:00
Dmitry Vyukov
9fe4bdc5f1 executor: overhaul
Make as much code as possible shared between all OSes.
In particular main is now common across all OSes.
Make more code shared between executor and csource
(in particular, loop function and threaded execution logic).
Also make loop and threaded logic shared across all OSes.
Make more posix/unix code shared across OSes
(e.g. signal handling, pthread creation, etc).
Plus other changes along similar lines.
Also support test OS in executor (based on portable posix)
and add 4 arches that cover all execution modes
(fork server/no fork server, shmem/no shmem).

This change paves way for testing of executor code
and allows to preserve consistency across OSes and executor/csource.
2018-07-24 12:04:27 +02:00