Commit Graph

134 Commits

Author SHA1 Message Date
Dmitry Vyukov
ba1af74335 Makefile: fix coverage collection condition 2018-12-31 13:14:11 +01:00
Dmitry Vyukov
70b2a9e855 Makefile: collect coverage report for codecov.io 2018-12-31 12:51:07 +01:00
shankarapailoor
6a60a19530 tools/syz-trace2syz: add tool to convert strace output to programs
* fixing weird merge error

* fixing presubmit

* fixing presubmit

* removing parsing code because of -Xraw option

* fix presubmit

* update

* deleting vma_call_handlers as we are currently skipping most vma calls. This simplifies memory_tracker as we don't need to keep track of vma allocations

* removing custom handling of bpf_instruction union

* removing ifconf parsing

* update

* removed all expression types and replaced them with constant types. removing ipv6_addr parsing while -Xraw is getting fixed. Removing constants.go

* removing ipv6 parsing

* presubmit

* moving direction check from ipv4_addr out to genUnion

* removing code that parses kcov

* removing redundant test

* removing custom code in generate unions to fill ipv4_addr

* proggen: changing order of imports to make external packages import first

fixing presubmit

* changing log messages to lower case to be consistent with other packages.

* removing pointer type and simplifying memory_tracker

removing comment

* moving context and return_cache to seaparate files

* deleting default argument generation when we should probably throw an error
2018-12-06 16:25:37 +01:00
Dmitry Vyukov
c3ff1810cf Makefile: constrain gometalinter memory usage
gometalinter gets OOM-killed on travis:
https://travis-ci.org/google/syzkaller/jobs/462546388

I've run 4 experiments locally and this combination
seems to make memory usage slightly better:

GOGC=50 GOMAXPROCS=1 TIME="%e %M" time gometalinter.v2 ./...
114.35 8274556
122.54 8293580
117.85 8349960

GOGC=50 GOMAXPROCS=1 TIME="%e %M" time gometalinter.v2 ./...
147.45 7512512
150.13 6820408
149.26 7909636

GOGC=100 GOMAXPROCS=2 TIME="%e %M" time gometalinter.v2 ./...
109.73 8332476
114.91 8563776
114.54 8478368

GOGC=50 GOMAXPROCS=2 TIME="%e %M" time gometalinter.v2 ./...
145.43 6726144
147.40 8247952
155.56 7815656
2018-12-03 13:29:57 +01:00
Dmitry Vyukov
13ab4beeef syz-manager: modernize web UI
1. Use dashboard style.
2. Allow sorting of tables.
3. Show old crashes in grey.
4. Use tables instead of text output for more pages.
5. Show corpus inputs on a separate page to allow copy-pasting.
6. Use standard JS sorting instead of custom bubble sort (much faster).
7. Fix off-by one in table sorting.

Fixes #694
2018-11-22 14:36:32 +01:00
Dmitry Vyukov
adf636a83b tools/syz-fmt: allow to reformat all OSes at once
We forgot trusty in Makefile.
Fix this once and for all by formatting all known targets.
2018-11-17 11:46:48 -08:00
Dmitry Vyukov
d1a8851085 sys/targest: introduce target.BuildOS
We can't cross-compile native binaries from just any OS to any other.
For most OSes we can do only native compilation.
Some can only be compiled from linux.
To date we avoided this problem completely (mostly assumed linux build OS).
Make this notion of what can build what explicit.
2018-11-17 11:42:22 -08:00
Dmitry Vyukov
0d872150de sys: initial trusty support
Trusty is a set of software components supporting
a Trusted Execution Environment (TEE) on mobile devices.

https://source.android.com/security/trusty

Add syscall descriptions and some boilerplate.
2018-11-07 17:52:06 -08:00
Greg Steuck
7df9db2eb2 Conditionally enable test -race when supported on the OS. 2018-10-29 12:28:37 +01:00
Julia Hansbrough
0f3e026191 Include generate_fidl as part of the extract step.
Anytime we're running `make extract/generate` with Fuchsia as a target, we
probably want to update FIDL descriptions too, so we don't fall behind
what upstream is up to.

This includes generate_fidl as part of the `make extract` workflow.

Also, corrects fidlgenPath to properly find the path for ARM targets (we
only have host_x64 right now).
2018-10-02 20:47:12 +02:00
Dmitry Vyukov
bc83c27668 Makefile: test openbsd build 2018-09-20 11:21:57 +02:00
Dmitry Vyukov
02d481205b Makefile: format openbsd syscall descriptions 2018-09-20 11:21:51 +02:00
Dokyung Song
e30d3b5240 sys/fuchsia: add Go script that generates fidl descriptions 2018-09-06 21:18:15 +02:00
Dmitry Vyukov
49312e6d5e Revert "Makefile: Enable working with symlinked directories"
This reverts commit 7ce4cd3a7e.

This breaks syzbot as it executes "cd " which gets it into
a completely unrelated dir and then it tries to format
everything there as Go code.
2018-09-05 11:34:56 +02:00
Zach Riggle
7ce4cd3a7e Makefile: Enable working with symlinked directories
Resolves #698
2018-08-30 22:04:51 -07:00
Dmitry Vyukov
2763e04c22 tools/syz-runtest: add tool for program unit testing
The tool is run as:

$ syz-runtest -config manager.config

This runs all programs from sys/*/test/* in different modes
on actual VMs and checks results.

Fixes #603
2018-08-03 21:08:02 +02:00
Dmitry Vyukov
78e3ad98f6 sys/test: add more tests
Add syz_errno syscall which sets errno to the argument,
and add a test with different errno values.
This mostly tests the testing infrastructure itself.

Add syz_compare syscall which compare two blobs,
this can be used for testing of argument memory layout.

Implement syz_mmap and fix Makefile to allow building syz-execprog for test OS.
Useful for debugging.

Update #603
2018-08-03 19:53:14 +02:00
Dmitry Vyukov
d9a893a554 Makefile: don't compile all targets into target binaries
Currently target binaries contain support for all OS/arch combinations.
However, obviously a fuchsia target binary won't test windows.
For target binaries we need support only for a single target
(with the exception of 386/arm target in amd64/arm64 binaries).
So compile in only _the_ target into target binaries.
This reduces akaros/amd64 fuzzer binary from 33 to 7 MB
and execprog from 28 to 2 MB.
2018-08-02 19:07:22 +02:00
Dmitry Vyukov
788351b1b0 pkg/csource: rewrite gen.sh in Go
Shell files cause portability problems.
On Linux it's hard to install /bin/sh,
/bin/bash is not present on *BSD.
Any solution is hard to test on Darwin.
Don't even want to mention Windows.

Just do it in Go.
2018-07-27 10:22:23 +02:00
Dmitry Vyukov
1ca246b8b2 Makefile: fix build 2018-07-25 08:39:05 +02:00
Dmitry Vyukov
da1670a69d Makefile: install /bin/sh
Installs /bin/sh required for pkg/csource/gen.sh
2018-07-25 08:36:42 +02:00
Dmitry Vyukov
d1ed573636 Makefile: disable 32-bit executor build
Fails on travis...
2018-07-24 14:04:43 +02:00
Dmitry Vyukov
9fe4bdc5f1 executor: overhaul
Make as much code as possible shared between all OSes.
In particular main is now common across all OSes.
Make more code shared between executor and csource
(in particular, loop function and threaded execution logic).
Also make loop and threaded logic shared across all OSes.
Make more posix/unix code shared across OSes
(e.g. signal handling, pthread creation, etc).
Plus other changes along similar lines.
Also support test OS in executor (based on portable posix)
and add 4 arches that cover all execution modes
(fork server/no fork server, shmem/no shmem).

This change paves way for testing of executor code
and allows to preserve consistency across OSes and executor/csource.
2018-07-24 12:04:27 +02:00
Dmitry Vyukov
f85123c954 Makefile: add akaros support 2018-07-05 13:04:37 +02:00
Dmitry Vyukov
bdc18a387e Makefile: format C sources before generating Go
If executor common headers are unformatted then go generate pkg/csource
will embed unformatted headers. Then format_cpp will re-format
only executor headers, but not embed versions. As the result
pkg/csource will end up with wrong embed headers.

Format C sources before generating Go to fix this.
2018-06-30 17:26:22 +02:00
Dmitry Vyukov
4cd4cbd625 Makefile: don't use target go to build build binaries
Build of host Go binaries is broken on fuchsia,
because we use target toolchain and it creates broken binaries.
Introduce HOSTGO and use it for host binaries.
2018-06-30 17:20:17 +02:00
Dmitry Vyukov
5adb8b7437 Makefile: simplify fuchsia Go build
$(SOURCEDIR)/scripts/devshell/go seems to work out of the box.
2018-06-29 11:16:50 +02:00
Dmitry Vyukov
eecefab32c Makefile: fix fuchsia build 2018-06-26 18:59:07 +02:00
Dmitry Vyukov
94ef62054c pkg/build: move from pkg/kernel
Rename pkg/kernel to pkg/build and prepare for multi-OS support.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
2a075d57ab pkg/report: allow to specify suppressions per OS
Currently all (linux-specific) suppressions are hardcoded in mgrconfig.
This is very wrong. Move them to pkg/report and allow to specify per OS.
Add gvisor-specific suppressions.
This required a bit of refactoring. Introduce mgrconfig.KernelObj finally.
Make report.NewReporter and vm.Create accept mgrconfig directly
instead of passing it as multiple scattered args.
Remove tools/syz-parse and it always did the same as tools/syz-symbolize.
Simplify global vars in syz-manager/cover.go.
Create reporter eagerly in manager. Use sort.Slice more.
Overall -90 lines removed.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
e0e534c6c2 docs: update fuchsia instructions 2018-06-06 20:55:40 +02:00
Dmitry Vyukov
6479ab2a75 Makefile, sys/targets: move all native compilation logic to sys/targets
We currently have native cross-compilation logic duplicated
in Makefile and in sys/targets. Some pieces are missed in one
place, some are in another. Only pkg/csource knows how to check
for -static support.

Move all CC/CFLAGS logic to sys/targets and pull results in Makefile.

This should make Makefile work on distros that have broken x86_64-linux-gnu-gcc,
now we will use just gcc. And this removes the need to define NOSTATIC,
as it's always auto-detected.

This also paves the way for making pkg/csource work on OSes other than Linux.
2018-06-06 10:02:03 +02:00
Dmitry Vyukov
c992b76711 Makefile: allow to extract consts only from specified files 2018-05-18 09:48:49 +02:00
Dmitry Vyukov
d3a7c28ef1 Makefile: don't run gometalinter in parallel
apt-get update seems to heal missing packages.
But gometalinter failed due to OOM:
https://travis-ci.org/google/syzkaller/jobs/378313344
Don't run it in parallel with other tests.
2018-05-13 11:59:44 +02:00
Dmitry Vyukov
c05b619dab Makefile: print uname in install_prerequisites
Travis is still broken with:

E: Unable to locate package gcc-aarch64-linux-gnu
E: Unable to locate package gcc-arm-linux-gnueabihf
E: Unable to locate package gcc-powerpc64le-linux-gnu

https://travis-ci.org/google/syzkaller/jobs/378034948

Let's at least see what's OS is there and what gcc packages present.
2018-05-13 11:36:13 +02:00
Dmitry Vyukov
e726f42b64 Makefile: don't fail on unknown packages
Travis CI started failing with:

E: Unable to locate package gcc-aarch64-linux-gnu
E: Unable to locate package gcc-arm-linux-gnueabihf
E: Unable to locate package gcc-powerpc64le-linux-gnu
make: *** [install_prerequisites] Error 100

https://travis-ci.org/google/syzkaller/jobs/377740347

Let's try to ignore these packages.
2018-05-12 10:10:02 +02:00
Dmitry Vyukov
08a0ef9cb0 Makefile: fix deps
syz-sysgen depends on prog which depends on pkg/ifuzz.
Currently we concurrently regenerate pkg/ifuzz and build
syz-sysgen, which can cause failures.
Make pkg/ifuzz dependent on syz-sysgen, so that it is
built before we regenerate ifuzz.
2018-05-05 14:27:27 +02:00
Dmitry Vyukov
b438ff2328 Makefile: parallelize
Use -j by default and parallelize targets as much as possible.
2018-05-04 20:07:51 +02:00
Theodore Ts'o
06db3cec94 Makefile: fix build in an i386 debian chroot
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2018-04-30 09:20:55 +02:00
Dmitry Vyukov
729f261057 Makefile: fix install_prerequisites
apt-get complains if all these packages are installed at once.
2018-04-02 20:09:30 +02:00
Dmitry Vyukov
d5473a248e Makefile: remove obsolete package
This package is not present on latest debian versions.
2018-03-31 16:31:48 +02:00
Julia Hansbrough
f63eeee99f fuchsia: Update syzkaller to build with current Fuchsia API. (#543)
* fuchsia: Fix the `extractor` tool.

The include path in Zircon has changed; updated syz-extract/fuchsia.go
to include this, and re-ran extract to get updated *.const files.

* fuchsia: Update syzkaller to build with current Fuchsia API.

Fuchsia doesn't have a stable API right now, so alas, this will probably
continue to change until that's nailed down.

But, useful to get this up-to-date at least.

Relevant notes:
* zx_channel_call_finish and _retry aren't technically public; leave
them out until we have a less-cludgy way to expose them
* musl supports setjmp/longjmp but not _setjmp/_longjump
* remove some unsupported syscalls
* update the build invocation
2018-03-21 10:26:33 +01:00
Dmitry Vyukov
e20831f6ce
Makefile: enable gometalinter on travis (#537) 2018-03-20 13:26:51 +01:00
Julia Hansbrough
c8b3f7c1d1 Makefile: Enable CGO and set ZIRCON_BUILD_DIR for Fuchsia targets.
While CGO should be disabled by default, it is necessary when building
FUCHSIA.

Additionally, the Fuchsia Go toolchain expects ZIRCON_BUILD_DIR to be
set (
https://github.com/fuchsia-mirror/third_party-go/blob/master/misc/fuchsia/gccwrap.sh
), so we set it here.
2018-02-15 07:43:44 +01:00
Dmitry Vyukov
e67d44e011 executor: compile with -O2
We don't frequently debug it and it does some intensive computations
on coverage, so no reason to not compile with -O2.
2018-02-10 12:14:13 +01:00
Julia Hansbrough
a1bc9d40ad syz/syz-extract: Update Fuchsia extractor with current Fuchsia paths.
A change in Zircon a while back moved around where sysroots are located
in Fuchsia; this update will allow for proper extraction.
2018-02-04 12:21:23 +01:00
Dmitry Vyukov
4198e58824 vm/vmimpl: unbreak freebsd build
Also add freebsd/netbsd host build to presubmit.
2018-01-16 06:41:23 +01:00
Ed Maste
81fe66b42a Makefile: fix git "modified tree" test
Makefile falsely detected a modified git tree when the first ifeq
argument was unquoted. Also switch to the $(shell ... style for
consistency.

As requested, add myself to AUTHORS and CONTRIBUTORS.
2017-12-21 13:53:39 +01:00
Dmitry Vyukov
90a46995a8 Makefile: fix Android builds
There are 2 known problems with current Android support:
1. It does not work with newer NDK
(happens on every NDK update).
2. Dynamic Go binaries do not start on Android emulator.

Drop special Android support and just build static linux binaries.

For context see:
https://groups.google.com/forum/#!msg/syzkaller/etg1ZJmTMzg/NYE-yjxxAQAJ
https://groups.google.com/d/msg/syzkaller/8KjCYWslTFY/1oTXn5tTAgAJ

Fixes #478
2017-12-20 10:52:19 +01:00
Dmitry Vyukov
019cf5f235 sys: move test syscalls to a separate target
We have them in linux solely for historical reasons.

Fixes #462
2017-12-17 11:39:14 +01:00