reactos/syzkaller
1
0
Fork 0
mirror of https://github.com/reactos/syzkaller.git synced 2024-11-26 21:00:30 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
bc15f7dbbc
syzkaller/Makefile

362 lines
14 KiB
Makefile
Raw Normal View History

makefile: cleanup - fix and reorder .PHONY targets - `make all` now builds everything, to build only essentials run `make main` - `make all-tools` rename to `make tools`
2017-07-27 14:59:09 +00:00
# Copyright 2017 syzkaller project authors. All rights reserved.
Add Makefile
2015-10-12 15:15:57 +00:00
# Use of this source code is governed by Apache 2 LICENSE that can be found in the LICENSE file.
Makefile, sys/targets: move all native compilation logic to sys/targets We currently have native cross-compilation logic duplicated in Makefile and in sys/targets. Some pieces are missed in one place, some are in another. Only pkg/csource knows how to check for -static support. Move all CC/CFLAGS logic to sys/targets and pull results in Makefile. This should make Makefile work on distros that have broken x86_64-linux-gnu-gcc, now we will use just gcc. And this removes the need to define NOSTATIC, as it's always auto-detected. This also paves the way for making pkg/csource work on OSes other than Linux.
2018-06-05 15:14:47 +00:00
# There are 3 OS/arch pairs:
Makefile: build target binaries into separate dirs We currently build binaries for all targets into bin. This makes mess in bin/ and does not allow testing of different archs. Build target binaries into bin/OS_ARCH/ subdirs. Host binaries are still built into bin/. Update #333 Update #324 Update #191
2017-09-19 09:00:40 +00:00
# - HOSTOS/HOSTARCH: pair where syz-manager will run.
# - TARGETOS/TARGETVMARCH: pair of the target OS under test.
# - TARGETOS/TARGETARCH: pair of the target test process.
#
# The last 2 differ for e.g. amd64 OS and 386 test processes (compat syscall testing).
# All pairs default to the current machine. All but BUILD can be overriden.
#
# For example, to test linux/amd64 on linux/amd64, you just run:
# make
# To test linux/arm64 from darwin/amd64 host, run:
# make HOSTOS=darwin HOSTARCH=amd64 TARGETOS=linux TARGETARCH=arm64
# To test x86 compat syscalls, run:
# make TARGETVMARCH=amd64 TARGETARCH=386
#
Makefile: fix Android builds There are 2 known problems with current Android support: 1. It does not work with newer NDK (happens on every NDK update). 2. Dynamic Go binaries do not start on Android emulator. Drop special Android support and just build static linux binaries. For context see: https://groups.google.com/forum/#!msg/syzkaller/etg1ZJmTMzg/NYE-yjxxAQAJ https://groups.google.com/d/msg/syzkaller/8KjCYWslTFY/1oTXn5tTAgAJ Fixes #478
2017-12-19 14:22:08 +00:00
# There is one special case for extracting constants for Android
# (you don't need this unless you update system call descriptions):
# make extract TARGETOS=android SOURCEDIR=/path/to/android/checkout
Makefile: build target binaries into separate dirs We currently build binaries for all targets into bin. This makes mess in bin/ and does not allow testing of different archs. Build target binaries into bin/OS_ARCH/ subdirs. Host binaries are still built into bin/. Update #333 Update #324 Update #191
2017-09-19 09:00:40 +00:00
Makefile, sys/targets: move all native compilation logic to sys/targets We currently have native cross-compilation logic duplicated in Makefile and in sys/targets. Some pieces are missed in one place, some are in another. Only pkg/csource knows how to check for -static support. Move all CC/CFLAGS logic to sys/targets and pull results in Makefile. This should make Makefile work on distros that have broken x86_64-linux-gnu-gcc, now we will use just gcc. And this removes the need to define NOSTATIC, as it's always auto-detected. This also paves the way for making pkg/csource work on OSes other than Linux.
2018-06-05 15:14:47 +00:00
define newline
Makefile: build target binaries into separate dirs We currently build binaries for all targets into bin. This makes mess in bin/ and does not allow testing of different archs. Build target binaries into bin/OS_ARCH/ subdirs. Host binaries are still built into bin/. Update #333 Update #324 Update #191
2017-09-19 09:00:40 +00:00
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
Makefile, sys/targets: move all native compilation logic to sys/targets We currently have native cross-compilation logic duplicated in Makefile and in sys/targets. Some pieces are missed in one place, some are in another. Only pkg/csource knows how to check for -static support. Move all CC/CFLAGS logic to sys/targets and pull results in Makefile. This should make Makefile work on distros that have broken x86_64-linux-gnu-gcc, now we will use just gcc. And this removes the need to define NOSTATIC, as it's always auto-detected. This also paves the way for making pkg/csource work on OSes other than Linux.
2018-06-05 15:14:47 +00:00
endef
all: replace TRAVIS env var with CI In preparation to running some tests as github actions. Both Travis and Github define CI env var, while TRAVIS is, well, too Travis-specific. Update #1699
2020-05-07 13:40:13 +00:00
ENV := $(subst \n,$(newline),$(shell CI=$(CI)\
Makefile, sys/targets: move all native compilation logic to sys/targets We currently have native cross-compilation logic duplicated in Makefile and in sys/targets. Some pieces are missed in one place, some are in another. Only pkg/csource knows how to check for -static support. Move all CC/CFLAGS logic to sys/targets and pull results in Makefile. This should make Makefile work on distros that have broken x86_64-linux-gnu-gcc, now we will use just gcc. And this removes the need to define NOSTATIC, as it's always auto-detected. This also paves the way for making pkg/csource work on OSes other than Linux.
2018-06-05 15:14:47 +00:00
SOURCEDIR=$(SOURCEDIR) HOSTOS=$(HOSTOS) HOSTARCH=$(HOSTARCH) \
TARGETOS=$(TARGETOS) TARGETARCH=$(TARGETARCH) TARGETVMARCH=$(TARGETVMARCH) \
targets: syz-extract: add Clang support for Linux builds Allow targets.go use Clang instead of the default Linux compiler by setting the SYZ_CLANG=1 env var. Doing so changes the compiler to "clang" and the linker to "ld.ldd", assuming they are in $PATH, and adds the --target and -ferror-limit CFLAGS. Target also exports KernelCompiler and KernelLinker fields now, which allows overriding the compiler and linker in the kernel make invocation. Signed-off-by: Alexander Potapenko <glider@google.com>
2020-05-14 10:12:53 +00:00
SYZ_CLANG=$(SYZ_CLANG) \
tools/syz-make: rename from syz-env syz-env will be a good, short name for the planned docker-ized build environment. So rename syz-env to syz-make. Maybe not the best name, but it's not visible basically anywhere.
2020-05-15 15:52:15 +00:00
go run tools/syz-make/make.go))
tools/syz-check: inject description compilation warnings Currently we print them as part of `make genereate`, but nobody reads them, too much output each time. Don't print them in `make generate` and instead print in syz-check, the warn files are a good mechanism to handle "known warnings".
2019-12-20 13:19:12 +00:00
# Uncomment in case of emergency.
# $(info $(ENV))
Makefile, sys/targets: move all native compilation logic to sys/targets We currently have native cross-compilation logic duplicated in Makefile and in sys/targets. Some pieces are missed in one place, some are in another. Only pkg/csource knows how to check for -static support. Move all CC/CFLAGS logic to sys/targets and pull results in Makefile. This should make Makefile work on distros that have broken x86_64-linux-gnu-gcc, now we will use just gcc. And this removes the need to define NOSTATIC, as it's always auto-detected. This also paves the way for making pkg/csource work on OSes other than Linux.
2018-06-05 15:14:47 +00:00
$(eval $(ENV))
sys/targets: better detection for missing/broken cross-compilers 1. Detect when compiler is present, but is not functioning (can't build a simple program, common for Linux distros). 2. Be more strict with skipping tests due to missing/broken compilers on CI (on CI they should work, so fail loudly if not). 3. Dedup this logic across syz-env and pkg/csource tests. 4. Add better error reporting for syz-env. Fixes #1606
2020-04-29 13:31:23 +00:00
ifneq ("$(SYZERROR)", "")
$(error $(SYZERROR))
endif
Makefile, sys/targets: move all native compilation logic to sys/targets We currently have native cross-compilation logic duplicated in Makefile and in sys/targets. Some pieces are missed in one place, some are in another. Only pkg/csource knows how to check for -static support. Move all CC/CFLAGS logic to sys/targets and pull results in Makefile. This should make Makefile work on distros that have broken x86_64-linux-gnu-gcc, now we will use just gcc. And this removes the need to define NOSTATIC, as it's always auto-detected. This also paves the way for making pkg/csource work on OSes other than Linux.
2018-06-05 15:14:47 +00:00
ifeq ("$(NCORES)", "")
tools/syz-make: rename from syz-env syz-env will be a good, short name for the planned docker-ized build environment. So rename syz-env to syz-make. Maybe not the best name, but it's not visible basically anywhere.
2020-05-15 15:52:15 +00:00
$(error syz-make failed)
Makefile: build target binaries into separate dirs We currently build binaries for all targets into bin. This makes mess in bin/ and does not allow testing of different archs. Build target binaries into bin/OS_ARCH/ subdirs. Host binaries are still built into bin/. Update #333 Update #324 Update #191
2017-09-19 09:00:40 +00:00
endif
Makefile: don't reset jobserver Re-exporting -j resets make jobserver. Don't do that. Only export -j in top level make invocation.
2020-06-19 07:27:59 +00:00
ifeq ("$(MAKELEVEL)", "0")
MAKEFLAGS += -j$(NCORES) --no-print-directory
endif
Makefile: build target binaries into separate dirs We currently build binaries for all targets into bin. This makes mess in bin/ and does not allow testing of different archs. Build target binaries into bin/OS_ARCH/ subdirs. Host binaries are still built into bin/. Update #333 Update #324 Update #191
2017-09-19 09:00:40 +00:00
Makefile, sys/targets: move all native compilation logic to sys/targets We currently have native cross-compilation logic duplicated in Makefile and in sys/targets. Some pieces are missed in one place, some are in another. Only pkg/csource knows how to check for -static support. Move all CC/CFLAGS logic to sys/targets and pull results in Makefile. This should make Makefile work on distros that have broken x86_64-linux-gnu-gcc, now we will use just gcc. And this removes the need to define NOSTATIC, as it's always auto-detected. This also paves the way for making pkg/csource work on OSes other than Linux.
2018-06-05 15:14:47 +00:00
GO := go
Makefile: don't use target go to build build binaries Build of host Go binaries is broken on fuchsia, because we use target toolchain and it creates broken binaries. Introduce HOSTGO and use it for host binaries.
2018-06-30 15:20:17 +00:00
HOSTGO := go
Makefile: Enable CGO and set ZIRCON_BUILD_DIR for Fuchsia targets. While CGO should be disabled by default, it is necessary when building FUCHSIA. Additionally, the Fuchsia Go toolchain expects ZIRCON_BUILD_DIR to be set ( https://github.com/fuchsia-mirror/third_party-go/blob/master/misc/fuchsia/gccwrap.sh ), so we set it here.
2018-02-15 02:16:27 +00:00
# By default, build all Go binaries as static. We don't need cgo and it is
# known to cause problems at least on Android emulator.
Makefile: support overriding CGO_ENABLED We generally want CGO_ENABLED=0, but cgo is required for odroid. This change makes the following work: make manager CGO_ENABLED=1 GOTAGS=odroid
2019-10-11 15:10:11 +00:00
CGO_ENABLED ?= 0
export CGO_ENABLED
Makefile: add akaros support
2018-07-05 11:04:37 +00:00
TARGETGOOS := $(TARGETOS)
TARGETGOARCH := $(TARGETVMARCH)
go.mod: vendor golangci-lint
2020-07-04 09:12:55 +00:00
export GO111MODULE=on
Makefile: avoid non-portable realpath pwd -P works great and is portable. This fixes OpenBSD ci.
2020-07-07 04:37:50 +00:00
export GOBIN=$(shell pwd -P)/bin
Makefile: Enable CGO and set ZIRCON_BUILD_DIR for Fuchsia targets. While CGO should be disabled by default, it is necessary when building FUCHSIA. Additionally, the Fuchsia Go toolchain expects ZIRCON_BUILD_DIR to be set ( https://github.com/fuchsia-mirror/third_party-go/blob/master/misc/fuchsia/gccwrap.sh ), so we set it here.
2018-02-15 02:16:27 +00:00
Makefile: build target binaries into separate dirs We currently build binaries for all targets into bin. This makes mess in bin/ and does not allow testing of different archs. Build target binaries into bin/OS_ARCH/ subdirs. Host binaries are still built into bin/. Update #333 Update #324 Update #191
2017-09-19 09:00:40 +00:00
GITREV=$(shell git rev-parse HEAD)
Makefile: fix git "modified tree" test Makefile falsely detected a modified git tree when the first ifeq argument was unquoted. Also switch to the $(shell ... style for consistency. As requested, add myself to AUTHORS and CONTRIBUTORS.
2017-12-20 20:21:19 +00:00
ifeq ("$(shell git diff --shortstat)", "")
Makefile: build target binaries into separate dirs We currently build binaries for all targets into bin. This makes mess in bin/ and does not allow testing of different archs. Build target binaries into bin/OS_ARCH/ subdirs. Host binaries are still built into bin/. Update #333 Update #324 Update #191
2017-09-19 09:00:40 +00:00
REV=$(GITREV)
else
REV=$(GITREV)+
endif
dashboard/app: properly handle build failures Separate kernel and syzkaller build failures. Fix logic to understand when a build is fixed: look if kernel/syzkaller commit changes to understand if it's a new good build or re-upload of an old build. Fixes #1014
2019-03-22 17:38:28 +00:00
GITREVDATE=$(shell git log -n 1 --format="%ad")
Makefile: build target binaries into separate dirs We currently build binaries for all targets into bin. This makes mess in bin/ and does not allow testing of different archs. Build target binaries into bin/OS_ARCH/ subdirs. Host binaries are still built into bin/. Update #333 Update #324 Update #191
2017-09-19 09:00:40 +00:00
# Don't generate symbol table and DWARF debug info.
# Reduces build time and binary sizes considerably.
# That's only needed if you use gdb or nm.
# If you need that, build manually without these flags.
prog: dump orig prog if Deserialize panics We are seeing some one-off panics during Deserialization and it's unclear if it's machine memory corrpution or an actual bug in prog. I leam towards machine memory corruption but it's impossible to prove without seeing the orig program. Move git revision to prog and it's more base package (sys can import prog, prog can't import sys).
2020-02-21 09:22:07 +00:00
GOFLAGS := "-ldflags=-s -w -X github.com/google/syzkaller/prog.GitRevision=$(REV) -X 'github.com/google/syzkaller/prog.gitRevisionDate=$(GITREVDATE)'"
dashboard/app: properly handle build failures Separate kernel and syzkaller build failures. Fix logic to understand when a build is fixed: look if kernel/syzkaller commit changes to understand if it's a new good build or re-upload of an old build. Fixes #1014
2019-03-22 17:38:28 +00:00
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
GOHOSTFLAGS := $(GOFLAGS)
GOTARGETFLAGS := $(GOFLAGS)
Makefile: build target binaries into separate dirs We currently build binaries for all targets into bin. This makes mess in bin/ and does not allow testing of different archs. Build target binaries into bin/OS_ARCH/ subdirs. Host binaries are still built into bin/. Update #333 Update #324 Update #191
2017-09-19 09:00:40 +00:00
ifneq ("$(GOTAGS)", "")
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
GOHOSTFLAGS += "-tags=$(GOTAGS)"
endif
GOTARGETFLAGS += "-tags=syz_target syz_os_$(TARGETOS) syz_arch_$(TARGETVMARCH) $(GOTAGS)"
sys/test: add more tests Add syz_errno syscall which sets errno to the argument, and add a test with different errno values. This mostly tests the testing infrastructure itself. Add syz_compare syscall which compare two blobs, this can be used for testing of argument memory layout. Implement syz_mmap and fix Makefile to allow building syz-execprog for test OS. Useful for debugging. Update #603
2018-08-03 17:48:30 +00:00
ifeq ("$(TARGETOS)", "test")
TARGETGOOS := $(HOSTOS)
TARGETGOARCH := $(HOSTARCH)
endif
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
ifeq ("$(TARGETOS)", "akaros")
TARGETGOOS := $(HOSTOS)
TARGETGOARCH := $(HOSTARCH)
Makefile: allow to build executor without -static -static is not supported on some platforms
2016-01-25 09:57:56 +00:00
endif
all: convert Fuchsia to use "host fuzzing" mode Go support is not a priority for Fuchsia at the moment, so it's preferable to use host fuzzing mode for Fuchsia like currently done for Akaros. This commit basically looks for all the places where there was special logic for OS=="akaros" and extends the same logic for OS=="fuchsia".
2019-08-20 01:23:15 +00:00
ifeq ("$(TARGETOS)", "fuchsia")
TARGETGOOS := $(HOSTOS)
TARGETGOARCH := $(HOSTARCH)
endif
sys: initial trusty support Trusty is a set of software components supporting a Trusted Execution Environment (TEE) on mobile devices. https://source.android.com/security/trusty Add syscall descriptions and some boilerplate.
2018-11-08 01:52:06 +00:00
ifeq ("$(TARGETOS)", "trusty")
TARGETGOOS := $(HOSTOS)
TARGETGOARCH := $(HOSTARCH)
endif
Makefile: build target binaries into separate dirs We currently build binaries for all targets into bin. This makes mess in bin/ and does not allow testing of different archs. Build target binaries into bin/OS_ARCH/ subdirs. Host binaries are still built into bin/. Update #333 Update #324 Update #191
2017-09-19 09:00:40 +00:00
.PHONY: all host target \
tools/syz-runtest: add tool for program unit testing The tool is run as: $ syz-runtest -config manager.config This runs all programs from sys/*/test/* in different modes on actual VMs and checks results. Fixes #603
2018-07-25 14:45:57 +00:00
manager runtest fuzzer executor \
makefile: cleanup - fix and reorder .PHONY targets - `make all` now builds everything, to build only essentials run `make main` - `make all-tools` rename to `make tools`
2017-07-27 14:59:09 +00:00
ci hub \
tools/syz-trace2syz: add tool to convert strace output to programs * fixing weird merge error * fixing presubmit * fixing presubmit * removing parsing code because of -Xraw option * fix presubmit * update * deleting vma_call_handlers as we are currently skipping most vma calls. This simplifies memory_tracker as we don't need to keep track of vma allocations * removing custom handling of bpf_instruction union * removing ifconf parsing * update * removed all expression types and replaced them with constant types. removing ipv6_addr parsing while -Xraw is getting fixed. Removing constants.go * removing ipv6 parsing * presubmit * moving direction check from ipv4_addr out to genUnion * removing code that parses kcov * removing redundant test * removing custom code in generate unions to fill ipv4_addr * proggen: changing order of imports to make external packages import first fixing presubmit * changing log messages to lower case to be consistent with other packages. * removing pointer type and simplifying memory_tracker removing comment * moving context and return_cache to seaparate files * deleting default argument generation when we should probably throw an error
2018-12-06 15:25:37 +00:00
execprog mutate prog2c trace2syz stress repro upgrade db \
tools/syz-crush: Add feature to run C-program usage: syz-crush -config=config.json creprog.c Produced output is same as syz-manager produces. New directory called reproduced_with is added that lists all the reproducers that triggered the crash. Example: ~/workdir/crashes $ tree . ├── 2a8fff76c6267ecfeaf3530a602ed48afbc4a151 │   ├── description │   ├── log0 │   ├── log1 │   ├── log2 │   ├── report0 │   ├── report1 │   ├── report2 │   └── reproduced_with │   └── 17a6ed226769660f3e5dad7b22b13466b938e290.c ├── 2b6b921457afe8cebd0a96ca40f8569e6ee95174 │   ├── description │   ├── log0 │   ├── log1 │   ├── log2 │   ├── log3 │   ├── log4 │   ├── log5 │   ├── report0 │   ├── report1 │   ├── report2 │   ├── report3 │   ├── report4 │   ├── report5 │   └── reproduced_with │   ├── 9fb2f5b37bf4428382334e336bbbb201634c7c4c.c │   └── ab27002b46d19cafb1ebb8b040f0a3b0f8f88974.c Signed-off-by: Jukka Kaartinen <jukka.kaartinen@unikie.com>
2020-06-04 12:00:05 +00:00
usbgen symbolize crush \
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
bin/syz-extract bin/syz-fmt \
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
extract generate generate_go generate_sys \
format format_go format_cpp format_sys \
.github/workflows: check commit description format Check PR commit subjects for the standard format we use "pkg/name: description of the change". Check PR commit bodies for at most 120 characters.
2020-07-31 10:07:23 +00:00
tidy test test_race check_copyright check_language check_links check_diff check_commits \
Makefile: simplify arch build tests There is not much value in the current 2 levels of indirections.
2020-07-16 13:16:06 +00:00
presubmit presubmit_parallel clean
Add Makefile
2015-10-12 15:15:57 +00:00
Makefile: build target binaries into separate dirs We currently build binaries for all targets into bin. This makes mess in bin/ and does not allow testing of different archs. Build target binaries into bin/OS_ARCH/ subdirs. Host binaries are still built into bin/. Update #333 Update #324 Update #191
2017-09-19 09:00:40 +00:00
all: host target
Makefile: don't do go install We did it for caching purposes, but now (1) go command caches everything anyway, (2) we use custom tags and flags, so result of go install is not reused anyway.
2020-04-18 12:36:05 +00:00
host: manager runtest repro mutate prog2c db upgrade
target: fuzzer execprog stress executor
Add Makefile
2015-10-12 15:15:57 +00:00
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
executor: descriptions
Makefile: don't cross-compile executor if cross-compiler is missing We started building netbsd executor on linux. This fails on CI since cross-compiler is missing. But we don't want to exclude netbsd from presubmit entirely. So check if cross-compiler is missing and just produce a warning. This should unbreak CI and humans will see the warning.
2019-01-24 10:28:55 +00:00
ifneq ("$(BUILDOS)", "$(NATIVEBUILDOS)")
sys/targest: introduce target.BuildOS We can't cross-compile native binaries from just any OS to any other. For most OSes we can do only native compilation. Some can only be compiled from linux. To date we avoided this problem completely (mostly assumed linux build OS). Make this notion of what can build what explicit.
2018-11-17 19:20:17 +00:00
$(info ************************************************************************************)
sys/targets: better detection for missing/broken cross-compilers 1. Detect when compiler is present, but is not functioning (can't build a simple program, common for Linux distros). 2. Be more strict with skipping tests due to missing/broken compilers on CI (on CI they should work, so fail loudly if not). 3. Dedup this logic across syz-env and pkg/csource tests. 4. Add better error reporting for syz-env. Fixes #1606
2020-04-29 13:31:23 +00:00
$(info Executor will not be built)
$(info Building executor for ${TARGETOS} is not supported on ${BUILDOS})
sys/targest: introduce target.BuildOS We can't cross-compile native binaries from just any OS to any other. For most OSes we can do only native compilation. Some can only be compiled from linux. To date we avoided this problem completely (mostly assumed linux build OS). Make this notion of what can build what explicit.
2018-11-17 19:20:17 +00:00
$(info ************************************************************************************)
Makefile: don't cross-compile executor if cross-compiler is missing We started building netbsd executor on linux. This fails on CI since cross-compiler is missing. But we don't want to exclude netbsd from presubmit entirely. So check if cross-compiler is missing and just produce a warning. This should unbreak CI and humans will see the warning.
2019-01-24 10:28:55 +00:00
else
ifneq ("$(NO_CROSS_COMPILER)", "")
$(info ************************************************************************************)
sys/targets: better detection for missing/broken cross-compilers 1. Detect when compiler is present, but is not functioning (can't build a simple program, common for Linux distros). 2. Be more strict with skipping tests due to missing/broken compilers on CI (on CI they should work, so fail loudly if not). 3. Dedup this logic across syz-env and pkg/csource tests. 4. Add better error reporting for syz-env. Fixes #1606
2020-04-29 13:31:23 +00:00
$(info Executor will not be built)
$(info Native cross-compiler is missing/broken:)
$(info $(NO_CROSS_COMPILER))
Makefile: don't cross-compile executor if cross-compiler is missing We started building netbsd executor on linux. This fails on CI since cross-compiler is missing. But we don't want to exclude netbsd from presubmit entirely. So check if cross-compiler is missing and just produce a warning. This should unbreak CI and humans will see the warning.
2019-01-24 10:28:55 +00:00
$(info ************************************************************************************)
else
mkdir -p ./bin/$(TARGETOS)_$(TARGETARCH)
$(CC) -o ./bin/$(TARGETOS)_$(TARGETARCH)/syz-executor$(EXE) executor/executor.cc \
pkg/runtest: make tests pass on freebsd The problem is stupid: <endian.h> should be included as <sys/endian.h> on freebsd. Pass actual host OS to executor build as HOSTGOOS and use it to figure out how we should include this header.
2019-03-07 19:36:03 +00:00
$(ADDCFLAGS) $(CFLAGS) -DGOOS_$(TARGETOS)=1 -DGOARCH_$(TARGETARCH)=1 \
-DHOSTGOOS_$(HOSTOS)=1 -DGIT_REVISION=\"$(REV)\"
Makefile: don't cross-compile executor if cross-compiler is missing We started building netbsd executor on linux. This fails on CI since cross-compiler is missing. But we don't want to exclude netbsd from presubmit entirely. So check if cross-compiler is missing and just produce a warning. This should unbreak CI and humans will see the warning.
2019-01-24 10:28:55 +00:00
endif
sys/targest: introduce target.BuildOS We can't cross-compile native binaries from just any OS to any other. For most OSes we can do only native compilation. Some can only be compiled from linux. To date we avoided this problem completely (mostly assumed linux build OS). Make this notion of what can build what explicit.
2018-11-17 19:20:17 +00:00
endif
Makefile: don't generate DWARF for Go binaries Reduces build time and binary sizes considerably.
2017-05-26 13:32:26 +00:00
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
# .descriptions is a stub file that serves as a substitute for all files generated by syz-sysgen:
# sys/*/gen/*.go, executor/defs.h, executor/syscalls.h
# syz-sysgen generates them all at once, so we can't make each of them an independent target.
.PHONY: descriptions
descriptions:
tools/syz-linter: add custom linter For now we have 2 simple checks: 1. for multiline comments: /* */ -> // 2. for string len comparison with 0: len(str) != 0 -> str != "" Update #1876
2020-07-02 16:38:27 +00:00
go list -f '{{.Stale}}' ./sys/syz-sysgen | grep -q false || go install ./sys/syz-sysgen
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
$(MAKE) .descriptions
.descriptions: sys/*/*.txt sys/*/*.const bin/syz-sysgen
bin/syz-sysgen
touch .descriptions
manager: descriptions
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-manager github.com/google/syzkaller/syz-manager
Add Makefile
2015-10-12 15:15:57 +00:00
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
runtest: descriptions
tools/syz-runtest: add tool for program unit testing The tool is run as: $ syz-runtest -config manager.config This runs all programs from sys/*/test/* in different modes on actual VMs and checks results. Fixes #603
2018-07-25 14:45:57 +00:00
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-runtest github.com/google/syzkaller/tools/syz-runtest
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
fuzzer: descriptions
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
GOOS=$(TARGETGOOS) GOARCH=$(TARGETGOARCH) $(GO) build $(GOTARGETFLAGS) -o ./bin/$(TARGETOS)_$(TARGETVMARCH)/syz-fuzzer$(EXE) github.com/google/syzkaller/syz-fuzzer
Add Makefile
2015-10-12 15:15:57 +00:00
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
execprog: descriptions
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
GOOS=$(TARGETGOOS) GOARCH=$(TARGETGOARCH) $(GO) build $(GOTARGETFLAGS) -o ./bin/$(TARGETOS)_$(TARGETVMARCH)/syz-execprog$(EXE) github.com/google/syzkaller/tools/syz-execprog
Add 'all-tools' target to build tools Also remove leftover mention of 'master'
2015-12-21 12:38:27 +00:00
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
ci: descriptions
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-ci github.com/google/syzkaller/syz-ci
syz-ci: add continuous integration system
2017-06-19 15:23:03 +00:00
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
hub: descriptions
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-hub github.com/google/syzkaller/syz-hub
syz-hub: split client name and manager name This allows to reduce number of hub clients by grouping managers and creating one client per such group.
2017-06-29 14:01:03 +00:00
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
repro: descriptions
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-repro github.com/google/syzkaller/tools/syz-repro
tools/syz-repro: add reproduction tool
2015-12-23 18:19:45 +00:00
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
mutate: descriptions
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-mutate github.com/google/syzkaller/tools/syz-mutate
Add 'all-tools' target to build tools Also remove leftover mention of 'master'
2015-12-21 12:38:27 +00:00
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
prog2c: descriptions
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-prog2c github.com/google/syzkaller/tools/syz-prog2c
Add 'all-tools' target to build tools Also remove leftover mention of 'master'
2015-12-21 12:38:27 +00:00
tools/syz-crush: Add feature to run C-program usage: syz-crush -config=config.json creprog.c Produced output is same as syz-manager produces. New directory called reproduced_with is added that lists all the reproducers that triggered the crash. Example: ~/workdir/crashes $ tree . ├── 2a8fff76c6267ecfeaf3530a602ed48afbc4a151 │   ├── description │   ├── log0 │   ├── log1 │   ├── log2 │   ├── report0 │   ├── report1 │   ├── report2 │   └── reproduced_with │   └── 17a6ed226769660f3e5dad7b22b13466b938e290.c ├── 2b6b921457afe8cebd0a96ca40f8569e6ee95174 │   ├── description │   ├── log0 │   ├── log1 │   ├── log2 │   ├── log3 │   ├── log4 │   ├── log5 │   ├── report0 │   ├── report1 │   ├── report2 │   ├── report3 │   ├── report4 │   ├── report5 │   └── reproduced_with │   ├── 9fb2f5b37bf4428382334e336bbbb201634c7c4c.c │   └── ab27002b46d19cafb1ebb8b040f0a3b0f8f88974.c Signed-off-by: Jukka Kaartinen <jukka.kaartinen@unikie.com>
2020-06-04 12:00:05 +00:00
crush: descriptions
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-crush github.com/google/syzkaller/tools/syz-crush
tools/syz-reporter: Tool to share/visualize crashes Goes through crash folder that is stated in the workdir. Collects the crashes, counts and tags. usage: ./bin/syz-reporter -config manager.cfg Signed-off-by: Jukka Kaartinen <jukka.kaartinen@unikie.com>
2020-06-18 13:58:20 +00:00
reporter: descriptions
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-reporter github.com/google/syzkaller/tools/syz-reporter
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
stress: descriptions
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
GOOS=$(TARGETGOOS) GOARCH=$(TARGETGOARCH) $(GO) build $(GOTARGETFLAGS) -o ./bin/$(TARGETOS)_$(TARGETVMARCH)/syz-stress$(EXE) github.com/google/syzkaller/tools/syz-stress
Add 'all-tools' target to build tools Also remove leftover mention of 'master'
2015-12-21 12:38:27 +00:00
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
db: descriptions
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-db github.com/google/syzkaller/tools/syz-db
makefile: add db target to build syz-db
2017-06-13 17:39:39 +00:00
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
upgrade: descriptions
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-upgrade github.com/google/syzkaller/tools/syz-upgrade
tools/syz-upgrade: helper tool for corpus format upgrades
2015-12-28 11:49:19 +00:00
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
trace2syz: descriptions
tools/syz-trace2syz: add tool to convert strace output to programs * fixing weird merge error * fixing presubmit * fixing presubmit * removing parsing code because of -Xraw option * fix presubmit * update * deleting vma_call_handlers as we are currently skipping most vma calls. This simplifies memory_tracker as we don't need to keep track of vma allocations * removing custom handling of bpf_instruction union * removing ifconf parsing * update * removed all expression types and replaced them with constant types. removing ipv6_addr parsing while -Xraw is getting fixed. Removing constants.go * removing ipv6 parsing * presubmit * moving direction check from ipv4_addr out to genUnion * removing code that parses kcov * removing redundant test * removing custom code in generate unions to fill ipv4_addr * proggen: changing order of imports to make external packages import first fixing presubmit * changing log messages to lower case to be consistent with other packages. * removing pointer type and simplifying memory_tracker removing comment * moving context and return_cache to seaparate files * deleting default argument generation when we should probably throw an error
2018-12-06 15:25:37 +00:00
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-trace2syz github.com/google/syzkaller/tools/syz-trace2syz
Makefile: add target for syz-symbolize
2020-05-13 18:48:47 +00:00
expand: descriptions
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-expand github.com/google/syzkaller/tools/syz-expand
all: add basic USB fuzzing support This commits implements 4 syzcalls: syz_usb_connect, syz_usb_io_control, syz_usb_ep_write and syz_usb_disconnect. Those syzcalls are used to emit USB packets through a custom GadgetFS-like interface (currently exposed at /sys/kernel/debug/usb-fuzzer), which requires special kernel patches. USB fuzzing support is quite basic, as it mostly covers only the USB device enumeration process. Even though the syz_usb_ep_write syzcall does allow to communicate with USB endpoints after the device has been enumerated, no coverage is collected from that code yet.
2019-04-11 13:44:07 +00:00
usbgen:
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-usbgen github.com/google/syzkaller/tools/syz-usbgen
Makefile: add target for syz-symbolize
2020-05-13 18:48:47 +00:00
symbolize:
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o ./bin/syz-symbolize github.com/google/syzkaller/tools/syz-symbolize
tools: add syz-expand The syz-expand tools allows to parse a program and print it including all the default values. This is mainly useful for debugging, like doing manual program modifications while trying to come up with a reproducer for some particular kernel behavior.
2019-09-23 13:34:59 +00:00
Include generate_fidl as part of the `extract` step. Anytime we're running `make extract/generate` with Fuchsia as a target, we probably want to update FIDL descriptions too, so we don't fall behind what upstream is up to. This includes generate_fidl as part of the `make extract` workflow. Also, corrects fidlgenPath to properly find the path for ARM targets (we only have host_x64 right now).
2018-09-29 01:15:56 +00:00
# `extract` extracts const files from various kernel sources, and may only
# re-generate parts of files.
overhaul syscall description generation process This splits generation process into two phases: 1. Extract values of constants from linux kernel sources. 2. Generate Go code. Constant values are checked in. The advantage is that the second phase is now completely independent from linux source files, kernel version, presence of headers for particular drivers, etc. This allows to change what Go code we generate any time without access to all kernel headers (which in future won't be limited to only upstream headers). Constant extraction process does require proper kernel sources, but this can be done only once by the person who added the driver and has access to the required sources. Then the constant values are checked in for others to use. Consant extraction process is per-file/per-arch. That is, if I am adding a driver that is not present upstream and that works only on a single arch, I will check in constants only for that driver and for that arch.
2016-08-26 05:09:25 +00:00
extract: bin/syz-extract
Include generate_fidl as part of the `extract` step. Anytime we're running `make extract/generate` with Fuchsia as a target, we probably want to update FIDL descriptions too, so we don't fall behind what upstream is up to. This includes generate_fidl as part of the `make extract` workflow. Also, corrects fidlgenPath to properly find the path for ARM targets (we only have host_x64 right now).
2018-09-29 01:15:56 +00:00
ifeq ($(TARGETOS),fuchsia)
$(MAKE) generate_fidl TARGETARCH=amd64
$(MAKE) generate_fidl TARGETARCH=arm64
else
endif
Makefile: allow to extract consts only from specified files
2018-05-18 07:48:49 +00:00
bin/syz-extract -build -os=$(TARGETOS) -sourcedir=$(SOURCEDIR) $(FILES)
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
sys/sysparser: move from sysparser
2017-06-17 11:17:11 +00:00
bin/syz-extract:
Makefile: don't compile all targets into target binaries Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
2018-08-02 17:07:22 +00:00
GOOS=$(HOSTOS) GOARCH=$(HOSTARCH) $(HOSTGO) build $(GOHOSTFLAGS) -o $@ ./sys/syz-extract
overhaul syscall description generation process This splits generation process into two phases: 1. Extract values of constants from linux kernel sources. 2. Generate Go code. Constant values are checked in. The advantage is that the second phase is now completely independent from linux source files, kernel version, presence of headers for particular drivers, etc. This allows to change what Go code we generate any time without access to all kernel headers (which in future won't be limited to only upstream headers). Constant extraction process does require proper kernel sources, but this can be done only once by the person who added the driver and has access to the required sources. Then the constant values are checked in for others to use. Consant extraction process is per-file/per-arch. That is, if I am adding a driver that is not present upstream and that works only on a single arch, I will check in constants only for that driver and for that arch.
2016-08-26 05:09:25 +00:00
Include generate_fidl as part of the `extract` step. Anytime we're running `make extract/generate` with Fuchsia as a target, we probably want to update FIDL descriptions too, so we don't fall behind what upstream is up to. This includes generate_fidl as part of the `make extract` workflow. Also, corrects fidlgenPath to properly find the path for ARM targets (we only have host_x64 right now).
2018-09-29 01:15:56 +00:00
# `generate` does *not* depend on any kernel sources, and generates everything
# in one pass, for all arches. It can be run on a bare syzkaller checkout.
.github/workflows: add CI workflow Switch from Travis to Github Actions for testing. This is faster and is better integrated with github. Update #1699
2020-05-06 12:43:04 +00:00
generate:
$(MAKE) descriptions
Makefile: fix generate target descriptions target needs to be executed before generate_go, because the latter builds Go code which won't build without descriptions.
2020-05-15 15:58:17 +00:00
$(MAKE) generate_go
makefile: generate: do go generate && format
2017-06-13 17:39:21 +00:00
$(MAKE) format
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
generate_go: format_cpp
syz-manager: modernize web UI 1. Use dashboard style. 2. Allow sorting of tables. 3. Show old crashes in grey. 4. Use tables instead of text output for more pages. 5. Show corpus inputs on a separate page to allow copy-pasting. 6. Use standard JS sorting instead of custom bubble sort (much faster). 7. Fix off-by one in table sorting. Fixes #694
2018-11-22 13:27:37 +00:00
$(GO) generate ./pkg/csource ./executor ./pkg/ifuzz ./pkg/build ./pkg/html
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
sys/fuchsia: add Go script that generates fidl descriptions
2018-09-05 19:45:17 +00:00
generate_fidl:
Include generate_fidl as part of the `extract` step. Anytime we're running `make extract/generate` with Fuchsia as a target, we probably want to update FIDL descriptions too, so we don't fall behind what upstream is up to. This includes generate_fidl as part of the `make extract` workflow. Also, corrects fidlgenPath to properly find the path for ARM targets (we only have host_x64 right now).
2018-09-29 01:15:56 +00:00
ifeq ($(TARGETOS),fuchsia)
$(HOSTGO) generate ./sys/fuchsia
sys/fuchsia: add Go script that generates fidl descriptions
2018-09-05 19:45:17 +00:00
$(MAKE) format_sys
Include generate_fidl as part of the `extract` step. Anytime we're running `make extract/generate` with Fuchsia as a target, we probably want to update FIDL descriptions too, so we don't fall behind what upstream is up to. This includes generate_fidl as part of the `make extract` workflow. Also, corrects fidlgenPath to properly find the path for ARM targets (we only have host_x64 right now).
2018-09-29 01:15:56 +00:00
else
endif
sys/fuchsia: add Go script that generates fidl descriptions
2018-09-05 19:45:17 +00:00
tools/syz-trace2syz: add tool to convert strace output to programs * fixing weird merge error * fixing presubmit * fixing presubmit * removing parsing code because of -Xraw option * fix presubmit * update * deleting vma_call_handlers as we are currently skipping most vma calls. This simplifies memory_tracker as we don't need to keep track of vma allocations * removing custom handling of bpf_instruction union * removing ifconf parsing * update * removed all expression types and replaced them with constant types. removing ipv6_addr parsing while -Xraw is getting fixed. Removing constants.go * removing ipv6 parsing * presubmit * moving direction check from ipv4_addr out to genUnion * removing code that parses kcov * removing redundant test * removing custom code in generate unions to fill ipv4_addr * proggen: changing order of imports to make external packages import first fixing presubmit * changing log messages to lower case to be consistent with other packages. * removing pointer type and simplifying memory_tracker removing comment * moving context and return_cache to seaparate files * deleting default argument generation when we should probably throw an error
2018-12-06 15:25:37 +00:00
generate_trace2syz:
(cd tools/syz-trace2syz/parser; ragel -Z -G2 -o lex.go straceLex.rl)
(cd tools/syz-trace2syz/parser; goyacc -o strace.go -p Strace -v="" strace.y)
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
format: format_go format_cpp format_sys
format_go:
Revert "Makefile: Enable working with symlinked directories" This reverts commit 7ce4cd3a7e986b396dfce6540b04e9be5f084746. This breaks syzbot as it executes "cd " which gets it into a completely unrelated dir and then it tries to format everything there as Go code.
2018-09-05 09:34:56 +00:00
$(GO) fmt ./...
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
format_cpp:
tools: add fops_probe utility fops_probe utility helps to understand what file_operations callbacks are attached to a particular file. Requries KCOV and KALLSYMS. Build with: g++ tools/fops_probe/fops_probe.cc -Wall -static -o fops_probe Then copy the binary to target machine and run as: ./fops_probe /dev/fb0 You should see output similar to: ffffffff81bcccb9 vfs_read ................ ffffffff83af85c3 fb_read ffffffff83b52af5 cirrusfb_sync ffffffff81bcd219 vfs_write ................ ffffffff83af7fe2 fb_write ffffffff83b52af5 cirrusfb_sync ffffffff81c1b745 do_vfs_ioctl ffffffff83af7ea9 fb_ioctl ffffffff81a4ea44 do_mmap ................ ffffffff83af716c fb_mmap which allows to understand what callbacks are associated with /dev/fb0.
2019-12-03 10:34:27 +00:00
clang-format --style=file -i executor/*.cc executor/*.h \
executor: minor cleanup of android sandbox Fix code formatting, clang-tidy warnings, minor style nits.
2020-03-11 11:09:17 +00:00
executor/android/android_seccomp.h \
tools: add fops_probe utility fops_probe utility helps to understand what file_operations callbacks are attached to a particular file. Requries KCOV and KALLSYMS. Build with: g++ tools/fops_probe/fops_probe.cc -Wall -static -o fops_probe Then copy the binary to target machine and run as: ./fops_probe /dev/fb0 You should see output similar to: ffffffff81bcccb9 vfs_read ................ ffffffff83af85c3 fb_read ffffffff83b52af5 cirrusfb_sync ffffffff81bcd219 vfs_write ................ ffffffff83af7fe2 fb_write ffffffff83b52af5 cirrusfb_sync ffffffff81c1b745 do_vfs_ioctl ffffffff83af7ea9 fb_ioctl ffffffff81a4ea44 do_mmap ................ ffffffff83af716c fb_mmap which allows to understand what callbacks are associated with /dev/fb0.
2019-12-03 10:34:27 +00:00
tools/kcovtrace/*.c tools/kcovfuzzer/*.c tools/fops_probe/*.cc
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
format_sys: bin/syz-fmt
tools/syz-fmt: allow to reformat all OSes at once We forgot trusty in Makefile. Fix this once and for all by formatting all known targets.
2018-11-17 19:46:48 +00:00
bin/syz-fmt all
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
Makefile: enforce formatting of sys files in presubmit
2017-08-18 09:14:20 +00:00
bin/syz-fmt:
Include generate_fidl as part of the `extract` step. Anytime we're running `make extract/generate` with Fuchsia as a target, we probably want to update FIDL descriptions too, so we don't fall behind what upstream is up to. This includes generate_fidl as part of the `make extract` workflow. Also, corrects fidlgenPath to properly find the path for ARM targets (we only have host_x64 right now).
2018-09-29 01:15:56 +00:00
$(HOSTGO) build $(GOHOSTFLAGS) -o $@ ./tools/syz-fmt
executor, csource: share some common code between executor and csource
2016-08-28 12:59:48 +00:00
executor: fix clang-tidy warnings A single check is enabled for now (misc-definitions-in-headers). But it's always fixable and found 2 bugs in csource.
2017-06-13 15:21:33 +00:00
tidy:
executor: fix compiler warnings in test
2017-06-14 10:56:42 +00:00
# A single check is enabled for now. But it's always fixable and proved to be useful.
Makefile: enable clang-tidy in presubmit tests Fix all errors it points to. The one in netlink_send_ext is concerning.
2020-08-04 11:28:47 +00:00
clang-tidy -quiet -header-filter=.* -warnings-as-errors=* \
Makefile: fix tidy rule Building executor now requires GOOS/GOARCH macros.
2019-03-21 11:30:53 +00:00
-extra-arg=-DGOOS_$(TARGETOS)=1 -extra-arg=-DGOARCH_$(TARGETARCH)=1 \
Makefile: enable clang-tidy in presubmit tests Fix all errors it points to. The one in netlink_send_ext is concerning.
2020-08-04 11:28:47 +00:00
-extra-arg=-DHOSTGOOS_$(HOSTOS)=1 -extra-arg=-DGIT_REVISION=\"$(REV)\" \
-checks=-*,misc-definitions-in-headers,bugprone-macro-parentheses,clang-analyzer-*,-clang-analyzer-security.insecureAPI*,-clang-analyzer-optin.performance* \
Makefile: fix tidy rule Building executor now requires GOOS/GOARCH macros.
2019-03-21 11:30:53 +00:00
executor/*.cc
executor: fix clang-tidy warnings A single check is enabled for now (misc-definitions-in-headers). But it's always fixable and found 2 bugs in csource.
2017-06-13 15:21:33 +00:00
.golangci.yml: update Enable more linters. Tune settings for existings. Fix excludes. Update #977
2019-04-23 13:58:23 +00:00
lint:
go.mod: vendor golangci-lint
2020-07-04 09:12:55 +00:00
# This should install the command from our vendor dir.
tools/syz-linter: add custom linter For now we have 2 simple checks: 1. for multiline comments: /* */ -> // 2. for string len comparison with 0: len(str) != 0 -> str != "" Update #1876
2020-07-02 16:38:27 +00:00
CGO_ENABLED=1 $(HOSTGO) install github.com/golangci/golangci-lint/cmd/golangci-lint
CGO_ENABLED=1 $(HOSTGO) build -buildmode=plugin -o bin/syz-linter.so ./tools/syz-linter
bin/golangci-lint run ./...
.golangci.yml: update Enable more linters. Tune settings for existings. Fix excludes. Update #977
2019-04-23 13:58:23 +00:00
Makefile: simplify arch build tests There is not much value in the current 2 levels of indirections.
2020-07-16 13:16:06 +00:00
presubmit:
$(MAKE) presubmit_smoke
$(MAKE) presubmit_arch
$(MAKE) presubmit_race
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
Makefile: simplify arch build tests There is not much value in the current 2 levels of indirections.
2020-07-16 13:16:06 +00:00
presubmit_smoke:
$(MAKE) generate
Makefile: enable clang-tidy in presubmit tests Fix all errors it points to. The one in netlink_send_ext is concerning.
2020-08-04 11:28:47 +00:00
$(MAKE) -j100 check_commits check_diff check_copyright check_language check_links presubmit_build tidy
Makefile: simplify arch build tests There is not much value in the current 2 levels of indirections.
2020-07-16 13:16:06 +00:00
$(MAKE) test
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
Makefile: simplify arch build tests There is not much value in the current 2 levels of indirections.
2020-07-16 13:16:06 +00:00
presubmit_build:
# Run go build before lint for better error messages if build is broken.
# This does not check build of test files, but running go test takes too long (even for building).
$(GO) build ./...
$(MAKE) lint
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
Makefile: simplify arch build tests There is not much value in the current 2 levels of indirections.
2020-07-16 13:16:06 +00:00
presubmit_arch: descriptions
env HOSTOS=linux HOSTARCH=amd64 $(MAKE) host
env HOSTOS=freebsd HOSTARCH=amd64 $(MAKE) host
env HOSTOS=netbsd HOSTARCH=amd64 $(MAKE) host
env HOSTOS=openbsd HOSTARCH=amd64 $(MAKE) host
env HOSTOS=darwin HOSTARCH=amd64 $(MAKE) host
env TARGETOS=linux TARGETARCH=amd64 $(MAKE) target
Makefile: test executor build with clang To get more compiler coverage and to not let it rot. clang get detect more bugs statically.
2020-07-16 13:18:20 +00:00
env TARGETOS=linux TARGETARCH=amd64 SYZ_CLANG=yes $(MAKE) target
Makefile: enable 386 executor build in presubmit This hacky way to install 386 packages seems to unbreak executor build.
2019-05-07 12:14:38 +00:00
env TARGETOS=linux TARGETARCH=386 $(MAKE) target
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
env TARGETOS=linux TARGETARCH=arm64 $(MAKE) target
Makefile: enable arm executor build in presubmit arm build seems to work now on Travis since we switched to xenial.
2019-05-07 11:23:39 +00:00
env TARGETOS=linux TARGETARCH=arm $(MAKE) target
Implement basic support for MIPS64LE Add basic stuff to enable MIPS64ELR2 target: - build - make extract - make generate - qemu execution - system call parsing from /proc/kallsyms
2019-11-19 11:12:54 +00:00
env TARGETOS=linux TARGETARCH=mips64le $(MAKE) target
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
env TARGETOS=linux TARGETARCH=ppc64le $(MAKE) target
Makefile: add support for linux/riscv64 Install g++-riscv64-linux-gnu in install_prerequisites target and hook up arch_linux_riscv64_target. Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2020-06-26 10:08:18 +00:00
env TARGETOS=linux TARGETARCH=riscv64 $(MAKE) target
sys/linux: first 64bit big-endian architecture s390x * mmap syscall is special on Linux s390x because the parameters for this syscall are passed as a struct on user stack instead of registers. * Introduce the SyscallTrampolines table into targets.Target to address the above problem. * There is a bug in Linux kernel s390x which causes QEMU TCG to hang when KASAN is enabled. The bug has been fixed in the forthcoming Linux 5.8 version. Until then do not enable KASAN when using QEMU TCG, QEMU KVM shall have no problems with KASAN. Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-06-02 07:18:07 +00:00
env TARGETOS=linux TARGETARCH=s390x $(MAKE) target
Makefile: build netbsd binaries so that we don't regress
2017-10-23 08:19:56 +00:00
env TARGETOS=freebsd TARGETARCH=amd64 $(MAKE) target
sys/freebsd: Add support for fuzzing FreeBSD/i386 Add support to fuzz 32 bit FreeBSD system calls.
2019-05-22 16:36:58 +00:00
env TARGETOS=freebsd TARGETARCH=386 $(MAKE) target
Makefile: build netbsd binaries so that we don't regress
2017-10-23 08:19:56 +00:00
env TARGETOS=netbsd TARGETARCH=amd64 $(MAKE) target
Makefile: test openbsd build
2018-09-19 09:14:54 +00:00
env TARGETOS=openbsd TARGETARCH=amd64 $(MAKE) target
sys/targest: introduce target.BuildOS We can't cross-compile native binaries from just any OS to any other. For most OSes we can do only native compilation. Some can only be compiled from linux. To date we avoided this problem completely (mostly assumed linux build OS). Make this notion of what can build what explicit.
2018-11-17 19:20:17 +00:00
env TARGETOS=windows TARGETARCH=amd64 $(MAKE) target
tools: add syz-big-env On top of syz-env it provides akaros/fuchsia/netbsd toolchains and gcloud sdk. With this it's possible to run dashboard/app tests on CI and locally and test executor build and pkg/{csource,cover} for these OSes. Update #1765
2020-06-01 17:09:18 +00:00
env TARGETOS=akaros TARGETARCH=amd64 $(MAKE) executor
env TARGETOS=fuchsia TARGETARCH=amd64 $(MAKE) executor
env TARGETOS=fuchsia TARGETARCH=arm64 $(MAKE) executor
executor: overhaul Make as much code as possible shared between all OSes. In particular main is now common across all OSes. Make more code shared between executor and csource (in particular, loop function and threaded execution logic). Also make loop and threaded logic shared across all OSes. Make more posix/unix code shared across OSes (e.g. signal handling, pthread creation, etc). Plus other changes along similar lines. Also support test OS in executor (based on portable posix) and add 4 arches that cover all execution modes (fork server/no fork server, shmem/no shmem). This change paves way for testing of executor code and allows to preserve consistency across OSes and executor/csource.
2018-07-20 18:26:05 +00:00
env TARGETOS=test TARGETARCH=64 $(MAKE) executor
env TARGETOS=test TARGETARCH=64_fork $(MAKE) executor
Makefile: switch to golangci-lint Finally! Fixes #977 Fixes #1108
2019-05-15 08:12:35 +00:00
env TARGETOS=test TARGETARCH=32_shmem $(MAKE) executor
env TARGETOS=test TARGETARCH=32_fork_shmem $(MAKE) executor
executor: overhaul Make as much code as possible shared between all OSes. In particular main is now common across all OSes. Make more code shared between executor and csource (in particular, loop function and threaded execution logic). Also make loop and threaded logic shared across all OSes. Make more posix/unix code shared across OSes (e.g. signal handling, pthread creation, etc). Plus other changes along similar lines. Also support test OS in executor (based on portable posix) and add 4 arches that cover all execution modes (fork server/no fork server, shmem/no shmem). This change paves way for testing of executor code and allows to preserve consistency across OSes and executor/csource.
2018-07-20 18:26:05 +00:00
tools: add syz-big-env On top of syz-env it provides akaros/fuchsia/netbsd toolchains and gcloud sdk. With this it's possible to run dashboard/app tests on CI and locally and test executor build and pkg/{csource,cover} for these OSes. Update #1765
2020-06-01 17:09:18 +00:00
presubmit_big: descriptions
# This target runs on CI in syz-big-env,
# so we test packages that need GCloud SDK or OS toolchains.
$(GO) test -short -coverprofile=.coverage.txt ./dashboard/app ./pkg/csource ./pkg/cover
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
tools: add syz-big-env On top of syz-env it provides akaros/fuchsia/netbsd toolchains and gcloud sdk. With this it's possible to run dashboard/app tests on CI and locally and test executor build and pkg/{csource,cover} for these OSes. Update #1765
2020-06-01 17:09:18 +00:00
presubmit_race: descriptions
.github/workflows: add CI workflow Switch from Travis to Github Actions for testing. This is faster and is better integrated with github. Update #1699
2020-05-06 12:43:04 +00:00
# -race requires cgo
Makefile: fix double $(GO) $(GO)
2019-01-14 17:24:22 +00:00
env CGO_ENABLED=1 $(GO) test -race; if test $$? -ne 2; then \
env CGO_ENABLED=1 $(GO) test -race -short -bench=.* -benchtime=.2s ./... ;\
Conditionally enable test -race when supported on the OS.
2018-10-29 05:42:42 +00:00
fi
Makefile: parallelize Use -j by default and parallelize targets as much as possible.
2018-05-04 17:51:44 +00:00
tools: add syz-big-env On top of syz-env it provides akaros/fuchsia/netbsd toolchains and gcloud sdk. With this it's possible to run dashboard/app tests on CI and locally and test executor build and pkg/{csource,cover} for these OSes. Update #1765
2020-06-01 17:09:18 +00:00
test: descriptions
$(GO) test -short -coverprofile=.coverage.txt ./...
makefile: cleanup - fix and reorder .PHONY targets - `make all` now builds everything, to build only essentials run `make main` - `make all-tools` rename to `make tools`
2017-07-27 14:59:09 +00:00
clean:
Makefile: generate descriptions on-the-fly Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
2020-04-29 15:57:32 +00:00
rm -rf ./bin .descriptions sys/*/gen executor/defs.h executor/syscalls.h
sys/syz-extract: support fuchsia
2017-09-25 06:47:15 +00:00
Makefile: enable gometalinter on travis (#537)
2018-03-20 12:26:51 +00:00
# For a tupical Ubuntu/Debian distribution.
tools: add syz-env syz-env provides a canned dev environment with all tools necessary (Go, gcc, clang-format, etc) for development/testing in a form of a Docker image. Example use: $ syz-env make presubmit $ syz-env make extract SOURCEDIR=~/linux Update #1699
2020-05-16 05:14:45 +00:00
# We use "|| true" for apt-get install because packages are all different on different distros.
# Also see tools/syz-env for container approach.
sys/syz-extract: support fuchsia
2017-09-25 06:47:15 +00:00
install_prerequisites:
Makefile: print uname in install_prerequisites Travis is still broken with: E: Unable to locate package gcc-aarch64-linux-gnu E: Unable to locate package gcc-arm-linux-gnueabihf E: Unable to locate package gcc-powerpc64le-linux-gnu https://travis-ci.org/google/syzkaller/jobs/378034948 Let's at least see what's OS is there and what gcc packages present.
2018-05-13 09:34:03 +00:00
uname -a
sudo apt-get update
Makefile: fix install_prerequisites apt-get complains if all these packages are installed at once.
2018-04-02 17:38:11 +00:00
sudo apt-get install -y -q libc6-dev-i386 linux-libc-dev \
Implement basic support for MIPS64LE Add basic stuff to enable MIPS64ELR2 target: - build - make extract - make generate - qemu execution - system call parsing from /proc/kallsyms
2019-11-19 11:12:54 +00:00
gcc-aarch64-linux-gnu gcc-arm-linux-gnueabi gcc-powerpc64le-linux-gnu gcc-mips64el-linux-gnuabi64 || true
Makefile: allow to extract consts only from specified files
2018-05-18 07:48:49 +00:00
sudo apt-get install -y -q g++-aarch64-linux-gnu || true
sudo apt-get install -y -q g++-powerpc64le-linux-gnu || true
Makefile: install gnueabi arm compiler The arm compiler was changed in e5d1b3ac44472b1e7741da6bbcc0e655b931025e from gnueabihf to gnueabi. Install the corresponding packages. @Codysseus
2019-04-12 13:15:18 +00:00
sudo apt-get install -y -q g++-arm-linux-gnueabi || true
Implement basic support for MIPS64LE Add basic stuff to enable MIPS64ELR2 target: - build - make extract - make generate - qemu execution - system call parsing from /proc/kallsyms
2019-11-19 11:12:54 +00:00
sudo apt-get install -y -q g++-mips64el-linux-gnuabi64 || true
sys/linux: first 64bit big-endian architecture s390x * mmap syscall is special on Linux s390x because the parameters for this syscall are passed as a struct on user stack instead of registers. * Introduce the SyscallTrampolines table into targets.Target to address the above problem. * There is a bug in Linux kernel s390x which causes QEMU TCG to hang when KASAN is enabled. The bug has been fixed in the forthcoming Linux 5.8 version. Until then do not enable KASAN when using QEMU TCG, QEMU KVM shall have no problems with KASAN. Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-06-02 07:18:07 +00:00
sudo apt-get install -y -q g++-s390x-linux-gnu || true
Makefile: add support for linux/riscv64 Install g++-riscv64-linux-gnu in install_prerequisites target and hook up arch_linux_riscv64_target. Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2020-06-26 10:08:18 +00:00
sudo apt-get install -y -q g++-riscv64-linux-gnu || true
Makefile: test executor build with clang To get more compiler coverage and to not let it rot. clang get detect more bugs statically.
2020-07-16 13:18:20 +00:00
sudo apt-get install -y -q clang clang-format ragel
Makefile: set GO111MODULE=off for go get in install_prerequisites Otherwise the versions in go.mod are updated on running `make install_prerequisites` leading to inconsistent vendoring errors: go: inconsistent vendoring in /go/src/github.com/google/syzkaller: golang.org/x/net@v0.0.0-20200625001655-4c5254603344: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt golang.org/x/tools@v0.0.0-20200717024301-6ddee64345a6: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt golang.org/x/net@v0.0.0-20200520182314-0ba52f642ac2: is marked as explicit in vendor/modules.txt, but not explicitly required in go.mod golang.org/x/tools@v0.0.0-20200626171337-aa94e735be7f: is marked as explicit in vendor/modules.txt, but not explicitly required in go.mod Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2020-07-20 14:34:19 +00:00
GO111MODULE=off go get -u golang.org/x/tools/cmd/goyacc
docs, tools: add local link checker This commit adds tools/check_links.py script, that checks that all local links from documentation files are valid; fixes some of the invalid links that we had; and makes travis buildbot check them as well.
2017-10-26 16:48:54 +00:00
tools: add script that checks copyright headers Fixes #1604
2020-02-18 15:02:42 +00:00
check_copyright:
./tools/check-copyright.sh
all: use more respective language Some terms are normalised on the technical level but may be oppressive on a societal level. Replace them with more technically neutral terms. See the following doc for a longer version: https://tools.ietf.org/id/draft-knodel-terminology-00.html
2020-06-11 20:10:07 +00:00
check_language:
./tools/check-language.sh
.github/workflows: check commit description format Check PR commit subjects for the standard format we use "pkg/name: description of the change". Check PR commit bodies for at most 120 characters.
2020-07-31 10:07:23 +00:00
check_commits:
./tools/check-commits.sh
docs, tools: add local link checker This commit adds tools/check_links.py script, that checks that all local links from documentation files are valid; fixes some of the invalid links that we had; and makes travis buildbot check them as well.
2017-10-26 16:48:54 +00:00
check_links:
python ./tools/check_links.py $$(pwd) $$(ls ./*.md; find ./docs/ -name '*.md')
makefile: move diff check to makefile from travis script
2017-11-20 16:23:36 +00:00
# Check that the diff is empty. This is meant to be executed after generating
# and formatting the code to make sure that everything is committed.
check_diff:
Makefile: fix check_links 1. Disable pager. 2. Actually fail.
2020-05-13 09:42:59 +00:00
@if [ "$(shell git --no-pager diff --name-only)" != "" ]; then \
git --no-pager diff; \
Makefile: print errors in standard format Use the standard "file:line:col: error message" format. Update #1699
2020-05-17 04:53:43 +00:00
git --no-pager diff --name-only | \
sed "s#.*#&:1:1: The file is not formatted/regenerated. Run 'make generate' and include it into the commit.#g"; \
Makefile: fix check_links 1. Disable pager. 2. Actually fail.
2020-05-13 09:42:59 +00:00
false; \
Makefile: print what files are not formatted Move this part from .travis.yml to Makefile. It's useful to see it always and we generally don't have lots of logic in the travis file. Also move list of changed files to the bottom (that's what one generally sees at console and at the end of the log).
2020-05-06 11:38:02 +00:00
fi
Reference in New Issue Copy Permalink