Some terms are normalised on the technical level
but may be oppressive on a societal level.
Replace them with more technically neutral terms.
See the following doc for a longer version:
https://tools.ietf.org/id/draft-knodel-terminology-00.html
On top of syz-env it provides akaros/fuchsia/netbsd toolchains and gcloud sdk.
With this it's possible to run dashboard/app tests on CI and locally
and test executor build and pkg/{csource,cover} for these OSes.
Update #1765
syz-env provides a canned dev environment with all tools necessary
(Go, gcc, clang-format, etc) for development/testing in a form
of a Docker image. Example use:
$ syz-env make presubmit
$ syz-env make extract SOURCEDIR=~/linux
Update #1699
syz-env will be a good, short name for the planned
docker-ized build environment. So rename syz-env to syz-make.
Maybe not the best name, but it's not visible basically anywhere.
Allow targets.go use Clang instead of the default Linux compiler by
setting the SYZ_CLANG=1 env var. Doing so changes the compiler to
"clang" and the linker to "ld.ldd", assuming they are in $PATH, and adds
the --target and -ferror-limit CFLAGS.
Target also exports KernelCompiler and KernelLinker fields now, which allows
overriding the compiler and linker in the kernel make invocation.
Signed-off-by: Alexander Potapenko <glider@google.com>
In preparation to running some tests as github actions.
Both Travis and Github define CI env var, while TRAVIS is, well,
too Travis-specific.
Update #1699
Move this part from .travis.yml to Makefile.
It's useful to see it always and we generally
don't have lots of logic in the travis file.
Also move list of changed files to the bottom
(that's what one generally sees at console
and at the end of the log).
Run fewer builds in presubmit. Also run them sequentially instead of parallel.
The current presubmit OOMs on CI every other time, this is not useful.
Update #1699
Checking in the generated descriptions files makes few things simpler,
but causes pain for pull requests: (1) PRs that touch descriptions
_always_ conflict, (2) PRs are large and harder to review,
(3) people sometimes forget to add auto-generated files.
The proposed way does not require us to hardcode lots of dependencies
in the Makefile (which is nice) and seem to work.
Let's see how it works.
The main contributor-visible consequence is that the auto-generated
files do not need to be checked-in now.
Credit for figuring the Makefile magic goes to @melver.
Fixes#1291
1. Detect when compiler is present, but is not functioning
(can't build a simple program, common for Linux distros).
2. Be more strict with skipping tests due to missing/broken compilers on CI
(on CI they should work, so fail loudly if not).
3. Dedup this logic across syz-env and pkg/csource tests.
4. Add better error reporting for syz-env.
Fixes#1606
We did it for caching purposes, but now
(1) go command caches everything anyway,
(2) we use custom tags and flags, so result
of go install is not reused anyway.
We are seeing some one-off panics during Deserialization
and it's unclear if it's machine memory corrpution or
an actual bug in prog. I leam towards machine memory corruption
but it's impossible to prove without seeing the orig program.
Move git revision to prog and it's more base package
(sys can import prog, prog can't import sys).
Currently we print them as part of `make genereate`,
but nobody reads them, too much output each time.
Don't print them in `make generate` and instead
print in syz-check, the warn files are a good mechanism
to handle "known warnings".
fops_probe utility helps to understand what file_operations callbacks
are attached to a particular file. Requries KCOV and KALLSYMS.
Build with:
g++ tools/fops_probe/fops_probe.cc -Wall -static -o fops_probe
Then copy the binary to target machine and run as:
./fops_probe /dev/fb0
You should see output similar to:
ffffffff81bcccb9 vfs_read
................
ffffffff83af85c3 fb_read
ffffffff83b52af5 cirrusfb_sync
ffffffff81bcd219 vfs_write
................
ffffffff83af7fe2 fb_write
ffffffff83b52af5 cirrusfb_sync
ffffffff81c1b745 do_vfs_ioctl
ffffffff83af7ea9 fb_ioctl
ffffffff81a4ea44 do_mmap
................
ffffffff83af716c fb_mmap
which allows to understand what callbacks are associated with /dev/fb0.
These used to be required to build Fuchsia's Go toolchain to build
syz-fuzzer to run on Fuchsia. However, Fuchsia now uses HostFuzzer
mode, so only syz-executor (written in C) needs to run on Fuchsia; all
Go code runs on the Host OS (currently Linux).
The syz-expand tools allows to parse a program and print it including all
the default values. This is mainly useful for debugging, like doing manual
program modifications while trying to come up with a reproducer for some
particular kernel behavior.
Go support is not a priority for Fuchsia at the moment, so it's
preferable to use host fuzzing mode for Fuchsia like currently done
for Akaros.
This commit basically looks for all the places where there was special
logic for OS=="akaros" and extends the same logic for OS=="fuchsia".
This commit removes a dependency on fuchsia's tools/devshell/contrib/go,
which was removed recently[0].
Now to build go binaries for fuchsia we are not relying on the
fuchsia.git repo at all. The code just needs a copy of the fuchsia
//third_party/go repository, the fuchsia sdk and a copy of clang.
Users should download the sdk and clang beforehand (see documentation
for links), and then run (with the correct environment variables):
```
$ make TARGETOS=fuchsia TARGETARCH=amd64 \
SOURCEDIR=$HOME/fuchsia \
FX_SDK_PATH=${HOME}/sdk/fuchsia-sdk \
CLANG_PATH=${HOME}/sdk/clang \
fuchsia_go
```
After that, they will be able to build the syzkaller go binaries by
doing:
```
$ make TARGETOS=fuchsia TARGETARCH=amd64 \
SOURCEDIR=$HOME/fuchsia \
FX_SDK_PATH=${HOME}/sdk/fuchsia-sdk \
CLANG_PATH=${HOME}/sdk/clang
```
This commits adds two scripts to tools/fuchsia, one for building go in
fuchsia (just calls the ./make.all script in fuchsia's go repo), and one
that replaces the devshell/contrib/go script.
Given that go is unsupported in fuchsia, this change might break at any
point.
[0]: https://fuchsia-review.googlesource.com/c/fuchsia/+/291631
This commits implements 4 syzcalls: syz_usb_connect, syz_usb_io_control,
syz_usb_ep_write and syz_usb_disconnect. Those syzcalls are used to emit USB
packets through a custom GadgetFS-like interface (currently exposed at
/sys/kernel/debug/usb-fuzzer), which requires special kernel patches.
USB fuzzing support is quite basic, as it mostly covers only the USB device
enumeration process. Even though the syz_usb_ep_write syzcall does allow to
communicate with USB endpoints after the device has been enumerated, no
coverage is collected from that code yet.
Separate kernel and syzkaller build failures.
Fix logic to understand when a build is fixed:
look if kernel/syzkaller commit changes to understand
if it's a new good build or re-upload of an old build.
Fixes#1014
Differences in code formatting between Go versions cause constant
problems for us (https://github.com/golang/go/issues/25161).
Currently we support 1.9 and 1.10. Switch to newer 1.11 and 1.12.
Fixes#1013
The problem is stupid: <endian.h> should be included as <sys/endian.h> on freebsd.
Pass actual host OS to executor build as HOSTGOOS and use it to figure out
how we should include this header.
We started building netbsd executor on linux.
This fails on CI since cross-compiler is missing.
But we don't want to exclude netbsd from presubmit entirely.
So check if cross-compiler is missing and just produce a warning.
This should unbreak CI and humans will see the warning.
See discussion in PR #942.
Extend support for optional flags in sys/targets
as this flag is not supported by gcc 5.
Make flags consistent across Makefile and pkg/csource.
