* fixing weird merge error
* fixing presubmit
* fixing presubmit
* removing parsing code because of -Xraw option
* fix presubmit
* update
* deleting vma_call_handlers as we are currently skipping most vma calls. This simplifies memory_tracker as we don't need to keep track of vma allocations
* removing custom handling of bpf_instruction union
* removing ifconf parsing
* update
* removed all expression types and replaced them with constant types. removing ipv6_addr parsing while -Xraw is getting fixed. Removing constants.go
* removing ipv6 parsing
* presubmit
* moving direction check from ipv4_addr out to genUnion
* removing code that parses kcov
* removing redundant test
* removing custom code in generate unions to fill ipv4_addr
* proggen: changing order of imports to make external packages import first
fixing presubmit
* changing log messages to lower case to be consistent with other packages.
* removing pointer type and simplifying memory_tracker
removing comment
* moving context and return_cache to seaparate files
* deleting default argument generation when we should probably throw an error
* build/openbsd: minor cleanup (use tuples instead of maps)
* Grammar nits in comments.
* Simplify openbsd.Create, will defer when there's more than one error exit.
* pkg/build: Support copying kernel into GCE image
* Simple test for openbsd image copy build.
* Cleanup in case something failed before.
* Support multi-processor VMs on GCE.
* More debug
* Reformat
* OpenBSD gce image needs to be raw.
* GC
* Force format to GNU directly on Go 1.10 or newer.
* Use vmType passed as a parameter inside openbsd.go
* gofmt
* more fmt
* Can't use GENERIC.mp just yet.
* capitalize
* Copyright
We used to use len([]CallInfo) to check both, whether the slice is nil or
whether its length is zero. Since ProgInfo is not a slice, we need a
separate check for nil.
This patch add a new struct ProgInfo that for now holds info about each
call in a program []CallInfo, but in the future will be expanded with remote
coverage info. Update all the callers to use the new interface as well.
We can't cross-compile native binaries from just any OS to any other.
For most OSes we can do only native compilation.
Some can only be compiled from linux.
To date we avoided this problem completely (mostly assumed linux build OS).
Make this notion of what can build what explicit.
reportPrefixLen can become wrong after symbolization
if we symbolize any lines in the prefix.
Adjust reportPrefixLen during symbolization.
Automatic testing of this is problematic
because we would need to symbolize which requires
the object file with debug info.
Tested manually with syz-symbolize.
* Use cdn.openbsd.org not cloudflare. Install vmm-firmware explicitly.
* Use cdn.openbsd.org not cloudflare. Install vmm-firmware explicitly.
* set hw.smt=1: the underlying hw on VM is unknown so don't waste SMT.
We don't need consistent image after reboot since we always reimage.
We know of some use cases that don't use journalling, but don't know
of any that use journalling.
* tools/create-openbsd-gce-ci: use config from /syzkaller
* Use syzkaller copy of src instead of a separate one.
* Using /dev/null disk in vm.conf
* Use KVM, enable doas, no longer symlink.
* Use a dummmy.img
* Revert "Use a dummmy.img"
This reverts commit 656b24d5e4573dde5e95c6158852001c7241e65a.
Currently we choose block device to use (nbd/loop) based on SYZ_VM_TYPE.
Strictly saying these things are orthogonal.
losetup is broken on Ubuntu. qemu-nbd is broken on Debian.
Try to auto-detect what will work based on uname.
This updates the image creation tool to use Debian stretch (current stable)
instead of wheezy, which is very out of date. The only change needed here
was a hint to systemd to make the root filesystem read-write after booting.
Documentation has also been updated.
Upstream "selinux: fix mounting of cgroup2 under older policies"
commit fixes mounting of cgroup2 under wheezy selinux policy.
So don't disable selinux on start.
Create separate cmdline arguments that enable selinux and apparmor.
Support checking "blocked"/"unfinished" flags for calls.
Support test constanints, e.g. "requires: threaded" or "requires: -sandbox=setuid".
Some improvements in tools/syz-runtest.
Update #603
The tool is run as:
$ syz-runtest -config manager.config
This runs all programs from sys/*/test/* in different modes
on actual VMs and checks results.
Fixes#603
mgrconfig was used only by syz-manager initially,
but now it's used by a dozen of packages and it's
weird to import from under a binary dir.
pkg/ is much more reasonable dir for a widely used
helper package.
Make as much code as possible shared between all OSes.
In particular main is now common across all OSes.
Make more code shared between executor and csource
(in particular, loop function and threaded execution logic).
Also make loop and threaded logic shared across all OSes.
Make more posix/unix code shared across OSes
(e.g. signal handling, pthread creation, etc).
Plus other changes along similar lines.
Also support test OS in executor (based on portable posix)
and add 4 arches that cover all execution modes
(fork server/no fork server, shmem/no shmem).
This change paves way for testing of executor code
and allows to preserve consistency across OSes and executor/csource.
Only akaros needs OS, because the rest assume host OS.
But speciying OS for all OSes breaks patch testing on syzbot
because old execprog does not have os flag.
Provide stats and logs for failed repro and save it in manager.
In particular log is useful for failed repros,
currently there is no visibility into why bugs
failed to reproduce.
I had missed that once hardening is enabled, it automatically disables
any exposure of JITed addresses, therefore when crashes or warnings are
thrown we don't unwind beyond a helper function. For now disable hardening.
After merge window I'll see if it's possible to detangle the case where
kernel queries kallsyms internally to find function names whenever a WARN
or BUG is thrown. If that's not possible easily, we can potentially add a
harden mode 3 which does hardening but does not disable kallsyms exposure
and then set this here for tools like syzkaller.
Fixes: ac9b19d2e4 ("bpf: enable hardening mode 1 for jited images")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
[dvyukov: also updated dashboard/config/upstream.sysctl]
Currently all (linux-specific) suppressions are hardcoded in mgrconfig.
This is very wrong. Move them to pkg/report and allow to specify per OS.
Add gvisor-specific suppressions.
This required a bit of refactoring. Introduce mgrconfig.KernelObj finally.
Make report.NewReporter and vm.Create accept mgrconfig directly
instead of passing it as multiple scattered args.
Remove tools/syz-parse and it always did the same as tools/syz-symbolize.
Simplify global vars in syz-manager/cover.go.
Create reporter eagerly in manager. Use sort.Slice more.
Overall -90 lines removed.
Currently host feature detection/setup code is spread
across platform-independent fuzzer code, pkg/host, pkg/ipc
and executor.
Move this all into pkg/host and show readable info
about features on manager start.
Fixes#46
This will harden non-root programs from kernel side, but not
root-only ones. Helps also to increase coverage a bit since
syzkaller generates programs for both cases.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
We currently have native cross-compilation logic duplicated
in Makefile and in sys/targets. Some pieces are missed in one
place, some are in another. Only pkg/csource knows how to check
for -static support.
Move all CC/CFLAGS logic to sys/targets and pull results in Makefile.
This should make Makefile work on distros that have broken x86_64-linux-gnu-gcc,
now we will use just gcc. And this removes the need to define NOSTATIC,
as it's always auto-detected.
This also paves the way for making pkg/csource work on OSes other than Linux.
This probably can break some things.
I feel that tun setup can affect other things even if
syz_emit_ethernet/syz_extract_tcp_res are not used.
So it can make sense to setup tun even if they are not used.
But let's be more careful.
loop is much more standard than nbd and does not require additional modules.
nbd broke on Debian rolling.
loop also allows parallel execution thanks to losetup -f.
Use loop instead of nbd.
Also improve cleanup logic and add one missing sudo.
Update #501
1. Account for the fact that go can be already in path.
2. Unset GOROOT in case it is set already.
3. export variables (not sure how it worked for me).
There is no point in using sancov,
it does not do anything other than transforming
binary format to a useful text format.
Write out text format directly.
The new pseudo syscall allows opening sockets that can only
be created in init net namespace (BLUETOOTH, NFC, LLC).
Use it to open these sockets.
Unfortunately this only works with sandbox none at the moment.
The problem is that setns of a network namespace requires CAP_SYS_ADMIN
in the target namespace, and we've lost all privs in the init namespace
during creation of a user namespace.
syz-fuzzer organically grew from a small nice main function
into a huge single-file monster with tons of global state.
Start refactoring it into something more managable.
This change separates 2 things:
1. Proc: a single fuzzing process (ipc.Env wrapper).
2. WorkQueue: holds global non-fuzzing work items.
More work needed, but this is good first step.
Currently threaded/collide are global environment flags.
It can be useful to turn off collider during some executions
(minimization, triage, etc).
Make them per-program options.
Currently getting a complete report requires a complex,
multi-step dance (including getting information that
external users are not interested in -- guilty file).
Simplify interface down to 2 functions: Parse and Symbolize.
Parse does what it did before, Symbolize symbolizes report
and fills in maintainers. This simplifies both implementations
of Reporter interface and all users of the interface.
Potentially we could get this down to 1 function Parse
that does everything. However, (1) Symbolize can fail,
while Parse cannot, (2) usually we want to ignore (log)
Symbolize errors, but otherwise proceed with the report,
(3) repro does not need symbolization for all but the
last report.
Whole raw output is indivisble part of Report,
currently we always pass Output separately along with Report.
Make Output a Report field.
Then, put whole Report into manager Crash and repro context and Result.
There is little point in passing Report as aa bunch of separate fields.
This allows callers to get access to Report.Corrupted.
Better than adding 6-th return value and will allow
to pipe other report properties if necessary.
We currently have several names for crash attributes, which is disturbing.
E.g. crash title is called "Title" or "Desc". Name them consistently.
Title - single line bug identity.
Report - whole crash text.
Log - whole fuzzer/kernel output.
syz-execprog doesn't utilize info about fault injections from a prog log.
Since syz-execprog is used by the repro package to reproduce crashes,
crashes caused by fault injections might not reproduce.
This commit adds tools/check_links.py script, that checks that all local
links from documentation files are valid; fixes some of the invalid links
that we had; and makes travis buildbot check them as well.
During smashing we know what call gave new coverage,
so we can concentrate just on it.
This helps to reduce amount of hints generated (we have too many of them).
Now each prog function accepts the desired target explicitly.
No global, implicit state involved.
This is much cleaner and allows cross-OS/arch testing, etc.
Large overhaul moves syscalls and arg types from sys to prog.
Sys package now depends on prog and contains only generated
descriptions of syscalls.
Introduce prog.Target type that encapsulates all targer properties,
like syscall list, ptr/page size, etc. Also moves OS-dependent pieces
like mmap call generation from prog to sys.
Update #191
A hint is basically a tuple consisting of a pointer to an argument
in one of the syscalls of a program and a value, which should be
assigned to that argument.
A simplified version of hints workflow looks like this:
1. Fuzzer launches a program and collects all the comparisons' data
for every syscall in the program.
2. Next it tries to match the obtained comparison operands' values
vs. the input arguments' values.
3. For every such match the fuzzer mutates the program by
replacing the pointed argument with the saved value.
4. If a valid program is obtained, then fuzzer launches it and
checks if new coverage is obtained.
This commit includes:
1. All the code related to hints generation, parsing and mutations.
2. Fuzzer functions to launch the process.
3. Some new stats gathered by fuzzer and manager, related to hints.
4. An updated version of execprog to test the hints process.
The old parser in sys/sysparser is too hacky, difficult to extend
and drops debug info too early, so that we can't produce proper error messages.
Add a new parser that is build like a proper language parser
and preserves full debug info for every token.
On most distributions default grub target is i386-pc, which works.
However, on some default is x86_64-efi, which fails with:
grub-install: error: cannot find EFI directory.
Explicitly specify i386-pc target.
Exec total is affected by initial triage/minimize phase,
so two experiments can have the same execution speed
in the stable mode, but have constant diff due to the initial phase.
The one that is higher looks better, but that's not very important.
Provide execution speed characteristic that is not affected
by initial phase. It is not displayed by default.
If the script is aborted at an unfortunate point, it leaves the whole system broken.
E.g. we've seen that fdisk cannot update partition table until the next reboot.
If you really need to kill it, use a different signal. But better wait.
Currently we have unix permissions for new files/dirs
hardcoded throughout the code base. Some places use 0644,
some - 0640, some - 0600 and a variety of other constants.
Introduce osutil.MkdirAll/WriteFile that use the default
permissions and use them throughout the code base.
This makes permissions consistent and also allows to easily
change the permissions later if we change our minds.
Also merge pkg/fileutil into pkg/osutil as they become
dependent on each other. The line between them was poorly
defined anyway as both operate on files.
Currently syz-symbolize symbolizes whole input file.
Add a new mode (controlled with -report flag) when
it prints report as would be extracted by syz-manager.
vm/gce differs from other VM types in that it accepts image
in a weird, GCE-specific format (namely, image named disk.raw
is put into .tar.gz file). This makes it impossible to write
generic code that creates images for any VM types.
Make vm/gce accept just image like e.g. vm/qemu
and handle own specifics internally.
Sshkey is a property of image, which is in manager config.
Move sshkey to the same location as image.
The motivation for the move is as follows.
Continuous build produces an image and the key,
both need to be passed manager instance.
Continuous build system should not distinguish
different VM types and mess with their configs.
NOTE FOR USERS: this breaks manager configs again.
Hopefully the last time for now. Docs are updated.
We have 2 packages with the same name: pkg/config and syz-manager/config.
This leads to constant clashes. We either rename one to pkgconfig or
another to mgrconfig. This is not good and will become worse when/if
we have another program-specific config in a separate package.
Rename manager config to mgrconfig.
Other program-specific configs can use the same convention
in future -- fooconfig.
VM infrastructure currently has several problems:
- Config struct is complete mess with a superset of params for all VM types
- verification of Config is mess spread across several places
- there is no place where VM code could do global initialization
like creating GCE connection, uploading GCE image to GCS,
matching adb devices with consoles, etc
- it hard to add private VM implementations
such impl would need to add code to config package
which would lead to constant merge conflicts
- interface for VM implementation is mixed with interface for VM users
this does not allow to provide best interface for both of them
- there is no way to add common code for all VM implementations
This change solves these problems by:
- splitting VM interface for users (vm package) and VM interface
for VM implementations (vmimpl pacakge), this in turn allows
to add common code
- adding Pool concept that allows to do global initialization
and config checking at the right time
- decoupling manager config from VM-specific config
each VM type now defines own config
Note: manager configs need to be changed after this change:
VM-specific parts are moved to own "vm" subobject.
Note: this change also drops "local" VM type.
Its story was long unclear and there is now syz-stress which solves the same problem.
Introduce generic config.Load function that can be
reused across multiple programs (syz-manager, syz-gce, etc).
Move the generic config functionality to pkg/config package.
The idea is to move all helper (non-main) packages to pkg/ dir,
because we have more and more of them and they pollute the top dir.
Move the syz-manager config parts into syz-manager/config package.
This is mostly a cleanup change with little functional change.
In ipc.command.exec, remove the status fallback from the pipe to the
exit status. Once the executor is serving, it always writes the status
over the pipe; anything else is an error.
Remove the panic check in syz-stress, which is no longer needed.
Currently syz-symbolize uses report.Parse function
that extracts crash messages from console output.
Symbolize all console output instead.
E.g. there can be something on the console that is not crash.
If an external sandbox process wraps the executor, it may be helpful to
send a signal other than SIGKILL to the sandbox when the program times
out or fails to respond. This gives the sandbox the opportunity to emit
additional debugging information before exiting.
Add an 'abort' signal to ipc, which is sent to the executor before
SIGKILL. If the executor fails to exit within 5s, the signal is upgraded
to SIGKILL.
The default abort signal remains SIGKILL, maintaining existing behavior.
This commit adds Odroid C2 support to syzkaller.
It's now possible to specify "type": "odroid" in manager config.
Documentation on how to setup fuzzing with Odroid C2 board is here:
https://github.com/google/syzkaller/wiki/Setup:-Odroid-C2
Note, that after this change libusb-1.0-0-dev package should be
installed to build syzkaller.
Currently syzkaller uses per-call basic block (BB) coverage.
This change implements edge (not-per-call) coverage.
Edge coverage is more detailed than BB coverage as it captures
not-taken branches, looping, etc. So it provides better feedback signal.
This coverage is now called "signal" throughout the code.
BB code coverage is also collected as it is required for visualisation.
Not doing per-call coverage reduces corpus ~6-7x (from ~35K to ~5K),
this has profound effect on fuzzing efficiency.
kcovtrace is like strace but show kernel coverage collected with KCOV.
It is very simplistic at this point and does not support multithreaded processes, etc.
It can be used to understand, for example, exact location where kernel bails out
with an error for a particular syscall.
