golint suggests to comment all blank imports.
But actually we don't need whole sys, we can import only sys/linux.
Also rename target var to prevent name shadowing.
It's used only by 2 functions: genSockaddrNetlink and genIfrIfru.
Majority of functions just accept the straceType as argument,
which looks like a much more appropriate way to pass an argument to a function.
Amusingly, both functions already accept and use the straceType as argument.
This has number of advantages:
1. Tests are readable and writable.
The current checks [1] are neither.
2. Tests are much more compact.
3. Tests verify all aspects rather than just
1 aspect of the resulting program.
4. Tests are much less fragile.
5. Any diffs in the results will be more clearly visible.
[1]
switch a := p.Calls[1].Args[0].(type) {
case *prog.ResultArg:
if a.Res != p.Calls[0].Ret {
switch a := p.Calls[1].Args[0].(type) {
case *prog.ResultArg:
pipeSecondFd := p.Calls[0].Args[0].(*prog.PointerArg).Res.(*prog.GroupArg).Inner[1]
if a.Res != pipeSecondFd {
write := p.Calls[len(p.Calls)-2]
inotifyRmWatch := p.Calls[len(p.Calls)-1]
switch a := write.Args[0].Type().(type) {
case *prog.ResourceType:
if a.TypeName != "fd" {
t.Fatalf("expected first argument of write to have type fd, got: %s", a.TypeName)
}
default:
t.Fatalf("first argument of write is not resource type: %s", a.Name())
}
switch a := inotifyRmWatch.Args[1].(type) {
case *prog.ResultArg:
b := a.Type().(*prog.ResourceType)
if b.TypeName != "inotifydesc" {
t.Fatalf("expected second argument of inotify_rm_watch to have type inoitfydesc, got: %s", b.TypeName)
}
if a.Res != p.Calls[2].Ret {
t.Fatalf("inotify_rm_watch's second argument should match the result of inotify_add_watch.")
}
}
sockaddr, ok := a.(*prog.PointerArg).Res.(*prog.GroupArg)
if !ok {
t.Fatalf("%s", a.Type().Name())
}
ipv4Addr, ok := sockaddr.Inner[2].(*prog.UnionArg)
if !ok {
t.Fatalf("expected 3rd argument to be unionArg, got %s", sockaddr.Inner[2].Type().Name())
}
optName := ipv4Addr.Option.Type().FieldName()
if !strings.Contains(optName, "rand") {
t.Fatalf("expected ip option to be random opt, got: %s", optName)
}
ip, ok := ipv4Addr.Option.(*prog.ConstArg)
if !ok {
t.Fatalf("ipv4Addr option is not IntType")
}
if ip.Val != expectedIp {
t.Fatalf("parsed != expected, %d != %d", ip.Val, expectedIp)
}
There are 2 bugs:
1. We always allocate 1 page, even if use more.
2. VMA addresses are not aligned, so most mmap-like functions fail with EINVAL.
The added test currently panics with "unaligned vma address".
Current code structuring has 2 problems:
1. parsing anything with proggen requires complex multistep dance including
- parsing data with parser
- walking the resulting tree manually and calling proggen on each
- then for each context
- calling FillOutMemory (unclear why it's not part of parsing)
- calling prog.Finalize
- checking is the program is not too large
All of this duplicated across trace2syz and tests.
And any new tests or fuzzers we will write will need to duplicate
all of this logic too.
2. As the result of this structuring, lots of proggen guts
and implementation details are exposed.
While none of the callers are actually intersted in Context details,
they are not interested in Context itself whatsoever.
What every caller wants is "here is data to parse, give me programs".
Add such function.
We already printed file name of the trace in parseTraces,
no need to print it again and again.
Consequently we don't need Filename in TraceTree.
If needed, caller can always log it before parsing,
or pass along with the TraceTree.
Prog.Finalize combines assignSizesCall, SanitizeCall and validate.
Intended for users who build own programs,
so that we don't need to expose all individual methods.
These set_robust_list and set_tid_address are issued by glibc
for every process/thread start.
Normal programs don't use them and it's unlikely we build
something interesting with them (e.g. we won't get real robust list in memory).
Skip them.
This is needed for both tools/syz-db and tools/syz-trace2syz.
Also, remove code to resolve SHA1 collisions.
Also, don't set db version as we actually want to minimize
and smash these programs like anything else
(not minimizing nor smashing them is only useful during tool testing).
* fixing weird merge error
* fixing presubmit
* fixing presubmit
* removing parsing code because of -Xraw option
* fix presubmit
* update
* deleting vma_call_handlers as we are currently skipping most vma calls. This simplifies memory_tracker as we don't need to keep track of vma allocations
* removing custom handling of bpf_instruction union
* removing ifconf parsing
* update
* removed all expression types and replaced them with constant types. removing ipv6_addr parsing while -Xraw is getting fixed. Removing constants.go
* removing ipv6 parsing
* presubmit
* moving direction check from ipv4_addr out to genUnion
* removing code that parses kcov
* removing redundant test
* removing custom code in generate unions to fill ipv4_addr
* proggen: changing order of imports to make external packages import first
fixing presubmit
* changing log messages to lower case to be consistent with other packages.
* removing pointer type and simplifying memory_tracker
removing comment
* moving context and return_cache to seaparate files
* deleting default argument generation when we should probably throw an error
* build/openbsd: minor cleanup (use tuples instead of maps)
* Grammar nits in comments.
* Simplify openbsd.Create, will defer when there's more than one error exit.
* pkg/build: Support copying kernel into GCE image
* Simple test for openbsd image copy build.
* Cleanup in case something failed before.
* Support multi-processor VMs on GCE.
* More debug
* Reformat
* OpenBSD gce image needs to be raw.
* GC
* Force format to GNU directly on Go 1.10 or newer.
* Use vmType passed as a parameter inside openbsd.go
* gofmt
* more fmt
* Can't use GENERIC.mp just yet.
* capitalize
* Copyright
We used to use len([]CallInfo) to check both, whether the slice is nil or
whether its length is zero. Since ProgInfo is not a slice, we need a
separate check for nil.
This patch add a new struct ProgInfo that for now holds info about each
call in a program []CallInfo, but in the future will be expanded with remote
coverage info. Update all the callers to use the new interface as well.
We can't cross-compile native binaries from just any OS to any other.
For most OSes we can do only native compilation.
Some can only be compiled from linux.
To date we avoided this problem completely (mostly assumed linux build OS).
Make this notion of what can build what explicit.
reportPrefixLen can become wrong after symbolization
if we symbolize any lines in the prefix.
Adjust reportPrefixLen during symbolization.
Automatic testing of this is problematic
because we would need to symbolize which requires
the object file with debug info.
Tested manually with syz-symbolize.
* Use cdn.openbsd.org not cloudflare. Install vmm-firmware explicitly.
* Use cdn.openbsd.org not cloudflare. Install vmm-firmware explicitly.
* set hw.smt=1: the underlying hw on VM is unknown so don't waste SMT.
We don't need consistent image after reboot since we always reimage.
We know of some use cases that don't use journalling, but don't know
of any that use journalling.
* tools/create-openbsd-gce-ci: use config from /syzkaller
* Use syzkaller copy of src instead of a separate one.
* Using /dev/null disk in vm.conf
* Use KVM, enable doas, no longer symlink.
* Use a dummmy.img
* Revert "Use a dummmy.img"
This reverts commit 656b24d5e4573dde5e95c6158852001c7241e65a.
Currently we choose block device to use (nbd/loop) based on SYZ_VM_TYPE.
Strictly saying these things are orthogonal.
losetup is broken on Ubuntu. qemu-nbd is broken on Debian.
Try to auto-detect what will work based on uname.
This updates the image creation tool to use Debian stretch (current stable)
instead of wheezy, which is very out of date. The only change needed here
was a hint to systemd to make the root filesystem read-write after booting.
Documentation has also been updated.
Upstream "selinux: fix mounting of cgroup2 under older policies"
commit fixes mounting of cgroup2 under wheezy selinux policy.
So don't disable selinux on start.
Create separate cmdline arguments that enable selinux and apparmor.
Support checking "blocked"/"unfinished" flags for calls.
Support test constanints, e.g. "requires: threaded" or "requires: -sandbox=setuid".
Some improvements in tools/syz-runtest.
Update #603
The tool is run as:
$ syz-runtest -config manager.config
This runs all programs from sys/*/test/* in different modes
on actual VMs and checks results.
Fixes#603
mgrconfig was used only by syz-manager initially,
but now it's used by a dozen of packages and it's
weird to import from under a binary dir.
pkg/ is much more reasonable dir for a widely used
helper package.
Make as much code as possible shared between all OSes.
In particular main is now common across all OSes.
Make more code shared between executor and csource
(in particular, loop function and threaded execution logic).
Also make loop and threaded logic shared across all OSes.
Make more posix/unix code shared across OSes
(e.g. signal handling, pthread creation, etc).
Plus other changes along similar lines.
Also support test OS in executor (based on portable posix)
and add 4 arches that cover all execution modes
(fork server/no fork server, shmem/no shmem).
This change paves way for testing of executor code
and allows to preserve consistency across OSes and executor/csource.
Only akaros needs OS, because the rest assume host OS.
But speciying OS for all OSes breaks patch testing on syzbot
because old execprog does not have os flag.
Provide stats and logs for failed repro and save it in manager.
In particular log is useful for failed repros,
currently there is no visibility into why bugs
failed to reproduce.
I had missed that once hardening is enabled, it automatically disables
any exposure of JITed addresses, therefore when crashes or warnings are
thrown we don't unwind beyond a helper function. For now disable hardening.
After merge window I'll see if it's possible to detangle the case where
kernel queries kallsyms internally to find function names whenever a WARN
or BUG is thrown. If that's not possible easily, we can potentially add a
harden mode 3 which does hardening but does not disable kallsyms exposure
and then set this here for tools like syzkaller.
Fixes: ac9b19d2e4 ("bpf: enable hardening mode 1 for jited images")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
[dvyukov: also updated dashboard/config/upstream.sysctl]
Currently all (linux-specific) suppressions are hardcoded in mgrconfig.
This is very wrong. Move them to pkg/report and allow to specify per OS.
Add gvisor-specific suppressions.
This required a bit of refactoring. Introduce mgrconfig.KernelObj finally.
Make report.NewReporter and vm.Create accept mgrconfig directly
instead of passing it as multiple scattered args.
Remove tools/syz-parse and it always did the same as tools/syz-symbolize.
Simplify global vars in syz-manager/cover.go.
Create reporter eagerly in manager. Use sort.Slice more.
Overall -90 lines removed.
Currently host feature detection/setup code is spread
across platform-independent fuzzer code, pkg/host, pkg/ipc
and executor.
Move this all into pkg/host and show readable info
about features on manager start.
Fixes#46
This will harden non-root programs from kernel side, but not
root-only ones. Helps also to increase coverage a bit since
syzkaller generates programs for both cases.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
We currently have native cross-compilation logic duplicated
in Makefile and in sys/targets. Some pieces are missed in one
place, some are in another. Only pkg/csource knows how to check
for -static support.
Move all CC/CFLAGS logic to sys/targets and pull results in Makefile.
This should make Makefile work on distros that have broken x86_64-linux-gnu-gcc,
now we will use just gcc. And this removes the need to define NOSTATIC,
as it's always auto-detected.
This also paves the way for making pkg/csource work on OSes other than Linux.
This probably can break some things.
I feel that tun setup can affect other things even if
syz_emit_ethernet/syz_extract_tcp_res are not used.
So it can make sense to setup tun even if they are not used.
But let's be more careful.
loop is much more standard than nbd and does not require additional modules.
nbd broke on Debian rolling.
loop also allows parallel execution thanks to losetup -f.
Use loop instead of nbd.
Also improve cleanup logic and add one missing sudo.
Update #501
1. Account for the fact that go can be already in path.
2. Unset GOROOT in case it is set already.
3. export variables (not sure how it worked for me).
There is no point in using sancov,
it does not do anything other than transforming
binary format to a useful text format.
Write out text format directly.
The new pseudo syscall allows opening sockets that can only
be created in init net namespace (BLUETOOTH, NFC, LLC).
Use it to open these sockets.
Unfortunately this only works with sandbox none at the moment.
The problem is that setns of a network namespace requires CAP_SYS_ADMIN
in the target namespace, and we've lost all privs in the init namespace
during creation of a user namespace.
syz-fuzzer organically grew from a small nice main function
into a huge single-file monster with tons of global state.
Start refactoring it into something more managable.
This change separates 2 things:
1. Proc: a single fuzzing process (ipc.Env wrapper).
2. WorkQueue: holds global non-fuzzing work items.
More work needed, but this is good first step.
Currently threaded/collide are global environment flags.
It can be useful to turn off collider during some executions
(minimization, triage, etc).
Make them per-program options.
Currently getting a complete report requires a complex,
multi-step dance (including getting information that
external users are not interested in -- guilty file).
Simplify interface down to 2 functions: Parse and Symbolize.
Parse does what it did before, Symbolize symbolizes report
and fills in maintainers. This simplifies both implementations
of Reporter interface and all users of the interface.
Potentially we could get this down to 1 function Parse
that does everything. However, (1) Symbolize can fail,
while Parse cannot, (2) usually we want to ignore (log)
Symbolize errors, but otherwise proceed with the report,
(3) repro does not need symbolization for all but the
last report.
Whole raw output is indivisble part of Report,
currently we always pass Output separately along with Report.
Make Output a Report field.
Then, put whole Report into manager Crash and repro context and Result.
There is little point in passing Report as aa bunch of separate fields.
This allows callers to get access to Report.Corrupted.
Better than adding 6-th return value and will allow
to pipe other report properties if necessary.
We currently have several names for crash attributes, which is disturbing.
E.g. crash title is called "Title" or "Desc". Name them consistently.
Title - single line bug identity.
Report - whole crash text.
Log - whole fuzzer/kernel output.
syz-execprog doesn't utilize info about fault injections from a prog log.
Since syz-execprog is used by the repro package to reproduce crashes,
crashes caused by fault injections might not reproduce.
This commit adds tools/check_links.py script, that checks that all local
links from documentation files are valid; fixes some of the invalid links
that we had; and makes travis buildbot check them as well.
During smashing we know what call gave new coverage,
so we can concentrate just on it.
This helps to reduce amount of hints generated (we have too many of them).
Now each prog function accepts the desired target explicitly.
No global, implicit state involved.
This is much cleaner and allows cross-OS/arch testing, etc.
Large overhaul moves syscalls and arg types from sys to prog.
Sys package now depends on prog and contains only generated
descriptions of syscalls.
Introduce prog.Target type that encapsulates all targer properties,
like syscall list, ptr/page size, etc. Also moves OS-dependent pieces
like mmap call generation from prog to sys.
Update #191
