If we deserialized a huge blob (larger than max blob size),
then we can get a negative size in the "Insert random bytes" case at:
if r := int(maxLen) - len(data); n > r {
n = r
}
Don't insert bytes if data is already larger than maxLen.
Default value for ProcType is 0 (same for all PID's).
Usually 0 either does not make sense at all or make different PIDs collide
(since we use ProcType to separate value ranges for different PIDs).
So don't change ProcType to 0 unless the type is explicitly marked as opt
(in that case we will also generate 0 anyway).
Fix several nasty bugs in minimization that could lead
to almost arbitrary results. These bugs affected both
crash minimization and corpus population.
Extend the randomized test to catch these bugs.
Add additional asserts to code to catch similar bugs in future.
Reported-by @xairy
Providing additional info, especially regarding syscall arguments, in reproducers
can be helpful. An example is device numbers passed to mknod(2).
This commit introduces an optional annotate function on a per target basis.
Example for the OpenBSD target:
$ cat prog.in
mknod(0x0, 0x0, 0x4503)
getpid()
$ syz-prog2c -prog prog.in
int main(void)
{
syscall(SYS_mmap, 0x20000000, 0x1000000, 3, 0x1012, -1, 0, 0);
syscall(SYS_mknod, 0, 0, 0x4503); /* major = 69, minor = 3 */
syscall(SYS_getpid);
return 0;
}
We may be in createResource but have no resources at all because of ANYRES
that are not in target.Resources.
This is actually the case for some test targets. We have resources there,
but syscalls that create them are disabled.
In such case we crash in Intn(0).
Check that we have some resources before calling Intn.
gometalinter says:
pkg/compiler/consts.go:192:⚠️ internal error: no range for "n" (vetshadow)
pkg/compiler/consts.go:197:⚠️ internal error: no range for "n" (vetshadow)
prog/encoding.go:862:⚠️ declaration of "v" shadows declaration at prog/encoding.go:852 (vetshadow)
This somehow happens only with Go1.11 but not 1.12 so wasn't detected locally.
The prog warnings looks legit.
The pkg/compiler warning was amusingly introduced to please golangci-lint checker,
revert that fix for now.
Differences in code formatting between Go versions cause constant
problems for us (https://github.com/golang/go/issues/25161).
Currently we support 1.9 and 1.10. Switch to newer 1.11 and 1.12.
Fixes#1013
Make pseudo checksums depend (via csumUses) on the arg it requires for
calculation. Otherwise we fail to assign addrs to those args during encoding
for execution. Also add a test.
C's \xHH hex constants in strings accept any number of hex digits
(not just 2 or 4). So later non-hex escaped chars glue to the \x construct.
Use \OOO instead as it accepts at most 3 octal digits.
Non-determinism is bad:
- it leads to flaky coverage reports
- it makes test failures non-reproducible
Remove 4 sources of non-determinism related to maps:
- file name generation
- string generation
- resource generation
- hints generation
All a test that ensures all main operations are fully deterministic.
We don't specify trailing unused args for some syscalls
(e.g. ioctl that does not use its arg).
Executor always filled tailing unsed args with 0's
but pkg/csource didn't. Some such syscalls actually
check that the unsed arg is 0 and as the result failed with C repro.
We could statically check and eliminate all such cases,
but it turns out the warning fires in 1500+ cases:
a3ace5a63f/gistfile1.txt
So instead fill such args with 0's in pkg/csource too.
Without this check programs may end up panicing in places far away
from the real cause. E.g.
worker# ./syz-fuzzer -executor=./syz-executor -name=vm-0 -arch=amd64 -manager=10.128.0.101:21386 -sandbox=setuid -procs=2 -v=0 -cover=true -debug=false -test=false
2004/02/03 12:11:11 fuzzer started
2004/02/03 12:11:11 dialing manager at 10.128.0.101:21386
2004/02/03 12:11:12 syscalls: 1
2004/02/03 12:11:12 code coverage: enabled
2004/02/03 12:11:12 comparison tracing: support is not implemented in syzkaller
2004/02/03 12:11:12 setuid sandbox: support is not implemented in syzkaller
2004/02/03 12:11:12 namespace sandbox: support is not implemented in syzkaller
2004/02/03 12:11:12 Android sandbox: support is not implemented in syzkaller
2004/02/03 12:11:12 fault injection: support is not implemented in syzkaller
2004/02/03 12:11:12 leak checking: support is not implemented in syzkaller
2004/02/03 12:11:12 net packet injection: enabled
2004/02/03 12:11:12 net device setup: support is not implemented in syzkaller
panic: invalid argument to Intn
goroutine 27 [running]:
math/rand.(*Rand).Intn(0xc000dff530, 0x0, 0x40)
/usr/local/go/src/math/rand/rand.go:169 +0x9c
github.com/google/syzkaller/prog.(*ChoiceTable).Choose(0xc000d92ec0, 0xc000dff530, 0xffffffffffffffff, 0xc000dff650)
/syzkaller/gopath/src/github.com/google/syzkaller/prog/prio.go:241 +0x1a0
github.com/google/syzkaller/prog.(*randGen).generateCall(0xc000e145a0, 0xc000c2a200, 0xc000ce7f80, 0x2348f1940, 0xc000ce3440, 0xc000e6ee01)
/syzkaller/gopath/src/github.com/google/syzkaller/prog/rand.go:451 +0x69
github.com/google/syzkaller/prog.(*Target).Generate(0xc00007f1e0, 0x8f8680, 0xc000ce3440, 0x1e, 0xc000d92ec0, 0x0)
/syzkaller/gopath/src/github.com/google/syzkaller/prog/generation.go:19 +0x2b2
main.(*Proc).loop(0xc000d92f40)
/syzkaller/gopath/src/github.com/google/syzkaller/syz-fuzzer/proc.go:93 +0x2a1
created by main.main
/syzkaller/gopath/src/github.com/google/syzkaller/syz-fuzzer/fuzzer.go:236 +0xfe2
AUTO arguments can be used for:
- consts
- lens
- pointers
For const's and len's AUTO is replaced with the natural value,
addresses for AUTO pointers are allocated linearly.
This greatly simplifies writing test programs by hand
as most of the time we want these natural values.
Update tests to use AUTO.
Add bulk of checks for strict parsing mode.
Probably not complete, but we can extend then in future as needed.
Turns out we can't easily use it for serialized programs
as they omit default args and during deserialization it looks like missing args.
Over time we relaxed parsing to handle all kinds of invalid programs
(excessive/missing args, wrong types, etc).
This is useful when reading old programs from corpus.
But this is harmful for e.g. reading test inputs as they can become arbitrary outdated.
For runtests which creates additional problem of executing not
what is actually written in the test (or at least what author meant).
Add strict parsing mode that does not tolerate any errors.
For now it just checks excessive syscall arguments.
Move target and vars into parser and make all
parsing functions methods of the parser.
This reduces number of args that we need to pass around
and eases adding more state that needs to be passed around.
There are 2 bugs:
1. We always allocate 1 page, even if use more.
2. VMA addresses are not aligned, so most mmap-like functions fail with EINVAL.
The added test currently panics with "unaligned vma address".
Prog.Finalize combines assignSizesCall, SanitizeCall and validate.
Intended for users who build own programs,
so that we don't need to expose all individual methods.
* fixing weird merge error
* fixing presubmit
* fixing presubmit
* removing parsing code because of -Xraw option
* fix presubmit
* update
* deleting vma_call_handlers as we are currently skipping most vma calls. This simplifies memory_tracker as we don't need to keep track of vma allocations
* removing custom handling of bpf_instruction union
* removing ifconf parsing
* update
* removed all expression types and replaced them with constant types. removing ipv6_addr parsing while -Xraw is getting fixed. Removing constants.go
* removing ipv6 parsing
* presubmit
* moving direction check from ipv4_addr out to genUnion
* removing code that parses kcov
* removing redundant test
* removing custom code in generate unions to fill ipv4_addr
* proggen: changing order of imports to make external packages import first
fixing presubmit
* changing log messages to lower case to be consistent with other packages.
* removing pointer type and simplifying memory_tracker
removing comment
* moving context and return_cache to seaparate files
* deleting default argument generation when we should probably throw an error
Filename generated escaping paths in the past.
The reason for the check during validation is to
wipe old program from corpuses. Now that they are
hopefully wiped everywhere, we can relax the check
to restrict only filename to not produce escaping paths,
but allow existing programs with escaping paths.
This is useful in particular if we generate syzkaller
programs from strace output.
Currently when we get target consts with target.ConstMap["name"]
during target initialization, we just get 0 for missing consts.
This is error-prone as we can mis-type a const, or a const may
be undefined only on some archs (as we have common unix code
shared between several OSes).
Check that all the consts are actually defined.
The check detects several violations, to fix them:
1. move mremap to linux as it's only defined on linux
2. move S_IFMT to openbsd, as it's only defined and used on openbsd
3. define missing MAP_ANONYMOUS for freebsd and netbsd
4. fix extract for netbsd
This avoids the issue of "android" not having any registered configurations
or syscalls / ioctls / etc, when built with GOOS=android.
This occurs when building in Google3, since --config=android_arm64 selects
the Android toolchain.
Currently we only generate either valid user-space pointers or NULL.
Extend NULL to a set of special pointers that we will use in programs.
All targets now contain 3 special values:
- NULL
- 0xfffffffffffffff (invalid kernel pointer)
- 0x999999999999999 (non-canonical address)
Each target can add additional special pointers on top of this.
Also generate NULL/special pointers for non-opt ptr's.
This restriction was always too restrictive. We may want to generate
them with very low probability, but we do want to generate them.
Also change pointers to NULL/special during mutation
(but still not in the opposite direction).
TestSerializeDeserializeRandom fails from time to time
because program is different after we serialize/deserialize it.
Turns out openbsd SanitizeCall is not idempotent.
Add a test for this and disable the logic for now.
Check that argument types match expected static types.
I.e. detect when, say, syscall argument is a resource,
but actual generated argument is a pointer.
Write coverage from unfinished syscalls.
Also detect when a syscall was blocked during execution,
even if it finished. Helpful for fallback coverage.
Fixes#580
Make as much code as possible shared between all OSes.
In particular main is now common across all OSes.
Make more code shared between executor and csource
(in particular, loop function and threaded execution logic).
Also make loop and threaded logic shared across all OSes.
Make more posix/unix code shared across OSes
(e.g. signal handling, pthread creation, etc).
Plus other changes along similar lines.
Also support test OS in executor (based on portable posix)
and add 4 arches that cover all execution modes
(fork server/no fork server, shmem/no shmem).
This change paves way for testing of executor code
and allows to preserve consistency across OSes and executor/csource.
Hints mutation could produce unsanitized calls.
Sanitize calls after hints mutation.
Also sanitize on load (in validate), because bad programs
can already be in corpuses. And it's just the right thing
to do because sanitization rules can change over time.
First we emitted fallbackSignalFlags inside of the loop,
while we need to this outside of the loop.
Second, make flags signal weaker otherwise we get all 256
signals for open, chmod, etc.
Third, simplify and speedup code.
Currently a call that both accepts and creates a resource
self-justifies itself and thus is always enabled.
A good example is accept call. Accepts are always self-enable
and thus enable all other syscalls that work with the socket.
Calculate TransitivelyEnabledCalls in the opposite direction
to resolve this. Start with empty set of enable syscalls,
then enable syscalls that don't accept any resources,
then enable syscalls that accept resources created by the
previous batch of syscalls, and so on.
This prevents self-enablement of accept.
The current code is total, unstructured mess.
Since we now have 1:1 type -> arg correspondence,
rework validation around args. This makes code
much cleaner and 30% shorter.
In resources.go, haveGettime is False when SyscallMap["clock_gettime"]
is nil.
In this code, there's a branch that's entered only if Gettime is False,
which appends SyscallMap["clock_gettime"] to resourceCtors. That is, it
appends nil to resourceCtors, then iterates through resourceCtors and
tries to dereference the .Name of each time, in this case, nil.Name.
This was causing a page fault on Fuchsia.
I'm not certain how the "standard" flow is supposed to work, since it
seems like any code that enters the `if cantCreate == "" && !haveGettime`
should fail... but, removing that section causes test failures, so let's
just enforce that SyscallMap["clock_gettime"] is non-nil.
If there's a better way to solve this, I'm open to suggestions.
Now file names become:
string[filename]
with a possibility of using other string features:
stringnoz[filename]
string[filename, CONST_SIZE]
and filename is left as type alias as it is commonly used:
type filename string[filename]
Use big-endian match/replace for both blobs and ints.
Sometimes we have unmarked blobs (no little/big-endian info);
for ANYBLOBs we intentionally lose all marking;
but even for marked ints we may need this too.
Consider that kernel code does not convert the data
(i.e. not ntohs(pkt->proto) == ETH_P_BATMAN),
but instead converts the constant (i.e. pkt->proto == htons(ETH_P_BATMAN)).
In such case we will see dynamic operand that does not
match what we have in the program.
Handle most of type changes, e.g. const is changed to struct,
or struct to pointers. In all these cases we create default args.
They may not give the coverage anymore, but still better than
losing them right away.
We now always create net namespace for testing,
so socket ports and other IDs do not overlap between
different test processes.
Proc types play badly with squashing packets to ANYBLOB.
To squash into a block we need concrete value, but it depends
on process id.
Removing proc also makes tun setup and address descriptions simpler.
We currently use -1 as default value for resources
when the actual value is not available.
-1 is good for fd's, but is not the right default
value for pointers/keys/etc.
Pass from prog and use in executor proper default
value for resources.
Even during mutation of a call we want to analyze whole program
to find all used addresses (rather then stop on the selected call).
Also update address during ANY mutation if size has increased.
Squash complex structs into flat byte array and mutate this array
with generic blob mutations. This allows to mutate what we currently
consider as paddings and add/remove paddings from structs, etc.
Fix alignemnt calculation for packed structs with alignment and bitfields.
Amusingly this affected only a single real struct -- ipv6_fragment_ext_header.
1. mmap all memory always, without explicit mmap calls in the program.
This makes lots of things much easier and removes lots of code.
Makes mmap not a special syscall and allows to fuzz without mmap enabled.
2. Change address assignment algorithm.
Current algorithm allocates unmapped addresses too frequently
and allows collisions between arguments of a single syscall.
The new algorithm analyzes actual allocations in the program
and places new arguments at unused locations.
There are 2 bugs currently:
1. mutationArgs recurses into special types,
even though they must be mutated as the whole only.
2. When mutationArgs is called from Gen.MutateArg,
it included the top special type as well,
it must not because at this point only the subargs
must be mutated.
Fix both problems.
Make Foreach* callback accept the arg and a context struct
that can contain lots of aux info.
This (1) removes lots of unuser base/parent args,
(2) provides foundation for stopping recursion,
(3) allows to merge foreachSubargOffset.
Put the underflow entry at the end.
Entries must end on an unconditional, non-goto entry,
otherwise fallthrough from the last entry is invalid.
Add arp tables support.
Split unspec matches/targets to unspec and inet.
Reset ipv6 and arp tables in executor.
Fix number of counters in tables.
Plus a bunch of assorted fixes for matches/targets.
