There are 2 bugs currently:
1. mutationArgs recurses into special types,
even though they must be mutated as the whole only.
2. When mutationArgs is called from Gen.MutateArg,
it included the top special type as well,
it must not because at this point only the subargs
must be mutated.
Fix both problems.
Make Foreach* callback accept the arg and a context struct
that can contain lots of aux info.
This (1) removes lots of unuser base/parent args,
(2) provides foundation for stopping recursion,
(3) allows to merge foreachSubargOffset.
Put the underflow entry at the end.
Entries must end on an unconditional, non-goto entry,
otherwise fallthrough from the last entry is invalid.
Add arp tables support.
Split unspec matches/targets to unspec and inet.
Reset ipv6 and arp tables in executor.
Fix number of counters in tables.
Plus a bunch of assorted fixes for matches/targets.
This reduces size of a corpus in half.
We store corpus on manager and on hub,
so this will reduce their memory consumption.
But also makes large programs more readable.
Unions with only 1 field are not actually unions,
and can always be replaced with the option type.
However, they are still useful when there will be
more options in future but currently only 1 is described.
Alternatives are:
- not using union (but then all existing programs will be
broken when union is finally introduced)
- adding a fake field (ugly and reduces fuzzer efficiency)
Allow unions with only 1 field.
Consider the following example:
type len_templ1[DATA1, DATA2] {
data DATA1
inner len_temp2[DATA2]
}
type len_temp2[DATA] {
data DATA
len len[len_templ1, int8]
}
Here len refers to a parent struct, but the struct is a template,
so it's actual name is something like "len_templ1[int8, int16]".
Currently this does not work as compiler barks at incorrect
len target.
Make this work.
Few managers recently crashed with:
panic: syscall mknod$loop: per proc arg 'proc' has bad value '4294967295'
panic: sync: unlock of unlocked mutex
goroutine 35438 [running]:
sync.(*Mutex).Unlock(0xc42166e0c8)
sync/mutex.go:184 +0xc1
panic(0xb98980, 0xc448971aa0)
runtime/panic.go:491 +0x283
main.(*Manager).Connect(0xc42166e000, 0xc42056d060, 0xc42038f000, 0x0, 0x0)
syz-manager/manager.go:868 +0x11cc
And a similar issue was reported on mailing list.
It's unclear where these bogus programs come from.
It seems that hub was somehow involved here.
4294967295 is (uint32)-1 which is trucated special
value for proc types.
The test did not uncover any bugs, bug since I wrote it
and it looks like a useful test, let's commit it anyway.
Generated program always uses pid=0 even when there are multiple processes.
Make each process use own pid.
Unfortunately required to do quite significant changes to prog,
because the current format only supported fixed pid.
Fixes#490
Factor out program parsing from pkg/csource.
csource code that parses program and at the same time
formats output is very messy and complex.
New aproach also allows to understand e.g.
when a call has copyout instructions which is
useful for better C source output.
Introduce isUsed(arg) helper, use it in several places.
Move method definitions closer to their types.
Simplify presence check for ArgUsed.Used() in several places.
Fixes#188
We now will write just ""/1000 to denote a 1000-byte output buffer.
Also we now don't store 1000-byte buffer in memory just to denote size.
Old format is still parsed.
Fixes#460
File names, crypto algorithm names, etc in programs are completely unreadable:
bind$alg(r0, &(0x7f0000408000)={0x26, "6861736800000000000000000000",
0x0, 0x0, "6d6435000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000
00000000000"}, 0x58)
Introduce another format for printable strings.
New args are denoted by '' ("" for old args).
New format is enabled for printable chars, \x00
and \t, \r, \n.
Example:
`serialize(&(0x7f0000408000)={"6861736800000000000000000000", "4849000000"})`,
vs:
`serialize(&(0x7f0000408000)={'hash\x00', 'HI\x00'})`,
We now have a bunch of targets compiled into each binary.
All targets are initialized eagerly on startup time.
As the result a do nothing binary starts for ~0.58s and
consumes ~21MB.
Initialize targets lazily. Usually only 1 target is used.
This reduces startup time to ~0.00s and memory consumption
to ~5.4MB.
Clone program only once.
Preallocate slices in clone.
Remove the clone full mode.
Always mutate args in place.
Allocate replacers map lazily.
Don't allocate res map at all (calculate valus on the go).
Remove sliceToUint64, pad.
benchmark old ns/op new ns/op delta
BenchmarkHints 122100048 7466013 -93.89%
The race initially showed up on the new benchmark (see race report below).
The race indicated a wrong call passed to replaceArg,
as the result we sanitized the wrong call and left the new call un-sanitized.
Fix this.
Add test that exposes this.
Run benchmarks in race mode during presubmit
(benchmarks have higher chances of uncovering races than tests).
WARNING: DATA RACE
Write at 0x00c42000d3f0 by goroutine 18:
github.com/google/syzkaller/sys/linux.(*arch).sanitizeCall()
sys/linux/init.go:155 +0x256
github.com/google/syzkaller/sys/linux.(*arch).(github.com/google/syzkaller/sys/linux.sanitizeCall)-fm()
sys/linux/init.go:42 +0x4b
github.com/google/syzkaller/prog.(*Prog).replaceArg()
prog/prog.go:357 +0x239
github.com/google/syzkaller/prog.generateHints.func2()
prog/hints.go:105 +0x124
github.com/google/syzkaller/prog.checkConstArg()
prog/hints.go:128 +0xf3
github.com/google/syzkaller/prog.generateHints()
prog/hints.go:120 +0x495
github.com/google/syzkaller/prog.(*Prog).MutateWithHints.func1()
prog/hints.go:72 +0x67
github.com/google/syzkaller/prog.foreachSubargImpl.func1()
prog/analysis.go:86 +0x9f
github.com/google/syzkaller/prog.foreachSubargImpl()
prog/analysis.go:104 +0xc8
github.com/google/syzkaller/prog.foreachArgArray()
prog/analysis.go:113 +0x89
github.com/google/syzkaller/prog.foreachArg()
prog/analysis.go:121 +0x50
github.com/google/syzkaller/prog.(*Prog).MutateWithHints()
prog/hints.go:71 +0x18e
github.com/google/syzkaller/prog.BenchmarkHints.func1()
prog/hints_test.go:477 +0x77
testing.(*B).RunParallel.func1()
testing/benchmark.go:626 +0x156
Previous read at 0x00c42000d3f0 by goroutine 17:
github.com/google/syzkaller/prog.clone()
prog/clone.go:38 +0xbaa
github.com/google/syzkaller/prog.(*Prog).cloneImpl()
prog/clone.go:21 +0x17f
github.com/google/syzkaller/prog.generateHints()
prog/hints.go:95 +0xd0
github.com/google/syzkaller/prog.(*Prog).MutateWithHints.func1()
prog/hints.go:72 +0x67
github.com/google/syzkaller/prog.foreachSubargImpl.func1()
prog/analysis.go:86 +0x9f
github.com/google/syzkaller/prog.foreachSubargImpl()
prog/analysis.go:104 +0xc8
github.com/google/syzkaller/prog.foreachArgArray()
prog/analysis.go:113 +0x89
github.com/google/syzkaller/prog.foreachArg()
prog/analysis.go:121 +0x50
github.com/google/syzkaller/prog.(*Prog).MutateWithHints()
prog/hints.go:71 +0x18e
github.com/google/syzkaller/prog.BenchmarkHints.func1()
prog/hints_test.go:477 +0x77
testing.(*B).RunParallel.func1()
testing/benchmark.go:626 +0x156
Hints for data args don't work.
We do all the work, but at the final stage we patch
arg in the _old_ program, not in the _new_ one.
So programs passed to the callback are all the same
and don't contain any mutations.
Tests did not catch this because they work right before that point
(don't test the actual interface function MutateWithHints).
Fix that and add a test that catches this.
In some cases we need to extend a buffer by a large
margin to pass the next if in kernel (a size check).
Currently we only append a single byte, so we can
never pass the if incrementally (size is always
smaller than threshold, so 1-byte larger inputs
are not added to corpus).
Boot and minimally test images before declaring them as good
and switching to using them.
If image build/boot/test fails, upload report about this to dashboard.
For string[N] we successfully deserialize a string of any length.
Similarly for a fixed-size array[T, N] we successfully deserialize
an array of any size.
Such programs later crash in foreachSubargOffset because static size
Type.Size() does not match what we've calculated iterating over fields.
The crash happens only in SerializeForExec in syz-fuzzer,
which is especially bad.
Fix this from both sides:
1. Validate sizes of arrays/buffers in Validate.
2. Repair incorrect sizes in Deserialize.
There is effectively infinite number of possible crypto
algorithm names due to templates. Plus there is tricky
relation between algorithms and algorithm type names.
This change adds custom mutator for sockaddr_alg struct
to improve variance in generated algorithms.
