Commit Graph

74 Commits

Author SHA1 Message Date
Dmitry Vyukov
fa6c7b7080 sys/linux: prohibit opening /proc/self/exe
Fuzzer manages to open it and do bad things with it.
Prevent it from doing so.
2019-02-08 16:30:44 +01:00
Dmitry Vyukov
9e8a45fe27 tools/syz-trace2syz/proggen: replace memoryTracker with prog.memAlloc 2018-12-07 12:44:45 +01:00
Dmitry Vyukov
ab4b148b34 prog: add Prog.Finalize
Prog.Finalize combines assignSizesCall, SanitizeCall and validate.
Intended for users who build own programs,
so that we don't need to expose all individual methods.
2018-12-06 18:56:08 +01:00
Dmitry Vyukov
ceaec61a83 prog: export Type.DefaultArg
It's effectively exported anyway.
So export it the proper way.
2018-12-06 18:55:46 +01:00
Dmitry Vyukov
e8dd2c6713 prog: add concept of "special pointers"
Currently we only generate either valid user-space pointers or NULL.
Extend NULL to a set of special pointers that we will use in programs.
All targets now contain 3 special values:
 - NULL
 - 0xfffffffffffffff (invalid kernel pointer)
 - 0x999999999999999 (non-canonical address)
Each target can add additional special pointers on top of this.

Also generate NULL/special pointers for non-opt ptr's.
This restriction was always too restrictive. We may want to generate
them with very low probability, but we do want to generate them.

Also change pointers to NULL/special during mutation
(but still not in the opposite direction).
2018-08-30 21:45:03 -07:00
Dmitry Vyukov
c00da3df66 prog: collect all prog comments
Parse and collect and prog comments.
Will be needed for runtest annotations
(e.g. "requires threaded mode", etc).
2018-08-08 13:07:49 +02:00
Dmitry Vyukov
c8643744fe prog: refactor defaultArg/isDefaultArg
Refactor from single-big-switch to type methods.

Update #538
2018-08-02 16:57:31 +02:00
Dmitry Vyukov
97bce4e2ce prog: refactor program serialization
Make argument serialization Arg method.
This eliminates a very long function
that serializes all arguments.

Update #538
2018-07-31 18:43:50 +02:00
Dmitry Vyukov
68faa52582 prog: parse comments in serialized programs
Remember per-call comments, will be useful for annotating tests.
Also support this form:
call() # comment
2018-07-27 10:22:23 +02:00
Dmitry Vyukov
306ca0571c prog, pkg/compiler: support fmt type
fmt type allows to convert intergers and resources
to string representation.
2018-07-08 22:52:24 +02:00
Dmitry Vyukov
6a0382b543 prog: rework validation code
The current code is total, unstructured mess.
Since we now have 1:1 type -> arg correspondence,
rework validation around args. This makes code
much cleaner and 30% shorter.
2018-05-05 11:43:00 +02:00
Dmitry Vyukov
afe402d20a prog: make c.Ret optional
No reason to allocate return value if there is no return type.
c.Ret == nil is the reasonable indication that this is a "void" call.
2018-05-05 10:25:45 +02:00
Dmitry Vyukov
9dfb5efa91 prog: simplify code
Now that we don't have ReturnArg and only ResultArg's refer
to other ResultArg's we can remove ArgUser/ArgUsed and
devirtualize lots of code.
2018-05-05 10:13:04 +02:00
Dmitry Vyukov
5ca897bd50 prog: remove ReturnArg
It's not all that needed.
2018-05-05 09:08:48 +02:00
Dmitry Vyukov
36d1c4540a all: fix gometalinter warnings
Fix typos, non-canonical code, remove dead code, etc.
2018-03-08 18:48:26 +01:00
Dmitry Vyukov
d0790618dc prog: fix isDefaultArg
Test that isDefaultArg returns true for result of DefaultArg.
Fix few bugs uncovered by this test.
2018-03-08 12:02:17 +01:00
Dmitry Vyukov
4bad061006 prog: extend some error messages 2018-03-05 12:58:29 +01:00
Dmitry Vyukov
14dae29c2a executor: use proper default values for resources
We currently use -1 as default value for resources
when the actual value is not available.
-1 is good for fd's, but is not the right default
value for pointers/keys/etc.
Pass from prog and use in executor proper default
value for resources.
2018-02-26 15:00:46 +01:00
Dmitry Vyukov
9fe8aa42c5 prog: add arbitrary mutation of complex structs
Squash complex structs into flat byte array and mutate this array
with generic blob mutations. This allows to mutate what we currently
consider as paddings and add/remove paddings from structs, etc.
2018-02-25 18:22:02 +01:00
Dmitry Vyukov
44f66b4026 prog: return concrete arg types from Make functions
This removes a bunch of type assertions and makes code type safer.
2018-02-24 17:00:34 +01:00
Dmitry Vyukov
75a7c5e2d1 prog: rework address allocation
1. mmap all memory always, without explicit mmap calls in the program.
This makes lots of things much easier and removes lots of code.
Makes mmap not a special syscall and allows to fuzz without mmap enabled.

2. Change address assignment algorithm.
Current algorithm allocates unmapped addresses too frequently
and allows collisions between arguments of a single syscall.
The new algorithm analyzes actual allocations in the program
and places new arguments at unused locations.
2018-02-19 21:48:20 +01:00
Dmitry Vyukov
85d1218f41 prog: rework foreachArg
Make Foreach* callback accept the arg and a context struct
that can contain lots of aux info.
This (1) removes lots of unuser base/parent args,
(2) provides foundation for stopping recursion,
(3) allows to merge foreachSubargOffset.
2018-02-19 21:48:20 +01:00
Dmitry Vyukov
033b610ec9 sys/linux: improve netfilter descriptions
Put the underflow entry at the end.
Entries must end on an unconditional, non-goto entry,
otherwise fallthrough from the last entry is invalid.

Add arp tables support.

Split unspec matches/targets to unspec and inet.

Reset ipv6 and arp tables in executor.

Fix number of counters in tables.

Plus a bunch of assorted fixes for matches/targets.
2018-02-09 20:14:33 +01:00
Dmitry Vyukov
d973f28294 prog: don't serialize default arguments
This reduces size of a corpus in half.
We store corpus on manager and on hub,
so this will reduce their memory consumption.
But also makes large programs more readable.
2018-02-01 15:20:12 +01:00
Dmitry Vyukov
08146b1a84 sys/linux: extend netfilter descriptions 2018-01-27 17:08:43 +01:00
Dmitry Vyukov
5d7477249b prog: remove unused UnionArg.OptionType 2018-01-27 17:08:43 +01:00
Dmitry Vyukov
a8927abe6c prog: support opt for proc types 2018-01-06 17:40:49 +01:00
Dmitry Vyukov
3645389673 pkg/csource: fix handling of proc types
Generated program always uses pid=0 even when there are multiple processes.
Make each process use own pid.

Unfortunately required to do quite significant changes to prog,
because the current format only supported fixed pid.

Fixes #490
2017-12-22 11:59:46 +01:00
Dmitry Vyukov
dcfdc02b77 prog: minor refactoring around arguments
Introduce isUsed(arg) helper, use it in several places.
Move method definitions closer to their types.
Simplify presence check for ArgUsed.Used() in several places.
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
8ef0050706 prog: don't serialize output data args
Fixes #188

We now will write just ""/1000 to denote a 1000-byte output buffer.
Also we now don't store 1000-byte buffer in memory just to denote size.
Old format is still parsed.
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
0da2fed84f prog: simplify MakeResultArg
Fixes #445
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
5e7b20cfc3 prog: fix a data race
The race initially showed up on the new benchmark (see race report below).
The race indicated a wrong call passed to replaceArg,
as the result we sanitized the wrong call and left the new call un-sanitized.

Fix this.
Add test that exposes this.
Run benchmarks in race mode during presubmit
(benchmarks have higher chances of uncovering races than tests).

WARNING: DATA RACE
Write at 0x00c42000d3f0 by goroutine 18:
  github.com/google/syzkaller/sys/linux.(*arch).sanitizeCall()
      sys/linux/init.go:155 +0x256
  github.com/google/syzkaller/sys/linux.(*arch).(github.com/google/syzkaller/sys/linux.sanitizeCall)-fm()
      sys/linux/init.go:42 +0x4b
  github.com/google/syzkaller/prog.(*Prog).replaceArg()
      prog/prog.go:357 +0x239
  github.com/google/syzkaller/prog.generateHints.func2()
      prog/hints.go:105 +0x124
  github.com/google/syzkaller/prog.checkConstArg()
      prog/hints.go:128 +0xf3
  github.com/google/syzkaller/prog.generateHints()
      prog/hints.go:120 +0x495
  github.com/google/syzkaller/prog.(*Prog).MutateWithHints.func1()
      prog/hints.go:72 +0x67
  github.com/google/syzkaller/prog.foreachSubargImpl.func1()
      prog/analysis.go:86 +0x9f
  github.com/google/syzkaller/prog.foreachSubargImpl()
      prog/analysis.go:104 +0xc8
  github.com/google/syzkaller/prog.foreachArgArray()
      prog/analysis.go:113 +0x89
  github.com/google/syzkaller/prog.foreachArg()
      prog/analysis.go:121 +0x50
  github.com/google/syzkaller/prog.(*Prog).MutateWithHints()
      prog/hints.go:71 +0x18e
  github.com/google/syzkaller/prog.BenchmarkHints.func1()
      prog/hints_test.go:477 +0x77
  testing.(*B).RunParallel.func1()
      testing/benchmark.go:626 +0x156

Previous read at 0x00c42000d3f0 by goroutine 17:
  github.com/google/syzkaller/prog.clone()
      prog/clone.go:38 +0xbaa
  github.com/google/syzkaller/prog.(*Prog).cloneImpl()
      prog/clone.go:21 +0x17f
  github.com/google/syzkaller/prog.generateHints()
      prog/hints.go:95 +0xd0
  github.com/google/syzkaller/prog.(*Prog).MutateWithHints.func1()
      prog/hints.go:72 +0x67
  github.com/google/syzkaller/prog.foreachSubargImpl.func1()
      prog/analysis.go:86 +0x9f
  github.com/google/syzkaller/prog.foreachSubargImpl()
      prog/analysis.go:104 +0xc8
  github.com/google/syzkaller/prog.foreachArgArray()
      prog/analysis.go:113 +0x89
  github.com/google/syzkaller/prog.foreachArg()
      prog/analysis.go:121 +0x50
  github.com/google/syzkaller/prog.(*Prog).MutateWithHints()
      prog/hints.go:71 +0x18e
  github.com/google/syzkaller/prog.BenchmarkHints.func1()
      prog/hints_test.go:477 +0x77
  testing.(*B).RunParallel.func1()
      testing/benchmark.go:626 +0x156
2017-12-08 11:33:30 +01:00
Dmitry Vyukov
1808de66ce prog: repair arrays/buffers with incorrect size in Deserialize
For string[N] we successfully deserialize a string of any length.
Similarly for a fixed-size array[T, N] we successfully deserialize
an array of any size.
Such programs later crash in foreachSubargOffset because static size
Type.Size() does not match what we've calculated iterating over fields.
The crash happens only in SerializeForExec in syz-fuzzer,
which is especially bad.
Fix this from both sides:
1. Validate sizes of arrays/buffers in Validate.
2. Repair incorrect sizes in Deserialize.
2017-11-28 19:15:28 +01:00
Dmitry Vyukov
ddf7b3e065 sys/linux: improve AF_ALG alg name generation
There is effectively infinite number of possible crypto
algorithm names due to templates. Plus there is tricky
relation between algorithms and algorithm type names.

This change adds custom mutator for sockaddr_alg struct
to improve variance in generated algorithms.
2017-11-24 13:56:20 +01:00
Dmitry Vyukov
7e076b78b4 prog: export MakeData/UnionArg as we do for other arg types
Target code can use these to generate special structs.
2017-11-22 11:46:26 +01:00
Dmitry Vyukov
52a33fd516 prog: remove default target and all global state
Now each prog function accepts the desired target explicitly.
No global, implicit state involved.
This is much cleaner and allows cross-OS/arch testing, etc.
2017-09-15 16:02:37 +02:00
Dmitry Vyukov
ffe7e17368 prog, sys: move types to prog
Large overhaul moves syscalls and arg types from sys to prog.
Sys package now depends on prog and contains only generated
descriptions of syscalls.
Introduce prog.Target type that encapsulates all targer properties,
like syscall list, ptr/page size, etc. Also moves OS-dependent pieces
like mmap call generation from prog to sys.

Update #191
2017-09-05 15:52:42 +02:00
Dmitry Vyukov
4fc4702694 prog: dot-import sys
In preparation for moving sys types to prog to reduce later diffs.
2017-09-05 10:46:34 +02:00
Dmitry Vyukov
5db39ab953 sys: rename Call to Syscall
In preparation for moving sys types to prog
to avoid confusion between sys.Call and prog.Call.
2017-09-05 10:38:22 +02:00
Dmitry Vyukov
1c0d4caf7c sys: change BitfieldLast to BitfieldMiddle
That's the condition we always want.
Currently we always check:
t.BitfieldOffset() == 0 || t.BitfieldLast()
now can check just:
!t.BitfieldMiddle()
2017-09-04 20:51:56 +02:00
Dmitry Vyukov
399addc875 sys, pkg/compiler: move padding computation to compiler
This makes types constant during execution, everything is precomputed.
2017-09-04 20:25:23 +02:00
Dmitry Vyukov
838e336594 sys, prog: switch values to to uint64
We currently use uintptr for all values.
This won't work for 32-bit archs.
Moreover in some cases we use uintptr but assume
that it is always 64-bits (e.g. in encodingexec).
Switch everything to uint64.

Update #324
2017-08-19 10:16:23 +02:00
Dmitry Vyukov
0939075822 prog: reuse defaultArg
Reuse defaultArg in generateArg. There is code that does the same.
Also, don't generate pointer value for output arguments.
2017-08-09 10:38:38 +02:00
Dmitry Vyukov
c3ba5e72f5 prog: fix restoration of default arguments
Currently fails on:
 - pointers
 - VMAs
 - structs
 - fixed-size structs
2017-08-09 10:28:10 +02:00
Andrey Konovalov
1517bd9548 prog: generate missing syscall args when decoding
After a change in syscall description the number of syscall arguments
might change and some of the programs in corpus get invalidated.

This change makes syzkaller to generate missing arguments when decoding a
program as an attempt to fix and keep more programs from corpus.
2017-08-01 19:19:05 +02:00
Andrey Konovalov
cfc46d9d0b prog: split Arg into smaller structs
Right now Arg is a huge struct (160 bytes), which has many different fields
used for different arg kinds. Since most of the args we see in a typical
corpus are ArgConst, this results in a significant memory overuse.

This change:
- makes Arg an interface instead of a struct
- adds a SomethingArg struct for each arg kind we have
- converts all *Arg pointers into just Arg, since interface variable by
  itself contains a pointer to the actual data
- removes ArgPageSize, now ConstArg is used instead
- consolidates correspondence between arg kinds and types, see comments
  before each SomethingArg struct definition
- now LenType args that denote the length of VmaType args are serialized as
  "0x1000" instead of "(0x1000)"; to preserve backwards compatibility
  syzkaller is able to parse the old format for now
- multiple small changes all over to make the above work

After this change syzkaller uses twice less memory after deserializing a
typical corpus.
2017-07-17 14:34:09 +02:00
Andrey Konovalov
f919224c44 sys, executor: extract tcp sequence numbers from /dev/net/tun
This commit adds a new pseudo syscall syz_extract_tcp_res, that reads
a packet from /dev/net/tun and extracts tcp sequence numbers to be used
in subsequent packets.

As a result this syzkaller program:

mmap(&(0x7f0000000000/0x10000)=nil, (0x10000), 0x3, 0x32, 0xffffffffffffffff, 0x0)
r0 = socket$inet_tcp(0x2, 0x1, 0x0)
bind$inet(r0, &(0x7f0000001000)={0x2, 0x0, @empty=0x0, [0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0]}, 0x10)
listen(r0, 0x5)
syz_emit_ethernet(0x36, &(0x7f0000002000)={@local={[0xaa, 0xaa, 0xaa, 0xaa, 0xaa], 0x0}, @random="4c6112cc15d8", [], {{0x800, @ipv4={{0x5, 0x4, 0x0, 0x0, 0x28, 0x0, 0x0, 0x0, 0x6, 0x0, @remote={0xac, 0x14, 0x0, 0xbb}, @local={0xac, 0x14, 0x0, 0xaa}, {[]}}, @tcp={{0x1, 0x0, 0x42424242, 0x42424242, 0x0, 0x0, 0x5, 0x2, 0x0, 0x0, 0x0, {[]}}, {""}}}}}})
syz_extract_tcp_res(&(0x7f0000003000)={<r1=>0x42424242, <r2=>0x42424242}, 0x1, 0x0)
syz_emit_ethernet(0x38, &(0x7f0000004000)={@local={[0xaa, 0xaa, 0xaa, 0xaa, 0xaa], 0x0}, @remote={[0xbb, 0xbb, 0xbb, 0xbb, 0xbb], 0x0}, [], {{0x800, @ipv4={{0x5, 0x4, 0x0, 0x0, 0x2a, 0x0, 0x0, 0x0, 0x6, 0x0, @remote={0xac, 0x14, 0x0, 0xbb}, @local={0xac, 0x14, 0x0, 0xaa}, {[]}}, @tcp={{0x1, 0x0, r2, r1, 0x0, 0x0, 0x5, 0x10, 0x0, 0x0, 0x0, {[]}}, {"0c10"}}}}}})
r3 = accept$inet(r0, &(0x7f0000005000)={0x0, 0x0, @multicast1=0x0, [0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0]}, &(0x7f0000006000)=0x10)

established a TCP connection:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:20000           0.0.0.0:*               LISTEN      5477/a.out
tcp        2      0 172.20.0.170:20000      172.20.0.187:20001      ESTABLISHED 5477/a.out

Similar program for IPv6:

mmap(&(0x7f0000000000/0x10000)=nil, (0x10000), 0x3, 0x32, 0xffffffffffffffff, 0x0)
r0 = socket$inet6_tcp(0xa, 0x1, 0x0)
bind$inet6(r0, &(0x7f0000000000)={0xa, 0x1, 0x0, @empty={[0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0]}, 0x0}, 0x1c)
listen(r0, 0x5)
syz_emit_ethernet(0x4a, &(0x7f0000001000)={@local={[0xaa, 0xaa, 0xaa, 0xaa, 0xaa], 0x0}, @random="de895db1468d", [], {{0x86dd, @ipv6={0x0, 0x6, "a228af", 0x14, 0x6, 0x0, @remote={0xfe, 0x80, [0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0], 0x0, 0xbb}, @local={0xfe, 0x80, [0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0], 0x0, 0xaa}, {[], @tcp={{0x0, 0x1, 0x42424242, 0x42424242, 0x0, 0x0, 0x5, 0x2, 0x0, 0x0, 0x0, {[]}}, {""}}}}}}})
syz_extract_tcp_res(&(0x7f0000002000)={<r1=>0x42424242, <r2=>0x42424242}, 0x1, 0x0)
syz_emit_ethernet(0x4a, &(0x7f0000003000)={@local={[0xaa, 0xaa, 0xaa, 0xaa, 0xaa], 0x0}, @random="de895db1468d", [], {{0x86dd, @ipv6={0x0, 0x6, "a228af", 0x14, 0x6, 0x0, @remote={0xfe, 0x80, [0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0], 0x0, 0xbb}, @local={0xfe, 0x80, [0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0], 0x0, 0xaa}, {[], @tcp={{0x0, 0x1, r2, r1, 0x0, 0x0, 0x5, 0x10, 0x0, 0x0, 0x0, {[]}}, {""}}}}}}})
r3 = accept$inet6(r0, &(0x7f0000004000)={0x0, 0x0, 0x0, @empty={[0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0]}, 0x0}, &(0x7f0000005000)=0x1c)

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp6       0      0 :::20001                :::*                    LISTEN      5527/a.out
tcp6       0      0 fe80::aa:20001          fe80::bb:20000          ESTABLISHED 5527/a.out
2017-05-26 14:28:09 +02:00
Andrey Konovalov
ac0c70f74a prog, executor: move checksum computation to executor
This commit moves checksum computation to executor. This will allow to embed
dynamically generated values (like TCP sequence numbers) into packets.
2017-05-12 15:47:59 +02:00
Andrey Konovalov
63b16a5d5c prog, sys: add csum type, embed checksums for ipv4 packets
This change adds a `csum[kind, type]` type.
The only available kind right now is `ipv4`.
Using `csum[ipv4, int16be]` in `ipv4_header` makes syzkaller calculate
and embed correct checksums into ipv4 packets.
2017-01-25 20:31:13 +01:00
Andrey Konovalov
872e436375 prog, sys: fix padding varlen structs 2017-01-19 15:22:04 +01:00