68 Commits

Author SHA1 Message Date
Dmitry Vyukov
17b3eb97de prog: speed up foreachArgImpl
Don't generate garbage for ctx.
This speeds up mutation tests by 20%.
2020-05-18 11:34:42 +02:00
Dmitry Vyukov
58da4c35b1 prog: introduce Field type
Remvoe FieldName from Type and add a separate Field type
that holds field name. Use Field for struct fields, union options
and syscalls arguments, only these really have names.

Reduces size of sys/linux/gen/amd64.go from 5665583 to 5201321 (-8.2%).
Allows to not create new type for squashed any pointer.
But main advantages will follow, e.g. removing StructDesc,
using TypeRef in Arg, etc.

Update 
2020-05-02 12:16:06 +02:00
Dmitry Vyukov
e54e9781a4 prog: remove Dir from Type
Having Dir is Type is handy, but forces us to duplicate lots of types.
E.g. if a struct is referenced as both in and out, then we need to
have 2 copies and 2 copies of structs/types it includes.
If also prevents us from having the struct type as struct identity
(because we can have up to 3 of them).

Revert to the old way we used to do it: propagate Dir as we walk
syscall arguments. This moves lots of dir passing from pkg/compiler
to prog package.
Now Arg contains the dir, so once we build the tree, we can use dirs
as before.

Reduces size of sys/linux/gen/amd64.go from 6058336 to 5661150 (-6.6%).

Update 
2020-05-01 13:31:17 +02:00
Dmitry Vyukov
91db3ed8ce prog: add ignore_return and breaks_returns call attribtues
We had these hard-coded for fuchsia and linux accordingly.
Replace with call attributes.
2020-04-19 10:26:57 +02:00
Dmitry Vyukov
4272e40b21 prog: don't add fallback coverage after clone+ptrace
clone+ptrace combo cause fallback coverage explosion under gvisor.
Mechanics of that are unclear, but effect is very clear.
2020-01-15 18:04:21 +01:00
Dmitry Vyukov
b803944b58 prog: don't add fallback coverage after prctl
The same reason as with seccomp.
2020-01-15 17:35:31 +01:00
Dmitry Vyukov
20c6855484 prog: refactor bitfields representation
All callers of BitfieldMiddle just want static size (0 for middle).
Make it so: Size for middle bitfields just returns 0. Removes lots of if's.
Introduce Type.UnitSize, which now holds the underlying type for bitfields.
This will be needed to fix  b/c even if UnitSize=4 for last bitfield
Size can be anywhere from 0 to 4 (not necessary equal to UnitSize due to overlapping).
2019-12-19 19:32:45 +01:00
Veronica Radu
dbd627eb61 prog: add implementation for resource centric 2019-09-03 16:30:45 +02:00
Dmitry Vyukov
c84501fe70 prog: fix a bunch of bugs in parsing
Add fuzzer for Deserialize and fix 5 or so bugs it found.

Fixes 
2019-03-29 08:56:02 +01:00
Greg Steuck
0b29b7f952 prog: prevent sandbox escaping files from entering s.files 2018-11-27 13:16:05 +01:00
Dmitry Vyukov
e8dd2c6713 prog: add concept of "special pointers"
Currently we only generate either valid user-space pointers or NULL.
Extend NULL to a set of special pointers that we will use in programs.
All targets now contain 3 special values:
 - NULL
 - 0xfffffffffffffff (invalid kernel pointer)
 - 0x999999999999999 (non-canonical address)
Each target can add additional special pointers on top of this.

Also generate NULL/special pointers for non-opt ptr's.
This restriction was always too restrictive. We may want to generate
them with very low probability, but we do want to generate them.

Also change pointers to NULL/special during mutation
(but still not in the opposite direction).
2018-08-30 21:45:03 -07:00
Dmitry Vyukov
2dc4378f02 prog: don't add fallback signal after seccomp
seccomp filter can produce arbitrary errno values for subsequent syscalls.
Don't trust anything afterwards.
2018-08-19 00:13:24 -07:00
Dmitry Vyukov
1a381291a3 executor: collect coverage from unfinished syscalls
Write coverage from unfinished syscalls.
Also detect when a syscall was blocked during execution,
even if it finished. Helpful for fallback coverage.

Fixes 
2018-07-29 19:46:05 +02:00
Dmitry Vyukov
57799a834f prog: fix and improve fallback coverage
First we emitted fallbackSignalFlags inside of the loop,
while we need to this outside of the loop.
Second, make flags signal weaker otherwise we get all 256
signals for open, chmod, etc.
Third, simplify and speedup code.
2018-06-30 13:28:11 +02:00
Dmitry Vyukov
f7498af7af prog: add stronger fallback signal
Also mixin resource constructors and some signature
of flags values for successful calls into fallback coverage.
2018-06-29 20:34:43 +02:00
Dmitry Vyukov
9dfb5efa91 prog: simplify code
Now that we don't have ReturnArg and only ResultArg's refer
to other ResultArg's we can remove ArgUser/ArgUsed and
devirtualize lots of code.
2018-05-05 10:13:04 +02:00
Dmitry Vyukov
3e679c51c0 pkg/compiler: support non-zero terminated filenames
Now file names become:

string[filename]

with a possibility of using other string features:

stringnoz[filename]
string[filename, CONST_SIZE]

and filename is left as type alias as it is commonly used:

type filename string[filename]
2018-04-02 20:10:07 +02:00
Dmitry Vyukov
002cecf202 pkg/compiler: allow specifying static size for filename's
Sometimes filenames are embed into structs and need to take fixed space.
2018-03-05 12:10:27 +01:00
Dmitry Vyukov
41f6f2579b prog: fix address analysis
Even during mutation of a call we want to analyze whole program
to find all used addresses (rather then stop on the selected call).
Also update address during ANY mutation if size has increased.
2018-02-26 13:33:11 +01:00
Dmitry Vyukov
9fe8aa42c5 prog: add arbitrary mutation of complex structs
Squash complex structs into flat byte array and mutate this array
with generic blob mutations. This allows to mutate what we currently
consider as paddings and add/remove paddings from structs, etc.
2018-02-25 18:22:02 +01:00
Dmitry Vyukov
2145057cb8 pkg/compiler: fix alignment corner case
Fix alignemnt calculation for packed structs with alignment and bitfields.
Amusingly this affected only a single real struct -- ipv6_fragment_ext_header.
2018-02-25 14:44:29 +01:00
Dmitry Vyukov
75a7c5e2d1 prog: rework address allocation
1. mmap all memory always, without explicit mmap calls in the program.
This makes lots of things much easier and removes lots of code.
Makes mmap not a special syscall and allows to fuzz without mmap enabled.

2. Change address assignment algorithm.
Current algorithm allocates unmapped addresses too frequently
and allows collisions between arguments of a single syscall.
The new algorithm analyzes actual allocations in the program
and places new arguments at unused locations.
2018-02-19 21:48:20 +01:00
Dmitry Vyukov
6051a5b552 prog: combine RequiresBitmasks and RequiresChecksums into RequiredFeatures 2018-02-19 21:48:20 +01:00
Dmitry Vyukov
4eef71bb2d prog: merge foreachSubargOffset into foreachArgImpl 2018-02-19 21:48:20 +01:00
Dmitry Vyukov
85d1218f41 prog: rework foreachArg
Make Foreach* callback accept the arg and a context struct
that can contain lots of aux info.
This (1) removes lots of unuser base/parent args,
(2) provides foundation for stopping recursion,
(3) allows to merge foreachSubargOffset.
2018-02-19 21:48:20 +01:00
Dmitry Vyukov
08146b1a84 sys/linux: extend netfilter descriptions 2018-01-27 17:08:43 +01:00
Dmitry Vyukov
8ef0050706 prog: don't serialize output data args
Fixes 

We now will write just ""/1000 to denote a 1000-byte output buffer.
Also we now don't store 1000-byte buffer in memory just to denote size.
Old format is still parsed.
2017-12-17 11:39:14 +01:00
Dmitry Vyukov
52a33fd516 prog: remove default target and all global state
Now each prog function accepts the desired target explicitly.
No global, implicit state involved.
This is much cleaner and allows cross-OS/arch testing, etc.
2017-09-15 16:02:37 +02:00
Dmitry Vyukov
ffe7e17368 prog, sys: move types to prog
Large overhaul moves syscalls and arg types from sys to prog.
Sys package now depends on prog and contains only generated
descriptions of syscalls.
Introduce prog.Target type that encapsulates all targer properties,
like syscall list, ptr/page size, etc. Also moves OS-dependent pieces
like mmap call generation from prog to sys.

Update 
2017-09-05 15:52:42 +02:00
Dmitry Vyukov
4fc4702694 prog: dot-import sys
In preparation for moving sys types to prog to reduce later diffs.
2017-09-05 10:46:34 +02:00
Dmitry Vyukov
1c0d4caf7c sys: change BitfieldLast to BitfieldMiddle
That's the condition we always want.
Currently we always check:
t.BitfieldOffset() == 0 || t.BitfieldLast()
now can check just:
!t.BitfieldMiddle()
2017-09-04 20:51:56 +02:00
Dmitry Vyukov
399addc875 sys, pkg/compiler: move padding computation to compiler
This makes types constant during execution, everything is precomputed.
2017-09-04 20:25:23 +02:00
Dmitry Vyukov
838e336594 sys, prog: switch values to to uint64
We currently use uintptr for all values.
This won't work for 32-bit archs.
Moreover in some cases we use uintptr but assume
that it is always 64-bits (e.g. in encodingexec).
Switch everything to uint64.

Update 
2017-08-19 10:16:23 +02:00
Andrey Konovalov
cfc46d9d0b prog: split Arg into smaller structs
Right now Arg is a huge struct (160 bytes), which has many different fields
used for different arg kinds. Since most of the args we see in a typical
corpus are ArgConst, this results in a significant memory overuse.

This change:
- makes Arg an interface instead of a struct
- adds a SomethingArg struct for each arg kind we have
- converts all *Arg pointers into just Arg, since interface variable by
  itself contains a pointer to the actual data
- removes ArgPageSize, now ConstArg is used instead
- consolidates correspondence between arg kinds and types, see comments
  before each SomethingArg struct definition
- now LenType args that denote the length of VmaType args are serialized as
  "0x1000" instead of "(0x1000)"; to preserve backwards compatibility
  syzkaller is able to parse the old format for now
- multiple small changes all over to make the above work

After this change syzkaller uses twice less memory after deserializing a
typical corpus.
2017-07-17 14:34:09 +02:00
Andrey Konovalov
b3ea23c379 repro: always minimize over EnableTun 2017-06-12 19:48:23 +02:00
Andrey Konovalov
4d1df73af9 csource: force enable tun flag when required 2017-06-12 19:48:23 +02:00
Andrey Konovalov
ac0c70f74a prog, executor: move checksum computation to executor
This commit moves checksum computation to executor. This will allow to embed
dynamically generated values (like TCP sequence numbers) into packets.
2017-05-12 15:47:59 +02:00
Dmitry Vyukov
bba1519958 prog: fix mknod sanitization
mknod mode also includes ownership flags, so filter out the node type.
Also allow creation of loop nodes.
Remove mount$fs as it does not seem to make any sense.
2017-02-14 11:56:41 +01:00
Andrey Konovalov
63b16a5d5c prog, sys: add csum type, embed checksums for ipv4 packets
This change adds a `csum[kind, type]` type.
The only available kind right now is `ipv4`.
Using `csum[ipv4, int16be]` in `ipv4_header` makes syzkaller calculate
and embed correct checksums into ipv4 packets.
2017-01-25 20:31:13 +01:00
Andrey Konovalov
c8d03a05f3 prog: move size-related functions to size.go 2017-01-25 16:33:37 +01:00
Andrey Konovalov
1cf6a05e0e sys, prog: add length of parent of parent to templates
Example:
```
struct s1 {
	f0	len[s2] # length of s2
}

struct s2 {
	f0	s1
	f1	array[int32]
}
```
2017-01-23 18:13:11 +01:00
Andrey Konovalov
b323c5aaa9 prog: add FieldName to Type
FieldName() is the name of the struct field or union option with this type.
TypeName() is now always the name of the type.
2017-01-23 18:13:06 +01:00
Andrey Konovalov
243c4bf89c prog: fix bytesizeN for nonarray fields 2017-01-19 20:46:26 +01:00
Andrey Konovalov
8625843eeb prog: fix calculating parent length in struct with bitfields 2017-01-19 20:46:26 +01:00
Dmitry Vyukov
c377a6514d prog: reformat source 2017-01-09 20:26:34 +01:00
Andrey Konovalov
df98b6bde5 prog: add bytesizeN types 2016-12-20 18:12:07 +01:00
Dmitry Vyukov
578ee4fa8d prog: sanitize mknodat the same way as mknod 2016-11-18 10:24:13 +01:00
Dmitry Vyukov
8b731ed4b7 sys: replace FilenameType with BufferType{Kind: BufferFilename}
FilenameType is effectively a buffer, there is no need for a separate type.
2016-11-11 14:32:19 -08:00
Dmitry Vyukov
b40d502736 prog: remote Type argument from Arg.Size/Value
They are not necessary since we now always have types attached to args.
Also remove sys.Type.InnerType as it is not necessary now as well.
2016-11-11 14:31:55 -08:00
Dmitry Vyukov
1838728cc1 prog: simplify assignSizes
Now that we always have types attached to args,
assignSizes can be considerably simplified.
2016-11-11 14:30:20 -08:00