292 Commits

Author SHA1 Message Date
Dmitry Vyukov
3e679c51c0 pkg/compiler: support non-zero terminated filenames
Now file names become:

string[filename]

with a possibility of using other string features:

stringnoz[filename]
string[filename, CONST_SIZE]

and filename is left as type alias as it is commonly used:

type filename string[filename]
2018-04-02 20:10:07 +02:00
Dmitry Vyukov
185ac3525e prog: support big-endian during hints matching
Use big-endian match/replace for both blobs and ints.
Sometimes we have unmarked blobs (no little/big-endian info);
for ANYBLOBs we intentionally lose all marking;
but even for marked ints we may need this too.
Consider that kernel code does not convert the data
(i.e. not ntohs(pkt->proto) == ETH_P_BATMAN),
but instead converts the constant (i.e. pkt->proto == htons(ETH_P_BATMAN)).
In such case we will see dynamic operand that does not
match what we have in the program.
2018-04-01 15:28:01 +02:00
Dmitry Vyukov
0174c6c8f7 prog: reduce nesting level in shrinkExpand 2018-03-31 18:55:59 +02:00
Dmitry Vyukov
7c923cf8d4 sys/linux: add support for mounting filesystem images 2018-03-30 19:51:27 +02:00
Dmitry Vyukov
7c62f71804 sys/linux: add netlink fou descriptions 2018-03-21 12:18:36 +01:00
Dmitry Vyukov
36d1c4540a all: fix gometalinter warnings
Fix typos, non-canonical code, remove dead code, etc.
2018-03-08 18:48:26 +01:00
Dmitry Vyukov
d0790618dc prog: fix isDefaultArg
Test that isDefaultArg returns true for result of DefaultArg.
Fix few bugs uncovered by this test.
2018-03-08 12:02:17 +01:00
Dmitry Vyukov
4bad061006 prog: extend some error messages 2018-03-05 12:58:29 +01:00
Dmitry Vyukov
bc09be4253 prog: fix 32-bit build
Currently fails with:
prog/mutation.go:442:24: constant 4294967296 overflows int
2018-03-05 12:10:27 +01:00
Dmitry Vyukov
70a1ddb939 prog: harden program parsing against description changes more
Handle most of type changes, e.g. const is changed to struct,
or struct to pointers. In all these cases we create default args.
They may not give the coverage anymore, but still better than
losing them right away.
2018-03-05 12:10:27 +01:00
Dmitry Vyukov
b99c76c405 prog: recover after type changes during program deserialization
Make program deserialization handle and recover after type changes
in descriptions.
2018-03-05 12:10:27 +01:00
Dmitry Vyukov
bd5df8f49b prog: handle excessive args and fields during program parsing
Tolerate excessive args and fields during program parsing.
This is useful after description changes to not lose corpus.
2018-03-05 12:10:27 +01:00
Dmitry Vyukov
002cecf202 pkg/compiler: allow specifying static size for filename's
Sometimes filenames are embed into structs and need to take fixed space.
2018-03-05 12:10:27 +01:00
Dmitry Vyukov
5ef8dbdf5a pkg/compiler: support size attribute for unions 2018-03-05 12:10:27 +01:00
Dmitry Vyukov
a339951e5f pkg/compiler: add size attribute for structs
The size attribute allows to pad a struct up to the specified size.
2018-03-05 12:10:27 +01:00
Dmitry Vyukov
e28ba02d9d prog: harden program parsing
This fixes crash during parsing of existing programs in corpus
after vma<->ptr type change in descriptions.
2018-03-05 12:10:27 +01:00
Dmitry Vyukov
6ceee05ad9 prog: extend a TODO about big-endian hints 2018-02-27 12:06:28 +01:00
Dmitry Vyukov
6284466bc9 prog: properly squash bitfields 2018-02-26 18:21:51 +01:00
Dmitry Vyukov
d1322dff4c prog: remove stale TODOs 2018-02-26 17:46:44 +01:00
Dmitry Vyukov
b37b65b0e6 sys/linux: remove proc type from network descriptions
We now always create net namespace for testing,
so socket ports and other IDs do not overlap between
different test processes.
Proc types play badly with squashing packets to ANYBLOB.
To squash into a block we need concrete value, but it depends
on process id.
Removing proc also makes tun setup and address descriptions simpler.
2018-02-26 16:48:31 +01:00
Dmitry Vyukov
14dae29c2a executor: use proper default values for resources
We currently use -1 as default value for resources
when the actual value is not available.
-1 is good for fd's, but is not the right default
value for pointers/keys/etc.
Pass from prog and use in executor proper default
value for resources.
2018-02-26 15:00:46 +01:00
Dmitry Vyukov
41f6f2579b prog: fix address analysis
Even during mutation of a call we want to analyze whole program
to find all used addresses (rather then stop on the selected call).
Also update address during ANY mutation if size has increased.
2018-02-26 13:33:11 +01:00
Dmitry Vyukov
17ad67b472 prog: better error message when failed to create a resource 2018-02-26 12:15:08 +01:00
Dmitry Vyukov
9fe8aa42c5 prog: add arbitrary mutation of complex structs
Squash complex structs into flat byte array and mutate this array
with generic blob mutations. This allows to mutate what we currently
consider as paddings and add/remove paddings from structs, etc.
2018-02-25 18:22:02 +01:00
Dmitry Vyukov
2145057cb8 pkg/compiler: fix alignment corner case
Fix alignemnt calculation for packed structs with alignment and bitfields.
Amusingly this affected only a single real struct -- ipv6_fragment_ext_header.
2018-02-25 14:44:29 +01:00
Dmitry Vyukov
1f4ae3f413 pkg/compiler: don't assign call IDs statically
IDs change whenever a call is added or removed,
this leads to large diffs unnecessarly.
Assign IDs dynamically.
2018-02-25 14:31:40 +01:00
Dmitry Vyukov
44f66b4026 prog: return concrete arg types from Make functions
This removes a bunch of type assertions and makes code type safer.
2018-02-24 17:00:34 +01:00
Dmitry Vyukov
7a7c747c6f pkg/compiler: allow unions as syscall arguments
If all union options can be syscall arguments,
allow the union itself as syscall argument.
2018-02-23 13:50:02 +01:00
Dmitry Vyukov
3be86de046 sys/linux: prevent programs from doing arbitrary writes with ARCH_SET_FS 2018-02-23 11:55:37 +01:00
Dmitry Vyukov
c1f526e3e5 prog: mutate len fields based on hints
With the new address allocation logic,
the reason to not touch len has gone.
2018-02-19 21:48:20 +01:00
Dmitry Vyukov
a793204dd2 pkg/compiler: fix corner case in alignment calculation 2018-02-19 21:48:20 +01:00
Dmitry Vyukov
75a7c5e2d1 prog: rework address allocation
1. mmap all memory always, without explicit mmap calls in the program.
This makes lots of things much easier and removes lots of code.
Makes mmap not a special syscall and allows to fuzz without mmap enabled.

2. Change address assignment algorithm.
Current algorithm allocates unmapped addresses too frequently
and allows collisions between arguments of a single syscall.
The new algorithm analyzes actual allocations in the program
and places new arguments at unused locations.
2018-02-19 21:48:20 +01:00
Dmitry Vyukov
90fd650313 prog: fix PhysicalAddr for NULL addresses
Turns out we never produced NULL pointers because
what's meant to be NULL pointer was actually encoded
as pointer to beginning of the data region.
2018-02-19 21:48:20 +01:00
Dmitry Vyukov
6e89f94756 prog: fix mutationArgs for special types
There are 2 bugs currently:
1. mutationArgs recurses into special types,
even though they must be mutated as the whole only.
2. When mutationArgs is called from Gen.MutateArg,
it included the top special type as well,
it must not because at this point only the subargs
must be mutated.

Fix both problems.
2018-02-19 21:48:20 +01:00
Dmitry Vyukov
6051a5b552 prog: combine RequiresBitmasks and RequiresChecksums into RequiredFeatures 2018-02-19 21:48:20 +01:00
Dmitry Vyukov
4eef71bb2d prog: merge foreachSubargOffset into foreachArgImpl 2018-02-19 21:48:20 +01:00
Dmitry Vyukov
85d1218f41 prog: rework foreachArg
Make Foreach* callback accept the arg and a context struct
that can contain lots of aux info.
This (1) removes lots of unuser base/parent args,
(2) provides foundation for stopping recursion,
(3) allows to merge foreachSubargOffset.
2018-02-19 21:48:20 +01:00
Dmitry Vyukov
76ab9250a7 prog: move Minimize tests to an own file 2018-02-19 21:48:20 +01:00
Dmitry Vyukov
dd768bf1c6 prog: reorder Minimize arguments
Make the predicate the last argument.
It's more common and convenient (arguments are not separated by multiple lines).
2018-02-19 21:48:20 +01:00
Dmitry Vyukov
4f4f70406c prog: unexport Args and PyPhysicalAddr 2018-02-17 19:02:12 +01:00
Dmitry Vyukov
a08436c973 prog: give special type generators access to target 2018-02-17 19:02:12 +01:00
Dmitry Vyukov
5204f75a58 prog: fix test output
Print programs as strings, rather than byte sequences.
2018-02-17 19:02:12 +01:00
Dmitry Vyukov
dd4fcef5a2 pkg/compiler: allow len of var-len arrays
All netfilter subsystems use this unfortunately,
so demote this to a warning.
2018-02-17 19:02:12 +01:00
Dmitry Vyukov
033b610ec9 sys/linux: improve netfilter descriptions
Put the underflow entry at the end.
Entries must end on an unconditional, non-goto entry,
otherwise fallthrough from the last entry is invalid.

Add arp tables support.

Split unspec matches/targets to unspec and inet.

Reset ipv6 and arp tables in executor.

Fix number of counters in tables.

Plus a bunch of assorted fixes for matches/targets.
2018-02-09 20:14:33 +01:00
Dmitry Vyukov
11fa8bb3de prog: fix mutation of special types
Caught by existing tests, just happens very infrequently.
2018-02-01 15:20:37 +01:00
Dmitry Vyukov
d973f28294 prog: don't serialize default arguments
This reduces size of a corpus in half.
We store corpus on manager and on hub,
so this will reduce their memory consumption.
But also makes large programs more readable.
2018-02-01 15:20:12 +01:00
Dmitry Vyukov
ccac2d6f42 prog: fix TestSpecialStructs
We should not try to generate/mutate output arguments.
SpecialTypes generators are meant to accept inout arguments.
2018-01-31 15:50:36 +01:00
Dmitry Vyukov
08146b1a84 sys/linux: extend netfilter descriptions 2018-01-27 17:08:43 +01:00
Dmitry Vyukov
5d7477249b prog: remove unused UnionArg.OptionType 2018-01-27 17:08:43 +01:00
Dmitry Vyukov
e8b4970547 pkg/compiler: allow unions with only 1 field
Unions with only 1 field are not actually unions,
and can always be replaced with the option type.
However, they are still useful when there will be
more options in future but currently only 1 is described.
Alternatives are:
 - not using union (but then all existing programs will be
   broken when union is finally introduced)
 - adding a fake field (ugly and reduces fuzzer efficiency)

Allow unions with only 1 field.
2018-01-27 17:08:43 +01:00