Commit Graph

744 Commits

Author SHA1 Message Date
Dmitry Vyukov
8b2c1cb5bb syz-hub: don't flush global corpus after every addition
We already flush in addInputs, no need to flush in addInput.
Also add some logging on start as it can be slow.
2017-01-30 11:02:17 +01:00
Dmitry Vyukov
4ee789185b sys: improve kvm description
Allow fuzzer to change types of segment descriptors.
Alter more flags.
Allow fuzzer to do a random vmwrite.
2017-01-28 19:58:31 +01:00
Dmitry Vyukov
caef676b0a sys: improve key descriptions 2017-01-28 18:43:52 +01:00
Dmitry Vyukov
6f722a0348 sys: improve ipc descriptions 2017-01-28 17:52:51 +01:00
Dmitry Vyukov
e00c6a347b syz-manager: print at most 10 deleted programs
Printing thousands of programs is slow and unnecessary.
2017-01-28 17:52:14 +01:00
Dmitry Vyukov
e0e5de91ec sys: move ipc syscalls into separate file 2017-01-28 16:14:01 +01:00
Dmitry Vyukov
24362e5bc7 manager: make disabledHashes map 2017-01-27 21:48:36 +01:00
Dmitry Vyukov
7165b3a9b1 gce: revert to terminate on migration
No, it does not work:
Error 400: Invalid value for field 'resource.scheduling.preemptible': 'true'. Scheduling must have preemptible be false when OnHostMaintenance isn't TERMINATE
2017-01-27 20:59:43 +01:00
Dmitry Vyukov
9a1fbc16be Merge pull request #121 from google/new_cover
New cover
2017-01-27 20:53:49 +01:00
Dmitry Vyukov
83cf8e3924 csource, syz-gce: regenerate and reformat 2017-01-27 20:51:41 +01:00
Dmitry Vyukov
8365c3838d all: implement edge coverage
Currently syzkaller uses per-call basic block (BB) coverage.
This change implements edge (not-per-call) coverage.
Edge coverage is more detailed than BB coverage as it captures
not-taken branches, looping, etc. So it provides better feedback signal.
This coverage is now called "signal" throughout the code.
BB code coverage is also collected as it is required for visualisation.
Not doing per-call coverage reduces corpus ~6-7x (from ~35K to ~5K),
this has profound effect on fuzzing efficiency.
2017-01-27 20:46:18 +01:00
Dmitry Vyukov
1c190bb963 executor: fix KVM test
SMM is now supported for real code instead of prot16.
2017-01-27 20:46:09 +01:00
Dmitry Vyukov
362f1a61a1 gce: don't terminate instances on migration
Seems to be working without termination.
2017-01-27 20:46:09 +01:00
Dmitry Vyukov
86c334f459 vm/qemu: use static network config in 9p mode
Static config is faster than dhcp.
2017-01-27 20:46:09 +01:00
Dmitry Vyukov
edb0141529 Merge pull request #119 from billy-lau/fix_adb_reboot2
adb: executor: Revert to adb reboot
2017-01-26 16:14:12 +01:00
Dmitry Vyukov
51ed7072cd Merge pull request #120 from xairy/ipv4-options
sys: improve ipv4 options descriptions
2017-01-26 15:30:51 +01:00
Andrey Konovalov
b6f983b7de sys: improve ipv4 options descriptions 2017-01-26 15:12:42 +01:00
Billy Lau
a0a4a8ed95 adb: executor: Revert to adb reboot
Using `adb shell syz-executor reboot` to reboot devices has stopped
working with the recent Android update, probably due to the intro
of seccomp. I have reverted the device reboot logic to use `adb
shell reboot` although it can be flaky at times so that we can
continue to fuzz on devices, until a more reliable solution can be
sought out.
2017-01-26 13:19:22 +00:00
Andrey Konovalov
72353dc4f6 Merge pull request #117 from xairy/checksums
Embed ipv4 checksums
2017-01-25 20:33:34 +01:00
Andrey Konovalov
63b16a5d5c prog, sys: add csum type, embed checksums for ipv4 packets
This change adds a `csum[kind, type]` type.
The only available kind right now is `ipv4`.
Using `csum[ipv4, int16be]` in `ipv4_header` makes syzkaller calculate
and embed correct checksums into ipv4 packets.
2017-01-25 20:31:13 +01:00
Andrey Konovalov
c8d03a05f3 prog: move size-related functions to size.go 2017-01-25 16:33:37 +01:00
Dmitry Vyukov
13134c3cf7 syz-prog2c: fix a typo in error message 2017-01-25 11:05:19 +01:00
Dmitry Vyukov
f810d08444 executor: protect against memory corruptions better
Fuzzer has figured out how to corrupt input/output shmem regions
abusing the text memcpy in syz_kvm_setup_cpu. It guessed a negative
text_size value that causes the memcpy to overwrite shmem regions.
Protect better against such cases:
1. Make text_size unsigned (there is already a check that it is less than 1000).
2. Map input region as readable only, we don't write to it.
3. Add address sanity check to segv_handler, if we see that we are writing
   into executable data, it's better to crash instantly.
2017-01-25 11:01:30 +01:00
Dmitry Vyukov
40723a067e prog: validate deserialized programs
The optimization change removed validation too aggressively.
We do need program validation during deserialization,
because we can get bad programs from corpus or hub.
Restore program validation after deserialization.
2017-01-24 10:53:21 +01:00
Dmitry Vyukov
40c6a8ebf5 tools/create-image.sh: add psmisc package for killall
+ raise console output level, otherwise stack traces may be dropped
2017-01-24 10:03:42 +01:00
Dmitry Vyukov
495b91f119 report: ignore info message about slow nmi handlers 2017-01-24 10:03:42 +01:00
Dmitry Vyukov
90f7c6d7e1 Merge pull request #116 from xairy/net-consts-fix
sys: add missing consts for arm64 and ppc64
2017-01-23 21:41:39 +01:00
Andrey Konovalov
a962426be5 sys: add missing consts for arm64 and ppc64 2017-01-23 20:45:32 +01:00
Dmitry Vyukov
936725c942 Merge pull request #115 from xairy/better-tun
sys: add ethernet and ipv4 packet descriptions
2017-01-23 20:00:59 +01:00
Andrey Konovalov
e1f6013383 sys: add ethernet and ipv4 packet descriptions 2017-01-23 18:19:45 +01:00
Andrey Konovalov
f8ecf0862d executor: change tun subnet to 172.20.* 2017-01-23 18:17:15 +01:00
Andrey Konovalov
bb1ff0b559 Merge pull request #113 from xairy/parent-parent
Make it possible to specify length of parent of parent
2017-01-23 18:15:39 +01:00
Andrey Konovalov
07880f3c01 csource: use 0x%x format for printing bitfield addr and arg 2017-01-23 18:13:11 +01:00
Andrey Konovalov
1cf6a05e0e sys, prog: add length of parent of parent to templates
Example:
```
struct s1 {
	f0	len[s2] # length of s2
}

struct s2 {
	f0	s1
	f1	array[int32]
}
```
2017-01-23 18:13:11 +01:00
Andrey Konovalov
b323c5aaa9 prog: add FieldName to Type
FieldName() is the name of the struct field or union option with this type.
TypeName() is now always the name of the type.
2017-01-23 18:13:06 +01:00
Dmitry Vyukov
cd23722cf2 Merge pull request #114 from xairy/makefile-fix
makefile: fix syz-extract dependencies
2017-01-23 17:33:26 +01:00
Andrey Konovalov
e7e90813b4 makefile: fix syz-extract dependencies 2017-01-23 17:27:59 +01:00
Dmitry Vyukov
a7e4a49fae all: spot optimizations
A bunch of spot optmizations after cpu/memory profiling:
1. Optimize hot-path coverage comparison in fuzzer.
2. Don't allocate and copy serialized program, serialize directly into shmem.
3. Reduce allocations during parsing of output shmem (encoding/binary sucks).
4. Don't allocate and copy coverage arrays, refer directly to the shmem region
   (we are not going to mutate them).
5. Don't validate programs outside of tests, validation allocates tons of memory.
6. Replace the choose primitive with simpler switches.
   Choose allocates fullload of memory (for int, func, and everything the func refers).
7. Other minor optimizations.
2017-01-20 23:55:25 +01:00
Dmitry Vyukov
a8632569bf executor: reduce syscall blocking delay from 100ms to 20ms
Syscalls frequently block and this affects fuzzing speed.
20ms should be more than enough for a normal syscall to finish.
2017-01-20 23:53:40 +01:00
Dmitry Vyukov
02702eeef3 syz-fuzzer: print total number of executed programs 2017-01-20 18:06:38 +01:00
Dmitry Vyukov
cde826031a syz-fuzzer: expose pprof profiling 2017-01-20 18:04:20 +01:00
Dmitry Vyukov
0298b1afb6 config: don't mess with image
We have "9p" special value, also GCE images are not local paths.
Don't mess with it.
2017-01-20 15:01:53 +01:00
Dmitry Vyukov
758a06c51f prog: generate larger arrays
Currently we generate arrays of size [0,5] with equal probability.
Generate [0,10] with bias towards smaller arrays. But 0 has the lowest probability.
I've benchmark a slightly different change with max array size of 20,
results are somewhat inconclusive: it was better than baseline almost all way,
but baseline suddenly caught up at the end. It also considerably reduced
executions per second (by ~20%). So increasing array size to 10 should be a win...
2017-01-20 14:56:20 +01:00
Dmitry Vyukov
1451a9b65d syz-fuzzer: prefer to mutate instead of generate
Currently we generate 1 program per 9 mutations.
Do it per 99 mutations.
Benchmark shows both coverage increase and corpus reduction:

                    baseline     generate100            diff
coverage               65467           65569             102
corpus                 35423           35363             -60
exec total           5474879         5030990         -443889
2017-01-20 14:56:20 +01:00
Dmitry Vyukov
c4901df5c3 prog: mutate programs more aggressively
Currently we stop mutating with 50% probability.
Stop mutating with 33% probability instead.
Benchmark shows both coverage increase and corpus reduction:

                    baseline          oneof3            diff
coverage               65467           65604             137
corpus                 35423           35354             -69
exec total           5474879         5023268         -451611
2017-01-20 14:56:20 +01:00
Dmitry Vyukov
b218a25ecb prog: mutate int arguments
Mutate int arguments instead of regenerating.
Benchmark shows strong increase of coverage:

                    baseline     mutateconst            diff
coverage               65467           65744            +277
corpus                 35423           35638            +215
exec total           5474879         5197932         -276947
2017-01-20 14:56:20 +01:00
Dmitry Vyukov
3db67897de tools/kcovtrace: add KCOV-based tracing utility
kcovtrace is like strace but show kernel coverage collected with KCOV.
It is very simplistic at this point and does not support multithreaded processes, etc.
It can be used to understand, for example, exact location where kernel bails out
with an error for a particular syscall.
2017-01-20 14:56:20 +01:00
Dmitry Vyukov
f8b6a5831c tools/syz-benchcmp: add utility for visualization of syz-manager benchmarking results 2017-01-20 14:56:20 +01:00
Dmitry Vyukov
652ac3731d syz-manager: add benchmarking mode
In benchmarking mode (if the new -bench flag is specified)
syz-manager writes execution statistics into the specified file.
This allows later comparison of different runs (baseline vs some experiment).
For example, verify that some fuzzing modification indeed leads to larger coverage.
2017-01-20 14:56:20 +01:00
Dmitry Vyukov
f9869965cf syz-manager: don't create legacy corpus dir if it does not exist 2017-01-20 14:56:20 +01:00