Commit Graph

2371 Commits

Author SHA1 Message Date
Dmitry Vyukov
738d58ade0 pkg/csource: minimize netdevices and net reset
Add separate options to minimize netdevices setup and net namespace reset.

Fixes #581
2018-05-17 19:57:54 +02:00
Dmitry Vyukov
3717901c10 pkg/csource: serialize options into json
Current format is painful to parse after changes.
Switch to json.
2018-05-17 19:07:46 +02:00
Dmitry Vyukov
9753d3be5e tools/syz-prog2c: add missing csource option 2018-05-17 19:07:33 +02:00
Dmitry Vyukov
be47ebb0eb pkg/kernel: allow to split full make output
Currently kernel build failures are insanely verbose
(contain full kernel build output) and there is no
way to separate short descriptions from full output.
Make it possible.
Also try to extract failure root cause froom build log.
Use this in pkg/bisect to not pollute log on build failures.

Update #501
2018-05-17 17:18:24 +02:00
Dmitry Vyukov
28cbff8c7d pkg/bisect: stop at Linux v3.8
Update #501
2018-05-17 16:40:00 +02:00
Dmitry Vyukov
8dd3159f51 pkg/bisect: increase number of tries to 8
With 5 tries sometimes only 1 fails,
and sometimes we probably have false negatives.
Increase number of tries to 8 and compress
results if they all are the same.

Update #501
2018-05-17 16:30:20 +02:00
Dmitry Vyukov
edbe39a26d pkg/git: extract relevant email addresses from commits
Update #501
2018-05-17 15:58:58 +02:00
Dmitry Vyukov
4e1c0dbaea pkg/bisect: add bisection logic
Add first version of bisection package
(supports both bug and fix bisection).
And tools/syz-bisect wrapper for testing.

Update #501
2018-05-17 14:52:39 +02:00
Dmitry Vyukov
6595937c34 tools/create-gce-image.sh: support both nbd and loop
Pass target OS/arch and VM type to kernel.CreateImage.
Use nbd for gce and loop for qemu VM type.
2018-05-17 14:50:18 +02:00
Dmitry Vyukov
9c84f7c395 pkg/osutil: allow disabling sandboxing with env var
If SYZ_DISABLE_SANDBOXING=yes is set, don't do user sandboxing.
Will be usefule for bisection tool which runs locally,
but needs to build kernel.

Update #501
2018-05-17 14:03:46 +02:00
Dmitry Vyukov
be1a56115b sys/linux: use optional in tun descriptions 2018-05-17 12:16:18 +02:00
Dmitry Vyukov
031c7941d7 pkg/compiler: add optional builtin template
type optional[T] [
	val	T
	void	void
] [varlen]
2018-05-17 12:02:30 +02:00
Dmitry Vyukov
90c54c496b pkg/compiler: detect duplicate fields in template structs 2018-05-17 11:46:45 +02:00
Dmitry Vyukov
256b70f9cf sys/linux: fix duplicate field in struct 2018-05-17 11:45:51 +02:00
Dmitry Vyukov
a367c1d7a8 tools/create-gce-image.sh: support MKE2FS_CONFIG 2018-05-16 17:21:29 +02:00
Dmitry Vyukov
9e377df0c7 docs: describe syzbot moderation queue 2018-05-16 14:59:33 +02:00
Dmitry Vyukov
c7c5314649 dashboard/app: make it possible to provide links to subsections
E.g.:

https://syzkaller.appspot.com/#upstream-open
https://syzkaller.appspot.com/#upstream-pending
2018-05-16 14:58:46 +02:00
Billy Lau
75b2448a26 vm/adb: disable kptr_restrict for Android
echo 0 to kptr_restrict in /proc/sys/kernel to unhide
kernel pointers when fuzzing for more reliable crash
dedup and easier debugging when analyzing crash.
2018-05-16 10:33:18 +02:00
Dmitry Vyukov
68ce85f1a9 dashboard/app: reformat code 2018-05-15 13:24:37 +02:00
Murat
78d847c63e Updated file and fixed typo.
Fixed typo for JSON config file. It has a comma after "false". If this comma exist, it always gives issue about object thing.
2018-05-15 10:28:30 +02:00
Dmitry Vyukov
03258b9e7f dashboard/app: remove most details from email footer
Exact details of syzbot communication change over time,
so it's a bad idea to hardcode them in each email.
Full details are also quite lengthy, so we can't express
all of them in each email header. And providing partial
info may be worse than just providing a link to full info.

So remove most details from email footer,
and just give links to documentation.

Also clarify docs re when "syz fix" should/can be used.
2018-05-15 10:14:51 +02:00
Dmitry Vyukov
661fd7b988 executor: revert protection of kcov region
Turns out it's protection of kcov region that causes coverage drop.
Revert it and re-enable coverage filtering.
2018-05-15 09:45:50 +02:00
Dmitry Vyukov
d2284dda40 executor: revert coverage filtering
This causes very significant drop in corpus size. Needs additional debugging.
Revert for now.
2018-05-14 19:34:16 +02:00
Dmitry Vyukov
86ea19e4c2 tools/create-gce-image.sh: revert loop to nbd
loop devices are nice, but unfortunately this creates images
that does not boot on GCE. Reason is unclear.
Revert back to nbd for now.
2018-05-14 19:32:39 +02:00
Dmitry Vyukov
9467cacb1c executor: protect most of kcov region
We only write to the first page of kcov region.
Protect the rest from the fuzzer.
2018-05-14 15:31:14 +02:00
Dmitry Vyukov
364ad1502f executor: terminate on corrupted coverage
New measures still does not help against fuzzer producing
fake corrupted coverage.
Try to terminate instantly as we detect bad coverage.
2018-05-14 15:30:13 +02:00
Dmitry Vyukov
d131f1b9fc pkg/instance: relax syzkaller path check
Only check that syzkaller path is in GOPATH if we are going to build it.
syz-ci image testing does not have syzkaller path in GOPATH,
but it also does not build syzkaller.
2018-05-14 13:47:44 +02:00
Dmitry Vyukov
477728038c syz-ci: fix manager config creation
SyzFuzzerBin/SyzExecprogBin/SyzExecutorBin are ended up being empty.
Fix this.
2018-05-14 12:17:34 +02:00
Dmitry Vyukov
2407e7407a pkg/instance: add package for testing of images/patches/bisection
Move helper image/patch testing code from syz-ci/testing.go
to a separate package so that it can be reused during bisection.

Update #501
2018-05-14 11:32:56 +02:00
Dmitry Vyukov
faf3e3d229 executor: filter out invalid PCs on linux/x86_64
Fuzzer manages to corrupt output region and write random coverage again and again.
Do a sanity range check on coverage PCs to filter out invalid ones.
2018-05-14 11:17:58 +02:00
Dmitry Vyukov
d5dc4006db tools/kcovtrace: add missing include 2018-05-14 11:17:41 +02:00
Dmitry Vyukov
aa5027ae05 executor: randomize output region addr
Fuzzer manages to corrupt output region and write random coverage again and again.
Randomize output region addr to make it harder.
2018-05-14 11:17:24 +02:00
Dmitry Vyukov
9fb3ae4dfd pkg/osutil: introduce TempFile helper
Introduce TempFile helper and use it in several packages.
2018-05-14 11:17:23 +02:00
Dmitry Vyukov
acfd774f46 pkg/git: add bisection functionality
Bisect bisects good..bad commit range against the provided predicate (wrapper around git bisect).
The predicate should return an error only if there is no way to proceed
(it will abort the process), if possible it should prefer to return BisectSkip.
Progress of the process is streamed to the provided trace.
Returns the first commit on which the predicate returns BisectBad.

Update #501
2018-05-14 11:17:23 +02:00
Dmitry Vyukov
b4df103f78 pkg/git: add PreviousReleaseTags
PreviousReleaseTags returns list of preceding release tags
that are reachable from the given commit.

Update #501
2018-05-14 11:17:23 +02:00
Dmitry Vyukov
5060be1a90 pkg/git: add SwitchCommit and GetCommit
Will be useful for bisection logic.

Update #501
2018-05-14 11:17:23 +02:00
Dmitry Vyukov
d39e52252c pkg/kernel: use oldconfig instead of olddefconfig
See the added comment for motivation.

Update #501
2018-05-14 11:17:23 +02:00
Dmitry Vyukov
2a287a4dfc pkg/kernel: accept kernel config as byte slice
We usually have the config in memory,
so saving it to disk is an unnecessary step.
Accept byte slice directly.

Update #501
2018-05-14 11:17:23 +02:00
Dmitry Vyukov
1944eeb95d pkg/kernel: run distclean with -jNCPU 2018-05-14 11:17:18 +02:00
Dmitry Vyukov
4fce9bf152 syz-manager: add concept of partial configs
syz-ci uses partial (incomplete) manager config in several places.
Currently it is implemented in some ugly way.
Provide better support and unexport DefaultValues and SplitTarget.

Update #501
2018-05-14 11:16:57 +02:00
Dmitry Vyukov
ea9496cdbd tools/create-gce-image.sh: use loop instead of nbd
loop is much more standard than nbd and does not require additional modules.
nbd broke on Debian rolling.
loop also allows parallel execution thanks to losetup -f.

Use loop instead of nbd.
Also improve cleanup logic and add one missing sudo.

Update #501
2018-05-14 11:16:36 +02:00
Ioana Ciornei
481f030ccd executor: fix strncpy compile error
gcc8 is stricter when dealing with strings and strncpy and demands that
the size of the actual string to be copied to be explicitly smaller than
the size of the destination, just to make sure the NULL terminator is
taken into considerantion. This patch fixes the issue.

Signed-off-by: Ioana Ciornei <ciorneiioana@gmail.com>
2018-05-13 16:27:22 +02:00
Dmitry Vyukov
d3a7c28ef1 Makefile: don't run gometalinter in parallel
apt-get update seems to heal missing packages.
But gometalinter failed due to OOM:
https://travis-ci.org/google/syzkaller/jobs/378313344
Don't run it in parallel with other tests.
2018-05-13 11:59:44 +02:00
Dmitry Vyukov
c05b619dab Makefile: print uname in install_prerequisites
Travis is still broken with:

E: Unable to locate package gcc-aarch64-linux-gnu
E: Unable to locate package gcc-arm-linux-gnueabihf
E: Unable to locate package gcc-powerpc64le-linux-gnu

https://travis-ci.org/google/syzkaller/jobs/378034948

Let's at least see what's OS is there and what gcc packages present.
2018-05-13 11:36:13 +02:00
Dmitry Vyukov
e726f42b64 Makefile: don't fail on unknown packages
Travis CI started failing with:

E: Unable to locate package gcc-aarch64-linux-gnu
E: Unable to locate package gcc-arm-linux-gnueabihf
E: Unable to locate package gcc-powerpc64le-linux-gnu
make: *** [install_prerequisites] Error 100

https://travis-ci.org/google/syzkaller/jobs/377740347

Let's try to ignore these packages.
2018-05-12 10:10:02 +02:00
Dmitry Vyukov
63ef537b14 sys/syz-extract: improve compier error messages
Print each constant on separate line.
When compiler prints error messages it prints whole line
containing an error. So currently it prints all constants (thousands).
With this change it prints just the problematic constant.
2018-05-11 15:48:33 +02:00
Dmitry Vyukov
c8229620ff sys/linux: regenerate video4linux consts on the right tree
Now generated on:
https://source.codeaurora.org/quic/la/kernel/msm-4.9 msm-4.9
2018-05-11 15:09:33 +02:00
Dmitry Vyukov
2d35915b13 sys/linux: add missing consts for video4linux.txt
For now other arches are regenerated on upstream tree.
It does not contain a bunch of consts, so we define them to 1 for now.
arm64 consts are left intact.
video4linux.txt is added to "android" files in syz-extract,
so that future make extract runs don't overwrite arm64 consts.

Also fix VIDIOC_G_FBUF argument direction, currently tests crash with:

panic: call ioctl$VIDIOC_G_FBUF: pointer arg 'buffer' has output direction [recovered]
	panic: call ioctl$VIDIOC_G_FBUF: pointer arg 'buffer' has output direction
2018-05-11 14:24:16 +02:00
srikanth007m
ace16942e5 sys/linux: add Video4Linux IOCTL Descriptors
* Adding Video4Linux IOCTL Descriptors

This will cover the ioctl() function used to program V4L2 devices

* Const file for video4linux_arm64.const
2018-05-11 14:11:10 +02:00
Jean-Baptiste Theou
12c7428ad2 docs: add link for PH-1 kernel
Signed-off-by: Jean-Baptiste Theou <jb@essential.com>
2018-05-09 09:31:39 +02:00