Commit Graph

5249 Commits

Author SHA1 Message Date
Dmitry Vyukov
9ebcc5b1a8
docs: add another research paper 2020-07-12 16:09:44 +02:00
Dmitry Vyukov
115e19300f tools/syz-linter: check comments format
Check for capitalization, dots at the end
and two spaces after a period.

Update #1876
2020-07-12 08:22:44 +02:00
Dmitry Vyukov
c1147c8df7 all: fix comments format
Fix capitalization, dots at the end
and two spaces after a period.

Update #1876
2020-07-12 08:22:44 +02:00
Dmitry Vyukov
0faffd0438 syz-ci: set Timeout for bisections
Add Timeout config param for bisections.
Specify timeout in syz-ci as 8h based on the following data.

Out of 1049 cause bisections that we have now:
-  891 finished under  6h (84.9%)
-  957 finished under  8h (91.2%)
-  980 finished under 10h (93.4%)
-  989 finished under 12h (94.3%)
- 1011 finished under 18h (96.3%)
- 1025 finished under 24h (97.7%)

There is also a significant increase in errors/inconclusive bisections after ~8h.
Out of 4075 fix bisections:
- 4015 finished under  6h (98.5%)
- 4020 finished under  8h (98.7%)
- 4026 finished under 10h (98.8%)
- 4032 finished under 12h (98.9%)
Significant increase in errors starts after ~12h.

The current timeout also take into account that bisection jobs
compete with patch testing jobs (it's bad delaying patch testing).
When/if bisection jobs don't compete with patch testing,
it makes sense to increase this to 12-24h.

Fixes #1923
2020-07-12 08:22:14 +02:00
Dmitry Vyukov
1ad470c265 pkg/vcs: don't sandbox syzkaller repo
Currently we sandbox all repos b/c we assumed
that all builds are also sandboxes. But this causes
havoc for bisection/patch testing b/c syzkaller build
is not actually sandboxed anywhere. Build creates
root-owned files and then git can't do anything with them
but don't report errors either:

$ git checkout 8eda0b95 && echo OK
error: unable to unlink old 'sys/linux/gen/386.go': Permission denied
error: unable to unlink old 'sys/linux/gen/ppc64le.go': Permission denied
...
HEAD is now at 8eda0b957e
OK

We trust own sources and we don't test syzkaller patches,
so don't sandbox syzkaller repos.
2020-07-12 08:22:14 +02:00
Mark Johnston
7ba05d2dd6 pkg/build: always truncate loader.conf on FreeBSD
Otherwise we append the same lines to the file upon each build.
2020-07-11 08:49:28 +02:00
Mark Johnston
ca5272b763 pkg/build: load the SCTP and IPSec stacks on FreeBSD
IPSec is not compiled into GENERIC kernels and must be loaded.  SCTP
currently is compiled into GENERIC, but this is expected to change.
2020-07-11 08:49:28 +02:00
Cheng-Min Chiang
18d18b591b sys/linux: add descriptions for IP tunnels
Add descriptions related to IPv4/v6 tunnels
(include/uapi/linux/if_tunnel.h).
Descriptions of netlink actions are not included in this commit.
Update #533.
2020-07-10 13:02:07 -07:00
Cheng-Min Chiang
146a394c19 CONTRIBUTORS: add Cheng-Min Chiang
Add Cheng-Min Chiang (me) to the contributors list.
2020-07-10 13:02:07 -07:00
Dmitry Vyukov
78178cfb8c tools/syz-linter: fix comments check
Turns out ast.Inspect does not visit most comments.
Walk file.Comments manually.

Update #1876
2020-07-10 19:15:46 +02:00
Dmitry Vyukov
d4c58caef7 tools/syz-linter: enable some standard linters
Enable some x/tools linters that are not enabled in vet.

Update #1876
2020-07-10 18:16:58 +02:00
Alexander Egorenkov
56d01184e3 executor: fix bitfields for big-endian arch
Add bitfield tests for big-endian arch

Issue: #1885

Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-07-10 10:25:58 +02:00
Jiaheng Hu
edf162e8c3 sys/linux: add SECCOMP_IOCTL_NOTIF_ADDFD
Added the ioctl decsription for adding a fd to a seccomp notifier,
currently in linux next.
(https://lore.kernel.org/lkml/20200617220327.3731559-7-keescook@chromium.org/)
2020-07-09 11:39:39 -07:00
Dmitry Vyukov
fd3bba535d dashboard/app: cache per-namespace bug stats
We used to show number of fixed bugs at the top of the main page.
However, now with the button nagivation, "fixed" is shown on every page.
Fetching and processing all bugs on every page would be unwise.
Cache these stats in memcache. It will be useful to show more stats in future.
2020-07-09 19:40:12 +02:00
Dmitry Vyukov
a1aebcca7f vendor: add google.golang.org/appengine/memcache 2020-07-09 19:40:12 +02:00
Dmitry Vyukov
c063196348 dashboard/app: add memcache to admin interface
Add memcache statistics and flush action to the admin page
in preparation for actually using memcache in future commits.
2020-07-09 19:40:12 +02:00
Dmitry Vyukov
353d190191 dashboard/app: add navigation buttons
Currently we have only "fixed" link at the top of the page.
"invalid" is missing and this is not scalable.
Add natigation buttons (tabs) for main pages.
2020-07-09 19:40:12 +02:00
Dmitry Vyukov
894db56cb2 pkg/html: add function to format JS time 2020-07-09 19:40:12 +02:00
Dmitry Vyukov
66bbaf2d59 syz-ci: rebuild syzkaller if latest build does not exist 2020-07-09 19:40:12 +02:00
Andrey Konovalov
3ea3b66344 Update syzbot.md 2020-07-09 18:43:28 +02:00
Tobias Klauser
e2d3226c15 docs/linux: use OpenSBI jump firmware in QEMU for linux/riscv64
Use the OpenSBI "jump" firmware with the QEMU -bios parameter to boot
linux/riscv64. This will allow to boot the kernel directly without
having to recompile OpenSBI every time the kernel is recompiled.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2020-07-09 11:57:12 +02:00
Tobias Klauser
a864f3d20c docs/linux: add not about default OpenSBI firmware in QEMU for linux/riscv64
Mention the possibility to use `-bios default` to use OpenSBI provided
with QEMU. However, this might not work on all distributions yet, so
make it optional.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2020-07-09 11:43:36 +02:00
Tobias Klauser
1bac9ec6ea docs/linux: drop dhcpcd from buildroot options in linux/riscv64 instructions
buildroot already installs udhcpc as a DHCP client.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2020-07-09 11:43:36 +02:00
Tobias Klauser
e3dcfa19aa docs/linux: mention QEMU version in linux/riscv64 instructions
Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2020-07-09 11:43:36 +02:00
Dmitry Vyukov
5b6bfb32b0
docs: add explanation of flags type
* docs: add explanation of flags type
* Apply suggestions from code review
Co-authored-by: Marco Elver <marco.elver@gmail.com>
2020-07-09 11:42:24 +02:00
Necip Fazil Yildiran
ff85e8be1a sys/linux: improved the desc for ioprio priority values
ioprio priority values (as passed to ioprio_set()) take values from
a limited range. This fact was reflected in the descriptions.
2020-07-09 09:06:26 +02:00
Jiaheng Hu
bc238812ae sys/linux: add close_range
Added description for close_range which is currently in linux-next
(https://lore.kernel.org/lkml/20200602204219.186620-2-christian.brauner@ubuntu.com/).
2020-07-08 13:37:34 -07:00
Necip Fazil Yildiran
6154150efc sys/linux: enhanced socket netlink ethool descriptions 2020-07-08 22:34:46 +02:00
Dmitry Vyukov
9f9845eb28 tools/syz-check: improve warning for NLA_REJECT 2020-07-08 17:14:54 +02:00
Dmitry Vyukov
3fe0c531c7 executor: clarify debug/debug_verbose use 2020-07-08 15:56:59 +02:00
Dmitry Vyukov
f5b4cca65c executor: prohibit malloc 2020-07-08 15:55:10 +02:00
Dmitry Vyukov
0219d4e7da docs: explain pseudo-syscall argument type 2020-07-08 15:14:01 +02:00
Dmitry Vyukov
f1996f0507 docs: add tips on writing descriptions 2020-07-08 14:31:45 +02:00
Dmitry Vyukov
f7ae27af2f docs: add note that pseudo-syscalls are discouraged 2020-07-08 14:27:01 +02:00
Dmitry Vyukov
bae5742c50 pkg/report: tune synchronize_srcu name for older kernels 2020-07-08 09:26:48 +02:00
Jiaheng Hu
5962a2dc88 sys/linux: add process_madvise
Adde description for process_madvise syscall in
linux-next (https://lore.kernel.org/lkml/20200622192900.22757-4-minchan@kernel.org).
2020-07-07 18:08:11 -07:00
Jiaheng Hu
789dd0861d sys/linux: add new flags into clone3
Added the CLONE_INTO_CGROUP and CLONE_CLEAR_SIGHAND flag for the clone3
syscall.
2020-07-07 12:34:11 -07:00
Dmitry Vyukov
08fc4ef14d pkg/report: make smp_call_function anchor instead of on_each_cpu
Right after committing the on_each_cpu change,
another report come in where smp_call_function
is not called from on_each_cpu. And there are
actually more such callers in code, and also
as existing tests show. smp_call_function seems
to be the better root cause indication.
2020-07-07 12:04:22 +02:00
Dmitry Vyukov
f7b01f08a3 pkg/vcs: fix config bisection tests more
config-bisect.pl uses bash-isms and can't run on non-linux.
It also silently ignores all errors which made failures
very obscure -- the script happily succeeds in presence
of any errors. So the test failed later reading .config.
Use "set -eu" to not fail silently.
Also trace all config-bisect.pl invocations and output.
good/bad decisions are important and we always log them
in the normal bisection.
2020-07-07 10:40:55 +02:00
Dmitry Vyukov
e419f4e6d8 pkg/report: consider on_each_cpu as stall anchor function
Kernel frequently hangs in on_each_cpu,
it seems to be more indicative of the root cause
than subsequent frames.
2020-07-07 10:40:55 +02:00
Dmitry Vyukov
3f3be050c0 pkg/vcs: fix config bisection test
/bin/bash is usually installed on linux'es,
but not on other OSes, e.g. *BSDs.
Use /bin/sh instead of /bin/bash.
2020-07-07 08:47:36 +02:00
Dmitry Vyukov
ef6b94b522 pkg/vcs: fix config minimization test
The test requires new git binary.
It's not necessary installed everywhere.
Skip the test if git binary does not support bisection.

Also reoder functions from more important to less important.
Mark the test as parallel.
Move individual tests into subtests.
Split trace per subtest.
2020-07-07 08:36:58 +02:00
Greg Steuck
4272335555 Makefile: avoid non-portable realpath
pwd -P works great and is portable.
This fixes OpenBSD ci.
2020-07-07 07:28:07 +02:00
Dmitry Vyukov
ef278c737f pkg/report: use rtnetlink_rcv_msg as stall anchor frame
It also hash high branching factor and a bug is more likely in the callback.
For the added test we used to say:
INFO: rcu detected stall in __sys_sendmsg
now we say more useful:
INFO: rcu detected stall in tc_modify_qdisc
2020-07-06 17:26:25 +02:00
Dmitry Vyukov
36db93ffb9 pkg/osutil: remove appengine tag
Appengine doesn't provide appengine tag anymore.
Resort to use of syscall package for ExitStatus instead.
2020-07-06 15:59:35 +02:00
Dmitry Vyukov
4e3820bfb2 pkg/osutil: don't use os.ProcessState.ExitCode on appengine
It's not present in Go 1.11.
2020-07-06 15:04:35 +02:00
Dmitry Vyukov
ac5a135b30 tools/syz-linter: introduce helper Pass type
Introduce a helper Pass type so that we can have
some helper methods on it.

Update #1876
2020-07-06 08:23:03 +02:00
Dmitry Vyukov
3e0ce485a1 tools/syz-linter: check variable declarations
Warn about:

var x int = foo

In most cases this can be expressed shorter as:

var x int
x := foo
x := int(foo)

Update #1876
2020-07-06 08:23:03 +02:00
Dmitry Vyukov
ed2ced4c69 tools/syz-linter: check for \n at the end of log/error text
Update #1876
2020-07-06 08:23:03 +02:00
Dmitry Vyukov
22f8756722 tools/syz-linter: check log/error text ending with dot
Update #1876
2020-07-05 10:55:16 +02:00