Commit Graph

5442 Commits

Author SHA1 Message Date
Mark Johnston
d0a4af89ac sys/freebsd: switch to the new .const format 2020-09-13 17:15:57 +02:00
Alexander Egorenkov
d56d957c29 docs/linux: describe how to setup s390x test environment
Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-09-13 16:06:07 +02:00
Alexander Egorenkov
86bd8f9d48 tools/create-image.sh: support for foreign architectures
Also update sy-env to be able to build the root image inside.

Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-09-13 16:05:25 +02:00
Dmitry Vyukov
e5f38ffe77 syz-manager: fix nil defer
Don't leave nil fuzzers in the map.
2020-09-13 13:42:03 +02:00
Dmitry Vyukov
455eff3ca1 pkg/report: skip rb tree functions in linux reports
RB tree is just a container (like list we already skip),
the bug is usually in the caller. Skip RB frames.
The new titles are much more informative and have lower chances of collisions.
2020-09-13 09:26:44 +02:00
Dmitry Vyukov
ad54dc7a6d syz-manager: cosmetic refactoring
1. Don't duplicate VM name generation.
2. Always "shutdown" fuzzers to not keep stale machine info.
2020-09-12 17:38:54 +02:00
Dmitry Vyukov
01622de2d0 pkg/host: move machine info functionality from syz-fuzzer
It's better to keep functionality in packages rather than in main.
It makes it reusable and better organized.
Move machine info functionality to pkg/host and do some cosmetic refactoring.
2020-09-12 17:38:54 +02:00
bobogei81123
7aa6bd6859
syz-manager: collect machine information
* syz-manager: finish a prototype

Extract machine info from /proc/cpuinfo and /sys/kvm*/parameters/* and
send it from syz-fuzzer to syz-manager. Append the machine info after
crash reports.

* syz-manager: refactor the code

- Add kvm parameters machine info.
- Store the machine info in the RPCServer instead of the manager.
- Store the machine info in another field instead of appending it after
  the original report
- Save the machine info locally in machineInfo*.

* syz-manager: fix coding-style problems

* syz-fuzzer: improve the output from /proc/cpuinfo

Improve the machine info extracted from /proc/cpuinfo by grouping lines
with the same key.

* syz-manager: fix race condition in runInstance

* syz-fuzzer: add tests for collecting machine info

- Add some tests to test collecting machine information.
- Split readCPUInfo into scanCPUInfo so that we can test it.

* syz-fuzzer: refactor scanCPUInfo

Refactor scanCPUInfo so that no sorting is needed.

* syz-fuzzer: refactor some code

Fix some issue that was pointed out on Github.
2020-09-12 17:17:22 +02:00
Dmitry Vyukov
9296c80bbc tools/docker: mirror images on github
Some users don't have access to the gcr.io registry.
Mirror images on github as well.
2020-09-12 16:26:58 +02:00
Anton Lindqvist
ce441f065b executor: improve opendir(3) error handling
While investigating an OpenBSD reproducer[1][2] I discovered the
following:

* All threads are stuck on the last `sleep(1000000)` syscall in main(),
  hence no output for the test machine.

* Each executor process created in loop() performs one iteration but
  exits abnormally during the call to remove_dir().

* Calling remove_dir() will eventually invoke itself recursively since
  one of the executed syscall is `mkdir("./file0", 0)` meaning that it
  will try to remove the directory created by execute_one(). However,
  `opendir(3)` fails with `EACCES` due to the permissions passed to
  `mkdir(2)` is zero.

Instead of exiting, trying to remove the problematic directory in a best
effort manner makes the reproducer continue executing the generated
syscalls. This work around might be considered to narrow. Another option
would be to replace the `sleep(1000000)` with `waitpid(-1, NULL, 0)`
until ECHILD is hit.

[1] https://syzkaller.appspot.com/bug?id=6f7ce2a0536580a94f65f44e478732ec505e88af
[2] https://syzkaller.appspot.com/text?tag=ReproC&x=10fd1a71900000
2020-09-12 13:21:21 +02:00
Dmitry Vyukov
7b0683780a executor: detect gvisor
gvisor coverage is not in the range of linux kernel coverage.
So the coverage filter does not work. Detect if running under gvisor
and skip the coverage filter.
2020-09-12 13:03:35 +02:00
Dmitry Vyukov
c38fcca50d pkg/repro: fix execution of non-repeatig C programs
If we have a non-repeating C reproducer with timeout > vm.NoOutputTimeout and it hangs
(the reproducer itself does not terminate on its own, note: it does not have builtin timeout),
then we will falsely detect "not output from test machine" kernel bug.
We could fix it by adding a builtin timeout to such reproducers (like we have in all other cases).
However, then it will exit within few seconds and we will finish the test without actually waiting
for full vm.NoOutputTimeout, which breaks the whole reason of using vm.NoOutputTimeout in the first
place. So we would need something more elaborate: let the program exist after few seconds, but
continue waiting for kernel hang errors for minutes, but at the same time somehow ignore "no output"
error because it will be false in this case.
Instead we simply prohibit !Repeat with long timeouts.
It makes sense on its own to some degree: if we are chasing an elusive bug, repeating the test
will increase chances of reproducing it and can make the reproducer less flaky.
Syz repros does not have this problem because they always have internal timeout, however
(1) it makes sense on its own, (2) we will either not use the whole timeout or waste the remaining
time as mentioned above, (3) if we remove repeat for syz repro, we won't be able to handle it
when/if we switch to C repro (we can simplify options, but we can't "complicate" them back).
2020-09-12 13:03:27 +02:00
Dmitry Vyukov
306464056c sys/test/test: add a hanging test
Ensure that we can handle hanging syscalls in all modes.
2020-09-12 13:03:27 +02:00
Dmitry Vyukov
cc8045ff1f tools/syz-prog2c: add -repro flag
For manual testing of what pkg/repro uses.
2020-09-12 13:03:27 +02:00
Dmitry Vyukov
21d289c28f pkg/report: add gvisor "stuck task" pattern 2020-09-12 12:04:32 +02:00
Dmitry Vyukov
79fb24e23d
docs: clarify what is coverage 2020-09-11 11:59:39 +02:00
Dmitry Vyukov
adfb8b4e39
docs: clarify that Go 1.14 is required for contributors 2020-09-11 08:32:44 +02:00
bobogei81123
409809d8a7
sys/linux: improve descriptions of bpf tracing (#2076)
- Rename bpf_lsm to bpf_trace and put all bpf program types
  that use BPF_RAW_TRACEPOINT_OPEN here.
- Add descriptions for types RAW_TRACEPOINT(_WRITABLE), BPF_TRACING and
  BPF_EXT.
- Add the hook names for RAW_TRACEPOINT(_WRITABLE).
2020-09-09 14:02:04 -07:00
Albert van der Linde
ac7ca78e73 dashboard/config: enable CONFIG_FAULT_INJECTION_USERCOPY
As CONFIG_FAULT_INJECTION_USERCOPY is already included in linux-next,
enable the config.

Fixes #1979
2020-09-09 14:39:22 +02:00
Andrew Donnellan
0ea7a8875d dashboard/app: add missing index for job_poll API call
At least on my dashboard setup, the API call 'job_poll' results in a
NEED_INDEX datastore error.

Add the index definition that the development server generates if you make
the job_poll API call.

Signed-off-by: Andrew Donnellan <ajd@linux.ibm.com>
2020-09-09 07:14:48 +02:00
Alexander Egorenkov
a86a864df5 pkg/build: support Linux s390x
Signed-off-by: Alexander Egorenkov <Alexander.Egorenkov@ibm.com>
2020-09-09 07:13:52 +02:00
Greg Steuck
abf9ba4fc7
tools/create-openbsd: auto-detect snapshot version (#2082)
Removed some never-used knobs as syzkaller always uses snapshots.
Resolved some ShellCheck'd lint.

TESTED=by installing Aug31 6.8 beta.
2020-09-01 10:54:55 -07:00
Cheng-Min Chiang
d5a3ae1f76 sys/linux: add descriptions of qat_adf_ctl
Add descriptions for qat_adf_ctl. It might need an intel qat device.
Update #533.
2020-08-28 16:09:22 +02:00
Stefano Duo
816e0689d7 executor/common_linux.h: add missing FUSE opcodes
Add the following missing FUSE opcodes to the syz_fuse_handle_req
pseudo-syscall: FUSE_COPY_FILE_RANGE, FUSE_UNLINK, FUSE_DESTROY and
FUSE_BATCH_FORGET.
2020-08-27 07:47:59 +02:00
Tetsuo Handa
318430cbb3 executor/linux: change mount propagation type to private
unshare(CLONE_NEWNS) might not be sufficient for making all test processes run in
separate mount namespace, for "mount --make-rshared /" request issued by systemd
causes mount operations issued by test processes visible from outside of test
processes. Issue "mount --make-rprivate /" request after unshare(CLONE_NEWNS).
2020-08-26 12:25:46 +02:00
Ioana-Ruxandra Stăncioi
344da168cb Revert "sys/linux: add description for fsinfo"
The fsinfo() system call is under heavy discussion upstream, and it
might be totally changed. As a result it was dropped from linux-next
starting in next-20200819, and the pull request for 5.9
(https://lkml.kernel.org/r/1845353.1596469795@warthog.procyon.org.uk/)
wasn't accepted. Therefore revert the syscall description (commit
4b0871d) for now.
2020-08-24 16:12:41 -07:00
Andrey Konovalov
9006857c45
docs/usb: fix raw-gadget instructions 2020-08-25 01:06:05 +02:00
bobogei81123
622e52f44c
sys/linux: make bpf_lsm_btf_id optional (#2054)
Pull request #1971 add the resource bpf_lsm_btf_id and make that a
required resource for bpf$BPF_LSM_PROG_LOAD. However, we need #2035
merged to get a bpf_lsm_btf_id, and the pull request is currently
blocked by a pahole issue. Thus, bpf$BPF_LSM_PROG_LOAD will be disabled
for now.

This pull request makes bpf_lsm_btf_id optional for
bpf$BPF_LSM_PROG_LOAD, so we can test this syscall before the issue is
resolved.
2020-08-24 13:36:18 -07:00
Dmitry Vyukov
9e08308fe1 Makefile: fix clang-tidy comment
We now run more than one check.
2020-08-24 17:10:02 +02:00
Dmitry Vyukov
67b599d167 syz-manager: better handle errors in HTML templates
I periodically see:
2020/08/23 13:33:21 http: superfluous response.WriteHeader
	call from main.(*Manager).httpSummary (html.go:72)
which suggest that there are some erros during template execution.
But currently we don't seem to show them properly.
Show them properly and also log.
2020-08-24 13:28:22 +02:00
Dmitry Vyukov
4b4ae700b3
docs/linux: update found bugs 2020-08-24 13:26:15 +02:00
Andy Nguyen
cef5ae6814 executor/common_linux.h: make rfkill_unblock_all static 2020-08-23 12:31:52 +02:00
Andy Nguyen
5bde7c3b04 executor/common_linux.h: unblock rfkill and retry HCIDEVUP on ERFKILL 2020-08-23 12:31:52 +02:00
Dmitry Vyukov
a6d5f3ad13 prog: prune recursion for resourceCentric/createResource
Currently we only prune recursion for createResource.
But createResource can still recurse into resourceCentric,
this can produce huge programs and may be very slow
(esp. if we already in createResource doing failing attempts).

With this change:
$ CI=yes go test -run TestMutateCorpus -count=2 ./prog
passes in ~3 seconds. If I comment out inGenerateResource
logic, it times out after 100 seconds. So this can indeed
have significant effect on generation/mutation performance.
2020-08-23 11:46:14 +02:00
Dmitry Vyukov
1da71ab086 pkg/ipc: fix cross-arch build in test
BrokenCompiler is only initialized after Get.
2020-08-22 21:46:43 +02:00
Dmitry Vyukov
df5c8fa25f tools/docker: add old-env
old-env is based on Ubuntu 16.04 and allows to test
executor build on older distributions.

Fixes #2055
2020-08-22 21:42:13 +02:00
Dmitry Vyukov
49a5a1ab2f pkg/cover: fix calculation of prev PC for i386
Make the report generation test more realistic to use PCs
we will use in real life. This shows that PreviousInstructionPC
for 386 is broken. Fix it.

Reported-by: Alexander Lochmann <flipreverse>
See #2067
2020-08-22 21:26:42 +02:00
Dmitry Vyukov
7288521c2a pkg/ipc: run cross-arch executor tests
We may run some cross-arch tests using qemu-user.
2020-08-22 21:26:42 +02:00
Dmitry Vyukov
0dba77942f tools/docker/env: install qemu-user
qemu-user allows to run cross-arch binaries in tests.
2020-08-22 21:26:42 +02:00
clslgrnc
6436ce4bcd docs/linux: fix KMEMLEAK variable in suggested kernel configs 2020-08-21 08:12:55 +02:00
Cheng-Min Chiang
1d75fe458f sys/linux: add descriptions of vga_arbiter
Add desciptions for testing /dev/vga_arbiter.
Update #533
2020-08-20 13:39:39 -07:00
Tree Davies
7016057751 docs/usage.md: correct grammatical error
'...a way connect them' -> '...a way to connect them'

Signed-off-by: Tree Davies <tdavies@darkphysics.net>
2020-08-20 19:54:30 +02:00
Jiaheng Hu
ed282a3a90 sys/linux: add BTRFS specific ioctls
Add the ioctls specific to BTRFS (linux v5.8).
2020-08-19 09:29:36 -07:00
ThePraeceps
94b457068c Missing space at end of line of command
There is a missing space on the command at line 131, which when copied causes it not to execute properly.
2020-08-19 16:46:57 +02:00
Ioana-Ruxandra Stăncioi
db7879025a sys/linux: add lwtunnel_encap_seg6 description
Uncommented because the kernel fixed the seg6_iptunnel.h uapi header
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=88fab21c691bb1ff164e540735237a385e3afeaf
2020-08-19 13:28:33 +02:00
Marco Vanotti
a7d4a4f9e7 sys/targets: update fuchsia CFLAGS 2020-08-18 18:50:34 -07:00
Dmitry Vyukov
e1c29030da Revert "vm/vmimpl: disallow ssh authentication agent"
This reverts commit b9683dbdfd.

Majority of syzbot instances started failing with:

failed to run ["ssh" "-p" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "IdentityAgent=none" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/managers/upstream-bpf-next-kasan-gce/latest.tmp/key" "root@10.128.15.195" "pwd"]: exit status 255
command-line: line 0: Bad configuration option: identityagent
command-line: line 0: Bad configuration option: identityagent
2020-08-18 15:06:05 +02:00
Stefano Duo
6fe61c6bc7 executor/common_linux.go: fix build breakage for Linux 4.14
Locally define structures and constants required by
syz_fuse_handle_req() and remove linux/fuse.h. dependency.

Reported-by: syzbot+f5f98e1608b272285aa5@syzkaller.appspotmail.com
2020-08-18 14:18:38 +02:00
Marco Vanotti
b9683dbdfd vm/vmimpl: disallow ssh authentication agent
This commit adds a new option to SSH options, disallowing the
authentication agent. This is specially useful when you are testing
in a machine that sets the `SSH_AUTH_SOCK` environment variable, as
ssh will try to use that authentication agent on each ssh connection.
2020-08-18 12:18:12 +02:00
Mark Johnston
635f68f680 pkg/build: add configuration to /boot/loader.conf.local on FreeBSD
With commit 7ba05d2dd6 we always write a
fresh loader.conf on each build, but this clobbers any pre-existing
settings that may be required for a given setup.  This went unnoticed by
me for a while since bhyve requires no additional preconfiguration, but
clearly syzbot is affected.  On the other hand, before that commit we
were appending the same lines upon each build.  Use
/boot/loader.conf.local instead.
2020-08-18 11:22:05 +02:00