mirror of
https://github.com/reactos/syzkaller.git
synced 2024-11-23 03:19:51 +00:00
docs: move parts of README to docs
This commit is contained in:
Andrey Konovalov
parent
fbec6b14d4
commit
15826f5087
243
README.md
243
README.md
@ -1,242 +1,19 @@
|
|||||||
# syzkaller - linux syscall fuzzer
|
# syzkaller - linux kernel fuzzer
|
||||||
|
|
||||||
`syzkaller` is an unsupervised, coverage-guided Linux syscall fuzzer.
|
`syzkaller` is an unsupervised coverage-guided Linux kernel fuzzer.
|
||||||
It is meant to be used with
|
|
||||||
[KASAN](https://kernel.org/doc/html/latest/dev-tools/kasan.html) (available upstream with `CONFIG_KASAN=y`),
|
|
||||||
[KTSAN](https://github.com/google/ktsan) (prototype available),
|
|
||||||
[KMSAN](https://github.com/google/kmsan) (prototype available),
|
|
||||||
or [KUBSAN](https://kernel.org/doc/html/latest/dev-tools/ubsan.html) (available upstream with `CONFIG_UBSAN=y`).
|
|
||||||
|
|
||||||
Project mailing list: [syzkaller@googlegroups.com](https://groups.google.com/forum/#!forum/syzkaller).
|
The project mailing list is [syzkaller@googlegroups.com](https://groups.google.com/forum/#!forum/syzkaller).
|
||||||
You can subscribe to it with a google account or by sending an email to syzkaller+subscribe@googlegroups.com.
|
You can subscribe to it with a google account or by sending an email to syzkaller+subscribe@googlegroups.com.
|
||||||
|
|
||||||
List of [found bugs](docs/found_bugs.md).
|
[List of found bugs](docs/found_bugs.md).
|
||||||
|
|
||||||
How to [report Linux kernel bugs](docs/linux_kernel_reporting_bugs.md).
|
## Documentation
|
||||||
|
|
||||||
How to [contribute](docs/contributing.md).
|
- [How to install syzkaller](docs/setup.md)
|
||||||
|
- [How to use syzkaller](docs/usage.md)
|
||||||
## Usage
|
- [How syzkaller works](docs/internals.md)
|
||||||
|
- [How to contribute to syzkaller](docs/contributing.md)
|
||||||
The following components are needed to use syzkaller:
|
- [How to report Linux kernel bugs](docs/linux_kernel_reporting_bugs.md)
|
||||||
|
|
||||||
- C compiler with coverage support
|
|
||||||
- Linux kernel with coverage additions
|
|
||||||
- Virtual machine or a physical device
|
|
||||||
- syzkaller itself
|
|
||||||
|
|
||||||
Generic steps to set up syzkaller are described below.
|
|
||||||
More specific information (like the exact steps for a particular host system, VM type and a kernel architecture) can be found in [the documentation](docs/).
|
|
||||||
|
|
||||||
### C Compiler
|
|
||||||
|
|
||||||
Syzkaller is a coverage-guided fuzzer and therefore it needs the kernel to be built with coverage support, which requires a recent GCC version.
|
|
||||||
Coverage support was submitted to GCC in revision `231296`, released in GCC v6.0.
|
|
||||||
|
|
||||||
### Linux Kernel
|
|
||||||
|
|
||||||
Besides coverage support in GCC, you also need support for it on the kernel side.
|
|
||||||
KCOV was committed upstream in Linux kernel version 4.6 and can be enabled by configuring the kernel with `CONFIG_KCOV=y`.
|
|
||||||
For older kernels you need to backport commit [kernel: add kcov code coverage](https://github.com/torvalds/linux/commit/5c9a8750a6409c63a0f01d51a9024861022f6593).
|
|
||||||
|
|
||||||
To enable more syzkaller features and improve bug detection abilities, it's recommended to use additional config options.
|
|
||||||
See [this page](docs/linux_kernel_configs.md) for details.
|
|
||||||
|
|
||||||
### VM Setup
|
|
||||||
|
|
||||||
Syzkaller performs kernel fuzzing on slave virtual machines or physical devices.
|
|
||||||
These slave enviroments are referred to as VMs.
|
|
||||||
Out-of-the-box syzkaller supports QEMU, kvmtool and GCE virtual machines, Android devices and Odroid C2 boards.
|
|
||||||
|
|
||||||
These are the generic requirements for a syzkaller VM:
|
|
||||||
|
|
||||||
- The fuzzing processes communicate with the outside world, so the VM image needs to include
|
|
||||||
networking support.
|
|
||||||
- The program files for the fuzzer processes are transmitted into the VM using SSH, so the VM image
|
|
||||||
needs a running SSH server.
|
|
||||||
- The VM's SSH configuration should be set up to allow root access for the identity that is
|
|
||||||
included in the `syz-manager`'s configuration. In other words, you should be able to do `ssh -i
|
|
||||||
$SSHID -p $PORT root@localhost` without being prompted for a password (where `SSHID` is the SSH
|
|
||||||
identification file and `PORT` is the port that are specified in the `syz-manager` configuration
|
|
||||||
file).
|
|
||||||
- The kernel exports coverage information via a debugfs entry, so the VM image needs to mount
|
|
||||||
the debugfs filesystem at `/sys/kernel/debug`.
|
|
||||||
|
|
||||||
To use QEMU syzkaller VMs you have to install QEMU on your host system, see [QEMU docs](http://wiki.qemu.org/Manual) for details.
|
|
||||||
The [create-image.sh](tools/create-image.sh) script can be used to create a suitable Linux image.
|
|
||||||
Detailed steps for setting up syzkaller with QEMU on a Linux host are avaialble for [x86-64](docs/setup_ubuntu-host_qemu-vm_x86-64-kernel.md) and [arm64](docs/setup_linux-host_qemu-vm_arm64-kernel.md) kernels.
|
|
||||||
|
|
||||||
For some details on fuzzing the kernel on an Android device check out [this page](docs/setup_linux-host_android-device_arm64-kernel.md) and the explicit instructions for an Odroid C2 board are available [here](docs/setup_ubuntu-host_odroid-c2-board_arm64-kernel.md).
|
|
||||||
|
|
||||||
### Syzkaller
|
|
||||||
|
|
||||||
The syzkaller tools are written in [Go](https://golang.org), so a Go compiler (>= 1.8) is needed
|
|
||||||
to build them.
|
|
||||||
|
|
||||||
Go distribution can be downloaded from https://golang.org/dl/.
|
|
||||||
Unpack Go into a directory, say, `$HOME/go`.
|
|
||||||
Then, set `GOROOT=$HOME/go` env var.
|
|
||||||
Then, add Go binaries to `PATH`, `PATH=$HOME/go/bin:$PATH`.
|
|
||||||
Then, set `GOPATH` env var to some empty dir, say `GOPATH=$HOME/gopath`.
|
|
||||||
Then, run `go get -u -d github.com/google/syzkaller/...` to checkout syzkaller sources with all dependencies.
|
|
||||||
Then, `cd $GOPATH/src/github.com/google/syzkaller` and
|
|
||||||
build with `make`, which generates compiled binaries in the `bin/` folder.
|
|
||||||
|
|
||||||
To build additional syzkaller tools run `make all-tools`.
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
The operation of the syzkaller `syz-manager` process is governed by a configuration file, passed at
|
|
||||||
invocation time with the `-config` option. This configuration can be based on the
|
|
||||||
[example](syz-manager/config/testdata/qemu.cfg); the file is in JSON format with the
|
|
||||||
following keys in its top-level object:
|
|
||||||
|
|
||||||
- `http`: URL that will display information about the running `syz-manager` process.
|
|
||||||
- `workdir`: Location of a working directory for the `syz-manager` process. Outputs here include:
|
|
||||||
- `<workdir>/crashes/*`: crash output files (see [Crash Reports](#crash-reports))
|
|
||||||
- `<workdir>/corpus.db`: corpus with interesting programs
|
|
||||||
- `<workdir>/instance-x`: per VM instance temporary files
|
|
||||||
- `syzkaller`: Location of the `syzkaller` checkout.
|
|
||||||
- `vmlinux`: Location of the `vmlinux` file that corresponds to the kernel being tested.
|
|
||||||
- `procs`: Number of parallel test processes in each VM (4 or 8 would be a reasonable number).
|
|
||||||
- `leak`: Detect memory leaks with kmemleak.
|
|
||||||
- `image`: Location of the disk image file for the QEMU instance; a copy of this file is passed as the
|
|
||||||
`-hda` option to `qemu-system-x86_64`.
|
|
||||||
- `sandbox` : Sandboxing mode, the following modes are supported:
|
|
||||||
- "none": don't do anything special (has false positives, e.g. due to killing init)
|
|
||||||
- "setuid": impersonate into user nobody (65534), default
|
|
||||||
- "namespace": use namespaces to drop privileges
|
|
||||||
(requires a kernel built with `CONFIG_NAMESPACES`, `CONFIG_UTS_NS`,
|
|
||||||
`CONFIG_USER_NS`, `CONFIG_PID_NS` and `CONFIG_NET_NS`)
|
|
||||||
- `enable_syscalls`: List of syscalls to test (optional).
|
|
||||||
- `disable_syscalls`: List of system calls that should be treated as disabled (optional).
|
|
||||||
- `suppressions`: List of regexps for known bugs.
|
|
||||||
- `type`: Type of virtual machine to use, e.g. `qemu` or `adb`.
|
|
||||||
- `vm`: object with VM-type-specific parameters; for example, for `qemu` type paramters include:
|
|
||||||
- `count`: Number of VMs to run in parallel.
|
|
||||||
- `kernel`: Location of the `bzImage` file for the kernel to be tested;
|
|
||||||
this is passed as the `-kernel` option to `qemu-system-x86_64`.
|
|
||||||
- `cmdline`: Additional command line options for the booting kernel, for example `root=/dev/sda1`.
|
|
||||||
- `sshkey`: Location (on the host machine) of an SSH identity to use for communicating with
|
|
||||||
the virtual machine.
|
|
||||||
- `cpu`: Number of CPUs to simulate in the VM (*not currently used*).
|
|
||||||
- `mem`: Amount of memory (in MiB) for the VM; this is passed as the `-m` option to `qemu-system-x86_64`.
|
|
||||||
|
|
||||||
See also [config.go](syz-manager/config/config.go) for all config parameters.
|
|
||||||
|
|
||||||
|
|
||||||
## Running syzkaller
|
|
||||||
|
|
||||||
Start the `syz-manager` process as:
|
|
||||||
```
|
|
||||||
./bin/syz-manager -config my.cfg
|
|
||||||
```
|
|
||||||
|
|
||||||
The `-config` command line option gives the location of the configuration file [described above](#configuration).
|
|
||||||
|
|
||||||
The `syz-manager` process will wind up QEMU virtual machines and start fuzzing in them.
|
|
||||||
Found crashes, statistics and other information is exposed on the HTTP address provided in manager config.
|
|
||||||
|
|
||||||
|
|
||||||
## Process Structure
|
|
||||||
|
|
||||||
The process structure for the syzkaller system is shown in the following diagram;
|
|
||||||
red labels indicate corresponding configuration options.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
The `syz-manager` process starts, monitors and restarts several VM instances (support for
|
|
||||||
physical machines is not implemented yet), and starts a `syz-fuzzer` process inside of the VMs.
|
|
||||||
It is responsible for persistent corpus and crash storage. As opposed to `syz-fuzzer` processes,
|
|
||||||
it runs on a host with stable kernel which does not experience white-noise fuzzer load.
|
|
||||||
|
|
||||||
The `syz-fuzzer` process runs inside of presumably unstable VMs (or physical machines under test).
|
|
||||||
The `syz-fuzzer` guides fuzzing process itself (input generation, mutation, minimization, etc)
|
|
||||||
and sends inputs that trigger new coverage back to the `syz-manager` process via RPC.
|
|
||||||
It also starts transient `syz-executor` processes.
|
|
||||||
|
|
||||||
Each `syz-executor` process executes a single input (a sequence of syscalls).
|
|
||||||
It accepts the program to execute from the `syz-fuzzer` process and sends results back.
|
|
||||||
It is designed to be as simple as possible (to not interfere with fuzzing process),
|
|
||||||
written in C++, compiled as static binary and uses shared memory for communication.
|
|
||||||
|
|
||||||
## Crash Reports
|
|
||||||
|
|
||||||
When `syzkaller` finds a crasher, it saves information about it into `workdir/crashes` directory. The directory contains one subdirectory per unique crash type. Each subdirectory contains a `description` file with a unique string identifying the crash (intended for bug identification and deduplication); and up to 100 `logN` and `reportN` files, one pair per test machine crash:
|
|
||||||
```
|
|
||||||
- crashes/
|
|
||||||
- 6e512290efa36515a7a27e53623304d20d1c3e
|
|
||||||
- description
|
|
||||||
- log0
|
|
||||||
- report0
|
|
||||||
- log1
|
|
||||||
- report1
|
|
||||||
...
|
|
||||||
- 77c578906abe311d06227b9dc3bffa4c52676f
|
|
||||||
- description
|
|
||||||
- log0
|
|
||||||
- report0
|
|
||||||
...
|
|
||||||
```
|
|
||||||
|
|
||||||
Descriptions are extracted using a set of [regular expressions](report/report.go#L33). This set may need to be extended if you are using a different kernel architecture, or are just seeing a previously unseen kernel error messages.
|
|
||||||
|
|
||||||
`logN` files contain raw `syzkaller` logs and include kernel console output as well as programs executed before the crash. These logs can be fed to `syz-repro` tool for [crash location and minimization](docs/tools_syz-execprog_syz-prog2c_syz-repro.md), or to `syz-execprog` tool for [manual localization](docs/executing_syzkaller_programs.md). `reportN` files contain post-processed and symbolized kernel crash reports (e.g. a KASAN report). Normally you need just 1 pair of these files (i.e. `log0` and `report0`), because they all presumably describe the same kernel bug. However, `syzkaller` saves up to 100 of them for the case when the crash is poorly reproducible, or if you just want to look at a set of crash reports to infer some similarities or differences.
|
|
||||||
|
|
||||||
There are 3 special types of crashes:
|
|
||||||
- `no output from test machine`: the test machine produces no output whatsoever
|
|
||||||
- `lost connection to test machine`: the ssh connection to the machine was unexpectedly closed
|
|
||||||
- `test machine is not executing programs`: the machine looks alive, but no test programs were executed for long period of time
|
|
||||||
Most likely you won't see `reportN` files for these crashes (e.g. if there is no output from the test machine, there is nothing to put into report). Sometimes these crashes indicate a bug in `syzkaller` itself (especially if you see a Go panic message in the logs). However, frequently they mean a kernel lockup or something similarly bad (here are just a few examples of bugs found this way: [1](https://groups.google.com/d/msg/syzkaller/zfuHHRXL7Zg/Tc5rK8bdCAAJ), [2](https://groups.google.com/d/msg/syzkaller/kY_ml6TCm9A/wDd5fYFXBQAJ), [3](https://groups.google.com/d/msg/syzkaller/OM7CXieBCoY/etzvFPX3AQAJ)).
|
|
||||||
|
|
||||||
## Syscall description
|
|
||||||
|
|
||||||
`syzkaller` uses declarative description of syscalls to generate, mutate, minimize,
|
|
||||||
serialize and deserialize programs (sequences of syscalls). See details about the
|
|
||||||
format and extending the descriptions [here](docs/syscall_descriptions.md).
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
Here are some things to check if there are problems running syzkaller.
|
|
||||||
|
|
||||||
- Check that QEMU can successfully boot the virtual machine. For example,
|
|
||||||
if `IMAGE` is set to the VM's disk image (as per the `image` config value)
|
|
||||||
and `KERNEL` is set to the test kernel (as per the `kernel` config value)
|
|
||||||
then something like the following command should start the VM successfully:
|
|
||||||
|
|
||||||
```qemu-system-x86_64 -hda $IMAGE -m 256 -net nic -net user,host=10.0.2.10,hostfwd=tcp::23505-:22 -enable-kvm -kernel $KERNEL -append root=/dev/sda```
|
|
||||||
|
|
||||||
- Check that inbound SSH to the running virtual machine works. For example, with
|
|
||||||
a VM running and with `SSHKEY` set to the SSH identity (as per the `sshkey` config value) the
|
|
||||||
following command should connect:
|
|
||||||
|
|
||||||
```ssh -i $SSHKEY -p 23505 root@localhost```
|
|
||||||
|
|
||||||
- Check that the `CONFIG_KCOV` option is available inside the VM:
|
|
||||||
- `ls /sys/kernel/debug # Check debugfs mounted`
|
|
||||||
- `ls /sys/kernel/debug/kcov # Check kcov enabled`
|
|
||||||
- Build the test program from `Documentation/kcov.txt` and run it inside the VM.
|
|
||||||
|
|
||||||
- Check that debug information (from the `CONFIG_DEBUG_INFO` option) is available
|
|
||||||
- Pass the hex output from the kcov test program to `addr2line -a -i -f -e $VMLINUX` (where
|
|
||||||
`VMLINUX` is the vmlinux file, as per the `vmlinux` config value), to confirm
|
|
||||||
that symbols for the kernel are available.
|
|
||||||
|
|
||||||
- Use the `-v N` command line option to increase the amount of logging output, from both
|
|
||||||
the `syz-manager` top-level program and the `syz-fuzzer` instances (which go to the
|
|
||||||
output files in the `crashes` subdirectory of the working directory). Higher values of
|
|
||||||
N give more output.
|
|
||||||
|
|
||||||
- If logging indicates problems with the executor program (e.g. `executor failure`),
|
|
||||||
try manually running a short sequence of system calls:
|
|
||||||
- Build additional tools with `make all-tools`
|
|
||||||
- Copy `syz-executor` and `syz-execprog` into a running VM.
|
|
||||||
- In the VM run `./syz-execprog -executor ./syz-executor -debug sampleprog` where
|
|
||||||
sampleprog is a simple system call script (e.g. just containing `getpid()`).
|
|
||||||
- For example, if this reports that `clone` has failed, this probably indicates
|
|
||||||
that the test kernel does not include support for all of the required namespaces.
|
|
||||||
In this case, running the `syz-execprog` test with the `-nobody=0` option fixes the problem,
|
|
||||||
so the main configuration needs to be updated to set `dropprivs` to `false`.
|
|
||||||
|
|
||||||
## External Articles
|
## External Articles
|
||||||
|
|
||||||
|
|||||||
@ -1,4 +1,4 @@
|
|||||||
# Tools: syz-hub
|
# How to connect several managers via Hub
|
||||||
|
|
||||||
`syz-hub` program can be used to connect several `syz-manager`'s together and allow them to exchange programs.
|
`syz-hub` program can be used to connect several `syz-manager`'s together and allow them to exchange programs.
|
||||||
|
|
||||||
@ -1,3 +1,15 @@
|
|||||||
## Contributing
|
## Contributing
|
||||||
|
|
||||||
If you want to contribute to the project, you need to [sign Google CLA](https://cla.developers.google.com/) and add yourself to [AUTHORS](AUTHORS)/[CONTRIBUTORS](CONTRIBUTORS) files in the first pull request. Extending/improving [system call descriptions](sys/sys.txt) is always a good idea. If you want to work on something non-trivial, please briefly describe it on [syzkaller@googlegroups.com](https://groups.google.com/forum/#!forum/syzkaller) mailing list first so that there is agreement on high level approach and no duplication of work between contributors.
|
If you want to contribute to the project, feel free to send a pull request.
|
||||||
|
|
||||||
|
Before sending a pull request you need to [sign Google CLA](https://cla.developers.google.com/) (if you don't a bot will ask you to do that)
|
||||||
|
and add yourself to [AUTHORS](../AUTHORS)/[CONTRIBUTORS](../CONTRIBUTORS) files (in case this is your first pull request to syzkaller).
|
||||||
|
|
||||||
|
Some guildelines to follow:
|
||||||
|
|
||||||
|
- Prepend each commit with a `package:` prefix, where `package` is the package/tool this commit changes (look at examples in the [commit history](https://github.com/google/syzkaller/commits/master))
|
||||||
|
- Rebase your pull request onto the master branch before submitting
|
||||||
|
- If you're asked to add some fixes to your pull requested, please squash them into the commit being fixed
|
||||||
|
|
||||||
|
Extending/improving [system call descriptions](syscall_descriptions.md) is always a good idea.
|
||||||
|
If you want to work on something non-trivial, please briefly describe it on the [syzkaller@googlegroups.com](https://groups.google.com/forum/#!forum/syzkaller) mailing list first so that there is agreement on high level approach and no duplication of work between contributors.
|
||||||
|
|||||||
28
docs/crash_reports.md
Normal file
28
docs/crash_reports.md
Normal file
@ -0,0 +1,28 @@
|
|||||||
|
# Crash Reports
|
||||||
|
|
||||||
|
When `syzkaller` finds a crasher, it saves information about it into `workdir/crashes` directory. The directory contains one subdirectory per unique crash type. Each subdirectory contains a `description` file with a unique string identifying the crash (intended for bug identification and deduplication); and up to 100 `logN` and `reportN` files, one pair per test machine crash:
|
||||||
|
```
|
||||||
|
- crashes/
|
||||||
|
- 6e512290efa36515a7a27e53623304d20d1c3e
|
||||||
|
- description
|
||||||
|
- log0
|
||||||
|
- report0
|
||||||
|
- log1
|
||||||
|
- report1
|
||||||
|
...
|
||||||
|
- 77c578906abe311d06227b9dc3bffa4c52676f
|
||||||
|
- description
|
||||||
|
- log0
|
||||||
|
- report0
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
Descriptions are extracted using a set of [regular expressions](report/report.go#L33). This set may need to be extended if you are using a different kernel architecture, or are just seeing a previously unseen kernel error messages.
|
||||||
|
|
||||||
|
`logN` files contain raw `syzkaller` logs and include kernel console output as well as programs executed before the crash. These logs can be fed to `syz-repro` tool for [crash location and minimization](reproducing_crashes.md), or to `syz-execprog` tool for [manual localization](executing_syzkaller_programs.md). `reportN` files contain post-processed and symbolized kernel crash reports (e.g. a KASAN report). Normally you need just 1 pair of these files (i.e. `log0` and `report0`), because they all presumably describe the same kernel bug. However, `syzkaller` saves up to 100 of them for the case when the crash is poorly reproducible, or if you just want to look at a set of crash reports to infer some similarities or differences.
|
||||||
|
|
||||||
|
There are 3 special types of crashes:
|
||||||
|
- `no output from test machine`: the test machine produces no output whatsoever
|
||||||
|
- `lost connection to test machine`: the ssh connection to the machine was unexpectedly closed
|
||||||
|
- `test machine is not executing programs`: the machine looks alive, but no test programs were executed for long period of time
|
||||||
|
Most likely you won't see `reportN` files for these crashes (e.g. if there is no output from the test machine, there is nothing to put into report). Sometimes these crashes indicate a bug in `syzkaller` itself (especially if you see a Go panic message in the logs). However, frequently they mean a kernel lockup or something similarly bad (here are just a few examples of bugs found this way: [1](https://groups.google.com/d/msg/syzkaller/zfuHHRXL7Zg/Tc5rK8bdCAAJ), [2](https://groups.google.com/d/msg/syzkaller/kY_ml6TCm9A/wDd5fYFXBQAJ), [3](https://groups.google.com/d/msg/syzkaller/OM7CXieBCoY/etzvFPX3AQAJ)).
|
||||||
5
docs/internals.md
Normal file
5
docs/internals.md
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
### Internals
|
||||||
|
|
||||||
|
- [Process structure](process_structure.md)
|
||||||
|
- [Crash reports](crash_reports.md)
|
||||||
|
- [Syscall descriptions](syscall_descriptions.md)
|
||||||
@ -34,7 +34,13 @@ If your kernel doesn't have commits [arm64: setup: introduce kaslr_offset()](htt
|
|||||||
|
|
||||||
## Bug detection configs
|
## Bug detection configs
|
||||||
|
|
||||||
`KASAN` for use-after-free and out-of-bounds detection:
|
Syzkaller is meant to be used with
|
||||||
|
[KASAN](https://kernel.org/doc/html/latest/dev-tools/kasan.html) (available upstream with `CONFIG_KASAN=y`),
|
||||||
|
[KTSAN](https://github.com/google/ktsan) (prototype available),
|
||||||
|
[KMSAN](https://github.com/google/kmsan) (prototype available),
|
||||||
|
or [KUBSAN](https://kernel.org/doc/html/latest/dev-tools/ubsan.html) (available upstream with `CONFIG_UBSAN=y`).
|
||||||
|
|
||||||
|
Enable `KASAN` for use-after-free and out-of-bounds detection:
|
||||||
```
|
```
|
||||||
CONFIG_KASAN=y
|
CONFIG_KASAN=y
|
||||||
CONFIG_KASAN_INLINE=y
|
CONFIG_KASAN_INLINE=y
|
||||||
|
|||||||
21
docs/process_structure.md
Normal file
21
docs/process_structure.md
Normal file
@ -0,0 +1,21 @@
|
|||||||
|
# Process Structure
|
||||||
|
|
||||||
|
The process structure for the syzkaller system is shown in the following diagram;
|
||||||
|
red labels indicate corresponding configuration options.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
The `syz-manager` process starts, monitors and restarts several VM instances (support for
|
||||||
|
physical machines is not implemented yet), and starts a `syz-fuzzer` process inside of the VMs.
|
||||||
|
It is responsible for persistent corpus and crash storage. As opposed to `syz-fuzzer` processes,
|
||||||
|
it runs on a host with stable kernel which does not experience white-noise fuzzer load.
|
||||||
|
|
||||||
|
The `syz-fuzzer` process runs inside of presumably unstable VMs (or physical machines under test).
|
||||||
|
The `syz-fuzzer` guides fuzzing process itself (input generation, mutation, minimization, etc)
|
||||||
|
and sends inputs that trigger new coverage back to the `syz-manager` process via RPC.
|
||||||
|
It also starts transient `syz-executor` processes.
|
||||||
|
|
||||||
|
Each `syz-executor` process executes a single input (a sequence of syscalls).
|
||||||
|
It accepts the program to execute from the `syz-fuzzer` process and sends results back.
|
||||||
|
It is designed to be as simple as possible (to not interfere with fuzzing process),
|
||||||
|
written in C++, compiled as static binary and uses shared memory for communication.
|
||||||
|
Before Width: | Height: | Size: 42 KiB After Width: | Height: | Size: 42 KiB |
@ -1,4 +1,4 @@
|
|||||||
# Tools: syz-execprog, syz-prog2c, syz-repro
|
# How to reproduce crashes
|
||||||
|
|
||||||
The process of creating reproducer programs for syzkaller bugs is automated, however it's not perfect, so syzkaller provides a few tools for executing and reproducing programs manually.
|
The process of creating reproducer programs for syzkaller bugs is automated, however it's not perfect, so syzkaller provides a few tools for executing and reproducing programs manually.
|
||||||
|
|
||||||
116
docs/setup.md
Normal file
116
docs/setup.md
Normal file
@ -0,0 +1,116 @@
|
|||||||
|
# Setup
|
||||||
|
|
||||||
|
## Install
|
||||||
|
|
||||||
|
The following components are needed to use syzkaller:
|
||||||
|
|
||||||
|
- C compiler with coverage support
|
||||||
|
- Linux kernel with coverage additions
|
||||||
|
- Virtual machine or a physical device
|
||||||
|
- syzkaller itself
|
||||||
|
|
||||||
|
Generic steps to set up syzkaller are described below.
|
||||||
|
More specific information (like the exact steps for a particular host system, VM type and a kernel architecture) can be found on the following pages:
|
||||||
|
|
||||||
|
- [Setup: Ubuntu host, QEMU vm, x86-64 kernel](docs/setup_ubuntu-host_qemu-vm_x86-64-kernel.md)
|
||||||
|
- [Setup: Ubuntu host, Odroid C2 board, arm64 kernel](docs/setup_ubuntu-host_odroid-c2-board_arm64-kernel.md)
|
||||||
|
- [Setup: Linux host, QEMU vm, arm64 kernel](docs/setup_linux-host_qemu-vm_arm64-kernel.md)
|
||||||
|
- [Setup: Linux host, Android device, arm64 kernel](docs/setup_linux-host_android-device_arm64-kernel.md)
|
||||||
|
|
||||||
|
If you encounter any troubles, check the [troubleshooting](troubleshooting.md) page.
|
||||||
|
|
||||||
|
### C Compiler
|
||||||
|
|
||||||
|
Syzkaller is a coverage-guided fuzzer and therefore it needs the kernel to be built with coverage support, which requires a recent GCC version.
|
||||||
|
Coverage support was submitted to GCC in revision `231296`, released in GCC v6.0.
|
||||||
|
|
||||||
|
### Linux Kernel
|
||||||
|
|
||||||
|
Besides coverage support in GCC, you also need support for it on the kernel side.
|
||||||
|
KCOV was committed upstream in Linux kernel version 4.6 and can be enabled by configuring the kernel with `CONFIG_KCOV=y`.
|
||||||
|
For older kernels you need to backport commit [kernel: add kcov code coverage](https://github.com/torvalds/linux/commit/5c9a8750a6409c63a0f01d51a9024861022f6593).
|
||||||
|
|
||||||
|
To enable more syzkaller features and improve bug detection abilities, it's recommended to use additional config options.
|
||||||
|
See [this page](linux_kernel_configs.md) for details.
|
||||||
|
|
||||||
|
### VM Setup
|
||||||
|
|
||||||
|
Syzkaller performs kernel fuzzing on slave virtual machines or physical devices.
|
||||||
|
These slave enviroments are referred to as VMs.
|
||||||
|
Out-of-the-box syzkaller supports QEMU, kvmtool and GCE virtual machines, Android devices and Odroid C2 boards.
|
||||||
|
|
||||||
|
These are the generic requirements for a syzkaller VM:
|
||||||
|
|
||||||
|
- The fuzzing processes communicate with the outside world, so the VM image needs to include
|
||||||
|
networking support.
|
||||||
|
- The program files for the fuzzer processes are transmitted into the VM using SSH, so the VM image
|
||||||
|
needs a running SSH server.
|
||||||
|
- The VM's SSH configuration should be set up to allow root access for the identity that is
|
||||||
|
included in the `syz-manager`'s configuration. In other words, you should be able to do `ssh -i
|
||||||
|
$SSHID -p $PORT root@localhost` without being prompted for a password (where `SSHID` is the SSH
|
||||||
|
identification file and `PORT` is the port that are specified in the `syz-manager` configuration
|
||||||
|
file).
|
||||||
|
- The kernel exports coverage information via a debugfs entry, so the VM image needs to mount
|
||||||
|
the debugfs filesystem at `/sys/kernel/debug`.
|
||||||
|
|
||||||
|
To use QEMU syzkaller VMs you have to install QEMU on your host system, see [QEMU docs](http://wiki.qemu.org/Manual) for details.
|
||||||
|
The [create-image.sh](tools/create-image.sh) script can be used to create a suitable Linux image.
|
||||||
|
Detailed steps for setting up syzkaller with QEMU on a Linux host are avaialble for [x86-64](setup_ubuntu-host_qemu-vm_x86-64-kernel.md) and [arm64](setup_linux-host_qemu-vm_arm64-kernel.md) kernels.
|
||||||
|
|
||||||
|
For some details on fuzzing the kernel on an Android device check out [this page](setup_linux-host_android-device_arm64-kernel.md) and the explicit instructions for an Odroid C2 board are available [here](setup_ubuntu-host_odroid-c2-board_arm64-kernel.md).
|
||||||
|
|
||||||
|
### Syzkaller
|
||||||
|
|
||||||
|
The syzkaller tools are written in [Go](https://golang.org), so a Go compiler (>= 1.8) is needed
|
||||||
|
to build them.
|
||||||
|
|
||||||
|
Go distribution can be downloaded from https://golang.org/dl/.
|
||||||
|
Unpack Go into a directory, say, `$HOME/go`.
|
||||||
|
Then, set `GOROOT=$HOME/go` env var.
|
||||||
|
Then, add Go binaries to `PATH`, `PATH=$HOME/go/bin:$PATH`.
|
||||||
|
Then, set `GOPATH` env var to some empty dir, say `GOPATH=$HOME/gopath`.
|
||||||
|
Then, run `go get -u -d github.com/google/syzkaller/...` to checkout syzkaller sources with all dependencies.
|
||||||
|
Then, `cd $GOPATH/src/github.com/google/syzkaller` and
|
||||||
|
build with `make`, which generates compiled binaries in the `bin/` folder.
|
||||||
|
|
||||||
|
To build additional syzkaller tools run `make all-tools`.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
The operation of the syzkaller `syz-manager` process is governed by a configuration file, passed at
|
||||||
|
invocation time with the `-config` option. This configuration can be based on the
|
||||||
|
[example](syz-manager/config/testdata/qemu.cfg); the file is in JSON format with the
|
||||||
|
following keys in its top-level object:
|
||||||
|
|
||||||
|
- `http`: URL that will display information about the running `syz-manager` process.
|
||||||
|
- `workdir`: Location of a working directory for the `syz-manager` process. Outputs here include:
|
||||||
|
- `<workdir>/crashes/*`: crash output files (see [Crash Reports](#crash-reports))
|
||||||
|
- `<workdir>/corpus.db`: corpus with interesting programs
|
||||||
|
- `<workdir>/instance-x`: per VM instance temporary files
|
||||||
|
- `syzkaller`: Location of the `syzkaller` checkout.
|
||||||
|
- `vmlinux`: Location of the `vmlinux` file that corresponds to the kernel being tested.
|
||||||
|
- `procs`: Number of parallel test processes in each VM (4 or 8 would be a reasonable number).
|
||||||
|
- `leak`: Detect memory leaks with kmemleak.
|
||||||
|
- `image`: Location of the disk image file for the QEMU instance; a copy of this file is passed as the
|
||||||
|
`-hda` option to `qemu-system-x86_64`.
|
||||||
|
- `sandbox` : Sandboxing mode, the following modes are supported:
|
||||||
|
- "none": don't do anything special (has false positives, e.g. due to killing init)
|
||||||
|
- "setuid": impersonate into user nobody (65534), default
|
||||||
|
- "namespace": use namespaces to drop privileges
|
||||||
|
(requires a kernel built with `CONFIG_NAMESPACES`, `CONFIG_UTS_NS`,
|
||||||
|
`CONFIG_USER_NS`, `CONFIG_PID_NS` and `CONFIG_NET_NS`)
|
||||||
|
- `enable_syscalls`: List of syscalls to test (optional).
|
||||||
|
- `disable_syscalls`: List of system calls that should be treated as disabled (optional).
|
||||||
|
- `suppressions`: List of regexps for known bugs.
|
||||||
|
- `type`: Type of virtual machine to use, e.g. `qemu` or `adb`.
|
||||||
|
- `vm`: object with VM-type-specific parameters; for example, for `qemu` type paramters include:
|
||||||
|
- `count`: Number of VMs to run in parallel.
|
||||||
|
- `kernel`: Location of the `bzImage` file for the kernel to be tested;
|
||||||
|
this is passed as the `-kernel` option to `qemu-system-x86_64`.
|
||||||
|
- `cmdline`: Additional command line options for the booting kernel, for example `root=/dev/sda1`.
|
||||||
|
- `sshkey`: Location (on the host machine) of an SSH identity to use for communicating with
|
||||||
|
the virtual machine.
|
||||||
|
- `cpu`: Number of CPUs to simulate in the VM (*not currently used*).
|
||||||
|
- `mem`: Amount of memory (in MiB) for the VM; this is passed as the `-m` option to `qemu-system-x86_64`.
|
||||||
|
|
||||||
|
See also [config.go](syz-manager/config/config.go) for all config parameters.
|
||||||
@ -184,7 +184,7 @@ index 9576775a86f6..8bc4eb36fc1b 100644
|
|||||||
ifeq ($(call cc-option, $(CFLAGS_KASAN_MINIMAL) -Werror),)
|
ifeq ($(call cc-option, $(CFLAGS_KASAN_MINIMAL) -Werror),)
|
||||||
```
|
```
|
||||||
|
|
||||||
Configure the kernel (you might wan't to enable more configs as listed [here](docs/linux_kernel_configs.md)):
|
Configure the kernel (you might wan't to enable more configs as listed [here](linux_kernel_configs.md)):
|
||||||
``` bash
|
``` bash
|
||||||
make defconfig
|
make defconfig
|
||||||
# Edit .config to enable the following configs:
|
# Edit .config to enable the following configs:
|
||||||
|
|||||||
@ -78,7 +78,7 @@ CONFIG_KASAN=y
|
|||||||
CONFIG_KASAN_INLINE=y
|
CONFIG_KASAN_INLINE=y
|
||||||
```
|
```
|
||||||
|
|
||||||
You might also want to enable some other kernel configs as described [here](docs/linux-kernel-configs.md).
|
You might also want to enable some other kernel configs as described [here](linux-kernel-configs.md).
|
||||||
|
|
||||||
Since enabling these options results in more sub options being available, we need to regenerate config. Run this and press enter each time when prompted for some config value to leave it as default:
|
Since enabling these options results in more sub options being available, we need to regenerate config. Run this and press enter each time when prompted for some config value to leave it as default:
|
||||||
``` bash
|
``` bash
|
||||||
|
|||||||
44
docs/troubleshooting.md
Normal file
44
docs/troubleshooting.md
Normal file
@ -0,0 +1,44 @@
|
|||||||
|
# Troubleshooting
|
||||||
|
|
||||||
|
Here are some things to check if there are problems running syzkaller.
|
||||||
|
|
||||||
|
- Check that QEMU can successfully boot the virtual machine. For example,
|
||||||
|
if `IMAGE` is set to the VM's disk image (as per the `image` config value)
|
||||||
|
and `KERNEL` is set to the test kernel (as per the `kernel` config value)
|
||||||
|
then something like the following command should start the VM successfully:
|
||||||
|
|
||||||
|
```qemu-system-x86_64 -hda $IMAGE -m 256 -net nic -net user,host=10.0.2.10,hostfwd=tcp::23505-:22 -enable-kvm -kernel $KERNEL -append root=/dev/sda```
|
||||||
|
|
||||||
|
- Check that inbound SSH to the running virtual machine works. For example, with
|
||||||
|
a VM running and with `SSHKEY` set to the SSH identity (as per the `sshkey` config value) the
|
||||||
|
following command should connect:
|
||||||
|
|
||||||
|
```ssh -i $SSHKEY -p 23505 root@localhost```
|
||||||
|
|
||||||
|
- Check that the `CONFIG_KCOV` option is available inside the VM:
|
||||||
|
- `ls /sys/kernel/debug # Check debugfs mounted`
|
||||||
|
- `ls /sys/kernel/debug/kcov # Check kcov enabled`
|
||||||
|
- Build the test program from `Documentation/kcov.txt` and run it inside the VM.
|
||||||
|
|
||||||
|
- Check that debug information (from the `CONFIG_DEBUG_INFO` option) is available
|
||||||
|
- Pass the hex output from the kcov test program to `addr2line -a -i -f -e $VMLINUX` (where
|
||||||
|
`VMLINUX` is the vmlinux file, as per the `vmlinux` config value), to confirm
|
||||||
|
that symbols for the kernel are available.
|
||||||
|
|
||||||
|
- Use the `-v N` command line option to increase the amount of logging output, from both
|
||||||
|
the `syz-manager` top-level program and the `syz-fuzzer` instances (which go to the
|
||||||
|
output files in the `crashes` subdirectory of the working directory). Higher values of
|
||||||
|
N give more output.
|
||||||
|
|
||||||
|
- If logging indicates problems with the executor program (e.g. `executor failure`),
|
||||||
|
try manually running a short sequence of system calls:
|
||||||
|
- Build additional tools with `make all-tools`
|
||||||
|
- Copy `syz-executor` and `syz-execprog` into a running VM.
|
||||||
|
- In the VM run `./syz-execprog -executor ./syz-executor -debug sampleprog` where
|
||||||
|
sampleprog is a simple system call script (e.g. just containing `getpid()`).
|
||||||
|
- For example, if this reports that `clone` has failed, this probably indicates
|
||||||
|
that the test kernel does not include support for all of the required namespaces.
|
||||||
|
In this case, running the `syz-execprog` test with the `-nobody=0` option fixes the problem,
|
||||||
|
so the main configuration needs to be updated to set `dropprivs` to `false`.
|
||||||
|
|
||||||
|
If none of the above helps, file a bug on [the bug tracker](https://github.com/google/syzkaller/issues) or ask us directly on the syzkaller@googlegroups.com mailing list.
|
||||||
15
docs/usage.md
Normal file
15
docs/usage.md
Normal file
@ -0,0 +1,15 @@
|
|||||||
|
## Running syzkaller
|
||||||
|
|
||||||
|
Start the `syz-manager` process as:
|
||||||
|
```
|
||||||
|
./bin/syz-manager -config my.cfg
|
||||||
|
```
|
||||||
|
|
||||||
|
The `-config` command line option gives the location of the configuration file [described above](#configuration).
|
||||||
|
|
||||||
|
The `syz-manager` process will wind up QEMU virtual machines and start fuzzing in them.
|
||||||
|
Found crashes, statistics and other information is exposed on the HTTP address provided in manager config.
|
||||||
|
|
||||||
|
- [How to execute syzkaller programs](executing_syzkaller_programs.md)
|
||||||
|
- [How to reproduce crashes](reproducing_crashes.md)
|
||||||
|
- [How to connect several managers via Hub](connecting_several_managers.md)
|
||||||
Loading…
Reference in New Issue
Block a user