* Modify uiBug type. Rename BisectCause to BisectCauseDone. Introduce
BisectFixDone.
* Modify createUIBug() and MergeUIBug() to set the above fields
appropriately.
* Modify bug_list to display the bisection status; remove yesSort() as
it is not used anymore. Adjust ".list_table .stat" to appropriate width.
* Add TestBugBisectionStatus() to check bisection status on main page.
* Add file from running "make generate": pkg/html/generated.go
Better coverage reports with hierarchical coverage information,
number of programs covering each line,
handling of partially covered lines,
links to programs covering lines.
Fixes#682
After f613a7c4 ("pkg/cover: fix prefix computation") we stopped trimming
starting "/" from linux path names. This broke get_maintainers.pl script
which is now given non-existent absolute paths (e.g. "/mm/mmap.c"),
as the result syzbot can't find any maintainers for bug reports
and mails them into lkml limbo.
Trim starting slashes and dots. Extend tests for catch this.
We now pass 5 arguments through a bunch of functions,
this is quite inconvinient when the set of arguments changes.
Incapsulate all arguments in a struct and pass/store it as a whole.
This change modifies the way fuchsia is built to add the ssh keys to the
zbi image after building it. Previously that was done via the
"extra_ssh_keys" argument to the build system, but that was removed
recently.
A process with CAP_SYS_NICE can bring kernel down by asking for too high SCHED_DEADLINE priority,
as the result rcu and other system services that use kernel threads will stop functioning.
Some parameters for SCHED_DEADLINE should be OK, but we don't have means to enforce
values of indirect syscall arguments. Peter Zijlstra proposed sysctl_deadline_period_{min,max}
which could be used to enfore safe limits without droppping CAP_SYS_NICE, but we don't have it yet.
See the following bug for details:
https://groups.google.com/forum/#!topic/syzkaller-bugs/G6Wl_PKPIWI
Running sys/*/test programs requires real machines and kernels for each OS.
We can't do that in unit tests, but at least try to deserialize these programs
so that they don't get rotten.
Ensure that we have at least 1GB per Makefile job.
Go compiler/linker can consume significant amount of memory
(observed to consume at least 600MB). See #1276 for context.
Update #1276
One instance we observed that objdump hanged due to stdout
pipe overflow due to panic in archCallInsn.
The reason for the original panic is still unclear,
but fix the objdump hang. We need to terminate objdump
and propagate the panic.
Also extend the panic messages.
* sys/fuchsia: update zx_clock_get.
zx_clock_get was deprecated and replaced by zx_clock_get_new. In a
recent CL[0], they replaced the zx_clock_get by zx_clock_get_new and
moved all client. This commit updates syzkaller to use the new function.
[0]: https://fuchsia-review.googlesource.com/c/fuchsia/+/298575
* run make extract && make generate
1. Change HID descriptions to allow devices to have two interrupt endpoints.
2. Remove unneeded responses to OUT control requests from descriptions.
3. Add some debugging code to detect and report missing descriptions.
On OpenBSD, the executor sometimes manages to set the memory resource
limit 0 causing any following memory allocation to fail. Since threads
are potentially created from such a thread which cannot allocate any
memory, the executor will exit non-zero which in turn will cause
false-positive panics to be reported. For more info see the
discussion[1] in PR #1243.
Instead, if hitting a fatal error during thread creation exit zero.
[1] https://github.com/google/syzkaller/pull/1243
* Add support for non-obvious open commands
* Adding Michael Specter to CONTRIBUTORS
* update
* /pkg/host/host_linux.go: leverage .txt for OpenAt
Allows users to specify OpenAt flags. Currently, Syzkaller will
automatically attempt to open device nodes at startup, and ignore
descriptions that do not exist. Unfortunately, Syzkaller only opens with
read-only permissions, and some device drivers will
return -EINVAL or likewise if the correct flags are not specified.
Add retry logic that detects and ignores episodic flakes.
This test episodically flakes on syzbot.
We run with the default timeout, but require a test
to pass in 50+% of cases.
Running 72 test binaries in parallel I am getting 35-44 failures out of 72
with 1 retry. With 3 retries it drops to ~7. With 5 it is close to 0.
Use 7 retries for now. Let's see if it still flakes.
* Remove the environment variable: AKAROS_XCC_ROOT
(refer to commit: brho/akaros bdab8c3a)
* Add environment variable AKAROS_TOOLCHAINS which
points to the parent directory of the old AKAROS_XCC_ROOT.
(refer to commit: brho/akaro 62ec5c3c)
* Add "$AKAROS_TOOLCHAINS/x86_64-ucb-akaros-gcc/bin" to PATH.
Signed-off-by: Aditya Basu <mitthu@google.com>
For extractProgSingle() lastEntries are already reversed, so no need to
call reverseEntries().
Also for extractProgBisect() there's no point in reversing the entries,
as the idea is to try connecting multiple programs in chronological order.
Providing additional info, especially regarding syscall arguments, in reproducers
can be helpful. An example is device numbers passed to mknod(2).
This commit introduces an optional annotate function on a per target basis.
Example for the OpenBSD target:
$ cat prog.in
mknod(0x0, 0x0, 0x4503)
getpid()
$ syz-prog2c -prog prog.in
int main(void)
{
syscall(SYS_mmap, 0x20000000, 0x1000000, 3, 0x1012, -1, 0, 0);
syscall(SYS_mknod, 0, 0, 0x4503); /* major = 69, minor = 3 */
syscall(SYS_getpid);
return 0;
}
The csource options lack any relation to the tested OS,
while they do depend on the tested OS (most are linux-specific).
As the result we enable binfmt_misc, but its setup fails on anything
other than linux. As a work-around don't enable binfmt_misc on non-linux.
Currently we have them duplicated in docs/configuration.md
and the actual source in pkg/mgrconfig/config.go.
Documentation is missing in one place or another,
some is outdated, some is phrased slightly differently.
Move all docs to pkg/mgrconfig/config.go and reference it
from docs/configuration.md instead.
Leak checking support was half done and did not really work.
This is heavy-lifting to make it work.
1. Move leak/fault setup into executor.
pkg/host was a wrong place for them because we need then in C repros too.
The pkg/host periodic callback functionality did not work too,
we need it in executor so that we can reuse it in C repros too.
Remove setup/callback functions in pkg/host entirely.
2. Do leak setup/checking in C repros.
The way leak checking is invoked is slightly different from fuzzer,
but much better then no support at all.
At least the checking code is shared.
3. Add Leak option to pkg/csource and -leak flag to syz-prog2c.
4. Don't enalbe leak checking in fuzzer while we are triaging initial corpus.
It's toooo slow.
5. Fix pkg/repro to do something more sane for leak bugs.
Few other minor fixes here and there.
We duplicated the no output timeout in the repro package,
and it got out of sync. It's not 3 mins now, but 5 mins.
Remove the duplication and fix this.
In several places we do special handling for some crash types.
Currently we compare report title with magic strings,
which is error-prone. Add explicit Type to reports.
A const can be used as array size. Then if the const is not present
on all arches, compiler will produce an error about 0-sized-array.
There is no easy way to work around this for a user.
Use value of 1 for missing consts. It's just a bit safer.
Using a build tag to exclude files for golangci-lint
reduces memory consumption (it does not parse them).
The naive attempt with skip-dirs did not work.
So add codeanalysis build tag and use it in auto-generated files.
Update #977
This change adds compiler support for complex path
expressions in len targets. E.g. it allows to refer
to a sibling field as len[parent_struct:field:another_field].
See the docs change for details.
This is just a compiler change.
The feature is not yet supported by the prog package.
This prepared for handling of bytesize[parent:foo:bar] expressions
by allowing multiple identifiers after colon.
No functional changes for now, just preparation for storing more
than one identifier after colon.
For some bug types kernel produces traceback of all CPUs
(which is presumably useful for these crashes).
However, we currently throw it away because it has context
different from the oops context.
Detect such traceback and preserve output.
On powerpc, the "stat" syscall is implemented by "sys_newstat" entry point.
This causes a test failure as we can't find "sys_stat" in kallsyms.
Add "stat" -> "newstat" to the kallsyms rename map to work around this.
Closes: #1083 ("pkg/host: TestSupportedSyscalls fails on ppc64le")
Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
After commit 9ad9ef29ca
we started saying "your command '3' is accepted"
because we use numbers now. Keep string representation
of the command when parsing and use it in reply emails.
2 recent commits conflict and cause test 380 to fail:
pkg/report: improve warning titles
pkg/report: Handle powerpc stack traces correctly
Currently 380 is detected as "WARNING in program_check_exception"
rather than the expected "WARNING in assert_slb_presence".
The reason is that we started parsing WARNING stack trace and applying
proper skip patterns to frames.
Adjust WARNING matching and skip common powerpc WARNING frames.
powerpc stack traces are printed a bit differently from x86 stack traces.
Adjust the regexes accordingly to cope with this format.
Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Enable the Linux builder for ppc64le.
For ppc64le, we use zImage rather than bzImage as our kernel target. Pass
through the target architecture to buildKernel() so we can pick the right
target based on arch.
Closes: #1084 ("pkg/build: Support building Linux on ppc64le")
Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Add support for generating ppc64le pseries disk images. This will create a
disk image with a PowerPC PReP boot partition at the start, and install
the IEEE1275/OpenFirmware version of GRUB.
If we ever support the powernv platform in the future we're going to have
to do something different here.
Add a command line argument to specify architecture, defaulting to amd64.
Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Limit the amount of bisection chunks to 8. Going over this value probably
means that we are bisection a flaky crash, and continuing bisection would
just take a lot of time and likely produce no result.
Not using `elif GOOS_freebsd' since it could cause breakage on other *BSDs
due to unused variables.
Regression introduced in commit c7c3f772 (executor: improve setup for packet
handling on *BSD).
Most probably limited to input validation for now. In the future, it
could be extended to provide a bootable kernel during vm create (/bsd)
and turn vmid into a proper resource.
The OpenBSD VMs on GCE does support vmm(4).
Improve the handling of packets by:
* setting the local MAC address.
* configuring the local IPv4 address with prefix /24.
* adding an entry in the arp cache for the remote IPv4 address.
* adding an entry in the IPv6 neighbour cache for the remote
IPv6 address.
Allow 2000 ms of waiting time for syz_usb_connect and and the same time for
the whole program is this syzkall is present. Allow 200 ms of waiting time
for syz_usb_disconnect. Remove sleep from syz_usb_control_io.
This commits implements 4 syzcalls: syz_usb_connect, syz_usb_io_control,
syz_usb_ep_write and syz_usb_disconnect. Those syzcalls are used to emit USB
packets through a custom GadgetFS-like interface (currently exposed at
/sys/kernel/debug/usb-fuzzer), which requires special kernel patches.
USB fuzzing support is quite basic, as it mostly covers only the USB device
enumeration process. Even though the syz_usb_ep_write syzcall does allow to
communicate with USB endpoints after the device has been enumerated, no
coverage is collected from that code yet.
Instead of always closing open fds (number 3 to 30) after each program,
add an options called EnableCloseFds. It can be passed to syz-execprog,
syz-prog2c and syz-stress via the -enable and -disable flags. Set the
default value to true. Also minimize C repros over it, except for when
repeat is enabled.
We now have too many namespaces and bugs.
Main page takes infinity to load.
Also almost nobody is interested in more than 1 namespace.
So split main page per-namespaces.
pkg/host/host_linux.go: Add "arm" to kallsyms support test.
pkg/host/host_test.go: TestKallsymsParse(): Add entry to test arm kallsyms parsing support.
This commit moves the definition of the `syz_execute_func` after the
block of code that imports all the OS specific common headers.
This is required because after commit
dfd3394d42 `syz_execute_func` started
using the `NONFAILING` macro, which is defined in those header files for
each OS.
I also ran `make generate`.
TEST=I only tested that the executor works for Fuchsia with:
```shell
$ make executor TARGETOS=fuchsia TARGETARCH=amd64 SOURCEDIR=~/fuchsia
```
This commit modifies the common_fuchsia.h file changing the behavior of
the `syz_future_time function`. Before, the function used to have a switch
case that would fallthrough, making it always set the delta_ms to 10000.
The fix is to add a `break;` statement after each switch case.
We cannot create folders in other directories anymore, for now, we only
have access to create directories in /data/. This change is temporary as
we move to a component-based world.
The fuzzer gained control over host machines again with something like:
syz_execute_func(&(0x7f00000000c0)="c4827d5a6e0d5e57c3c3b7d95a91914e424a2664f0ff065b460f343030062e67660f50e900004681e400000100440fe531feabc4aba39d6c450754ddea420fae9972b571112d02")
Let's see if perturbing syz_execute_func a bit and wiping registers
will stop the outbreak.
The 1 hour timeout is quite high. But we've seen false positives with 20 mins
on the first build after bazel/deps update. Also other gvisor instances running
on the same machine contribute to longer build times.
We are hitting quotas for operation completion polling.
Not surprising since we create/delete instances all the time.
Slightly decrease rate of polling.
Ptr type has special handling of direction (pointers are always input).
But buffer type missed this special case all the time.
Make buffer less special by aliasing to the ptr[array[int8]] type.
As the result buffer type can't have optional trailing "opt" attribute
because we don't have such support for templates yet.
Change such cases to use ptr type directly.
Fixes#1097
gometalinter says:
pkg/compiler/consts.go:192:⚠️ internal error: no range for "n" (vetshadow)
pkg/compiler/consts.go:197:⚠️ internal error: no range for "n" (vetshadow)
prog/encoding.go:862:⚠️ declaration of "v" shadows declaration at prog/encoding.go:852 (vetshadow)
This somehow happens only with Go1.11 but not 1.12 so wasn't detected locally.
The prog warnings looks legit.
The pkg/compiler warning was amusingly introduced to please golangci-lint checker,
revert that fix for now.
Final reproducers are crafty. Sometimes they just stop working.
In that case the detailed listings of the programs executed by syz-repro
are very helpful. Let's print them at the loglevel 3.
We currently manually call extractRootCause in few selected places
to denote kernel build errors that we want to report to developers.
The rest are considered infra errors that we don't report.
This does not work well. We are missing fuchsia and gvisor build errors.
Treat all external command exection failures as kernel build errors instead.
Let's see how this works in practice.
Also add bazel-specfic error patterns and tests.
This commit adds the "//bundles:tools" packages to the fuchsia build
used for syzkaller. This includes ssh tools, which includes scp.
TEST=I have tested this on syz-ci. Fuchsia is now able to pass the image
test.
We are seeing some flakes during bisection and image testing.
Hard to tell what's the root cause because they are episodic.
But using non-preemptible VMs for bisection and image testing
looks good on all fronts.
Update #501
This CL changes the invocation of the `fx set` command to match the new
format[0].
This change involves removing "board" and "product" flags. Instead, now one has to use
`fx set product.board` to get the same result. In the case of syzkaller,
this would be "core.${ARCH}" where arch could be x64 or arm64.
Also, the default build directory now is `out/default`, and for now, it
is still not possible to have multiple archs built altogether. I set the
build directory manually to `out/${ARCH}` (as was before). This would
allow syzkaller to also build the kernel for arm if required.
[0]: https://fuchsia.googlesource.com/fuchsia/+/b814211ab33eeb4e8f02962d5c81d042b2464ac7
TEST=Tested this in syz-ci. It worked.
Separate kernel and syzkaller build failures.
Fix logic to understand when a build is fixed:
look if kernel/syzkaller commit changes to understand
if it's a new good build or re-upload of an old build.
Fixes#1014
We currently skip a commit iff all 10 instances errored.
But if, say, only 9 errored we consider it as OK,
but this significnalty reduces chances of detecting flaky crashes.
So skip if more than 2/3 errored.
Update #501
Add "#syz uncc" command as a safety handle.
The command allows sender to unsubscribe from all future communication on the bug.
Linus mentioned possibility of saying "I'm not the right person for this report"
in the context of bug reminders:
https://groups.google.com/d/msg/syzkaller/zYlQ-b-QPHQ/AJzpeObcBAAJ
Such error popped up during bisection on older kernels.
It's already detected properly, yet build failed with just "make failed"...
Unclear why that happened, but the test won't harm.
Update #501
Running bisection using an older repro failed with:
<stdin>: In function ‘syz_mount_image.constprop’:
<stdin>:298:3: error: argument 1 null where non-null expected [-Werror=nonnull]
In file included from <stdin>:26:0:
/usr/include/x86_64-linux-gnu/sys/stat.h:320:12: note: in a call to function ‘mkdir’ declared here
extern int mkdir (const char *__path, __mode_t __mode)
Let's be safe and ignore warnings during repro/bisect.
Everything that runs during tests still has all warnings.
Update #501
The added test triggers warnings like these:
<stdin>: In function ‘syz_mount_image.constprop’:
<stdin>:298:3: error: argument 1 null where non-null expected [-Werror=nonnull]
In file included from <stdin>:26:0:
/usr/include/x86_64-linux-gnu/sys/stat.h:320:12: note: in a call to function ‘mkdir’ declared here
extern int mkdir (const char *__path, __mode_t __mode)
^~~~~
cc1: all warnings being treated as errors
<stdin>: In function ‘syz_open_procfs.constprop’:
<stdin>:530:41: error: ‘%s’ directive argument is null [-Werror=format-truncation=]
<stdin>:85:110: note: in definition of macro ‘NONFAILING’
<stdin>:532:41: error: ‘%s’ directive argument is null [-Werror=format-truncation=]
<stdin>:85:110: note: in definition of macro ‘NONFAILING’
<stdin>:534:41: error: ‘%s’ directive argument is null [-Werror=format-truncation=]
<stdin>:85:110: note: in definition of macro ‘NONFAILING’
Use volatile for all arguments of syz_ functions to prevent
compiler from treating the arguments as constants in reproducers.
Popped up during bisection that used a repro that previously worked.
Update #501
This commit changes the fuchsia git repo to make it point to the main
repo instead of just "zircon".
Previously, zircon had its own git repo, so syzkaller had a wrapper that
would issue jiri commands for the whole fuchsia repo, but would just
report commits in the zircon repo.
Recently, fuchsia merged most of its repositories in just one big git
repo. So now, zircon is part of the fuchsia git repo. I think the code
continued working because the old zircon repo is still part of a git
repository, so git commands worked there.
The new code now doesn't have any special casing for Zircon, and now
(almost) everything is part of the fuchsia repo. The other repos are now
git submodules.
Use CheckoutCommit instead of SwitchCommitto get the crashing commit.
The problem is with trees like linux-next. They require at least fetching tags
and maybe even different tree (-history).
Use CheckoutCommit which at least fetches tags which is enough
for recent commits.
Update #501
The Fuchsia team is going to remove the `lib/fdio/util.h` library. They
have already moved all the functions to new header files.
I have seen that fuchsia uses `fdio_service_connect`, which has been
moved to the `lib/fdio/directory.h` header file.
This commit just changes the import path in the fuchsia executor, and in
the corresponding generated go file (I made that change by running `make
generate`).
Recently the fuchsia team decided to merge all their subrepos into one
big git repo. This meant that the "zircon" repo doesn't exist anymore.
Instead almost everything is under the fuchsia repo.
This change updates all the reference I could find in the code that
point to the zircon and docs repo to make them point to the new fuchsia
repo.
We already have a pattern to match "Fatal trap N: ..." panics, but the
format of our backtraces has changed since it was added. Add another
pattern which correctly matches a few instances generated by syzbot.
In some cases we start scp, which starts ssh,
then kill scp but the ssh subprocess is not killed.
As the result cmd.Wait hangs waiting for EOF on the stdout/stderr,
which are still kept alive by ssh subprocess. But ssh just hangs forever.
Create a process group for each command and kill whole process group.
Hopefully this will help.
This adds bulk of support for bisection to dashboard/app and syz-ci:
- APIs to send bisection jobs and accept results
- syz-ci logic to execute bisection jobs
- formatting of emails with results
- showing of results on dashboard
Some difficulties we have to overcome:
- since linux is frequently build/boot broken, lots of bisections are inconclusive,
need to present such results too
- git bisect is poorly suitable for automation, have to resort to output parsing (is output stable?)
- git bisect turns out to fail (exit with non-0 status) when bisection is inconclusive
(multiple potential cause commits)
- older syzkaller revisions can't be built with newer (broken) kernel header, e.g.:
ebtables.h:197:19: error: invalid conversion from ‘void*’ to ‘ebt_entry_target*’
- newer compilers produce more warnings and break old syzkaller builds, e.g.:
kvm.S.h:6:12: error: ‘kvm_asm64_vm86’ defined but not used [-Werror=unused-const-variable=]
- figuring relevant emails to CC from a commit is non-trivial:
besides commit author, there can be some emails in commit tags, or not,
which tags to use is an interesting question (some may include irrelevant emails)
we can also run get_maintainers.pl on the commit, but this can produce too wide
list if commit touches lots of files, it can also produce too small list,
and then we need to resort to blame
- for inconclusive bisection we probably don't need to include emails referenced
in the commits (there can be too many of these commits)
- need to be careful to exclude own syzbot email from commit CC list,
now syzbot emails are referenced in some commits (Reported-by/Tested-by/etc)
(can cause some kind of infinite recursion)
- lots of commits reference stable mailing list,
we should not include it in CC because it's referenced for backports rather then bug reports
- since we add new Bug entity fields which we use in queries,
whole datastore need to be upgrades to add the new field to index
- we must not discard the crash that was used for bisection
(treat it as a reported crash)
- bisection results need 2 forms of reports:
one when we add bisection results to already reported bug
another when we report a bug first time with bisection results
- when reporting a bug with bisection results we need to use the crash
that was used for bisection
- some fraction of bisections will probably fail with various errors
and we will need some mechanism to retry bisection after the root cause is resolved
this is not implemented yet
- linux-next is problematic for 2 reasons:
fix bisection can't possibly run on linux-next as commits are not reachable from HEAD
lots of commits are missing in linux-next (even in linux-next-history)
e.g. we have some c63e9e91a254a52 which is now missing in linux-next/linux-next-history
- older kernels can't be build with fresh gcc/binutils/perl/make/glibc
for now we have to stop at v3.9 (this only requires switching gcc several times along the way)
- kernels past v4.11 do not build with gcc 7 and 8 (undefined reference to `____ilog2_NaN')
- v4.1 and back have only compiler-gcc5.h
- v3.17 and back have only compiler-gcc4.h
- v3.6 and back do not have make olddefconfig
- compat socket calls can't be bisected past "x86/entry/syscalls: Wire up 32-bit
direct socket calls" (v4.10) because of
https://syzkaller.appspot.com/bug?id=b5b150e322d5f48c869bcf1528cdbee08d1421cb
- v2.6.28 and below does not work with modern make:
*** mixed implicit and normal rules: deprecated syntax
- v3.8 build fails:
Can't use 'defined(@array)' (Maybe you should just omit the defined()?) at kernel/timeconst.pl line 373.
kernel/Makefile:134: recipe for target 'kernel/timeconst.h' failed
- make 3.81 works for v2.6.28.
3.81 almost works with current HEAD, you need to run make twice because first run spuriously fails with:
- v2.6.28 with gcc-4.9.4 broken with:
include/linux/kvm.h:240:9: error: duplicate member ‘padding’
- but even defconfig fails:
VDSO arch/x86/vdso/vdso.so.dbg
gcc: error: elf_x86_64: No such file or directory
gcc: error: unrecognized command line option ‘-m’
It seems that we also need old binutils.
- for v3.8 and below we need perl-5.14.4.
Unfortunately this or any manually built perl doesn't work for later kernels:
Can't locate strict.pm in @INC
- kernels starting from 4.14 and older are boot broken:
https://lkml.org/lkml/2018/9/7/648
- kernels older than 4.12 are broken during netdev setup
(fixed by commit 675c8da049fd6556eb2d6cdd745fe812752f07a8)
Update #501
Currently we truncate output up to rep.EndPos after unexpected reboot.
But report sets EndPos to the _last_ report in output,
so if there are any other errors they are all skipped after
truncation to EndPos. Truncate just one line instead.
If we cherry-pick some fixes during bisection
we need to "git reset" repo before "git bisect reset".
Otherwise it will fail. Reset repo in more points.
Update #501
Linux kernel is frequently build/boot broken. Even on release tags.
Everything past 4.15 is broken with our config for multiple reasons.
This makes bisection process almost infeasible.
Try to work around some breakages by disabling configs are we go back in time.
Update #501
A bunch of improvements after more wide bisection testing.
Improve logging.
Support returning several commits for inconclusive bisection.
Return Report with the final crash.
Remove code that was moved to pkg/vcs.
Update #501
In preparation for syz-ci bisection:
- move bisection function into a separate interface
they look out of place in vcs.Repo because most OSes
don't implement it and most users don't case
- extract author name and more CC emails for commits
- move linux-specific PreviousReleaseTags into linux.go
- fix inconclusive bisection (more than 1 potential commits)
- add tests fr bisection
- add maintainers returned from get_maintainers.pl for commits
that don't have enough emails (e.g. only author email)
Update #501
Since we can be building very old revisions for bisection here,
make the build as permissive as possible.
Newer compilers tend to produce more warnings also kernel headers may be broken, e.g.:
ebtables.h:197:19: error: invalid conversion from ‘void*’ to ‘ebt_entry_target*’
Update #501
Turns out git bisect fails to communicate some outcomes (multiple potential commits),
and the only way to understand what happened is parsing plain English output.
Extracting the output from the error is quite unhandy.
So return it with the error.
Update #501
Differences in code formatting between Go versions cause constant
problems for us (https://github.com/golang/go/issues/25161).
Currently we support 1.9 and 1.10. Switch to newer 1.11 and 1.12.
Fixes#1013
This CL splits the fx commands into two: one that runs `fx set` setting
the same arguments as before, and another one right after, running `fx
clean-build` to build fuchsia.
Recently[0], fuchsia made it impossible to run `fx clean-build`
specifying the arguments. Instead, one should run `fx set` before.
The benefit is that `clean-build` now preserves the build arguments
across runs. For syzkaller, it doesn't change anything though.
[0]: https://fuchsia-review.googlesource.com/c/fuchsia/+/262674
This CL changes the supported vmType for fuchsia to be QEMU, instead of
GCE.
This was broken by commit b4e5a74e4, which changed the behavior of
getBuilder. It was part of a 6-commit PR trying to add support for
FreeBSD.
That commit has caused syzkaller to stop being able to build the fuchsia
image, getting the following error:
`kernel build failed: unsupported image type fuchsia/amd64/qemu`
For now we must assume that we're building on FreeBSD; cross-compilation
support isn't quite there yet.
During a build, we first build an updated kernel toolchain if required,
generate a custom kernel configuration file with KCOV enabled, and build
a new kernel. (When running on the custom kernel, "uname -i" will print
"SYZKALLER" rather than "GENERIC".) Then, the image is mounted under a
tmpdir and we install the custom kernel to /boot/kernel.
The problem is stupid: <endian.h> should be included as <sys/endian.h> on freebsd.
Pass actual host OS to executor build as HOSTGOOS and use it to figure out
how we should include this header.
This change makes all syz-execprog, syz-prog2c and syz-stress accept
-enable and -disable flags to enable or disable additional features
(tun, net_dev, net_reset, cgroups and binfmt_misc) instead of having
a separate flag for each of them.
The default (without any flags) behavior isn't changed: syz-execprog
and syz-stress enabled all the features (provided the runtime supports
them) and syz-prog2c disables all of them.
Fuchsia recently changed such that zx_vmar_map can't be declared
executable and writeable at the same time; use a new syscall for this
purpose.
Also made a few errors more informative.
1. Implement clean since we do incremental build we actually need no-op clean.
2. Don't copy netbsd to output dir, we don't need it there (we create full-fledged image).
3. Copy netbsd.gdb to obj/ dir, that's where it is expected.
4. Fix boot disk name for gce (it's sd0 instead of wd0).
* Modified pkg/build/netbsd.go
1. Made kernel build incremental
2. Added code to copy kernel to disk image
* Fix formating issues
* Fixed basic issues
* Fixed copy kernel to disk
1. Added CPU option to default
2. Added snapshot option to vm/qemu/qemu.go
3. Hacky solution to get the build working
* Fixed issues and added vm.MonitorExecution
* Added sync instead of poweroff
This implements 2 features:
- syz-ci polls a set of additional repos to discover fixing commits sooner
(e.g. it can now discover a fixing commit in netfilter tree before
it reaches any of the tested trees).
- syz-ci uploads info about commits to dashboard.
For example, a user marks a bug as fixed by commit "foo: bar".
syz-ci will find this commit in the main namespace repo
and upload commmit hash/date/author to dashboard. This in turn
allows to show links to fixing commits.
Fixes#691Fixes#610
Currently we always pass cover=false during instance testing
and as the result covereage is not tested. This can result in
broken images. Pass the actual value of coverage for testing.
SKIP/BROKEN distract too much attention from FAIL tests
and are not usually interesting. Add Verbose flag that
controls printing of SKIP/BROKEN tests. Enable it in
pkf/runtest/run_test.go by default and disable in tools/syz-runtest.
* pkg/build: share extractRootCause with openbsd
This should get kernel build errors reported in syz-ci console.
* Add a test
* lint
* pkb/build: extractRootCause for NetBSD too
A line length of 79 in the ddb output does not necessarily imply that the
following line is a continuation of the current line. Since there's no way
to distinguish between ordinary and continuation lines, it could end up
corrupting the report by joining two lines that are disjoint[1].
Instead, disable line wrapping in ddb. If we want some kind of wrapping in
the future it's easier done by pkg/report.
[1] https://syzkaller.appspot.com/bug?extid=03f7377a9848d7d008c9
Just appending the pid number can produce conflicting names
if the name itself ends with digits (standard temp file naming convention).
So append ".PID".
Also remove beginning from too long names instead of ending.
Temp files in tests has unique numbers at the end, we need to preserve them
to avoid file name conflicts.
