Currently we use the latest syzkaller commit that syz-ci uses itself.
As the result syz-execprog can fail to deserialize the reproducer.
Use the original syzkaller commit.
Support the new scheme of associating fixing commits with bugs.
Now we provide a tag along the lines of:
Reported-by: <syzbot+a4a91f6fc35e102@syzkaller.appspotmail.com>
The tag is supposed to be added to the commit.
Then we parse commit logs and extract these tags.
The final part on the dashboard is not ready yet,
but syz-ci should already parse and send the tags.
Builds with equal IDs are merged by dashboard,
so if only syzkaller commit has changed
dashboard won't create a new build and the new
syzkaller commit will be lost.
Forced syzkaller updates is the main reason for syz-ci restarts.
Reduce syzkaller freshness threshold from 6h to 1m.
Rebuilding syzkaller is relatively cheap anyway.
Make it possible to monitor health and operation
of all managers from dashboard.
1. Notify dashboard about internal syz-ci errors
(currently we don't know when/if they happen).
2. Send statistics from managers to dashboard.
Boot and minimally test images before declaring them as good
and switching to using them.
If image build/boot/test fails, upload report about this to dashboard.
Currently getting a complete report requires a complex,
multi-step dance (including getting information that
external users are not interested in -- guilty file).
Simplify interface down to 2 functions: Parse and Symbolize.
Parse does what it did before, Symbolize symbolizes report
and fills in maintainers. This simplifies both implementations
of Reporter interface and all users of the interface.
Potentially we could get this down to 1 function Parse
that does everything. However, (1) Symbolize can fail,
while Parse cannot, (2) usually we want to ignore (log)
Symbolize errors, but otherwise proceed with the report,
(3) repro does not need symbolization for all but the
last report.
Whole raw output is indivisble part of Report,
currently we always pass Output separately along with Report.
Make Output a Report field.
Then, put whole Report into manager Crash and repro context and Result.
There is little point in passing Report as aa bunch of separate fields.
This allows callers to get access to Report.Corrupted.
Better than adding 6-th return value and will allow
to pipe other report properties if necessary.
When manager is stopped there are sometimes runaway qemu
processes still running. Set PDEATHSIG for all subprocesses.
We never need child processes outliving parents.
1. Fetch last 200K commits instead of commits for past year.
For merged commits both author date and commit date can be
arbitrary long in past (e.g. we got a commit dated by 2014).
2. Strip some commit prefixes from commits.
We have some trees where backports are prefixed with "BACKPORT:".
Previously we could no match such commits.
We currently build binaries for all targets into bin.
This makes mess in bin/ and does not allow testing of different archs.
Build target binaries into bin/OS_ARCH/ subdirs.
Host binaries are still built into bin/.
Update #333
Update #324
Update #191
Add new "syzkaller_descriptions" config param that allows
to specify a dir with additional syscall descriptions
that will be copied on top of syzkaller checkout.
Dashboard needs to know when bug fixing commits reach
builders in order to fully close bugs.
Send commits that dashboard is interested in to dashboard.
If manager exits during start, it's hard to understand what happens.
Save the previous manager log as manager.log.old until we have a better solution.
Also log errors on 0 log level.
Don't write hub/dashboard params if they are not enabled
in syz-ci config. Otherwise manager gets name but not addr/key,
and that does not pass config validation.
Currently we have unix permissions for new files/dirs
hardcoded throughout the code base. Some places use 0644,
some - 0640, some - 0600 and a variety of other constants.
Introduce osutil.MkdirAll/WriteFile that use the default
permissions and use them throughout the code base.
This makes permissions consistent and also allows to easily
change the permissions later if we change our minds.
Also merge pkg/fileutil into pkg/osutil as they become
dependent on each other. The line between them was poorly
defined anyway as both operate on files.
We currently store 3 tags (compiler id, kernel commit and config hash).
But we also kernel git report/branch. To not store 2 more tag files,
combine everything into a single json file that holds all info about the build.
Will allow simpler extenstion in future as well.
