48 Commits

Author SHA1 Message Date
Dmitry Vyukov
906c09c71a dashboard/app: simplify reportCrash
Factor crash saving out of reportCrash into saveCrash.

Update #538
2018-08-02 16:57:31 +02:00
Dmitry Vyukov
c67a9331a4 gometalinter: clean up some errcheck warnings
Check some errors where relevant.
Unfortunately enabling errcheck does not look feasible, too many warnings.

Update #538
2018-08-02 16:57:31 +02:00
Dmitry Vyukov
3abee99f46 dashboard/app: fix crash selection for reporting
See the issue for the problem description.
Include repro level into reporting priority,
so that we can order by just it during selection
and ignore ReproC/ReproSyz.

Fixes #634
2018-07-27 21:10:01 +02:00
Dmitry Vyukov
9efa2eab69 dashboard/app: allow empty CompilerID in Build
Some OSes don't require a compiler.
2018-06-29 13:03:51 +02:00
Dmitry Vyukov
dba0b50e78 dashboard/app: fix gometalinter warning
dashboard/app/api.go:390:28⚠️ do not pass a nil Context, even if a function permits it; pass context.TODO if you are unsure about which Context to use (SA1012) (megacheck)
2018-06-28 10:02:52 +02:00
Dmitry Vyukov
9cff2eb96a dashboard/app: attach fixing commits to canonical bugs
Fixes #615
2018-06-11 12:12:01 +03:00
Dmitry Vyukov
b30b1136b6 dashboard/app: try 1 repro per day until we have at least syz repro
Bugs without repros are not actionable sometimes.

Fixes #627
2018-06-11 12:12:01 +03:00
Dmitry Vyukov
d6ae9b9739 dashboard/app: simplify config deployment
Currently one needs to switch between config_stub.go and prod
config back and forth on every deployment. This is very
inconvinient. Rework config, so that switching is not necessary.
2018-06-11 12:12:01 +03:00
Dmitry Vyukov
83f945db79 dashboard/app: fix old crash purging logic
NumRepro includes failed repro attempts,
so we should not look at it when purging old crashes.
2018-06-08 10:00:42 +02:00
Dmitry Vyukov
1319a7da09 dashboard/app: fix crash save throttling logic
bug.LastTime is updated on every crash, even if we don't save it.
As the result we did not save recent crashes for popular bugs at all.
Fix this by introducing bug.LastSavedCrash.
2018-06-08 09:34:02 +02:00
Dmitry Vyukov
2e579571e0 dashboard/app: refactor manager info in config
In preparation for future changes.
2018-04-24 13:58:56 +02:00
Dmitry Vyukov
9366d03f00 dashboard/app: allow testing fixes on exact commit and without patch
This implements 2 features:
1. It's now possible to specify exact commit when testing as:

2. It's possible to test without patch attached
assuming the patch is already committed to the tested tree.

Fixes #558
2018-04-24 13:23:01 +02:00
Dmitry Vyukov
e033c1f167 dashboard/app: use links instead of attachments in emails
As per discussion at:
https://groups.google.com/d/msg/syzkaller/zYlQ-b-QPHQ/AJzpeObcBAAJ
2018-03-25 12:46:05 +02:00
Dmitry Vyukov
ae36e6e584 dashboard/app: reduce noise is error log 2018-03-22 09:45:49 +01:00
Dmitry Vyukov
b389d9170c dashboard/app: don't log /api requests as errors
Not so interesting if it was just /api request.
2018-03-20 11:49:18 +01:00
Dmitry Vyukov
36d1c4540a all: fix gometalinter warnings
Fix typos, non-canonical code, remove dead code, etc.
2018-03-08 18:48:26 +01:00
Dmitry Vyukov
acd0caa5f7 dashboard/app: log api method/client
Since we now don't have them in URL, log them explicitly.
2018-03-08 13:18:32 +01:00
Dmitry Vyukov
8240eedfec dashboard/app: switch API to passing args as form-encoded values
This is slightly more secure and does not pollute URLs.
2018-03-08 13:00:10 +01:00
Dmitry Vyukov
afccdb6cff dashboard/app: preserve at least 1 crash per manager
When purging crashes we currently sort by priority (manager) first,
as the result if we have enough month-old crashes on upstream,
we stop saving any new crashes on other branches.
Sort by time first, but keep at least 1 crash per manager.
2018-02-26 12:08:01 +01:00
Dmitry Vyukov
9f8b1dde96 dashboard/app: save fewer crashes
Reduce threshold from 1000 to maxCrashes,
we don't save more than maxCrashes crashes,
so this looks like a good threshold to reduce
load on datastore.
2018-02-26 10:45:17 +01:00
Dmitry Vyukov
7daaa06d53 dashboard/app: restrict patch testing result CC list
Currently dashboard sends patch testing result to full bug CC list
(which includes kernel mailing lists). This is unnecessary and causes
problems with patchwork.
Reply only to people in the testing request CC list
(adding our mailing list if it was missing).

Fixes #526
2018-02-22 13:08:34 +01:00
Dmitry Vyukov
17061fc0f9 dashboard/app: finer-grained access control
This is the bulk of work for opening dashboard.
Implement finer-grained access control (admin/user/public).
Show only info designated to the current user access level.
Rework UIs for more clarity re bug statuses.
Show fixed bug.
2018-02-14 18:24:57 +01:00
Dmitry Vyukov
d7bc58204e dashboard/app: collect more info for better reports
Collect kernel build commit title/date.
Add support for kernel repo aliases (to be able
to say linux-next instead of full git repo address).
Collect on what managers a bug happened.
Reuse Crash.ReportLen as generic crash reporting priority.
Make it possible to prioritize reporting of particular
kernel repos and arches.

Fixes #473
2018-01-17 19:52:30 +01:00
Dmitry Vyukov
2129f66e2d dashboard/app: use Reported-by tags to fix bugs
Accept and use Reported-by tags in commits to mark bugs as fixed.
2018-01-17 19:52:30 +01:00
Dmitry Vyukov
c43cd72b20 dashboard/app: extend some log messages 2018-01-16 17:01:10 +01:00
Dmitry Vyukov
c66e766d03 dashboard/app: remember what crashes we reported for what bugs
We currently can silently switch crashes when report
to the next reporting, or test a patch using a repro
from a different crash.
Remember what crash we reported for a bug and use it
in both cases.
2018-01-16 14:56:29 +01:00
Dmitry Vyukov
6f03c35620 dashboard/app: extract fixing tags from commits
Support the new scheme of associating fixing commits with bugs.
Now we provide a tag along the lines of:

Reported-by: <syzbot+a4a91f6fc35e102@syzkaller.appspotmail.com>

The tag is supposed to be added to the commit.
Then we parse commit logs and extract these tags.

The final part on the dashboard is not ready yet,
but syz-ci should already parse and send the tags.
2017-12-27 09:09:18 +01:00
Dmitry Vyukov
73aba437a7 dashboard/app: bump max repros per bug to 10
We badly need repros. Developers complain.
2017-12-22 19:59:56 +01:00
Dmitry Vyukov
f2909d097e dashboard/app: add API for polling for closed bugs
External reporting may need to know when dashboard
is not interested in bugs anymore.
Add API that returns list of bugs dashboard considers closed.
2017-12-19 11:05:42 +01:00
Dmitry Vyukov
08c5c8bfdd dashboard/app: fix manager failed build reset
Only reset manager failed build if it uploaded _new_
successful build. On manager restart it uploads its
_old_ working build, and it should not reset a later
failed build.
2017-12-07 10:28:40 +01:00
Dmitry Vyukov
de212f1a05 dashboard/app: show manager addresses 2017-12-04 12:13:10 +01:00
Dmitry Vyukov
48359b9777 dashboard/app: improve needRepro
Don't reproduce corrupted reports.
Don't reproduce if canonical bug already has repro.
2017-12-01 17:27:08 +01:00
Dmitry Vyukov
948edf8055 dashboard/app: cosmetic changes to manager stats 2017-12-01 17:13:23 +01:00
Dmitry Vyukov
2fa91450df dashboard/app: add manager monitoring
Make it possible to monitor health and operation
of all managers from dashboard.
1. Notify dashboard about internal syz-ci errors
   (currently we don't know when/if they happen).
2. Send statistics from managers to dashboard.
2017-12-01 13:58:11 +01:00
Dmitry Vyukov
5153aeaffd syz-ci: test images before using them
Boot and minimally test images before declaring them as good
and switching to using them.

If image build/boot/test fails, upload report about this to dashboard.
2017-11-30 14:50:50 +01:00
Dmitry Vyukov
e338ada43c dashboard/app: fix purging of old crashes
We sorted by Report (which is a random int ID) instead of ReportLen.
Sort by ReportLen (order is opposite of what of queryCrashesForBug uses).
2017-11-28 17:52:24 +01:00
Dmitry Vyukov
fc5dca0a4a dashboard/app: reduce database contention
We frequently get "too much contention" errors when saving crashes.
Reduce contention by:
 - finding/creating bug before the transaction
 - saving crash outside of transaction
 - not saving crashes when we have too many of them already
2017-11-22 17:55:07 +01:00
Dmitry Vyukov
8c7cd9275d dashboard/app: support corrupted reports 2017-11-22 12:49:11 +01:00
Dmitry Vyukov
bf82068946 dashboard/app: assorted improvements/fixes for patch testing
Fix a bunch of bugs after testing.
Also update docs for patch testing.
2017-11-17 21:04:40 +01:00
Dmitry Vyukov
e928bb6b8a dashboard/app: test fix patches by request
Add initial support for testing fix patches.
A user on mailing list can request patch testing with:

and attach a patch.
2017-11-16 13:09:50 +03:00
Dmitry Vyukov
ed39afd6e8 dashboard/app: factor out uploadBuild from apiUploadBuild
Make it possible to upload builds from functions other than apiUploadBuild.
2017-11-16 09:47:38 +01:00
Dmitry Vyukov
d394531e95 dashboard/app, syz-ci: upload target OS/arch to dashboard 2017-09-19 17:00:26 +02:00
Dmitry Vyukov
06091e7473 dashboard/app: fix a bug with bugs with seq>0
See the added test.
2017-09-13 12:05:56 +02:00
Dmitry Vyukov
172189e955 dashboard/app: heavylifting of email reporting
- save Message-ID and use In-Reply-To in subsequent messages
- remember additional CC entries added manually
- don't mail to maintainers if maintainers list is empty
- improve mail formatting and add a footer
- implement upstream/fix/dup/invalid commands over email
- add tests
2017-08-17 19:42:11 +02:00
Dmitry Vyukov
0327584e99 dashboard/app: tell managers when dashboard needs a repro
Managers can't decide when a repro is required.
First, there can be bugs with seq>0.
Second, a repro can be already obtained on another manager.
2017-08-10 19:29:42 +02:00
Dmitry Vyukov
d23bf3250e dashboard/app: fix report_failed_repro api call
We are seeing the following errors in this handler:
cross-group transaction need to be explicitly specified, see TransactionOptions.Builder.withXG
2017-08-04 19:19:43 +02:00
Dmitry Vyukov
44347e8742 dashboard/app: purge at most 10 crashes at once
We see some episodic
"failed to delete old crash texts: Call error 11: Deadline exceeded (timeout)"
errors in logs.
Deleting at most 10 bugs should be enough since we do this check
after adding each new crash.
2017-08-02 16:37:42 +02:00
Dmitry Vyukov
61d1beb284 dashboard/app: new dashboard app
The new app is based on our experience with syz-dash
and is meant to supersede it. This app aims at full
automation of bug lifecycle: reporting, tracking updates,
closing fixed bugs.
The main differences are:
 - this app has support for reporting bugs
   either by email or using an arbitrary external reporting system
 - this app tracks status of bugs
 - this app captures more info about kernel builds
2017-08-02 15:11:03 +02:00