They should have been detected by "same binary" logic.
But the problem is that we may use different compilers
for different commits and they switch exactly at release commits.
So we can build the release with a differnet compiler than the
rest of commits and then obviously it won't be "same binary".
Detect release commits separately.
Update #1271
3/4 added tests currently fail.
The problem is that we don't collect results
from initial bisection range detection.
As the result we won't detect "same binary"
for release and first commit in a release.
Update #1271
Detect bisection to merge commits and to commits that don't affect
kernel binary (comments, other arches, whitespaces, etc).
Such bisections are not reported in emails (but shown on web).
Update #1271
* Modify Run() to return the commit object when a failure occurs on the
oldest tested release(for BisectCause) and latest tested
release(for BisectFix).
* Modify tests to check the returned commit object.
Add tests for the following cases(1 each for cause and fix):
- bisection returns the correct commit
- bisection failed due to crash not occuring on the original commit
- bisection failed as crash occurs on oldest/latest commit
- bisection is inconclusive
All callers of EnvForCommit need the compiler path,
so move this logic into EnvForCommit to avoid duplication.
Also simplifies tests because test impl can now return
an empty compiler (which should be unused).
(note: incomplete change)
Refactor existing code as follows:
* Move reusable test utility functions from git_repo_test.go to
pkg/vcs/test_util.go and make them exported.
* Split Run() into Run()+runImpl().
* Change type of bisect.go:env.inst to `instance.BuilderTester`.
Change usage inside syz-testbuild/testbuild.go accordingly.
* Move most of linux.PreviousReleaseTags() into vcs/git.go as
git.previousReleaseTags().
* Allow build.CompilerIdentity to be mocked.
Introduce the following changes:
* instance.BuilderTester is an interface with methods
BuildSyzkaller()
BuildKernel()
Test()
NewEnv() now returns this interface.
* type testEnv implements instance.BuilderTester.
* type testBuilder implements builder interface. Add a entry into table
inside pkg/build/build.go:getBuilder() to return testBuilder object.
We currently skip a commit iff all 10 instances errored.
But if, say, only 9 errored we consider it as OK,
but this significnalty reduces chances of detecting flaky crashes.
So skip if more than 2/3 errored.
Update #501
Use CheckoutCommit instead of SwitchCommitto get the crashing commit.
The problem is with trees like linux-next. They require at least fetching tags
and maybe even different tree (-history).
Use CheckoutCommit which at least fetches tags which is enough
for recent commits.
Update #501
A bunch of improvements after more wide bisection testing.
Improve logging.
Support returning several commits for inconclusive bisection.
Return Report with the final crash.
Remove code that was moved to pkg/vcs.
Update #501
In preparation for syz-ci bisection:
- move bisection function into a separate interface
they look out of place in vcs.Repo because most OSes
don't implement it and most users don't case
- extract author name and more CC emails for commits
- move linux-specific PreviousReleaseTags into linux.go
- fix inconclusive bisection (more than 1 potential commits)
- add tests fr bisection
- add maintainers returned from get_maintainers.pl for commits
that don't have enough emails (e.g. only author email)
Update #501
mgrconfig was used only by syz-manager initially,
but now it's used by a dozen of packages and it's
weird to import from under a binary dir.
pkg/ is much more reasonable dir for a widely used
helper package.
Currently kernel build failures are insanely verbose
(contain full kernel build output) and there is no
way to separate short descriptions from full output.
Make it possible.
Also try to extract failure root cause froom build log.
Use this in pkg/bisect to not pollute log on build failures.
Update #501
With 5 tries sometimes only 1 fails,
and sometimes we probably have false negatives.
Increase number of tries to 8 and compress
results if they all are the same.
Update #501
