27 Commits

Author SHA1 Message Date
Dmitry Vyukov
7dbfdaf076 pkg/build: improve extraction of build errors
1. Extract multiple error lines (up to 10).
Gives more complete picture, maybe the first one
is not the most indicative one.

2. Replace weird unicode quotes with normal quotes.
These may be mishandled by some systems that don't understand utf-8.
2019-12-11 10:36:35 +01:00
Dmitry Vyukov
424cf6e8a1 pkg/build: add build signatures
Add optional build signature for images, currently only implemented for linux.
This can be used in bisection process to detect changes that does not affect kernel.

Update #1271
2019-11-06 11:41:05 +01:00
Dmitry Vyukov
c487cd4633 pkg/build: refactor Image function arguments
Image takes too many arguments, so we need to do lots of forwarding,
adding new argumnets is painful and most OSes are not interested
in lots of arguments.
Combine all arguments into a params struct.
2019-11-06 11:41:05 +01:00
Dmitry Vyukov
084e388155 pkg/vcs: make EnvForCommit return full compiler path
All callers of EnvForCommit need the compiler path,
so move this logic into EnvForCommit to avoid duplication.
Also simplifies tests because test impl can now return
an empty compiler (which should be unused).
2019-10-22 15:41:03 +02:00
Zubin Mithra
a2bdbd8c37 pkg/bisect: add initial testing support for cause bisection
(note: incomplete change)

Refactor existing code as follows:
* Move reusable test utility functions from git_repo_test.go to
pkg/vcs/test_util.go and make them exported.
* Split Run() into Run()+runImpl().
* Change type of bisect.go:env.inst to `instance.BuilderTester`.
Change usage inside syz-testbuild/testbuild.go accordingly.
* Move most of linux.PreviousReleaseTags() into vcs/git.go as
git.previousReleaseTags().
* Allow build.CompilerIdentity to be mocked.

Introduce the following changes:
* instance.BuilderTester is an interface with methods
	BuildSyzkaller()
	BuildKernel()
	Test()
NewEnv() now returns this interface.
* type testEnv implements instance.BuilderTester.
* type testBuilder implements builder interface. Add a entry into table
inside pkg/build/build.go:getBuilder() to return testBuilder object.
2019-10-22 10:09:57 +02:00
Andrew Donnellan
90c8f82ae8 pkg/build: Add ppc64le Linux as a supported configuration
Enable the Linux builder for ppc64le.

For ppc64le, we use zImage rather than bzImage as our kernel target. Pass
through the target architecture to buildKernel() so we can pick the right
target based on arch.

Closes: #1084 ("pkg/build: Support building Linux on ppc64le")
Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
2019-05-03 10:00:16 +02:00
Dmitry Vyukov
4398bd5e97 pkg/build: detect "Permission denied" as build error
This happens when compiler has bad permissions.
2019-04-09 18:38:04 +02:00
Dmitry Vyukov
dae7a62f02 pkg/build: extract first build error instead of last
The first error is usually more informative and significant.
Extract first error instead of last.
2019-04-05 08:40:36 +02:00
Dmitry Vyukov
9633c5c8a1 pkg/build: extract bazel build errors
We currently manually call extractRootCause in few selected places
to denote kernel build errors that we want to report to developers.
The rest are considered infra errors that we don't report.
This does not work well. We are missing fuchsia and gvisor build errors.
Treat all external command exection failures as kernel build errors instead.
Let's see how this works in practice.
Also add bazel-specfic error patterns and tests.
2019-03-29 11:00:34 +01:00
Marco Vanotti
d09a902e9d pkg/build: fix getBuilder for fuchsia.
This CL changes the supported vmType for fuchsia to be QEMU, instead of
GCE.

This was broken by commit b4e5a74e4, which changed the behavior of
getBuilder. It was part of a 6-commit PR trying to add support for
FreeBSD.

That commit has caused syzkaller to stop being able to build the fuchsia
image, getting the following error:

`kernel build failed: unsupported image type fuchsia/amd64/qemu`
2019-03-14 07:13:58 +01:00
Mark Johnston
b8de2a60a0 pkg/build: add freebsd
For now we must assume that we're building on FreeBSD; cross-compilation
support isn't quite there yet.

During a build, we first build an updated kernel toolchain if required,
generate a custom kernel configuration file with KCOV enabled, and build
a new kernel.  (When running on the custom kernel, "uname -i" will print
"SYZKALLER" rather than "GENERIC".)  Then, the image is mounted under a
tmpdir and we install the custom kernel to /boot/kernel.
2019-03-12 14:30:21 +01:00
Mark Johnston
b4e5a74e4c pkg/build: reduce cyclomatic complexity of getBuilder() 2019-03-12 14:30:21 +01:00
Dmitry Vyukov
40311807ad pkg/build: netbsd build improvements
1. Implement clean since we do incremental build we actually need no-op clean.
2. Don't copy netbsd to output dir, we don't need it there (we create full-fledged image).
3. Copy netbsd.gdb to obj/ dir, that's where it is expected.
4. Fix boot disk name for gce (it's sd0 instead of wd0).
2019-02-23 17:01:44 +01:00
Siddharth M
7fa29a5895 pkg/build: added feature to copy kernel image for netbsd
* Modified pkg/build/netbsd.go
1. Made kernel build incremental
2. Added code to copy kernel to disk image

* Fix formating issues

* Fixed basic issues

* Fixed copy kernel to disk
1. Added CPU option to default
2. Added snapshot option to vm/qemu/qemu.go
3. Hacky solution to get the build working

* Fixed issues and added vm.MonitorExecution

* Added sync instead of poweroff
2019-02-22 13:18:44 +01:00
Greg Steuck
5e5bb28ebf pkg/build: share extractRootCause with openbsd
* pkg/build: share extractRootCause with openbsd

This should get kernel build errors reported in syz-ci console.

* Add a test

* lint
2019-02-11 11:09:05 +01:00
Siddharth M
ebf7a37c5b pkg/build: add basic support for NetBSD
* Add pkg/build support for NetBSD

* Fix length of characters
2019-01-25 18:16:51 +01:00
Michael Pratt
ec0147d47f Merge pull request #874 from prattmic/bazel_version
pkg/build: fix bazel version parsing
2018-12-13 09:38:59 +01:00
Greg Steuck
6419afbb77 openbsd: run on gce
* build/openbsd: minor cleanup (use tuples instead of maps)

* Grammar nits in comments.

* Simplify openbsd.Create, will defer when there's more than one error exit.

* pkg/build: Support copying kernel into GCE image

* Simple test for openbsd image copy build.

* Cleanup in case something failed before.

* Support multi-processor VMs on GCE.

* More debug

* Reformat

* OpenBSD gce image needs to be raw.

* GC

* Force format to GNU directly on Go 1.10 or newer.

* Use vmType passed as a parameter inside openbsd.go

* gofmt

* more fmt

* Can't use GENERIC.mp just yet.

* capitalize

* Copyright
2018-11-27 13:14:06 +01:00
Anton Lindqvist
43f25323dc pkg/build: OpenBSD support, currently limited to building a kcov enabled kernel 2018-09-17 10:49:33 +02:00
Dmitry Vyukov
ff2962ae80 syz-ci: de-hardcode kernel object file names
We used to have this list in syz-ci and syz-managers.
Now we have kernel object in sys/targets, so use it instead.

Update #712
2018-09-11 15:43:55 +02:00
Dmitry Vyukov
375a3e31e1 pkg/build: save kernel config early
Currently we don't capture kernel config for broken builds
because the config is saved as the last step.
Save it as early as possible.
2018-07-24 14:30:05 +02:00
Dmitry Vyukov
1c9d305894 pkg/build: add akaros support 2018-07-05 13:03:10 +02:00
Dmitry Vyukov
bdac5e17f1 pkg/build: don't fail without compiler
Some OSes don't need a compiler.
Just return an empty string in such case.
2018-06-29 12:03:04 +02:00
Dmitry Vyukov
68ce63c468 pkg/build: support fuchsia builds 2018-06-28 13:42:07 +02:00
Dmitry Vyukov
ea804a7120 pkg/build: pave way for multi-OS support
Unify kernel and image build, that distinction is really uninteresting.
Define interface that each OS needs to implement.
Add gvisor stub.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
8c9738f9c7 pkg/build: support bazel in CompilerIdentity 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
94ef62054c pkg/build: move from pkg/kernel
Rename pkg/kernel to pkg/build and prepare for multi-OS support.
2018-06-22 16:40:45 +02:00