50 Commits

Author SHA1 Message Date
Greg Steuck
16559f86f5
ci-openbsd-multicore: enable PF (#1033)
* pkg/build/openbsd: copy overlay files from userspaceDir into image

Fixes #1030
2019-03-05 09:08:17 -08:00
Julia Hansbrough
8845b00e1e pkg/build: update fuchsia fx arguments
We've once again changed the way you define a build via fx; update it in
Syzkaller too.
2019-03-01 06:20:22 +01:00
Julia Hansbrough
09aeeba49f fuchsia: Update Syzkaller to pull from Fuchsia monorepo (#1021)
Fuchsia and its repos now live in a new spot!
2019-02-27 21:44:58 -08:00
Dmitry Vyukov
9737e71ed1 pkg/build: fix netbsd ifconfig
! escaping seems not be required.
The current code writes / as well, which is wrong.
2019-02-23 19:18:12 +01:00
Dmitry Vyukov
30a3bd9f94 pkg/build: setup network interface on GCE for netbsd 2019-02-23 18:53:37 +01:00
Dmitry Vyukov
40311807ad pkg/build: netbsd build improvements
1. Implement clean since we do incremental build we actually need no-op clean.
2. Don't copy netbsd to output dir, we don't need it there (we create full-fledged image).
3. Copy netbsd.gdb to obj/ dir, that's where it is expected.
4. Fix boot disk name for gce (it's sd0 instead of wd0).
2019-02-23 17:01:44 +01:00
Dmitry Vyukov
baad4d3621 pkg/build: minor assorted improvements for netbsd
Create /fastboot and /var/db/entropy-file files.
Check that we copy kernel into the right location.
Fix code style.
2019-02-22 19:01:22 +01:00
Siddharth M
7fa29a5895 pkg/build: added feature to copy kernel image for netbsd
* Modified pkg/build/netbsd.go
1. Made kernel build incremental
2. Added code to copy kernel to disk image

* Fix formating issues

* Fixed basic issues

* Fixed copy kernel to disk
1. Added CPU option to default
2. Added snapshot option to vm/qemu/qemu.go
3. Hacky solution to get the build working

* Fixed issues and added vm.MonitorExecution

* Added sync instead of poweroff
2019-02-22 13:18:44 +01:00
Dmitry Vyukov
a39e52b1cd all: reformat with Go 1.10 2019-02-11 18:13:34 +01:00
Dan Robertson
13f1d0047a sys/linux: add NDISC packet formats to vnet.txt
Add the basic NDISC (RFC 4861) packet formats to sys/linux/vnet.txt.
2019-02-11 18:10:35 +01:00
Greg Steuck
73f5f45267 pkb/build: extractRootCause for NetBSD too
* pkg/build: share extractRootCause with openbsd

This should get kernel build errors reported in syz-ci console.

* Add a test

* lint

* pkb/build: extractRootCause for NetBSD too
2019-02-11 11:10:05 +01:00
Greg Steuck
5e5bb28ebf pkg/build: share extractRootCause with openbsd
* pkg/build: share extractRootCause with openbsd

This should get kernel build errors reported in syz-ci console.

* Add a test

* lint
2019-02-11 11:09:05 +01:00
Siddharth M
ebf7a37c5b pkg/build: add basic support for NetBSD
* Add pkg/build support for NetBSD

* Fix length of characters
2019-01-25 18:16:51 +01:00
Greg Steuck
a8b6d1af46 pkg/build/openbsd: always make clean to manage config in tree
clean build barely takes 3 minutes end-to-end on our CI machine.

Undo debug options which caused prevented kernels from booting on GCE.
2018-12-19 17:15:31 +01:00
Dmitry Vyukov
a9cc88c19f pkg/osutil: provide better Rename
os.Rename fails for cross-device renaming (e.g. to/from tmpfs).
This is quite unpleasant. Provide own version that falls back to copying.
2018-12-17 19:09:10 +01:00
Michael Pratt
ec0147d47f Merge pull request #874 from prattmic/bazel_version
pkg/build: fix bazel version parsing
2018-12-13 09:38:59 +01:00
Greg Steuck
610352d5d7 pkg/build/openbsd: require kernel config files as configuration data
Previously the config was generated directly, but testing multiple
configurations makes this cumbersome going forward. This makes
kernel_config a mandatory parameter.
2018-12-07 09:50:12 +01:00
Greg Steuck
6419afbb77 openbsd: run on gce
* build/openbsd: minor cleanup (use tuples instead of maps)

* Grammar nits in comments.

* Simplify openbsd.Create, will defer when there's more than one error exit.

* pkg/build: Support copying kernel into GCE image

* Simple test for openbsd image copy build.

* Cleanup in case something failed before.

* Support multi-processor VMs on GCE.

* More debug

* Reformat

* OpenBSD gce image needs to be raw.

* GC

* Force format to GNU directly on Go 1.10 or newer.

* Use vmType passed as a parameter inside openbsd.go

* gofmt

* more fmt

* Can't use GENERIC.mp just yet.

* capitalize

* Copyright
2018-11-27 13:14:06 +01:00
Dmitry Vyukov
9f6fc500f0 pkg/build: copy fuchsia ssh key
We've lost is somewhere along the way.
2018-11-21 08:16:32 +01:00
Julia Hansbrough
b08ee62aec [fuchsia] SSH fix and build invocation fix.
SSH keys are now included at the fx clean-build config.

A proper escape sequence looked weird so use a string literal to pass
that config.

Fixed some typos I found while debugging.
2018-11-16 11:58:29 -08:00
Julia Hansbrough
3a41052e6a [fuchsia] Fix typo in Fuchsia build.
Hey--so I *don't* know a particularly efficient way to test syz-ci right
now.

There's no good config I can find for just running it locally, and I
couldn't find any good unit tests for this function in particular, and
running it manually on a ci server isn't super-appealing since I'd have
to clobber the configuration there to point to my repo instead of the
canonical repo, pull from there, etc.

Do you have any good ideas for testing this before it rolls out?  In
particular I want the invocation to look exactly like:

./scripts/fx set x64 --packages garnet/packages/products/sshd --args 'extra_authorized_keys_file="//.ssh/authorized_keys"'

I thought I'd escaped it properly last time, but, since I couldn't test
it, clearly I goofed.  I *think* this does the right thing but I'd love
a sanity check.  Thanks!
2018-11-15 09:31:46 -08:00
Julia Hansbrough
5f5f6d14e8 [fuchsia] Fix Fuchsia build.
* Needed to clean out the build on the main host. We switched to a
"petal" layout which required a complete rebuild.

* Update SSH interface.  SSH keys have to be set at buildtime via
--args.

* Update fvm.blk location.
2018-11-12 19:07:01 -08:00
Dmitry Vyukov
1ba7fd7e65 all: fix code formatting
Reformat, remove debug leftovers, fix comment style.
2018-10-16 13:57:24 +02:00
Dmitry Vyukov
8cd30605ce tools/create-gce-image.sh: create ext4 image without journalling
We don't need consistent image after reboot since we always reimage.
We know of some use cases that don't use journalling, but don't know
of any that use journalling.
2018-10-15 18:56:55 +02:00
Dmitry Vyukov
8899d58437 tools/create-gce-image.sh: mount securityfs and configfs 2018-09-26 14:06:11 +02:00
Dmitry Vyukov
a0be1d1210 pkg/build: copy image to output dir
For vmm we need a pre-created image.
So if one is provided in the userspace dir,
copy it to output. This should be enough
to get vmm working.

Update #712
2018-09-17 10:53:06 +02:00
Anton Lindqvist
43f25323dc pkg/build: OpenBSD support, currently limited to building a kcov enabled kernel 2018-09-17 10:49:33 +02:00
Anton Lindqvist
e9b51a9024 pkg/build: fix typo in error messages 2018-09-17 10:49:33 +02:00
Dmitry Vyukov
ff2962ae80 syz-ci: de-hardcode kernel object file names
We used to have this list in syz-ci and syz-managers.
Now we have kernel object in sys/targets, so use it instead.

Update #712
2018-09-11 15:43:55 +02:00
Dmitry Vyukov
58f18e3f60 tools/create-gce-image.sh: disable ftrace_dump_on_oops
We don't have anything useful there.
But sometimes fuzzer somehow poppulates it with some nonsense,
and then it all dumped in crashes.
Disable it.
2018-09-10 16:19:40 +02:00
Dmitry Vyukov
6b5120a464 tools/create-gce-image.sh: auto-detect block device type
Currently we choose block device to use (nbd/loop) based on SYZ_VM_TYPE.
Strictly saying these things are orthogonal.
losetup is broken on Ubuntu. qemu-nbd is broken on Debian.
Try to auto-detect what will work based on uname.
2018-09-07 19:04:47 +02:00
Dmitry Vyukov
196410e4f5 dashboard/config: re-enable selinux
Upstream "selinux: fix mounting of cgroup2 under older policies"
commit fixes mounting of cgroup2 under wheezy selinux policy.
So don't disable selinux on start.
Create separate cmdline arguments that enable selinux and apparmor.
2018-09-05 12:50:53 +02:00
Dmitry Vyukov
98bfd6d34c tools/create-gce-image.sh: add default ext4 options
Set some realistic modern ext4 options when creating the image.
2018-09-03 16:42:59 +02:00
Dmitry Vyukov
95b5c82b28 pkg/build: do not use asan for fuchsia
If we build fuchsia with asan, syz-executor binary fails to run with:

$ /tmp/syz-executor
[00041.656] 01128.01275> dlsvc: could not open 'libc++.so.2'
[00041.656] 06074.06088> Error loading shared library libc++.so.2: ZX_ERR_NOT_FOUND (needed by
[00041.656] 06074.06088> /tmp/syz-executor
[00041.656] 06074.06088> )
[00041.656] 01128.01275> dlsvc: could not open 'libc++abi.so.1'
[00041.656] 06074.06088> Error loading shared library libc++abi.so.1: ZX_ERR_NOT_FOUND (needed by
[00041.656] 06074.06088> /tmp/syz-executor
[00041.656] 06074.06088> )
[00041.663] 01128.01275> dlsvc: could not open 'libunwind.so.1'
[00041.663] 06074.06088> Error loading shared library libunwind.so.1: ZX_ERR_NOT_FOUND (needed by
[00041.663] 06074.06088> /tmp/syz-executor
[00041.663] 06074.06088> )

Disable asan until we figure out how to fix this.
2018-08-20 11:40:44 -07:00
Dmitry Vyukov
4e1e8035f3 pkg/build: update gvisor race build process
Use separate target as this is something that can be upstreamed.
2018-08-18 22:04:17 -07:00
Julia Hansbrough
738da825b2 [fuchsia] Change QEMU args to use multiboot shim. 2018-08-17 07:02:12 -07:00
Dmitry Vyukov
5084a6252d pkg/build: build asan-ified fuchsia
Building with asan is useful to detect any bugs in C/C++ services.
2018-08-14 08:50:56 -07:00
Dmitry Vyukov
375a3e31e1 pkg/build: save kernel config early
Currently we don't capture kernel config for broken builds
because the config is saved as the last step.
Save it as early as possible.
2018-07-24 14:30:05 +02:00
Dmitry Vyukov
2e0e3130f9 pkg/build: update initrd name
Seems to have been renamed.
2018-07-10 11:14:36 +02:00
Dmitry Vyukov
7c31f7a78d pkg/build: always do clean build for fuchsia
We frequently see boot errors like:

[00010.201] 02991.03067> pkgsvr: 2018/06/30 23:39:41 system: failed to set system root from blob "ccbadb3901372b1e0fc5275f627f708bf3e5f3acfb0d4268638db0ff75fc7fd4": file does not exist

or:

[00003.691] 01126.01153> devmgr: launchpad /fs/blob/e66739acdd3d8efa3b7c9021e2107cf8431765c0b8eb0a1ec7f7dc7fd305f2f7 (pkgfs) failed: launchpad_vmo_from_file failure: -40

Presumably clean build may help.
2018-07-10 11:05:04 +02:00
Dmitry Vyukov
1c9d305894 pkg/build: add akaros support 2018-07-05 13:03:10 +02:00
Dmitry Vyukov
bdac5e17f1 pkg/build: don't fail without compiler
Some OSes don't need a compiler.
Just return an empty string in such case.
2018-06-29 12:03:04 +02:00
Dmitry Vyukov
68ce63c468 pkg/build: support fuchsia builds 2018-06-28 13:42:07 +02:00
Dmitry Vyukov
be3706ff1e pkg/build: better support for gvisor race build
Explicitly pass --features=race.
Expect linux_amd64_static_stripped as a potential output.
2018-06-26 15:36:46 +02:00
Dmitry Vyukov
665947c70a pkg/build: support gvisor race binaries 2018-06-23 09:46:28 +02:00
Dmitry Vyukov
67ce863533 pkg/build: save kernel config for gvisor
gvisor does not use it, but save it verbatim.
This can be useful to keep some additional description of the build.
2018-06-22 19:13:38 +02:00
Dmitry Vyukov
c97f0d7a86 pkg/build: add gvisor support 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
ea804a7120 pkg/build: pave way for multi-OS support
Unify kernel and image build, that distinction is really uninteresting.
Define interface that each OS needs to implement.
Add gvisor stub.
2018-06-22 16:40:45 +02:00
Dmitry Vyukov
8c9738f9c7 pkg/build: support bazel in CompilerIdentity 2018-06-22 16:40:45 +02:00
Dmitry Vyukov
94ef62054c pkg/build: move from pkg/kernel
Rename pkg/kernel to pkg/build and prepare for multi-OS support.
2018-06-22 16:40:45 +02:00