* pkg/compiler: Add error handler in `CollectUnused`.
This commit adds an error handler for the `CollectUnused` function. The
error handler just panics on any error, but is useful for debugging.
The error handler is used any time `comp` finds an error, and if it's
missing, it will panic due to a `nil` pointer dereference. At least now
we get a better understanding of the errors.
The only user of `CollectUnused` is `sys/fuchsia/fidlgen`, which is
failing now and will be fixed in a future commit.
The output message looks like this:
```
panic: could not collect unused nodes. fidl_net-stack.txt:110:15:
unknown type zx_chan_zircon_ethernet_Device_client
```
* pkg/compiler Better error handling in CollectUnused
This commit changes the default error handler for compiler to
`ast.LoggingHandler`, meaning that if `nil` is passed as an error
handler, `LoggingHandler` will be used instead.
`CollectUnused` now returns an error if any of the subfunctions produce errors.
`fidlgen` is the only caller of `CollectUnused`, and now checks for errors
as well.
* pkg/compiler Add tests for CollectUnused
This commit adds basic tests for the CollectUnused function. There's one
test that checks that it returns the right nodes, and another one that
makes sure that it returns errors when needed.
To make the test clearer, I had to add the error handler as an explicit
parameter in `CollectUnunsed`, instead of using the default one. This
avoid printing garbage in the logs. The `TestCollectUnusedError` function
uses a nopErrorHandler to avoid printing anything.
* pkg/compiler fix presubmit warnings
Squash of:
* Doc typo
* Ported some tun related functions.
* Copy vnet.txt from linux to openbsd.
* Simplified syz_emit_ethernet and stubbed out vnet.txt.
* Undo clang-format header sorting: headers are order sensitive.
* Uniquify tap devices by pid.
* clang-format off for includes
* Happier clang-format.
* Partially revert "Uniquify tap devices by pid."
Just rely on procid magic instead of getting it from a flag.
We can't cross-compile native binaries from just any OS to any other.
For most OSes we can do only native compilation.
Some can only be compiled from linux.
To date we avoided this problem completely (mostly assumed linux build OS).
Make this notion of what can build what explicit.
SSH keys are now included at the fx clean-build config.
A proper escape sequence looked weird so use a string literal to pass
that config.
Fixed some typos I found while debugging.
Hey--so I *don't* know a particularly efficient way to test syz-ci right
now.
There's no good config I can find for just running it locally, and I
couldn't find any good unit tests for this function in particular, and
running it manually on a ci server isn't super-appealing since I'd have
to clobber the configuration there to point to my repo instead of the
canonical repo, pull from there, etc.
Do you have any good ideas for testing this before it rolls out? In
particular I want the invocation to look exactly like:
./scripts/fx set x64 --packages garnet/packages/products/sshd --args 'extra_authorized_keys_file="//.ssh/authorized_keys"'
I thought I'd escaped it properly last time, but, since I couldn't test
it, clearly I goofed. I *think* this does the right thing but I'd love
a sanity check. Thanks!
* Needed to clean out the build on the main host. We switched to a
"petal" layout which required a complete rebuild.
* Update SSH interface. SSH keys have to be set at buildtime via
--args.
* Update fvm.blk location.
Sometimes the reorder_kernel error message is truncated causing the current
ignore pattern to fail. Instead, simply reject all lines containing
`reorder_kernel' in order to reduce noisy crash reports.
reportPrefixLen can become wrong after symbolization
if we symbolize any lines in the prefix.
Adjust reportPrefixLen during symbolization.
Automatic testing of this is problematic
because we would need to symbolize which requires
the object file with debug info.
Tested manually with syz-symbolize.
We don't need consistent image after reboot since we always reimage.
We know of some use cases that don't use journalling, but don't know
of any that use journalling.
My test harness for this code performed some steps that are not
performed when syz-executor is invoked directy.
Specifcally, we need to operate from a directory under /data/data,
and have the correct UID/GID set as the owner of the directory.
My test harness now correctly sets these, all sandbox operations
succeed, and loop() is invoked.
OpenBSD does not use "first line is title" convention,
so test that this is parsed properly.
Also test that multi-line fix/dup commands are properly parsed.
This is called from kfree in the added test.
We already ignore everything related to kmalloc/free
and e.g. arch/.*/mm/fault.c, so it looks reasonable
to ignore this one too.
EBUSY is returned when KMEMLEAK is in fact turned off,
not when scanning is already turned off.
That can't be recovered.
So instead check that we can write to the file in the check
function and give useful diagnostics.
The current memcg container seems to lead to lots of hangs/stalls.
Presumably the problem is with oom_score_adj and KASAN.
Executor process tree eats all memory and then the leaf process is killed
but the memory is not returned to memcg due to KASAN quarantine;
and the parent processes are protected from killing with oom_score_adj=-1000.
As the result the kernel locks up.
1. Don't use oom_score_adj=-1000. Instead bump leaf process score to 1000 (kill always).
2. Increase size of memcg to be larger than expected KASAN quarantine size.
Rewind kmemleak fd before reading it second time,
otherwise we will read truncated reports.
Auto-learn what leak reports we've already seen
and ignore them in future. This is required because
there are some false positives and some fire too frequently.
So now we will hit each leak only once per manager run,
but we still will try to reproduce them.
This sucks a lot, but ebtables.h is now broken too on Debian 4.17:
ebtables.h: In function ‘ebt_entry_target* ebt_get_target(ebt_entry*)’:
ebtables.h:197:19: error: invalid conversion from ‘void*’ to ‘ebt_entry_target*’
Improve go-fuzz fuzzer function and fix few new bugs it finds:
1. Panic in linux parser (bad).
2. Akaros can report empty rep.Report.
3. Fuchsia can return empty rep.Report.
Sometimes race conditions are reproduced by syz-execprog and are not
reproduced by the programs generated with syz-prog2c. In such cases
it's very helpful to know when exactly the fuzzing syscalls are executed.
Unfortunately, adding timestamps to the output of the original 'debug'
mode doesn't work. This mode provides very verbose output, which slows
down executor and breaks the repro.
So let's make the executor debug output less verbose and add
the timestamps.
Signed-off-by: Alexander Popov <alex.popov@linux.com>
On OpenBSD, nm(1) does not support the `-S' option.
The order of symbols from nm(1) varies between platforms, therefore rework the
test case to cope with that.
Fixes#723
Move debug_dump_data back to executor.cc.
debug_dump_data in common_linux.h does not play well
with pkg/csource debug stripping logic. It strips a large
random piece of code since it thinks debug_dump_data
definition is actually debug_dump_data call site.
Currently we have a global fixed set of sandboxes,
which makes it hard to add new OS-specific ones
(all OSes need to updated to say that they don't
support this sandbox).
Let it each OS say what sandboxes it supports instead.
executor: add support for android_untrusted_app sandbox
This adds a new sandbox type, 'android_untrusted_app', which restricts
syz-executor to the privileges which are available to third-party applications,
e.g. those installed from the Google Play store.
In particular, this uses the UID space reserved for applications (instead of
the 'setuid' sandbox, which uses the traditional 'nobody' user / 65534)
as well as a set of groups which the Android-specific kernels are aware of,
and finally ensures that the SELinux context is set appropriately.
Dependencies on libselinux are avoided by manually implementing the few
functions that are needed to change the context of the current process,
and arbitrary files. The underlying mechanisms are relatively simple.
Fixesgoogle/syzkaller#643
Test: make presubmit
Bug: http://b/112900774
For vmm we need a pre-created image.
So if one is provided in the userspace dir,
copy it to output. This should be enough
to get vmm working.
Update #712
We currently have this list in multiple places (somewhat diverged).
Specify this "overcommit" property in VM implementations.
In particular, we also want to allow overcommit for "vmm" type.
Update #712
After generating syscall description for fidl files using fidlgen, prune
all unused structs using the exact same mechanism used by the compiler's
check for unused structs. This allows the FIDL compiler to support
modular compilation; it does not need to have global knowledge of
whether each struct is used or not.
