Commit Graph

513 Commits

Author SHA1 Message Date
Dmitry Vyukov
22f8756722 tools/syz-linter: check log/error text ending with dot
Update #1876
2020-07-05 10:55:16 +02:00
Dmitry Vyukov
214e3e269c tools/syz-linter: check log/error text starting with capital letter
Update #1876
2020-07-05 10:55:16 +02:00
Dmitry Vyukov
24d7f50531 tools/syz-linter: add check for dup types in func args
Add check for:

func foo(a int, b int) -> func foo(a, b int)

Update #1876
2020-07-04 15:05:30 +02:00
Dmitry Vyukov
85b1d37b14 tools/syz-linter: add check for comment format
Update #1876
2020-07-04 15:05:30 +02:00
Dmitry Vyukov
c992206a1d tools/syz-linter: add custom linter
For now we have 2 simple checks:

1. for multiline comments:
/* */ -> //

2. for string len comparison with 0:
len(str) != 0 -> str != ""

Update #1876
2020-07-04 15:05:30 +02:00
Dmitry Vyukov
74cb4e09a5 go.mod: add go-fuzz-build
Otherwise go-fuzz-build now fails with:

+ go-fuzz-build -libfuzzer -func FuzzDeserialize -o fuzzer.a ./prog/test
-: cannot find package "." in:
	/syzkaller/gopath/src/github.com/google/syzkaller/vendor/github.com/dvyukov/go-fuzz/go-fuzz-dep
2020-07-04 15:05:30 +02:00
Dmitry Vyukov
c7d7f10bdf go.mod: vendor golangci-lint 2020-07-04 15:05:30 +02:00
Jouni Hogander
f8885dc4ce pkg/bisect: Implement config bisection
Implement Linux kernel configuration bisection. Use bisected minimalistic
configuration in commit bisection. Utilizes config_bisect.pl script from Linux
kernel tree in bisection.

Modify syz-bisect to read in kernel.baseline_config. This is used as a "good"
configuration when bisection is run.
2020-07-02 09:32:57 +02:00
Dmitry Vyukov
349959fbca tools/syz-check: support all arches 2020-06-26 17:34:52 +02:00
Dmitry Vyukov
adb7d9e67b tools/docker/env: add s390x and riscv64 toolchains
These arches are in-progress now. Need them in the image.
2020-06-25 19:27:01 +02:00
Jukka Kaartinen
1afe15356a tools/syz-reporter: Tool to share/visualize crashes
Goes through crash folder that is stated in the workdir.
Collects the crashes, counts and tags.

usage:
 ./bin/syz-reporter -config manager.cfg

Signed-off-by: Jukka Kaartinen <jukka.kaartinen@unikie.com>
2020-06-22 15:10:31 +02:00
Albert van der Linde
123cf502d3 tools/syz-headerparser/headerlib
Added __init__.py as it is required for headerparser.py to run.
Was removed by mistake in 672c10ad7c.
2020-06-19 14:44:08 +02:00
Dmitry Vyukov
b621d37e81 tools/syz-make: reduce number of make jobs
Ensure that we have at least 1GB per Go compiler/linker invocation.
Go compiler/linker can consume significant amount of memory
(observed to consume at least 600MB). See #1276 for context.
And we have parallelization both on make and on go levels,
this can severe oversubscribe RAM.
Note: the result can be significantly lower than the CPU number,
but this is fine because Go builds/tests are parallelized internally.
2020-06-19 11:56:47 +02:00
Alexander Egorenkov
bc258b506d pkg: support compiler triple for 'nm' and 'addr2line'
In preparation to support big-endian architectures.
2020-06-18 19:31:40 +02:00
Jukka Kaartinen
4ea9d9646e tools/syz-crush: Add debug option
Signed-off-by: Jukka Kaartinen <jukka.kaartinen@unikie.com>
2020-06-16 12:43:23 +02:00
Dmitry Vyukov
baca261187 tools/syz-crush: use standard log package
We always use verbosity 0, there is no point in using
our custom package. Switch to the standard one instead.
2020-06-15 18:57:58 +02:00
Dmitry Vyukov
911738ed1f tools/syz-crush: simplify and tidy
As discussed in #1798.
2020-06-15 18:57:58 +02:00
Jukka Kaartinen
f56b572638 tools/syz-crush: Add feature to run C-program
usage:
  syz-crush -config=config.json creprog.c

Produced output is same as syz-manager produces.
New directory called reproduced_with is added that lists all the
reproducers that triggered the crash.

Example:

~/workdir/crashes  $ tree
.
├── 2a8fff76c6267ecfeaf3530a602ed48afbc4a151
│   ├── description
│   ├── log0
│   ├── log1
│   ├── log2
│   ├── report0
│   ├── report1
│   ├── report2
│   └── reproduced_with
│       └── 17a6ed226769660f3e5dad7b22b13466b938e290.c
├── 2b6b921457afe8cebd0a96ca40f8569e6ee95174
│   ├── description
│   ├── log0
│   ├── log1
│   ├── log2
│   ├── log3
│   ├── log4
│   ├── log5
│   ├── report0
│   ├── report1
│   ├── report2
│   ├── report3
│   ├── report4
│   ├── report5
│   └── reproduced_with
│       ├── 9fb2f5b37bf4428382334e336bbbb201634c7c4c.c
│       └── ab27002b46d19cafb1ebb8b040f0a3b0f8f88974.c

Signed-off-by: Jukka Kaartinen <jukka.kaartinen@unikie.com>
2020-06-15 18:41:13 +02:00
Dmitry Vyukov
6378506674 tools: update ietf link to the latest version 2020-06-13 20:04:08 +02:00
Dmitry Vyukov
83af9a2cef sys/linux: regenerate .warn files
On latest kernel, with latest *.txt files and with kernels compiled with
-fno-eliminate-unused-debug-types -fno-eliminate-unused-debug-symbols flags.
2020-06-13 19:56:20 +02:00
Dmitry Vyukov
3637706c07 tools/syz-check: fix bug with struct name overwrite
We patched name in struct object, but the dwarf package
caches then and then can return in subsequent invocations.
This causes a struct name to be overwritten by typedef name.
Don't mutate returned struct objects.
2020-06-13 19:56:20 +02:00
Dmitry Vyukov
588020678f all: use more respective language
Some terms are normalised on the technical level
but may be oppressive on a societal level.
Replace them with more technically neutral terms.
See the following doc for a longer version:
https://tools.ietf.org/id/draft-knodel-terminology-00.html
2020-06-11 23:19:34 +02:00
Dmitry Vyukov
5c25fdab08 sys/targets: fail loudly if SOURCEDIR_GOOS is set but compiler is broken
Since we now have SOURCEDIR_{FUCHSIA,AKAROS,NETBSD} exported in the
syz-big-env docker image, this will make CI fail for broken cross-builds too.

Update instructions in the docker image to fix the current problem
with permissions in syz-big-env: we need to tar with --mode=go=u.
2020-06-11 18:06:49 +02:00
Ricardo Cañuelo
8648bd1f6d tools: add custom udev rules to generated images
Add a default udev rule file to the image creation process in
create-gce-image.sh and create-image.sh.

This change creates a default rule to make udev create a custom-named
symlink for the specific vim2m device.
2020-06-10 13:09:12 +02:00
Dmitry Vyukov
3f04838a10 .golangci.yml: make goconst checker more strict 2020-06-07 10:41:01 +02:00
Dmitry Vyukov
2ba71ec506 pkg/repro: don't enable features missing on the target
Manager has already checked what features are present on the target.
But if we detected that, say, USB is missing, we still enabled it
in the starting csource options. This is wrong, increases configuration
minimization time and may lead to some obscure bugs.
2020-06-03 11:35:08 +02:00
Dmitry Vyukov
52fd7b7d77 tools: add syz-big-env
On top of syz-env it provides akaros/fuchsia/netbsd toolchains and gcloud sdk.
With this it's possible to run dashboard/app tests on CI and locally
and test executor build and pkg/{csource,cover} for these OSes.

Update #1765
2020-06-02 13:39:30 +02:00
Dmitry Vyukov
fdf90f622b pkg/cover: add test for report generation
Test various combinations of no debug info,
no coverage instrumentation, no PCs, bad PCs, good PCs,
and what errors we produce for these.
Also implement support for cross-arch reports:
prefix objdump with cross-compile prefix
(e.g. aarch64-linux-gnu-objdump instead of objdump).
2020-05-27 14:38:28 +02:00
Dmitry Vyukov
0f54349fc6 tools/syz-check: restore handling of unions
Unions were dropped accidentially during removal StructDesc.
2020-05-25 18:06:29 +02:00
Dmitry Vyukov
82f3c7881f tools/syz-check: don't check kvm on arm 2020-05-25 18:06:29 +02:00
Dmitry Vyukov
fcec29826e tools/syz-check: fix field name in output 2020-05-25 18:06:29 +02:00
Dmitry Vyukov
5afa2ddd5a tools/syz-check: fix path calculation
pkg/ast now gives full file name in Pos.
2020-05-21 18:22:54 +02:00
Dmitry Vyukov
6d882fd26c tools/check_links.py: detect absolute links
Detect absolute links to our repo.
These are not checked and long and inconsistent with majority of links.
2020-05-19 12:52:04 +02:00
Dmitry Vyukov
24d9114275 .github/workflows: add CI workflow
Switch from Travis to Github Actions for testing.
This is faster and is better integrated with github.

Update #1699
2020-05-18 11:34:42 +02:00
Dmitry Vyukov
37137150cb tools: add syz-env
syz-env provides a canned dev environment with all tools necessary
(Go, gcc, clang-format, etc) for development/testing in a form
of a Docker image. Example use:

$ syz-env make presubmit
$ syz-env make extract SOURCEDIR=~/linux

Update #1699
2020-05-18 11:34:42 +02:00
Dmitry Vyukov
848ec2e87b tools/codecov.yml: fix format
Merge 2 coverage sections.
2020-05-18 11:34:42 +02:00
Dmitry Vyukov
4836c13c61 tools/syz-make: rename from syz-env
syz-env will be a good, short name for the planned
docker-ized build environment. So rename syz-env to syz-make.
Maybe not the best name, but it's not visible basically anywhere.
2020-05-18 11:34:42 +02:00
Dmitry Vyukov
672c10ad7c all: add missing copyright statements 2020-05-18 11:34:42 +02:00
Dmitry Vyukov
fc28aeb6b9 tools/check_links.py: print errors in standard format
Use the standard "file:line:col: error message" format.

Update #1699
2020-05-18 11:34:42 +02:00
Dmitry Vyukov
51c18193d3 tools/check-copyright.sh: print errors in standard format
Use the standard "file:line:col: error message" format.

Update #1699
2020-05-18 11:34:42 +02:00
Dmitry Vyukov
bb26bde008 tools/check-copyright.sh: check py and yml files
Update #1699
2020-05-18 11:34:42 +02:00
Dmitry Vyukov
50749f54cd pkg/report: add ParseAll
ParseAll returns all reports in output.
Use it in syz-symbolize.
2020-05-13 20:49:07 +02:00
Alexander Potapenko
88a97d1a7a Get rid of "Cross" in variable names
Renamed Target.BrokenCrossCompiler to Target.BrokenCompiler and
Target.CrossCFlags to Target.CFlags

"Everything in Target is about Cross now."

Signed-off-by: Alexander Potapenko <glider@google.com>
2020-05-13 13:05:41 +02:00
Dmitry Vyukov
8cbfd71747 tools/syz-db: implement memory benchmarking function
syz-db bench loads a corpus and measures memory consumption.
2020-05-05 14:01:52 +02:00
Dmitry Vyukov
a4d38b39a8 prog: support disabled attribute
Update #477
Update #502
2020-05-04 20:56:20 +02:00
Dmitry Vyukov
58ae5e1862 prog: remove StructDesc
Remove StructDesc, KeyedStruct, StructKey and all associated
logic/complexity in prog and pkg/compiler.
We can now handle recursion more generically with the Ref type,
and Dir/FieldName are not a part of the type anymore.
This makes StructType/UnionType simpler and more natural.

Reduces size of sys/linux/gen/amd64.go from 5201321 to 4180861 (-20%).

Update #1580
2020-05-03 12:55:42 +02:00
Dmitry Vyukov
5457883a51 tools/codecov.yml: add
Add configuratio file for codecov.io
2020-05-02 14:12:25 +02:00
Dmitry Vyukov
4d9915573a organize top level files
Remove .gitattributes, we don't have any of these files now.
Move fuzzit.sh -> tools/fuzzit.sh.
2020-05-02 14:12:25 +02:00
Dmitry Vyukov
58da4c35b1 prog: introduce Field type
Remvoe FieldName from Type and add a separate Field type
that holds field name. Use Field for struct fields, union options
and syscalls arguments, only these really have names.

Reduces size of sys/linux/gen/amd64.go from 5665583 to 5201321 (-8.2%).
Allows to not create new type for squashed any pointer.
But main advantages will follow, e.g. removing StructDesc,
using TypeRef in Arg, etc.

Update #1580
2020-05-02 12:16:06 +02:00
Dmitry Vyukov
bc734e7ada prog: rename {PtrType,ArrayType}.Type to Elem
Name "Type" is confusing when referring to pointer/array element type.
Frequently there are too many Type/typ/typ1/t and typ.Type is not very informative.
It _is_ a type, but what's usually more relevant is that it's an _element_ type.
Let's leave type checking to compiler and give it a more meaningful name.
2020-05-01 13:31:17 +02:00