Commit Graph

9 Commits

Author SHA1 Message Date
Dmitry Vyukov
ec42220e77 Makefile: generate descriptions on-the-fly
Checking in the generated descriptions files makes few things simpler,
but causes pain for pull requests: (1) PRs that touch descriptions
_always_ conflict, (2) PRs are large and harder to review,
(3) people sometimes forget to add auto-generated files.

The proposed way does not require us to hardcode lots of dependencies
in the Makefile (which is nice) and seem to work.
Let's see how it works.

The main contributor-visible consequence is that the auto-generated
files do not need to be checked-in now.

Credit for figuring the Makefile magic goes to @melver.

Fixes #1291
2020-04-30 17:18:29 +02:00
Dmitry Vyukov
0ce7569ee7 pkg/compiler: deduplicate Types in descriptions
Add prog.Ref Type that serves as a proxy for real types
and allows to deduplicate Types in generated descriptions.
The Ref type is effectively an index in an array of types.
Just before serialization pkg/compiler replaces real types
with the Ref types and prepares corresponding array of real types.
When a Target is registered in prog package, we do the opposite
operation and replace Ref's with the corresponding real types.

This brings improvements across the board:
compiler memory consumption is reduced by 15%,
test building time by 25%, descriptions size by 33%.

Before:
$ du -h sys/linux/gen
54M	sys/linux/gen

$ time GOMAXPROCS=1 go test -p=1 -c ./prog
real	0m54.200s
real	0m53.883s

$ time GOMAXPROCS=1 go install -p=1 ./tools/syz-execprog
real	0m27.911s
real	0m27.767s

$ TIME="%e %P %M" GOMAXPROCS=1 time go tool compile ./sys/linux/gen
20.59 100% 3200016
20.97 100% 3445976
20.25 100% 3209684

After:
$ du -h sys/linux/gen
36M	sys/linux/gen

$ time GOMAXPROCS=1 go test -p=1 -c ./prog
real	0m42.290s
real	0m43.230s

$ time GOMAXPROCS=1 go install -p=1 ./tools/syz-execprog
real	0m24.337s
real	0m24.727s

$ TIME="%e %P %M" GOMAXPROCS=1 time go tool compile ./sys/linux/gen
19.11 100% 2764952
19.66 100% 2787624
19.35 100% 2749376

Update #1580
2020-04-26 05:58:31 +02:00
Dmitry Vyukov
67234372ef prog: refactor target.MakeMmap
Make MakeMmap return more than 1 call.
This is a preparation for future changes.
Also remove addr/size as they are effectively
always the same and can be inferred from the target
(will also conflict with the future changes).
Also rename to MakeDataMmap to better represent
the new purpose: it's just some arbitrary mmap,
but rather mapping of the data segment.
2020-04-18 14:35:45 +02:00
Dmitry Vyukov
80d43738f1 prog: rename target.SanitizeCall to Neutralize
We will need a wrapper for target.SanitizeCall that will do more
than just calling the target-provided function. To avoid confusion
and potential mistakes, give the target function and prog function
different names. Prog package will continue to call this "sanitize",
which will include target's "neutralize" + more.
Also refactor API a bit: we need a helper function that sanitizes
the whole program because that's needed most of the time.

Fixes #477
Fixes #502
2020-03-17 21:19:13 +01:00
Dmitry Vyukov
55da6462ff pkg/serializer: do not write field names if it won't save space
If we are going to write all values, don't write field names.
This only increases size of generated files.
The change reduces size of generated files by 5.8%
(62870496-59410354=3460142 bytes saved).
2020-01-26 11:07:18 +01:00
Dmitry Vyukov
01d1494471 .golangci.yml: add codeanalysis build tag
Using a build tag to exclude files for golangci-lint
reduces memory consumption (it does not parse them).
The naive attempt with skip-dirs did not work.
So add codeanalysis build tag and use it in auto-generated files.

Update #977
2019-05-15 11:39:31 +02:00
Dmitry Vyukov
16c881ad85 pkg/compiler: generate complex len targets
Change the generated format for len type to support multiple path elements.
2019-05-14 19:28:01 +02:00
Dmitry Vyukov
89a1588519 sys: disable trusty for now
It crashes pkg/csource tests because
it does not have syz_mmap syscall but tries to use it.
2018-11-08 20:14:05 -08:00
Dmitry Vyukov
0d872150de sys: initial trusty support
Trusty is a set of software components supporting
a Trusted Execution Environment (TEE) on mobile devices.

https://source.android.com/security/trusty

Add syscall descriptions and some boilerplate.
2018-11-07 17:52:06 -08:00