syzkaller/tools
Dmitry Vyukov 0ce7569ee7 pkg/compiler: deduplicate Types in descriptions
Add prog.Ref Type that serves as a proxy for real types
and allows to deduplicate Types in generated descriptions.
The Ref type is effectively an index in an array of types.
Just before serialization pkg/compiler replaces real types
with the Ref types and prepares corresponding array of real types.
When a Target is registered in prog package, we do the opposite
operation and replace Ref's with the corresponding real types.

This brings improvements across the board:
compiler memory consumption is reduced by 15%,
test building time by 25%, descriptions size by 33%.

Before:
$ du -h sys/linux/gen
54M	sys/linux/gen

$ time GOMAXPROCS=1 go test -p=1 -c ./prog
real	0m54.200s
real	0m53.883s

$ time GOMAXPROCS=1 go install -p=1 ./tools/syz-execprog
real	0m27.911s
real	0m27.767s

$ TIME="%e %P %M" GOMAXPROCS=1 time go tool compile ./sys/linux/gen
20.59 100% 3200016
20.97 100% 3445976
20.25 100% 3209684

After:
$ du -h sys/linux/gen
36M	sys/linux/gen

$ time GOMAXPROCS=1 go test -p=1 -c ./prog
real	0m42.290s
real	0m43.230s

$ time GOMAXPROCS=1 go install -p=1 ./tools/syz-execprog
real	0m24.337s
real	0m24.727s

$ TIME="%e %P %M" GOMAXPROCS=1 time go tool compile ./sys/linux/gen
19.11 100% 2764952
19.66 100% 2787624
19.35 100% 2749376

Update #1580
2020-04-26 05:58:31 +02:00
..
android Android: Add simple test harness for Sandbox 2018-10-12 16:39:26 +02:00
fops_probe tools: add fops_probe utility 2019-12-03 18:48:14 +01:00
kcovfuzzer tools/kcovfuzzer: add simple KCOV/libfuzzer glue 2019-09-19 15:56:24 +02:00
kcovtrace tools/kcovtrace: Fix the format of kcovtrace.c 2019-06-11 15:25:32 +02:00
syz-benchcmp tools/syz-benchcmp: update name of coverage stat 2019-04-08 15:10:07 +02:00
syz-bisect pkg/bisect: detect wrong bisections 2019-11-07 11:07:03 +01:00
syz-check pkg/compiler: deduplicate Types in descriptions 2020-04-26 05:58:31 +02:00
syz-cover pkg/cover: hierarchical coverage reports 2019-07-26 19:32:14 +02:00
syz-crush syz/targets: add SyzExecutorCmd flag 2019-12-09 15:13:49 -08:00
syz-db prog: introduce strict parsing mode 2018-12-10 16:37:01 +01:00
syz-declextract sys/windows: add more descriptions 2017-09-27 20:17:09 +02:00
syz-env tools/syz-env: restrict Makefile parallelism based on RAM 2019-07-19 09:51:07 +02:00
syz-execprog pkg/host: add Features.Supported 2019-11-16 09:58:54 +01:00
syz-expand tools: add syz-expand 2019-09-23 17:13:23 +02:00
syz-fmt pkg/osutil: provide better Rename 2018-12-17 19:09:10 +01:00
syz-headerparser docs: fix spelling errors 2020-01-28 17:23:51 +01:00
syz-imagegen all: fix too long lines 2018-05-05 16:00:01 +02:00
syz-mutate prog: control program length 2020-03-13 13:16:53 +01:00
syz-prog2c csource, executor: add usb emulation feature 2020-04-03 12:42:06 +02:00
syz-repro pkg/log: rename -v to -vv 2019-07-16 11:18:12 +02:00
syz-reprolist tools/syz-reprolist: cache errors 2020-03-03 09:35:16 +01:00
syz-runtest runtest: fix using configs with namespace/setuid sandbox 2020-04-03 12:42:06 +02:00
syz-showprio tools/syz-showprio: add a tool to visualize the priority table 2019-09-23 15:35:26 +02:00
syz-stress prog: control program length 2020-03-13 13:16:53 +01:00
syz-symbolize tools/syz-symbolize: Store crash report if outdir is given 2020-04-03 11:01:08 +02:00
syz-testbuild pkg/bisect: detect wrong bisections 2019-11-07 11:07:03 +01:00
syz-trace2syz sys/linux: add some more ipv4/6 addresses 2020-04-14 09:11:27 +02:00
syz-tty Fix incorrect argument in syz-tty (#260) 2017-06-27 16:55:11 +02:00
syz-upgrade prog: introduce strict parsing mode 2018-12-10 16:37:01 +01:00
syz-usbgen docs: link github.com/xairy/raw-gadget 2020-02-06 19:34:17 +01:00
check_links.py tools: avoid hard-coding the path to interpreters 2020-02-24 10:18:37 +01:00
check-copyright.sh tools: specify the search path for find(1) 2020-02-24 10:18:37 +01:00
create-gce-image.sh tools/create-gce-image.sh: fix sysctl.kernel.hung_task_all_cpu_backtrace=1 2020-03-31 15:28:17 +02:00
create-image.sh tools/create-image.sh: make sure filesystem root has correct permissions 2020-04-03 12:42:06 +02:00
create-openbsd-gce-ci.sh tools/create-openbsd*: update to 6.7 2020-04-17 07:27:39 +02:00
create-openbsd-vmm-worker.sh tools/create-openbsd*: update to 6.7 2020-04-17 07:27:39 +02:00
demo_setup.sh tools: improve demo_setup.sh 2018-04-07 12:33:23 +02:00