27 Commits

Author SHA1 Message Date
Francis Visoiu Mistrih
f8806e59f5 [Remarks] Allow the RemarkStreamer to be used directly with a stream
The filename in the RemarkStreamer should be optional to allow clients
to stream remarks to memory or to existing streams.

This introduces a new overload of `setupOptimizationRemarks`, and avoids
enforcing the presence of a filename at different places.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@372195 91177308-0d34-0410-b5e6-96231b3b80d8
2019-09-18 01:04:45 +00:00
Jonas Devlieghere
114087caa6 [llvm] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@369013 91177308-0d34-0410-b5e6-96231b3b80d8
2019-08-15 15:54:37 +00:00
Fangrui Song
2ffd08308b Rename F_{None,Text,Append} to OF_{None,Text,Append}. NFC
F_{None,Text,Append} are kept for compatibility since r334221.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367800 91177308-0d34-0410-b5e6-96231b3b80d8
2019-08-05 05:43:48 +00:00
Francis Visoiu Mistrih
fcf99c532d Reland: [Remarks] Add an LLVM-bitstream-based remark serializer
Add a new serializer, using a binary format based on the LLVM bitstream
format.

This format provides a way to serialize the remarks in two modes:

1) Separate mode: the metadata is separate from the remark entries.
2) Standalone mode: the metadata and the remark entries are in the same
file.

The format contains:

* a meta block: container version, container type, string table,
external file path, remark version
* a remark block: type, remark name, pass name, function name, debug
file, debug line, debug column, hotness, arguments (key, value, debug
file, debug line, debug column)

A string table is required for this format, which will be dumped in the
meta block to be consumed before parsing the remark blocks.

On clang itself, we noticed a size reduction of 13.4x compared to YAML,
and a compile-time reduction of between 1.7% and 3.5% on CTMark.

Differential Revision: https://reviews.llvm.org/D63466

Original llvm-svn: 367364
Revert llvm-svn: 367370

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367372 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-31 00:13:51 +00:00
Francis Visoiu Mistrih
16956a6979 Revert "[Remarks] Add an LLVM-bitstream-based remark serializer"
This reverts commit r367364.

Breaks some bots: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-gn/builds/3161/steps/annotate/logs/stdio

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367370 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-31 00:01:34 +00:00
Francis Visoiu Mistrih
6038bea1b8 [Remarks] Add an LLVM-bitstream-based remark serializer
Add a new serializer, using a binary format based on the LLVM bitstream
format.

This format provides a way to serialize the remarks in two modes:

1) Separate mode: the metadata is separate from the remark entries.
2) Standalone mode: the metadata and the remark entries are in the same
file.

The format contains:

* a meta block: container version, container type, string table,
external file path, remark version
* a remark block: type, remark name, pass name, function name, debug
file, debug line, debug column, hotness, arguments (key, value, debug
file, debug line, debug column)

A string table is required for this format, which will be dumped in the
meta block to be consumed before parsing the remark blocks.

On clang itself, we noticed a size reduction of 13.4x compared to YAML,
and a compile-time reduction of between 1.7% and 3.5% on CTMark.

Differential Revision: https://reviews.llvm.org/D63466

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367364 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-30 23:11:57 +00:00
Francis Visoiu Mistrih
5093e0a4be [Remarks] Add two serialization modes for remarks: separate and standalone
The default mode is separate, where the metadata is serialized
separately from the remarks.

Another mode is the standalone mode, where the metadata is serialized
before the remarks, on the same stream.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367328 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-30 16:01:40 +00:00
Francis Visoiu Mistrih
6722c51b5b [Remarks][NFC] Rename remarks::Serializer to remarks::RemarkSerializer
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366939 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-24 19:47:57 +00:00
Francis Visoiu Mistrih
0294e50fe5 [Remarks] Simplify the creation of remark serializers
Introduce two new functions to create a serializer, and add support for
more combinations to the YAMLStrTabSerializer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366919 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-24 16:36:35 +00:00
Simon Pilgrim
4a5da46bcb Fix "control reaches end of non-void function" warning. NFCI.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366856 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-23 21:59:48 +00:00
Francis Visoiu Mistrih
f802102dc4 [Remarks] Introduce a new format: yaml-strtab
This exposes better support to use a string table with a format through
an actual new remark::Format, called yaml-strtab.

This can now be used with -fsave-optimization-record=yaml-strtab.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366849 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-23 20:42:46 +00:00
Francis Visoiu Mistrih
b06f154406 [Remarks][NFC] Move the YAML serializer to its own header
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366842 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-23 19:28:03 +00:00
Francis Visoiu Mistrih
00c0e0040f [Remarks] Simplify and refactor the RemarkParser interface
Before, everything was based on some kind of type erased parser
implementation which container a lot of boilerplate code when multiple
formats were to be supported.

This simplifies it by:

* the remark now owns its arguments
* *always* returning an error from the implementation side
* working around the way the YAML parser reports errors: catch them through
callbacks and re-insert them in a proper llvm::Error
* add a CParser wrapper that is used when implementing the C API to
avoid cluttering the C++ API with useless state
* LLVMRemarkParserGetNext now returns an object that needs to be
released to avoid leaking resources
* add a new API to dispose of a remark entry: LLVMRemarkEntryDispose

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366217 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-16 15:25:05 +00:00
Francis Visoiu Mistrih
ce4ba232b0 [Remarks][NFC] Combine ParserFormat and SerializerFormat
It's useless to have both.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366216 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-16 15:24:59 +00:00
Francis Visoiu Mistrih
1cb383ef62 [Remarks] Extend -fsave-optimization-record to specify the format
Use -fsave-optimization-record=<format> to specify a different format
than the default, which is YAML.

For now, only YAML is supported.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363573 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-17 16:06:00 +00:00
Francis Visoiu Mistrih
b5630562d8 [Remarks] Use the RemarkSetup error in setupOptimizationRemarks
Added the errors in r363415 but they were not used in the
RemarkStreamer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363439 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-14 18:18:26 +00:00
Francis Visoiu Mistrih
6058aa9ef5 Reland: [Remarks] Refactor optimization remarks setup
* Add a common function to setup opt-remarks
* Rename common options to the same names
* Add error types to distinguish between file errors and regex errors

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363415 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-14 16:20:51 +00:00
Francis Visoiu Mistrih
0f00485ae2 Revert "[Remarks] Refactor optimization remarks setup"
This reverts commit 6e6e3af55bb97e1a4c97375c15a2b0099120c5a7.

This breaks greendragon.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363343 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-14 00:05:56 +00:00
Francis Visoiu Mistrih
eb1d34aaa7 [Remarks] Refactor optimization remarks setup
* Add a common function to setup opt-remarks
* Rename common options to the same names
* Add error types to distinguish between file errors and regex errors

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363328 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-13 21:46:57 +00:00
Francis Visoiu Mistrih
890aba409a [Remarks][NFC] Move the serialization to lib/Remarks
Separate the remark serialization to YAML from the LLVM Diagnostics.

This adds a new serialization abstraction: remarks::Serializer. It's
completely independent from lib/IR and it provides an easy way to
replace YAML by providing a new remarks::Serializer.

Differential Revision: https://reviews.llvm.org/D62632

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@362160 91177308-0d34-0410-b5e6-96231b3b80d8
2019-05-30 21:45:59 +00:00
Francis Visoiu Mistrih
04dcbb5165 [Remarks] Add string deduplication using a string table
* Add support for uniquing strings in the remark streamer and emitting the string table in the remarks section.

* Add parsing support for the string table in the RemarkParser.

From this remark:

```
--- !Missed
Pass:     inline
Name:     NoDefinition
DebugLoc: { File: 'test-suite/SingleSource/UnitTests/2002-04-17-PrintfChar.c',
            Line: 7, Column: 3 }
Function: printArgsNoRet
Args:
  - Callee:   printf
  - String:   ' will not be inlined into '
  - Caller:   printArgsNoRet
    DebugLoc: { File: 'test-suite/SingleSource/UnitTests/2002-04-17-PrintfChar.c',
                Line: 6, Column: 0 }
  - String:   ' because its definition is unavailable'
...
```

to:

```
--- !Missed
Pass: 0
Name: 1
DebugLoc: { File: 3, Line: 7, Column: 3 }
Function: 2
Args:
  - Callee:   4
  - String:   5
  - Caller:   2
    DebugLoc: { File: 3, Line: 6, Column: 0 }
  - String:   6
...
```

And the string table in the .remarks/__remarks section containing:

```
inline\0NoDefinition\0printArgsNoRet\0
test-suite/SingleSource/UnitTests/2002-04-17-PrintfChar.c\0printf\0
will not be inlined into \0 because its definition is unavailable\0
```

This is mostly supposed to be used for testing purposes, but it gives us
a 2x reduction in the remark size, and is an incremental change for the
updates to the remarks file format.

Differential Revision: https://reviews.llvm.org/D60227

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@359050 91177308-0d34-0410-b5e6-96231b3b80d8
2019-04-24 00:06:24 +00:00
Francis Visoiu Mistrih
1e252811d6 Reland "[Remarks] Add -foptimization-record-passes to filter remark emission"
Currently we have -Rpass for filtering the remarks that are displayed as
diagnostics, but when using -fsave-optimization-record, there is no way
to filter the remarks while generating them.

This adds support for filtering remarks by passes using a regex.
Ex: `clang -fsave-optimization-record -foptimization-record-passes=inline`

will only emit the remarks coming from the pass `inline`.

This adds:

* `-fsave-optimization-record` to the driver
* `-opt-record-passes` to cc1
* `-lto-pass-remarks-filter` to the LTOCodeGenerator
* `--opt-remarks-passes` to lld
* `-pass-remarks-filter` to llc, opt, llvm-lto, llvm-lto2
* `-opt-remarks-passes` to gold-plugin

Differential Revision: https://reviews.llvm.org/D59268

Original llvm-svn: 355964

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355984 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-12 21:22:27 +00:00
Francis Visoiu Mistrih
a826459b5e Revert "[Remarks] Add -foptimization-record-passes to filter remark emission"
This reverts commit 20fff32b7d1f1a1bd417b22aa9f26ededd97a3e5.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355976 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-12 20:54:18 +00:00
Francis Visoiu Mistrih
1bd774409d [Remarks] Add -foptimization-record-passes to filter remark emission
Currently we have -Rpass for filtering the remarks that are displayed as
diagnostics, but when using -fsave-optimization-record, there is no way
to filter the remarks while generating them.

This adds support for filtering remarks by passes using a regex.
Ex: `clang -fsave-optimization-record -foptimization-record-passes=inline`

will only emit the remarks coming from the pass `inline`.

This adds:

* `-fsave-optimization-record` to the driver
* `-opt-record-passes` to cc1
* `-lto-pass-remarks-filter` to the LTOCodeGenerator
* `--opt-remarks-passes` to lld
* `-pass-remarks-filter` to llc, opt, llvm-lto, llvm-lto2
* `-opt-remarks-passes` to gold-plugin

Differential Revision: https://reviews.llvm.org/D59268

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355964 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-12 20:28:50 +00:00
Francis Visoiu Mistrih
78ee34a412 Reland "[Remarks] Refactor remark diagnostic emission in a RemarkStreamer"
This allows us to store more info about where we're emitting the remarks
without cluttering LLVMContext. This is needed for future support for
the remark section.

Differential Revision: https://reviews.llvm.org/D58996

Original llvm-svn: 355507

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355514 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-06 15:20:13 +00:00
Francis Visoiu Mistrih
4107715f75 Revert "[Remarks] Refactor remark diagnostic emission in a RemarkStreamer"
This reverts commit 2e8c4997a2089f8228c843fd81b148d903472e02.

Breaks bots.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355511 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-06 14:52:37 +00:00
Francis Visoiu Mistrih
aa137aa9ea [Remarks] Refactor remark diagnostic emission in a RemarkStreamer
This allows us to store more info about where we're emitting the remarks
without cluttering LLVMContext. This is needed for future support for
the remark section.

Differential Revision: https://reviews.llvm.org/D58996

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355507 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-06 14:32:08 +00:00