10 Commits

Author SHA1 Message Date
Francis Visoiu Mistrih
fcf99c532d Reland: [Remarks] Add an LLVM-bitstream-based remark serializer
Add a new serializer, using a binary format based on the LLVM bitstream
format.

This format provides a way to serialize the remarks in two modes:

1) Separate mode: the metadata is separate from the remark entries.
2) Standalone mode: the metadata and the remark entries are in the same
file.

The format contains:

* a meta block: container version, container type, string table,
external file path, remark version
* a remark block: type, remark name, pass name, function name, debug
file, debug line, debug column, hotness, arguments (key, value, debug
file, debug line, debug column)

A string table is required for this format, which will be dumped in the
meta block to be consumed before parsing the remark blocks.

On clang itself, we noticed a size reduction of 13.4x compared to YAML,
and a compile-time reduction of between 1.7% and 3.5% on CTMark.

Differential Revision: https://reviews.llvm.org/D63466

Original llvm-svn: 367364
Revert llvm-svn: 367370

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367372 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-31 00:13:51 +00:00
Francis Visoiu Mistrih
16956a6979 Revert "[Remarks] Add an LLVM-bitstream-based remark serializer"
This reverts commit r367364.

Breaks some bots: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-gn/builds/3161/steps/annotate/logs/stdio

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367370 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-31 00:01:34 +00:00
Francis Visoiu Mistrih
6038bea1b8 [Remarks] Add an LLVM-bitstream-based remark serializer
Add a new serializer, using a binary format based on the LLVM bitstream
format.

This format provides a way to serialize the remarks in two modes:

1) Separate mode: the metadata is separate from the remark entries.
2) Standalone mode: the metadata and the remark entries are in the same
file.

The format contains:

* a meta block: container version, container type, string table,
external file path, remark version
* a remark block: type, remark name, pass name, function name, debug
file, debug line, debug column, hotness, arguments (key, value, debug
file, debug line, debug column)

A string table is required for this format, which will be dumped in the
meta block to be consumed before parsing the remark blocks.

On clang itself, we noticed a size reduction of 13.4x compared to YAML,
and a compile-time reduction of between 1.7% and 3.5% on CTMark.

Differential Revision: https://reviews.llvm.org/D63466

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367364 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-30 23:11:57 +00:00
Francis Visoiu Mistrih
5093e0a4be [Remarks] Add two serialization modes for remarks: separate and standalone
The default mode is separate, where the metadata is serialized
separately from the remarks.

Another mode is the standalone mode, where the metadata is serialized
before the remarks, on the same stream.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367328 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-30 16:01:40 +00:00
Francis Visoiu Mistrih
02582ef3db Reland: [Remarks] Support parsing remark metadata in the YAML remark parser
This adds support to the yaml remark parser to be able to parse remarks
directly from the metadata.

This supports parsing separate metadata and following the external file
with the associated metadata, and also a standalone file containing
metadata + remarks all together.

Original llvm-svn: 367148
Revert llvm-svn: 367151

This has a fix for gcc builds.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367155 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-26 21:02:02 +00:00
Francis Visoiu Mistrih
a0eba5f776 Revert "[Remarks] Support parsing remark metadata in the YAML remark parser"
This reverts r367148.

Seems to fail on
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/27768.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367151 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-26 20:54:44 +00:00
Francis Visoiu Mistrih
aae2cef176 [Remarks] Support parsing remark metadata in the YAML remark parser
This adds support to the yaml remark parser to be able to parse remarks
directly from the metadata.

This supports parsing separate metadata and following the external file
with the associated metadata, and also a standalone file containing
metadata + remarks all together.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@367148 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-26 20:11:53 +00:00
Francis Visoiu Mistrih
f802102dc4 [Remarks] Introduce a new format: yaml-strtab
This exposes better support to use a string table with a format through
an actual new remark::Format, called yaml-strtab.

This can now be used with -fsave-optimization-record=yaml-strtab.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366849 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-23 20:42:46 +00:00
Francis Visoiu Mistrih
00c0e0040f [Remarks] Simplify and refactor the RemarkParser interface
Before, everything was based on some kind of type erased parser
implementation which container a lot of boilerplate code when multiple
formats were to be supported.

This simplifies it by:

* the remark now owns its arguments
* *always* returning an error from the implementation side
* working around the way the YAML parser reports errors: catch them through
callbacks and re-insert them in a proper llvm::Error
* add a CParser wrapper that is used when implementing the C API to
avoid cluttering the C++ API with useless state
* LLVMRemarkParserGetNext now returns an object that needs to be
released to avoid leaking resources
* add a new API to dispose of a remark entry: LLVMRemarkEntryDispose

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@366217 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-16 15:25:05 +00:00
Francis Visoiu Mistrih
498e1aac85 [docs][Remarks] Add documentation for remarks in LLVM
This adds documentation that describes remarks in LLVM.

It aims at explaining what remarks are, how to enable them, and what
users can do with the different modes.

It lists all the available flags in LLVM (excluding clang), and
describes the expected YAML structure as well as the tools that support
the YAML format today.

Differential Revision: https://reviews.llvm.org/D64355

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365578 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-09 23:16:59 +00:00