This adds various new functionality and cleanup surrounding the
use of the Stream library. Major changes include:
* Renaming of all classes for more consistency / meaningfulness
* Addition of some new methods for reading multiple values at once.
* Full suite of unit tests for reader / writer functionality.
* Full set of doxygen comments for all classes.
* Streams now store their own endianness.
* Fixed some bugs in a few of the classes that were discovered
by the unit tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296215 91177308-0d34-0410-b5e6-96231b3b80d8
This is part of a larger effort to get the Stream code moved
up to Support. I don't want to do it in one large patch, in
part because the changes are so big that it will treat everything
as file deletions and add, losing history in the process.
Aside from that though, it's just a good idea in general to
make small changes.
So this change only changes the names of the Stream related
source files, and applies necessary source fix ups.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296211 91177308-0d34-0410-b5e6-96231b3b80d8
This introduces the `analyze` subcommand. For now there is only
one option, to analyze hash collisions in the type streams. In
the future, however, we could add many more things here, such
as performing size analyses, compacting, and statistics about
the type of records etc.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293795 91177308-0d34-0410-b5e6-96231b3b80d8
This is not a list of pairs, it is a hash table data structure. We now
correctly parse this out and dump it from llvm-pdbdump.
We still need to understand the conditions that lead to a type
getting an entry in the hash adjuster table. That will be done
in a followup investigation / patch.
Differential Revision: https://reviews.llvm.org/D29090
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293090 91177308-0d34-0410-b5e6-96231b3b80d8
While the builder pattern has proven useful for certain other
larger types, in this case it was hampering the ability to use
the data structure, as for runtime access we need a map that
we can efficiently read from and write to. So the two are merged
into a single data structure that can efficiently be read to,
written from, deserialized from bytes, and serialized to bytes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292664 91177308-0d34-0410-b5e6-96231b3b80d8
Previously the type dumper itself was passed around to a lot of different
places and manipulated in ways that were more appropriate on the type
database. For example, the entire TypeDumper was passed into the symbol
dumper, when all the symbol dumper wanted to do was lookup the name of a
TypeIndex so it could print it. That's what the TypeDatabase is for --
mapping type indices to names.
Another example is how if the user runs llvm-pdbdump with the option to
dump symbols but not types, we still have to visit all types so that we
can print minimal information about the type of a symbol, but just without
dumping full symbol records. The way we did this before is by hacking it
up so that we run everything through the type dumper with a null printer,
so that the output goes to /dev/null. But really, we don't need to dump
anything, all we want to do is build the type database. Since
TypeDatabaseVisitor now exists independently of TypeDumper, we can do
this. We just build a custom visitor callback pipeline that includes a
database visitor but not a dumper.
All the hackery around printers etc goes away. After this patch, we could
probably even delete the entire CVTypeDumper class since really all it is
at this point is a thin wrapper that hides the details of how to build a
useful visitation pipeline. It's not a priority though, so CVTypeDumper
remains for now.
After this patch we will be able to easily plug in a different style of
type dumper by only implementing the proper visitation methods to dump
one-line output and then sticking it on the pipeline.
Differential Revision: https://reviews.llvm.org/D28524
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291724 91177308-0d34-0410-b5e6-96231b3b80d8
We were starting to get some name clashes between llvm-pdbdump
and the common CodeView framework, so I took this opportunity
to rename a bunch of files to more accurately describe their
usage. This also helps in llvm-pdbdump to distinguish
between different files and whether they are used for pretty
dump mode or raw dump mode.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291627 91177308-0d34-0410-b5e6-96231b3b80d8
Since we type-erase the Windows GUID structure, use unsigned bytes
rather than char, which may be signed (-fsigned-char). NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290765 91177308-0d34-0410-b5e6-96231b3b80d8
The original patch was broken due to some undefined behavior
as well as warnings that were triggering -Werror.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290000 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: The code we use to read PDBs assumed that streams we ask it to read exist, and would read memory outside a vector and crash if this wasn't the case. This would, for example, cause llvm-pdbdump to crash on PDBs generated by lld. This patch handles such cases more gracefully: the PDB reading code in LLVM now reports errors when asked to get a stream that is not present, and llvm-pdbdump will report missing streams and continue processing streams that are present.
Reviewers: ruiu, zturner
Subscribers: thakis, amccarth
Differential Revision: https://reviews.llvm.org/D27325
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@288722 91177308-0d34-0410-b5e6-96231b3b80d8
Previously support had been added for using CodeViewRecordIO
to read (deserialize) CodeView type records. This patch adds
support for writing those same records. With this patch,
reading and writing of CodeView type records finally uses a single
codepath.
Differential Revision: https://reviews.llvm.org/D26253
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@286304 91177308-0d34-0410-b5e6-96231b3b80d8
Using a pattern similar to that of YamlIO, this allows
us to have a single codepath for translating codeview
records to and from serialized byte streams. The
current patch only hooks this up to the reading of
CodeView type records. A subsequent patch will hook
it up for writing of CodeView type records, and then a
third patch will hook up the reading and writing of
CodeView symbols.
Differential Revision: https://reviews.llvm.org/D26040
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@285836 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: This adds support for dumping the globals stream from PDB files using llvm-pdbdump, similar to the support we have for the publics stream.
Reviewers: ruiu, zturner
Subscribers: beanz, mgorny, modocache
Differential Revision: https://reviews.llvm.org/D25801
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284861 91177308-0d34-0410-b5e6-96231b3b80d8
This is the first step towards round-tripping symbol information,
and thusly being able to write symbol information to a PDB.
This patch writes the symbol information for each compiland to
the Yaml when running in pdb2yaml mode. There's still some loose
ends, such as what to do about relocations (necessary in order to
print linkage names), how to print enums with friendly names, and
how to give the dumper access to the StringTable, but this is a
good first start.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283641 91177308-0d34-0410-b5e6-96231b3b80d8
Type visitor code had already been refactored previously to
decouple the visitor and the visitor callback interface. This
was necessary for having the flexibility to visit in different
ways (for example, dumping to yaml, reading from yaml, dumping
to ScopedPrinter, etc).
This patch merely implements the same visitation pattern for
symbol records that has already been implemented for type records.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@283609 91177308-0d34-0410-b5e6-96231b3b80d8
When we create a PDB file using PDBFileBuilder, the information
in the superblock, such as the size of the resulting file, is not
available.
Previously, PDBFileBuilder::initialize took a superblock assuming
that all the members of the struct are correct. That is useful when
you want to restore the exact information from a YAML file, but
that's probably the only use case in which that is useful.
When we are creating a PDB file on the fly, we have to backfill the
members.
This patch redefines PDBFileBuilder::initialize to take only a
block size. Now all the other members are left as default values,
so that they'll be updated when commit() is called.
Differential Revision: https://reviews.llvm.org/D25108
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282944 91177308-0d34-0410-b5e6-96231b3b80d8
WritableStream needs the exact file size to open a file, but
until we fix the final layout of a PDB file, we don't know the
size of the file.
This patch changes the parameter type of PDBFileBuilder::commit
to solve that chiecken-and-egg problem. Now the function opens
a file after fixing the layout, so it can create a file with the
exact size.
Differential Revision: https://reviews.llvm.org/D25107
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@282940 91177308-0d34-0410-b5e6-96231b3b80d8
The IPI stream is structurally identical to the TPI stream, but it
contains different record types. So we just re-use the TPI writing
code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281638 91177308-0d34-0410-b5e6-96231b3b80d8
The `CVType` had two redundant fields which were confusing and
error-prone to fill out. By treating member records as a distinct
type from leaf records, we are able to simplify this quite a bit.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D24432
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281556 91177308-0d34-0410-b5e6-96231b3b80d8
We have various command line options that print the type of a
stream, the size of a stream, etc but nowhere that it can all be
viewed together.
Since a previous patch introduced the ability to dump the bytes
of a stream, this seems like a good place to present a full view
of the stream's properties including its size, what kind of data
it represents, and the blocks it occupies. So I added the
ability to print that information to the -stream-data command
line option.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281077 91177308-0d34-0410-b5e6-96231b3b80d8
I ran into a situation where I wanted to print out the contents of
page 6 of a PDB as a binary blob, and there was no straightforward
way to do that.
In addition to adding that, this patch also adds the ability to dump
a stream by index as a binary blob, and it will stitch together all
the blocks and dump the whole thing as one seemingly contiguous
sequence of bytes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281070 91177308-0d34-0410-b5e6-96231b3b80d8
This simplifies a lot of code, and will actually be necessary for
an upcoming patch to serialize TPI record hash values.
The idea before was that visitors should be examining records, not
modifying them. But this is no longer true with a visitor that
constructs a CVRecord from Yaml. To handle this until now, we
were doing some fixups on CVRecord objects at a higher level, but
the code is really awkward, and it makes sense to just have the
visitor write the bytes into the CVRecord. In doing so I uncovered
a few bugs related to `Data` and `RawData` and fixed those.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D24362
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281067 91177308-0d34-0410-b5e6-96231b3b80d8
Previously we were making new instances of YamlTypeDumperCallbacks
in order to recurse down and serialize / deserialize nested
records such as field lists. This meant you could not pass
context from a higher operation to a lower operation because
it would be using a new instance of the visitor callback
delegate.
YAMLIO library was updated to support context-sensitive mappings,
so now we can reuse the same instance of the visitor callback
delegate even for nested operations.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280978 91177308-0d34-0410-b5e6-96231b3b80d8
This was originally submitted in r280549, and reverted in r280577
due to breaking one MSVC buildbot. The issue is that MSVC 2013
doesn't synthesize move constructors. So even though i was
writing std::move(A) it was copying it, leading to a bogus ArrayRef.
The solution here is to simply remove the std::vector<> from the
type, since it is unused and unnecessary. This way the ArrayRef
continues to point into the original memory backing the CVType.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280769 91177308-0d34-0410-b5e6-96231b3b80d8
Before we were kind of imitating the behavior of a Yaml sequence
by outputting each record one after the other. This makes it a
little cumbersome when we want to go the other direction -- from
Yaml to Pdb. So this treats FieldList records as no different than
any other list of records, by printing them as a Yaml sequence with
the exact same format.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280549 91177308-0d34-0410-b5e6-96231b3b80d8
Previously we were assuming that any visitation of types would
necessarily be against a type we had binary data for. Reasonable
assumption when were just reading PDBs and dumping them, but once
we start writing PDBs from Yaml this breaks down, because we have
no binary data yet, only Yaml, and from that we need to read the
record kind and perform the switch based on that.
So this patch does that. Instead of having the visitor switch
on the kind that is already in the CVType record, we change the
visitTypeBegin() method to return the Kind, and switch on the
returned value. This way, the default implementation can still
return the value from the CVType, but the implementation which
visits Yaml records and serializes binary PDB type records can
use the field in the Yaml as the source of the switch.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280307 91177308-0d34-0410-b5e6-96231b3b80d8
We were kind of hacking this together before by embedding the
ability to forward requests into the TypeDeserializer. When
we want to start adding more different kinds of visitor callback
interfaces though, this doesn't scale well and is very inflexible.
So introduce the notion of a pipeline, which itself implements
the TypeVisitorCallbacks interface, but which contains an internal
list of other callbacks to invoke in sequence.
Also update the existing uses of CVTypeVisitor to use this new
pipeline class for deserializing records before visiting them
with another visitor.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280293 91177308-0d34-0410-b5e6-96231b3b80d8
The original patch was breaking some buildbots due to an
incorrect ordering of function definitions which caused some
compilers to recognize a definition but others to not.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279089 91177308-0d34-0410-b5e6-96231b3b80d8
llvm-pdbdump already had code to retrieve column information in the line tables, but it wasn't using it.
Most Microsoft PDBs don't seem to have column info, so this wasn't missed. But Clang includes column info by default (at least for now), and being able to see that is useful for ensuring we get the column info correct.
Differential Revision: https://reviews.llvm.org/D23629
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@279001 91177308-0d34-0410-b5e6-96231b3b80d8
This is hitting a "use of undeclared identifier 'skipPadding' error
locally and on some bots.
This reverts r278869.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@278871 91177308-0d34-0410-b5e6-96231b3b80d8
Until now, our use case for the visitor has been to take a stream of bytes
representing a type stream, deserialize the records in sequence, and do
something with them, where "something" is determined by how the user
implements a particular set of callbacks on an abstract class.
For actually writing PDBs, however, we want to do the reverse. We have
some kind of description of the list of records in their in-memory format,
and we want to process each one. Perhaps by serializing them to a byte
stream, or perhaps by converting them from one description format (Yaml)
to another (in-memory representation).
This was difficult in the current model because deserialization and
invoking the callbacks were tightly coupled.
With this patch we change this so that TypeDeserializer is itself an
implementation of the particular set of callbacks. This decouples
deserialization from the iteration over a list of records and invocation
of the callbacks. TypeDeserializer is initialized with another
implementation of the callback interface, so that upon deserialization it
can pass the deserialized record through to the next set of callbacks. In
a sense this is like an implementation of the Decorator design pattern,
where the Deserializer is a decorator.
This will be useful for writing Pdbs from yaml, where we have a
description of the type records in Yaml format. In this case, the visitor
implementation would have each visitation callback method implemented in
such a way as to extract the proper set of fields from the Yaml, and it
could maintain state that builds up a list of these records. Finally at
the end we can pass this information through to another set of callbacks
which serializes them into a byte stream.
Reviewed By: majnemer, ruiu, rnk
Differential Revision: https://reviews.llvm.org/D23177
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277871 91177308-0d34-0410-b5e6-96231b3b80d8
MappedBlockSTream can work with any sequence of block data where
the ordering is specified by a list of block numbers. So rather
than manually stitch them together in the case of the FPM, reuse
this functionality so that we can treat the FPM as if it were
contiguous.
Reviewed By: ruiu
Differential Revision: https://reviews.llvm.org/D23066
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277609 91177308-0d34-0410-b5e6-96231b3b80d8
I examined a few PDBs and all of them treated pages for stream 0
are unused, thus they were unmarked in their free page bitmap.
I think we should do the same thing for compatibility.
Differential Revision: https://reviews.llvm.org/D23047
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277545 91177308-0d34-0410-b5e6-96231b3b80d8
The FPM is split at regular intervals across the MSF file, as the MS code
suggests. It turns out that the value of the interval is precisely the
block size. If the block size is 4096, then there are two Fpm pages every
4096 blocks.
So here we teach the PDBFile class to parse a split FPM, and also add more
options when dumping the FPM to display some additional information such
as orphaned pages (pages which the FPM says are allocated, but which
nothing appears to use), use after free pages (pages which the FPM says
are not allocated, but which are referenced by a stream), and multiple use
pages (pages which the FPM says are allocated but are used more than
once).
Reviewed By: ruiu
Differential Revision: https://reviews.llvm.org/D23022
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277388 91177308-0d34-0410-b5e6-96231b3b80d8
Previously this change was submitted from a Windows machine, so
changes made to the case of filenames and directory names did
not survive the commit, and as a result the CMake source file
names and the on-disk file names did not match on case-sensitive
file systems.
I'm resubmitting this patch from a Linux system, which hopefully
allows the case changes to make it through unfettered.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277213 91177308-0d34-0410-b5e6-96231b3b80d8
