Summary:
This considers module symbol streams and the global symbol stream to be
roots. Most types that this considers "unreferenced" are referenced by
LF_UDT_MOD_SRC_LINE id records, which VC seems to always include.
Essentially, they are types that the user can only find in the debugger
if they call them by name, they cannot be found by traversing a symbol.
In practice, around 80% of type information in a PDB is referenced by a
symbol. That seems like a reasonable number.
I don't really plan to do anything with this tool. It mostly just exists
for informational purposes, and to confirm that we probably don't need
to implement type reference tracking in LLD. We can continue to merge
all types as we do today without wasting space.
Reviewers: zturner, aganea
Subscribers: mgorny, hiraditya, arphaman, jdoerfert, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D59620
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@356692 91177308-0d34-0410-b5e6-96231b3b80d8
VarStreamArray was built on the assumption that it is backed by a
StreamRef, and offset 0 of that StreamRef is the first byte of the first
record in the array.
This is a logical and intuitive assumption, but unfortunately we have
use cases where it doesn't hold. Specifically, a PDB module's symbol
stream is prefixed by 4 bytes containing a magic value, and the first
byte of record data in the array is actually at offset 4 of this byte
sequence.
Previously, we would just truncate the first 4 bytes and then construct
the VarStreamArray with the resulting StreamRef, so that offset 0 of the
underlying stream did correspond to the first byte of the first record,
but this is problematic, because symbol records reference other symbol
records by the absolute offset including that initial magic 4 bytes. So
if another record wants to refer to the first record in the array, it
would say "the record at offset 4".
This led to extremely confusing hacks and semantics in loading code, and
after spending 30 minutes trying to get some math right and failing, I
decided to fix this in the underlying implementation of VarStreamArray.
Now, we can say that a stream is skewed by a particular amount. This
way, when we access a record by absolute offset, we can use the same
values that the records themselves contain, instead of having to do
fixups.
Differential Revision: https://reviews.llvm.org/D55344
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@348499 91177308-0d34-0410-b5e6-96231b3b80d8
When we're on the last bucket the computation is tricky.
We were failing when the last bucket contained multiple
matches. Added a new test for this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@344081 91177308-0d34-0410-b5e6-96231b3b80d8
When we're looking up a record in the last hash bucket chain, we
need to be careful with the end-offset calculation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@344001 91177308-0d34-0410-b5e6-96231b3b80d8
This works on Windows, but seems to be breaking tests that
use an external shell (e.g. bash) because backquote has special
meaning.
This particular argument wasn't crucial for the test, so I've
just removed it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343971 91177308-0d34-0410-b5e6-96231b3b80d8
The Globals table is a hash table keyed on symbol name, so
it's possible to lookup symbols by name in O(1) time. Add
a function to the globals stream to do this, and add an option
to llvm-pdbutil to exercise this, then use it to write some
tests to verify correctness.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343951 91177308-0d34-0410-b5e6-96231b3b80d8
These work a little differently because they are actually in
the globals stream and are treated as symbol records, even though
DIA presents them as types. So this also adds the necessary
infrastructure to cache records that live somewhere other than
the TPI stream as well.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343507 91177308-0d34-0410-b5e6-96231b3b80d8
We added support for dumping pointers but pointers to arrays
won't correctly dump until we add support for dumping arrays.
Instead of trying to dump everything, which this test isn't
even interested in, just dump enums and typedefs.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343398 91177308-0d34-0410-b5e6-96231b3b80d8
We didn't properly detect when a pointer was a member
pointer, and when that was the case we were not
properly returning class parent info. This caused
member pointers to render incorrectly in pretty mode.
However, we didn't even have pretty tests for pointers
in native mode, so those are also added now to ensure
this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343393 91177308-0d34-0410-b5e6-96231b3b80d8
This adds support for parsing function signature records and returning
them through the native DIA interface.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342780 91177308-0d34-0410-b5e6-96231b3b80d8
This allows the native reader to find records of class/struct/
union type and dump them. This behavior is tested by using the
diadump subcommand against golden output produced by actual DIA
SDK on the same PDB file, and again using pretty -native to
confirm that we actually dump the classes. We don't find class
members or anything like that yet, for now it's just the class
itself.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342779 91177308-0d34-0410-b5e6-96231b3b80d8
Some records point to an LF_CLASS, LF_UNION, LF_STRUCTURE, or LF_ENUM
which is a forward reference and doesn't contain complete debug
information. In these cases, we'd like to be able to quickly locate the
full record. The TPI stream stores an array of pre-computed record hash
values, one for each type record. If we pre-process this on startup, we
can build a mapping from hash value -> {list of possible matching type
indices}. Since hashes of full records are only based on the name and or
unique name and not the full record contents, we can then use forward
ref record to compute the hash of what *would* be the full record by
just hashing the name, use this to get the list of possible matches, and
iterate those looking for a match on name or unique name.
llvm-pdbutil is updated to resolve forward references for the purposes
of testing (plus it's just useful).
Differential Revision: https://reviews.llvm.org/D52283
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342656 91177308-0d34-0410-b5e6-96231b3b80d8
There were several issues with the previous implementation.
1) There were no tests.
2) We didn't support creating PDBSymbolTypePointer records for
builtin types since those aren't described by LF_POINTER
records.
3) We didn't support a wide enough variety of builtin types even
ignoring pointers.
This patch fixes all of these issues. In order to add tests,
it's helpful to be able to ignore the symbol index id hierarchy
because it makes the golden output from the DIA version not match
our output, so I've extended the dumper to disable dumping of id
fields.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342493 91177308-0d34-0410-b5e6-96231b3b80d8
Previously we would dump the names of enum types, but not their
enumerator values. This adds support for enumerator values. In
doing so, we have to introduce a general purpose mechanism for
caching symbol indices of field list members. Unlike global
types, FieldList members do not have a TypeIndex. So instead,
we identify them by the pair {TypeIndexOfFieldList, IndexInFieldList}.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342415 91177308-0d34-0410-b5e6-96231b3b80d8
Previously for cv-qualified types, we would just ignore them
and they would never get printed. Now we can enumerate them
and cache them like any other symbol type.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342414 91177308-0d34-0410-b5e6-96231b3b80d8
Currently if we got something like `const Foo` we'd ignore it and
just rely on printing the unmodified `Foo` later on. However,
for testing the native reading code we really would like to be able
to see these so that we can verify that the native reader can
actually handle them. Instead of printing out the full type though,
just print out the header.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342295 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
There are two registers encoded in the S_FRAMEPROC flags: one for locals
and one for parameters. The encoding is described by the
ExpandEncodedBasePointerReg function in cvinfo.h. Two bits are used to
indicate one of four possible values:
0: no register - Used when there are no variables.
1: SP / standard - Variables are stored relative to the standard SP
for the ISA.
2: FP - Variables are addressed relative to the ISA frame
pointer, i.e. EBP on x86. If realignment is required, parameters
use this. If a dynamic alloca is used, locals will be EBP relative.
3: Alternative - Variables are stored relative to some alternative
third callee-saved register. This is required to address highly
aligned locals when there are dynamic stack adjustments. In this
case, both the incoming SP saved in the standard FP and the current
SP are at some dynamic offset from the locals. LLVM uses ESI in
this case, MSVC uses EBX.
Most of the changes in this patch are to pass around the CPU so that we
can decode these into real, named architectural registers.
Subscribers: hiraditya
Differential Revision: https://reviews.llvm.org/D51894
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341999 91177308-0d34-0410-b5e6-96231b3b80d8
They were unintentionally calling DIA directly, which requires
Windows. We need to pass the -native flag, and this then required
fixing up one or two tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341731 91177308-0d34-0410-b5e6-96231b3b80d8
In order to start testing this, I've added a new mode to
llvm-pdbutil which is only really useful for writing tests.
It just dumps the value of raw fields in record format.
This isn't really ideal and it won't allow us to test some
important cases, but it's better than nothing for now.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341729 91177308-0d34-0410-b5e6-96231b3b80d8
Following D50807, and heading towards D50664, this intermediary change does the following:
1. Upgrade all custom Error types in llvm/trunk/lib/DebugInfo/ to use the new StringError behavior (D50807).
2. Implement std::is_error_code_enum and make_error_code() for DebugInfo error enumerations.
3. Rename GenericError -> PDBError (the file will be renamed in a subsequent commit)
4. Update custom error messages to follow the same formatting: (\w\s*)+\.
5. Keep generic "file not found" (ENOENT) errors as they are in PDB code. Previously, there used to be a custom enumeration for that purpose.
6. Remove a few extraneous LF in log() implementations. Printing LF is a responsability at a higher level, not at the error level.
Differential Revision: https://reviews.llvm.org/D51499
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341228 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This prefix was added in r333421, and it changed our dumper output to
say things like "CVRegEAX" instead of just "EAX". That's a functional
change that I'd rather avoid.
I tested GCC, Clang, and MSVC, and all of them support #pragma
push_macro. They don't issue warnings whem the macro is not defined
either.
I don't have a Mac so I can't test the real termios.h header, but I
looked at the termios.h sources online and looked for other conflicts.
I saw only the CR* macros, so those are the ones we work around.
Reviewers: zturner, JDevlieghere
Subscribers: hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D50851
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@339907 91177308-0d34-0410-b5e6-96231b3b80d8
Previously we emitted 20-byte SHA1 hashes. This is overkill
for identifying debug info records, and has the negative side
effect of making object files bigger and links slower. By
using only the last 8 bytes of a SHA1, we get smaller object
files and ~10% faster links.
This modifies the format of the .debug$H section by adding a new
value for the hash algorithm field, so that the linker will still
work when its object files have an old format.
Differential Revision: https://reviews.llvm.org/D46855
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332669 91177308-0d34-0410-b5e6-96231b3b80d8
The prefix includes type kind, which is important to preserve. Two
different type leafs can easily have the same interior record contents
as another type.
We ran into this issue in PR37492 where a bitfield type record collided
with a const modifier record. Their contents were bitwise identical, but
their kinds were different.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332664 91177308-0d34-0410-b5e6-96231b3b80d8
The DBI stream contains a list of module descriptors. At the
beginning of each descriptor is a structure representing the first
section contribution in the output file for that module. LLD
currently doesn't fill out this structure at all, but link.exe
does. So as a precursor to emitting this data in LLD, we first
need a way to dump it so that it can be checked.
This patch adds support for the dumping, and verifies via a test
that LLD emits bogus information.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330208 91177308-0d34-0410-b5e6-96231b3b80d8
There are two FPMs in an MSF file, the idea being that for
incremental updates you can write to the alternate one and then
atomically swap them on commit. LLVM defaulted to using FPM1
on the first commit, but this differs from Microsoft's behavior
which is to default to using FPM2 on the first commit. To
eliminate some byte-level file differences, this patch changes
LLVM's default to also be FPM2.
Additionally, LLVM was trying to be "smart" about marking FPM
pages allocated. In addition to marking every page belonging
to the alternate FPM as unallocated, LLVM also marked pages at
the end of the main FPM which were not needed as unallocated.
In order to match the behavior of Microsoft-generated PDBs, we
now always mark every FPM block as allocated, regardless of
whether it is in the main FPM or the alt FPM, and regardless of
whether or not it describes blocks which are actually in the file.
This has the side benefit of simplifying our code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@328812 91177308-0d34-0410-b5e6-96231b3b80d8
Since the local hash is a different number of bytes depending
on host architecture, we don't have a consistent value. I
will need to re-do this test for both x86 and x64. For now
it accepts any value for the local hash.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@319864 91177308-0d34-0410-b5e6-96231b3b80d8
This adds a new command line option, -udt-stats, which breaks
down the stats of S_UDT records. These are one of the biggest
contributors to the size of /DEBUG:FASTLINK PDBs, so they need
some additional tools to be able to analyze their usage. This
option will dig into each S_UDT record and determine what kind
of record it points to, and then break down the statistics by
the target type. The goal here is to identify how our object
files differ from MSVC object files in S_UDT records, so that
we can output fewer of them and reach size parity.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312276 91177308-0d34-0410-b5e6-96231b3b80d8
This adds support for dumping a summary of module symbols
and CodeView debug chunks. This option prints a table for
each module of all of the symbols that occurred in the module
and the number of times it occurred and total byte size. Then
at the end it prints the totals for the entire file.
Additionally, this patch adds the -jmc (just my code) option,
which suppresses modules which are from external libraries or
linker imports, so that you can focus only on the object files
and libraries that originate from your own source code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311338 91177308-0d34-0410-b5e6-96231b3b80d8
The %T lit expansion expands to a common directory shared between all the tests in the same directory, which is unexpected and unintuitive, and more importantly, it's been a source of subtle race conditions and flaky tests. In https://reviews.llvm.org/D35396, it was agreed that it would be best to simply ban %T and only keep %t, which is unique to each test. When a test needs a temporary directory, it can just create one using mkdir %t.
This patch removes %T in llvm.
Differential Revision: https://reviews.llvm.org/D36495
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310953 91177308-0d34-0410-b5e6-96231b3b80d8
These were pending in a separate patch but I forgot to squash them
before comitting, and this one didn't go through.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310764 91177308-0d34-0410-b5e6-96231b3b80d8
For some reason I didn't see this failure the first time. The
output format changed slightly, so we just have to update the
test for the new format.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310442 91177308-0d34-0410-b5e6-96231b3b80d8
This extends the native reader to enable llvm-pdbutil to list the enums in a
PDB and it includes a simple test. It does not yet list the values in the
enumerations, which requires an actual implementation of
NativeEnumSymbol::FindChildren.
To exercise this code, use a command like:
llvm-pdbutil pretty -native -enums foo.pdb
Differential Revision: https://reviews.llvm.org/D35738
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310144 91177308-0d34-0410-b5e6-96231b3b80d8
Image section headers are stored in the DBI stream, but we
had no way to dump them. This patch adds dumping support,
along with some tests that LLD actually dumps them correctly.
Differential Revision: https://reviews.llvm.org/D36332
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310107 91177308-0d34-0410-b5e6-96231b3b80d8
The PDB reserves certain blocks for the FPM that describe which
blocks in the file are allocated and which are free. We weren't
filling that out at all, and in some cases we were even stomping
it with incorrect data. This patch writes a correct FPM.
Differential Revision: https://reviews.llvm.org/D36235
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309896 91177308-0d34-0410-b5e6-96231b3b80d8
Recently problems have been discovered in the way we write the FPM
(free page map). In order to fix this, we first need to establish
a baseline about what a correct FPM looks like using an MSVC
generated PDB, so that we can then make our own generated PDBs
match. And in order to do this, the dumper needs a mode where it
can dump an FPM so that we can write tests for it.
This patch adds a command to dump the FPM, as well as a test against
a known-good PDB.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309894 91177308-0d34-0410-b5e6-96231b3b80d8
The PDB "symbol stream" actually contains symbol records for the publics
and the globals stream. The globals and publics streams are essentially
hash tables that point into a single stream of records. In order to
match cvdump's behavior, we need to only dump symbol records referenced
from the hash table. This patch implements that, and then implements
global stream dumping, since it's just a subset of public stream
dumping.
Now we shouldn't see S_PROCREF or S_GDATA32 records when dumping
publics, and instead we should see those record in the globals stream.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309066 91177308-0d34-0410-b5e6-96231b3b80d8
Reid Kleckner