Commit Graph

135 Commits

Author SHA1 Message Date
Zachary Turner
91d41f9825 [pdb] Re-add code to write PDB files.
Somehow all the functionality to write PDB files got removed,
probably accidentally when uploading the patch perhaps the wrong
one got uploaded.  This re-adds all the code, as well as the
corresponding test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@274248 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-30 17:43:00 +00:00
Zachary Turner
3a4681d580 Update llvm-pdbdump to use subcommands.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@274247 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-30 17:42:48 +00:00
Reid Kleckner
7abe77ab98 [pdb] Don't error on missing FPO streams
64-bit PDBs never have FPO data. They have xdata instead.

Also improve error recovery of stream summary dumping while I'm here.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@273046 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-17 20:38:01 +00:00
Zachary Turner
e8d381d02c Resubmit "[pdb] Change type visitor pattern to be dynamic."
There was a regression introduced during type stream merging when
visiting a field list record.  This has been fixed in this patch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272929 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-16 18:22:27 +00:00
Zachary Turner
575a9fbcc9 Revert "[pdb] Change type visitor pattern to be dynamic."
This reverts commit fb0dd311e1.

This breaks some llvm-readobj tests.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272927 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-16 18:09:04 +00:00
Zachary Turner
fb0dd311e1 [pdb] Change type visitor pattern to be dynamic.
This allows better catching of compiler errors since we can use
the override keyword to verify that methods are actually
overridden.

Also in this patch I've changed from storing a boolean Error
code everywhere to returning an llvm::Error, to propagate richer
error information up the call stack.

Reviewed By: ruiu, rnk
Differential Revision: http://reviews.llvm.org/D21410

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272926 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-16 18:00:28 +00:00
Zachary Turner
7d35b2d2b1 Resubmit "[pdb] Actually write a PDB to disk from YAML.""
Reviewed By: ruiu
Differential Revision: http://reviews.llvm.org/D21220

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272708 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-14 20:48:36 +00:00
Zachary Turner
bf3c011ba7 Revert "[pdb] Actually write a PDB to disk from YAML."
This reverts commit 879139e1c6.

This was committed accidentally when I blindly typed git svn
dcommit instead of the command to generate a patch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272693 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-14 18:51:35 +00:00
Zachary Turner
879139e1c6 [pdb] Actually write a PDB to disk from YAML.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272692 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-14 18:49:36 +00:00
Zachary Turner
90005b6d83 Make PDBFile take a StreamInterface instead of a MemBuffer.
This is the next step towards being able to write PDBs.
MemoryBuffer is immutable, and StreamInterface is our replacement
which can be any combination of read-only, read-write, or write-only
depending on the particular implementation.

The one place where we were creating a PDBFile (in RawSession) is
updated to subclass ByteStream with a simple adapter that holds
a MemoryBuffer, and initializes the superclass with the buffer's
array, so that all the functionality of ByteStream works
transparently.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272370 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-10 05:10:19 +00:00
Richard Smith
0eeb3d4004 Search for llvm-symbolizer binary in the same directory as argv[0], before
looking for it along $PATH. This allows installs of LLVM tools outside of
$PATH to find the symbolizer and produce pretty backtraces if they crash.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272232 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-09 00:53:21 +00:00
Zachary Turner
22ca3e86a7 [pdbdump-fuzzer] Try to fix build errors in fuzzer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272230 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-09 00:21:37 +00:00
Zachary Turner
99272168e0 [pdb] Handle stream index errors better.
Reviewed By: ruiu
Differential Revision: http://reviews.llvm.org/D21128

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272172 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-08 17:26:39 +00:00
Zachary Turner
44e9b3090f [pdb] Try to fix use after free.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272078 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-08 00:25:08 +00:00
Rui Ueyama
5ae402a784 [pdbdump] Print out # of hash buckets.
In the reference code, the field name is `cHashBuckets`.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272075 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-07 23:53:43 +00:00
Rui Ueyama
650053d1ce [pdbdump] Print out TPI hash key size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272073 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-07 23:44:27 +00:00
Zachary Turner
6e00ec14d5 [pdb] Use MappedBlockStream to parse the PDB directory.
In order to efficiently write PDBs, we need to be able to make a
StreamWriter class similar to a StreamReader, which can transparently deal
with writing to discontiguous streams, and we need to use this for all
writing, similar to how we use StreamReader for all reading.

Most discontiguous streams are the typical numbered streams that appear in
a PDB file and are described by the directory, but the exception to this,
that until now has been parsed by hand, is the directory itself.
MappedBlockStream works by querying the directory to find out which blocks
a stream occupies and various other things, so naturally the same logic
could not possibly work to describe the blocks that the directory itself
resided on.

To solve this, I've introduced an abstraction IPDBStreamData, which allows
the client to query for the list of blocks occupied by the stream, as well
as the stream length. I provide two implementations of this: one which
queries the directory (for indexed streams), and one which queries the
super block (for the directory stream).

This has the side benefit of vastly simplifying the code to parse the
directory. Whereas before a mini state machine was rolled by hand, now we
simply use FixedStreamArray to read out the stream sizes, then build a
vector of FixedStreamArrays for the stream map, all in just a few lines of
code.

Reviewed By: ruiu
Differential Revision: http://reviews.llvm.org/D21046

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271982 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-07 05:28:55 +00:00
Rui Ueyama
1895d4095b [pdbdump] Print section header flags.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271943 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-06 21:34:55 +00:00
Zachary Turner
4efa5e541f [llvm-pdbdump] Dump stream sizes and stream blocks to yaml.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271940 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-06 20:37:17 +00:00
Zachary Turner
61e0e2783c [llvm-pdbdump] Dump MSF headers to YAML.
This is the simplest possible patch to get some kind of YAML
output.  All it dumps is the MSF header fields so that in
theory an empty MSF file could be reconstructed.

Reviewed By: ruiu, majnemer
Differential Revision: http://reviews.llvm.org/D20971

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271939 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-06 20:37:05 +00:00
Rui Ueyama
24ef682bfb [pdbdump] Print out New FPO stream contents.
The data strucutre in the new FPO stream is described in the
PE/COFF spec. There is one record per function if frame pointer
is omitted.

Differential Revision: http://reviews.llvm.org/D20999

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271926 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-06 18:39:21 +00:00
Rui Ueyama
e5f15a26d7 pdbdump: print out TPI hashes.
Differential Revision: http://reviews.llvm.org/D20945

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271736 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-03 20:48:51 +00:00
Zachary Turner
041e9f2c56 [llvm-pdbdump] Introduce an abstraction for the output style.
This opens the door to introducing a YAML outputter which can be
used for machine consumption.  Currently the yaml output style
is unimplemented and returns an error if you try to use it.

Reviewed By: rnk, ruiu
Differential Revision: http://reviews.llvm.org/D20967

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271712 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-03 19:28:33 +00:00
Zachary Turner
f7d0c1f9cf [pdb] Print out file names instead of file offsets.
When printing line information and file checksums, we were printing
the file offset field from the struct header.  This teaches
llvm-pdbdump how to turn those numbers into the filename.  In the
case of file checksums, this is done by looking in the global
string table.  In the case of line contributions, this is done
by indexing into the file names buffer of the DBI stream.  Why
they use a different technique I don't know.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271630 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-03 05:52:57 +00:00
Zachary Turner
c5689fd37c [pdb] Dump file checksums from pdb codeview line info.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271622 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-03 04:01:48 +00:00
Zachary Turner
7e17f48869 [codeview] Dump line number and column information.
To facilitate this, a couple of changes had to be made:

1. `ModuleSubstream` got moved from `DebugInfo/PDB` to
`DebugInfo/CodeView`, and various codeview related types are defined
there.  It turns out `DebugInfo/CodeView/Line.h` already defines many of
these structures, but this is really old code that is not endian aware,
doesn't interact well with `StreamInterface` and not very helpful for
getting stuff out of a PDB.  Eventually we should migrate the old readobj
`COFFDumper` code to these new structures, or at least merge their
functionality somehow.

2. A `ModuleSubstream` visitor is introduced.  Depending on where your
module substream array comes from, different subsets of record types can
be expected.  We are already hand parsing these substream arrays in many
places especially in `COFFDumper.cpp`.  In the future we can migrate these
paths to the visitor as well, which should reduce a lot of code in
`COFFDumper.cpp`.

Differential Revision: http://reviews.llvm.org/D20936
Reviewed By: ruiu, majnemer

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271621 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-03 03:25:59 +00:00
Rui Ueyama
a4bbfe5926 Fix indentation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271620 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-03 02:42:30 +00:00
Zachary Turner
b36db5aff2 [llvm-pdbdump] Dump CodeView line information.
This first pass only splits apart the records and dumps the line
info kinds and binary data.  Subsequent patches will parse out
the binary data into more useful information and dump it in
detail.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271576 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-02 20:11:22 +00:00
Rui Ueyama
698e829b29 pdbdump: print out COFF section headers.
Unlike other sections that can grow to any size, the COFF section header
stream has maximum length because each record is fixed size and the COFF
file format limits the maximum number of sections. So I decided to not
create a specific stream class for it. Instead, I added a member function
to DbiStream class which returns a vector of COFF headers.

Differential Revision: http://reviews.llvm.org/D20717

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271557 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-02 18:20:20 +00:00
Zachary Turner
8839a0f88f [pdb] Parse and dump section map and section contribs
Differential Revision: http://reviews.llvm.org/D20876
Reviewed By: rnk, ruiu

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271488 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-02 05:07:49 +00:00
Kostya Serebryany
4d3c482d43 llvm-pdbdump-fuzzer: fix the build
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271352 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-31 23:39:31 +00:00
Reid Kleckner
c59038317a Speculative build fix for codeview type dumper API change
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271344 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-31 22:32:54 +00:00
Reid Kleckner
25565d97e7 [codeview] Improve readability of type record assembly
Adds the method MCStreamer::EmitBinaryData, which is usually an alias
for EmitBytes. In the MCAsmStreamer case, it is overridden to emit hex
dump output like this:
        .byte   0x0e, 0x00, 0x08, 0x10
        .byte   0x03, 0x00, 0x00, 0x00
        .byte   0x00, 0x00, 0x00, 0x00
        .byte   0x00, 0x10, 0x00, 0x00

Also, when verbose asm comments are enabled, this patch prints the dump
output for each comment before its record, like this:
        # ArgList (0x1000) {
        #   TypeLeafKind: LF_ARGLIST (0x1201)
        #   NumArgs: 0
        #   Arguments [
        #   ]
        # }
        .byte   0x06, 0x00, 0x01, 0x12
        .byte   0x00, 0x00, 0x00, 0x00

This should make debugging easier and testing more convenient.

Reviewers: aaboud

Subscribers: majnemer, zturner, amccarth, aaboud, llvm-commits

Differential Revision: http://reviews.llvm.org/D20711

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271313 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-31 18:45:36 +00:00
David Majnemer
692eabeda0 [llvm-pdbdump-fuzzer] Add a fuzzer for llvm-pdbdump
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271243 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-31 01:24:40 +00:00
David Majnemer
872acbc405 llvm-pdbdump should have a non-zero exit code on error
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271132 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-28 18:25:15 +00:00
Zachary Turner
5cfb6469b8 [pdb] Finish conversion to zero copy pdb access.
This converts remaining uses of ByteStream, which was still
left in the symbol stream and type stream, to using the new
StreamInterface zero-copy classes.

RecordIterator is finally deleted, so this is the only way left
now.  Additionally, more error checking is added when iterating
the various streams.

With this, the transition to zero copy pdb access is complete.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271101 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-28 05:21:57 +00:00
Zachary Turner
9cc0faee65 [codeview] Remove StreamReader copying method.
Since we want to move toward zero-copy access to stream data, we
want to remove all instances of copying operations.  So get rid
of some of those here.

Differential Revision: http://reviews.llvm.org/D20720
Reviewed By: ruiu

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270960 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-27 03:51:53 +00:00
Zachary Turner
0212bc82e0 [codeview,pdb] Try really hard to conserve memory when reading.
PDBs can be extremely large.  We're already mapping the entire
PDB into the process's address space, but to make matters worse
the blocks of the PDB are not arranged contiguously.  So, when
we have something like an array or a string embedded into the
stream, we have to make a copy.  Since it's convenient to use
traditional data structures to iterate and manipulate these
records, we need the memory to be contiguous.

As a result of this, we were using roughly twice as much memory
as the file size of the PDB, because every stream was copied
out and re-stitched together contiguously.

This patch addresses this by improving the MappedBlockStream
to allocate from a BumpPtrAllocator only when a read requires
a discontiguous read.  Furthermore, it introduces some data
structures backed by a stream which can iterate over both
fixed and variable length records of a PDB.  Since everything
is backed by a stream and not a buffer, we can read almost
everything from the PDB with zero copies.

Differential Revision: http://reviews.llvm.org/D20654
Reviewed By: ruiu

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270951 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-27 01:54:44 +00:00
Rui Ueyama
4c87e7cf1a pdbdump: print out the name of the stream 0.
Differential Revision: http://reviews.llvm.org/D20712

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270943 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-27 00:32:07 +00:00
Rui Ueyama
3438b38d19 pdbdump: Add -raw-all to enable all -raw-* flags.
Differential Revision: http://reviews.llvm.org/D20707

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270937 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 23:26:55 +00:00
Rui Ueyama
5b155a0d90 Fix typo.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270934 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-26 23:01:05 +00:00
Zachary Turner
213d3d3c81 [codeview] Move StreamInterface and StreamReader to libcodeview.
We have need to reuse this functionality, including making
additional generic stream types that are smarter about how and
when they copy memory versus referencing the original memory.
So all of these structures belong in the common library
rather than being pdb specific.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270751 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 20:37:03 +00:00
Zachary Turner
f50e8cdfbd [llvm-pdbdump] Dump raw stream contents as binary block.
Dumping it as ASCII makes it fairly useless.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270742 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 18:32:07 +00:00
Zachary Turner
4f4018ce2c [llvm-pdbdump] Decipher the remaining PDB streams.
We know at least know the meaning of every stream of the
PDB file.  Yay!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270669 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 05:49:48 +00:00
Zachary Turner
fbbe74464a [llvm-pdbdump] Dump the IPI stream and all records.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270661 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 04:35:22 +00:00
Zachary Turner
3b27e60206 [llvm-pdbdump] Stream 0 isn't actually the MSF superblock.
Oddly enough, I realized we don't actually know what stream
0 is (if anything).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270655 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 03:53:16 +00:00
Zachary Turner
ec842a3379 [llvm-pdbdump] Dump stream summary list.
Try to figure out what each stream is, and dump its name.

This gives us a better picture of what streams we still don't
understand.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270653 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-25 03:43:17 +00:00
Zachary Turner
2d4f33a508 [codeview] Add support for new type records.
This adds support for parsing and dumping the following
symbol types:

S_LPROCREF
S_ENVBLOCK
S_COMPILE2
S_REGISTER
S_COFFGROUP
S_SECTION
S_THUNK32
S_TRAMPOLINE

As of this patch, the test PDB files no longer have any unknown
symbol types.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270628 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-24 22:58:46 +00:00
Zachary Turner
45ab259f8d [llvm-pdbdump] Rework command line options.
When dumping huge PDB files, too many of the options were grouped
together so you would get neverending spew of output.  This patch
introduces more granular display options so you can only dump the
fields you actually care about.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270607 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-24 20:31:48 +00:00
Zachary Turner
653eb429f5 [codeview, pdb] Dump symbol records in publics stream
Differential Revision: http://reviews.llvm.org/D20580
Reviewed By: ruiu

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@270597 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-24 18:55:14 +00:00