28 Commits

Author SHA1 Message Date
Zachary Turner
e9f782a21b [PDB] Correctly use the target machine when writing DBI stream.
Using Config->is64() will treat ARM64 as Amd64, which is incorrect.
Furthermore, there are more esoteric architectures that could
theoretically be encountered.  Just set it directly to the machine
type, which we already know anyway.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330157 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-16 20:42:06 +00:00
Zachary Turner
9a56b66b35 Resubmit "Fix some incorrect fields in our generated PDBs."
This fixes the failing tests.  They simply hadn't been updated
to match the new output resulting from this patch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330145 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-16 18:17:13 +00:00
Zachary Turner
84d38622bd Revert "Fix some incorrect fields in our generated PDBs."
There are a couple of failing tests which slipped under my radar
so I'm reverting this while I attempt to fix.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330133 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-16 16:55:41 +00:00
Zachary Turner
b1bd1633f1 Fix some incorrect fields in our generated PDBs.
Most of these are pretty trivial and obvious. Setting the toolchain
version to 14.11 is perhaps a little questionable, but we've been bitten
in the past where one of our version fields sidn't match MSVC's, and I
definitely don't want to go through that diagnosis again as it was
pretty time consuming and hard to track down.

I found all of these by using llvm-pdbutil export to dump the dbi and
pdb streams to a file, then using fc followed by llvm-pdbutil explain to
explain the mismatched bytes.

There are still some more, these are just the low hanging fruit.

Differential Revision: https://reviews.llvm.org/D45276

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330130 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-16 16:27:49 +00:00
Zachary Turner
0995a92322 [PDB] Make our PDBs look more like MS PDBs.
When investigating bugs in PDB generation, the first step is
often to do the same link with link.exe and then compare PDBs.

But comparing PDBs is hard because two completely different byte
sequences can both be correct, so it hampers the investigation when
you also have to spend time figuring out not just which bytes are
different, but also if the difference is meaningful.

This patch fixes a couple of cases related to string table emission,
hash table emission, and the order in which we emit strings that
makes more of our bytes the same as the bytes generated by MS PDBs.

Differential Revision: https://reviews.llvm.org/D44810

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@328348 91177308-0d34-0410-b5e6-96231b3b80d8
2018-03-23 18:43:39 +00:00
Peter Collingbourne
95137e4939 COFF: PDB: Allow multiple modules with the same name.
It is possible for two modules to have the same name if they are
archive members with the same name, or if we are doing LTO (in which
case all modules will have the name "lto.tmp").

Differential Revision: https://reviews.llvm.org/D37589

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312744 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-07 20:39:46 +00:00
Reid Kleckner
63c839af68 [PDB] Fix section contributions
Summary:
PDB section contributions are supposed to use output section indices and
offsets, not input section indices and offsets.

This allows the debugger to look up the index of the module that it
should look up in the modules stream for symbol information. With this
change, windbg can now find line tables, but it still cannot print local
variables.

Fixes PR34048

Reviewers: zturner

Subscribers: hiraditya, ruiu, llvm-commits

Differential Revision: https://reviews.llvm.org/D36285

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309987 91177308-0d34-0410-b5e6-96231b3b80d8
2017-08-03 21:15:09 +00:00
Zachary Turner
548addb1f1 [lld/pdb] Add an empty globals stream.
We don't write any actual symbols to this stream yet, but for
now we just create the stream and hook it up to the appropriate
places and give it a valid header.

Differential Revision: https://reviews.llvm.org/D35290

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309608 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-31 19:36:08 +00:00
Zachary Turner
c1422a59e8 [lld/pdb] Create an empty public symbol record stream.
This is part of the continuing effort to increase parity between
LLD and MSVC PDBs.  link still doesn't like our PDBs, so the most
obvious thing to check was whether adding an empty publics stream
would get it to do something else.  It still fails in the same way
but at least this removes one more variable from the equation.
The next logical step would be to try creating an empty globals
stream.

Differential Revision: https://reviews.llvm.org/D35224

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@307598 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-10 22:40:20 +00:00
Zachary Turner
b85b70f834 Fix some differences between lld and MSVC generated PDBs.
A couple of things were different about our generated PDBs.

1) We were outputting the wrong Version on the PDB Stream.
   The version we were setting was newer than what MSVC is setting.
   It's not clear what the implications are, but we change LLD
   to use PdbImplVC70, as MSVC does.
2) For the optional debug stream indices in the DBI Stream, we
   were outputting 0 to mean "the stream is not present".  MSVC
   outputs uint16_t(-1), which is the "correct" way to specify
   that a stream is not present.  So we fix that as well.
3) We were setting the PDB Stream signature to 0.  This is supposed
   to be the result of calling time(nullptr).  Although this leads
   to non-deterministic builds, a better way to solve that is by
   having a command line option explicitly for generating a
   reproducible build, and have the default behavior of lld-link
   match the default behavior of link.

To test this, I'm making use of the new and improved `pdb diff`
sub command.  To make it suitable for writing tests against, I had
to modify the diff subcommand slightly to print less verbose output.
Previously it would always print | <column> | <value1> | <value2> |
which is quite verbose, and the values are fragile.  All we really
want to know is "did we produce the same value as link?"  So I added
command line options to print a single character representing the
result status (different, identical, equivalent), and another to
hide the value display.  Note that just inspecting the diff output
used to write the test, you can see some things that are obviously
wrong.  That is just reflective of the fact that this is the state
of affairs today, not that we're asserting that this is "correct".
We can use this as a starting point to discover differences, fix
them, and update the test.

Differential Revision: https://reviews.llvm.org/D35086

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@307422 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-07 18:45:56 +00:00
Zachary Turner
5974613ee5 [PDB] Teach libpdb to write DBI Stream ECNames.
Based strictly on the name, this seems to have something to do
width edit & continue.  The goal of this patch has nothing to do
with supporting edit and continue though.  msvc link.exe writes
very basic information into this area even when *not* compiling
with support for E&C, and so the goal here is to bring lld-link
to parity.  Since we cannot know what assumptions standard tools
make about the content of PDB files, we need to be as close as
possible.

This ECNames data structure is a standard PDB string hash table.
link.exe puts a single string into this hash table, which is the
full path to the PDB file on disk.  It then references this string
from the module descriptor for the compiler generated `* Linker *`
module.

With this patch, lld-link will generate the exact same sequence of
bytes as MSVC link for this subsection for a given object file
input (as reported by `llvm-pdbutil bytes -ec`).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@307356 91177308-0d34-0410-b5e6-96231b3b80d8
2017-07-07 05:04:36 +00:00
Zachary Turner
6ebce80ed1 [PDB] Don't write uninitialized bytes to a PDB file.
There were certain fields that we didn't know how to write, as
well as various padding bytes that we would ignore.  This leads
to garbage data in the PDB.  While not strictly necessary, we
should initialize these bytes to something meaningful, as it
makes for easier binary comparison between PDBs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@305819 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-20 18:50:55 +00:00
Reid Kleckner
d89466b34c [PDB] Start emitting source file and line information
Summary:
This is a first step towards getting line info to show up in VS and
windbg. So far, only llvm-pdbutil can parse the PDBs that we produce.
cvdump doesn't like something about our file checksum tables. I'll have
to dig into that next.

This patch adds a new DebugSubsectionRecordBuilder which takes bytes
directly from some other producer, such as a linker, and sticks it into
the PDB. Line tables only need to be relocated. No data needs to be
rewritten.

File checksums and string tables, on the other hand, need to be re-done.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D34257

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@305713 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-19 17:21:45 +00:00
Reid Kleckner
bd596b4de0 [PDB] Add a module descriptor for every object file
Summary:
Expose the module descriptor index and fill it in for section
contributions.

Reviewers: zturner

Subscribers: llvm-commits, ruiu, hiraditya

Differential Revision: https://reviews.llvm.org/D34126

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@305296 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-13 15:49:13 +00:00
Zachary Turner
19ca2b0f9d Move Object format code to lib/BinaryFormat.
This creates a new library called BinaryFormat that has all of
the headers from llvm/Support containing structure and layout
definitions for various types of binary formats like dwarf, coff,
elf, etc as well as the code for identifying a file from its
magic.

Differential Revision: https://reviews.llvm.org/D33843

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@304864 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-07 03:48:56 +00:00
Zachary Turner
a5be24d937 [PDB] Fix use after free.
Previously MappedBlockStream owned its own BumpPtrAllocator that
it would allocate from when a read crossed a block boundary.  This
way it could still return the user a contiguous buffer of the
requested size.  However, It's not uncommon to open a stream, read
some stuff, close it, and then save the information for later.
After all, since the entire file is mapped into memory, the data
should always be available as long as the file is open.

Of course, the exception to this is when the data isn't *in* the
file, but rather in some buffer that we temporarily allocated to
present this contiguous view.  And this buffer would get destroyed
as soon as the strema was closed.

The fix here is to force the user to specify the allocator, this
way it can provide an allocator that has whatever lifetime it
chooses.

Differential Revision: https://reviews.llvm.org/D33858

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@304623 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-03 00:33:35 +00:00
Bob Haarman
ae12b7b360 [pdb] pad source file name buffer at the end instead of the beginning
Summary:
DbiStreamBuilder calculated the offset of the source file names inside
the file info substream as the size of the file info substream minus
the size of the file names. Since the file info substream is padded to
a multiple of 4 bytes, this caused the first file name to be aligned
on a 4-byte boundary. By contrast, DbiModuleList would read the file
names immediately after the file name offset table, without skipping
to the next 4-byte boundary. This change makes it so that the file
names are written to the location where DbiModuleList expects them,
and puts any necessary padding for the file info substream after the
file names instead of before it.

Reviewers: amccarth, rnk, zturner

Reviewed By: amccarth, zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33475


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@303917 91177308-0d34-0410-b5e6-96231b3b80d8
2017-05-25 21:12:15 +00:00
Zachary Turner
4c78c0e348 [CodeView] Write CodeView line information.
Differential Revision: https://reviews.llvm.org/D32716

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@301882 91177308-0d34-0410-b5e6-96231b3b80d8
2017-05-01 23:27:42 +00:00
Zachary Turner
3220dd07ab Rename some PDB classes.
We have a lot of very similarly named classes related to
dealing with module debug info.  This patch has NFC, it just
renames some classes to be more descriptive (albeit slightly
more to type).  The mapping from old to new class names is as
follows:

   Old          |        New
ModInfo         | DbiModuleDescriptor
ModuleSubstream | ModuleDebugFragment
ModStream       | ModuleDebugStream

With the corresponding Builder classes renamed accordingly.

Differential Revision: https://reviews.llvm.org/D32506

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@301555 91177308-0d34-0410-b5e6-96231b3b80d8
2017-04-27 16:11:19 +00:00
Zachary Turner
bbf9606175 [pdb] Write the module info and symbol record streams.
Previously we did not have support for writing detailed
module information for each module, as well as the symbol
records.  This patch adds support for this, and in doing
so enables the ability to construct minimal PDBs from
just a few lines of YAML.  A test is added to illustrate
this functionality.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297900 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-15 22:18:53 +00:00
Zachary Turner
f3491b21c5 [Support] Move Stream library from MSF -> Support.
After several smaller patches to get most of the core improvements
finished up, this patch is a straight move and header fixup of
the source.

Differential Revision: https://reviews.llvm.org/D30266

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296810 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-02 20:52:51 +00:00
Zachary Turner
d9b4aa7542 [PDB] Make streams carry their own endianness.
Before the endianness was specified on each call to read
or write of the StreamReader / StreamWriter, but in practice
it's extremely rare for streams to have data encoded in
multiple different endiannesses, so we should optimize for the
99% use case.

This makes the code cleaner and more general, but otherwise
has NFC.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296415 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-28 00:04:07 +00:00
Zachary Turner
2219387eaf [PDB] Partial resubmit of r296215, which improved PDB Stream Library.
This was reverted because it was breaking some builds, and
because of incorrect error code usage.  Since the CL was
large and contained many different things, I'm resubmitting
it in pieces.

This portion is NFC, and consists of:

1) Renaming classes to follow a consistent naming convention.
2) Fixing the const-ness of the interface methods.
3) Adding detailed doxygen comments.
4) Fixing a few instances of passing `const BinaryStream& X`.  These
   are now passed as `BinaryStreamRef X`.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296394 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-27 22:11:43 +00:00
NAKAMURA Takumi
96678fa6d8 Revert r296215, "[PDB] General improvements to Stream library." and followings.
r296215, "[PDB] General improvements to Stream library."
r296217, "Disable BinaryStreamTest.StreamReaderObject temporarily."
r296220, "Re-enable BinaryStreamTest.StreamReaderObject."
r296244, "[PDB] Disable some tests that are breaking bots."
r296249, "Add static_cast to silence -Wc++11-narrowing."

std::errc::no_buffer_space should be used for OS-oriented errors for socket transmission.
(Seek discussions around llvm/xray.)

I could substitute s/no_buffer_space/others/g, but I revert whole them ATM.

Could we define and use LLVM errors there?

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296258 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-25 17:04:23 +00:00
Zachary Turner
5433d115a4 [PDB] General improvements to Stream library.
This adds various new functionality and cleanup surrounding the
use of the Stream library.  Major changes include:

* Renaming of all classes for more consistency / meaningfulness
* Addition of some new methods for reading multiple values at once.
* Full suite of unit tests for reader / writer functionality.
* Full set of doxygen comments for all classes.
* Streams now store their own endianness.
* Fixed some bugs in a few of the classes that were discovered
  by the unit tests.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296215 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-25 00:44:30 +00:00
Zachary Turner
40ea28492e [PDB] Rename Stream related source files.
This is part of a larger effort to get the Stream code moved
up to Support.  I don't want to do it in one large patch, in
part because the changes are so big that it will treat everything
as file deletions and add, losing history in the process.
Aside from that though, it's just a good idea in general to
make small changes.

So this change only changes the names of the Stream related
source files, and applies necessary source fix ups.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296211 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-25 00:33:34 +00:00
Zachary Turner
980cadce13 Don't assume little endian in StreamReader / StreamWriter.
In an effort to generalize this so it can be used by more than
just PDB code, we shouldn't assume little endian.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295525 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-18 01:35:33 +00:00
Adrian McCarthy
11e1d83ce5 NFC: Rename (PDB) RawSession to NativeSession
This eliminates one overload on the term Raw.

Differential Revision: https://reviews.llvm.org/D29098

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293104 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-25 22:38:55 +00:00