118 Commits

Author SHA1 Message Date
Zachary Turner
d9b4aa7542 [PDB] Make streams carry their own endianness.
Before the endianness was specified on each call to read
or write of the StreamReader / StreamWriter, but in practice
it's extremely rare for streams to have data encoded in
multiple different endiannesses, so we should optimize for the
99% use case.

This makes the code cleaner and more general, but otherwise
has NFC.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296415 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-28 00:04:07 +00:00
Zachary Turner
2219387eaf [PDB] Partial resubmit of r296215, which improved PDB Stream Library.
This was reverted because it was breaking some builds, and
because of incorrect error code usage.  Since the CL was
large and contained many different things, I'm resubmitting
it in pieces.

This portion is NFC, and consists of:

1) Renaming classes to follow a consistent naming convention.
2) Fixing the const-ness of the interface methods.
3) Adding detailed doxygen comments.
4) Fixing a few instances of passing `const BinaryStream& X`.  These
   are now passed as `BinaryStreamRef X`.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296394 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-27 22:11:43 +00:00
NAKAMURA Takumi
96678fa6d8 Revert r296215, "[PDB] General improvements to Stream library." and followings.
r296215, "[PDB] General improvements to Stream library."
r296217, "Disable BinaryStreamTest.StreamReaderObject temporarily."
r296220, "Re-enable BinaryStreamTest.StreamReaderObject."
r296244, "[PDB] Disable some tests that are breaking bots."
r296249, "Add static_cast to silence -Wc++11-narrowing."

std::errc::no_buffer_space should be used for OS-oriented errors for socket transmission.
(Seek discussions around llvm/xray.)

I could substitute s/no_buffer_space/others/g, but I revert whole them ATM.

Could we define and use LLVM errors there?

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296258 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-25 17:04:23 +00:00
Daniel Jasper
b646c0a82a Add static_cast to silence -Wc++11-narrowing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296249 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-25 07:53:36 +00:00
Zachary Turner
f18ff27042 [PDB] Disable some tests that are breaking bots.
This has to do with big endian, but I can't fix it until
Monday.  The code itself is fine, just the tests are wrong.
Disabling 3 tests for now.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296244 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-25 05:57:57 +00:00
Zachary Turner
0a1a6e3516 Re-enable BinaryStreamTest.StreamReaderObject.
I had an invalid pointer / size calculation that was causing
a stack smash.  Should be fixed now.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296220 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-25 01:20:08 +00:00
Zachary Turner
9e80319674 Disable BinaryStreamTest.StreamReaderObject temporarily.
This is crashing on some bots, so I need some time to investigate.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296217 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-25 00:52:59 +00:00
Zachary Turner
5433d115a4 [PDB] General improvements to Stream library.
This adds various new functionality and cleanup surrounding the
use of the Stream library.  Major changes include:

* Renaming of all classes for more consistency / meaningfulness
* Addition of some new methods for reading multiple values at once.
* Full suite of unit tests for reader / writer functionality.
* Full set of doxygen comments for all classes.
* Streams now store their own endianness.
* Fixed some bugs in a few of the classes that were discovered
  by the unit tests.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296215 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-25 00:44:30 +00:00
Zachary Turner
40ea28492e [PDB] Rename Stream related source files.
This is part of a larger effort to get the Stream code moved
up to Support.  I don't want to do it in one large patch, in
part because the changes are so big that it will treat everything
as file deletions and add, losing history in the process.
Aside from that though, it's just a good idea in general to
make small changes.

So this change only changes the names of the Stream related
source files, and applies necessary source fix ups.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296211 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-25 00:33:34 +00:00
Adrian McCarthy
0ca20ba516 Fix unit tests after r296049.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@296055 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-24 00:25:17 +00:00
Zachary Turner
980cadce13 Don't assume little endian in StreamReader / StreamWriter.
In an effort to generalize this so it can be used by more than
just PDB code, we shouldn't assume little endian.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295525 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-18 01:35:33 +00:00
Zachary Turner
3e9ce61e33 [pdb] Add the ability to resolve TypeServer PDBs.
Some PDBs or object files can contain references to other PDBs
where the real type information lives.  When this happens,
all type indices in the original PDB are meaningless because
their records are not there.

With this patch we add the ability to pull type info from those
secondary PDBs.

Differential Revision: https://reviews.llvm.org/D29973

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@295382 91177308-0d34-0410-b5e6-96231b3b80d8
2017-02-16 23:35:45 +00:00
Adrian McCarthy
60c9e9f95d Fix for r293104, which renamed a directory.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@293105 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-25 22:48:57 +00:00
Zachary Turner
9c1e7a6062 [pdb] Write the Named Stream mapping to Yaml and binary.
Differential Revision: https://reviews.llvm.org/D28919

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292665 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-20 22:42:09 +00:00
Zachary Turner
f213f1173e [PDB] Rename some files to be more intuitive.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292663 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-20 22:41:15 +00:00
Zachary Turner
1efbd52b3e [pdb] Add HashTable data structure.
This was being parsed / serialized ad-hoc inside the code
for a specific PDB stream.  But this data structure is used
in multiple ways / places within the PDB format.  To be able
to re-use it we need to raise this code out and make it more
generic.  In doing so, a number of bugs are fixed in the
original implementation, and support is added for growing
the hash table and deleting items from the hash table,
which had either been omitted or incorrect implemented in
the initial version.

Differential Revision: https://reviews.llvm.org/D28715

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292535 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-19 23:31:24 +00:00
Rui Ueyama
4b8b0ba562 PDB: Add a class to create the /names stream contents.
This patch adds a new class NameHashTableBuilder which creates /names streams.
This patch contains a test to confirm that a stream created by
NameHashTableBuilder can be read by NameHashTable reader class.

Differential Revision: https://reviews.llvm.org/D28707

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@292040 91177308-0d34-0410-b5e6-96231b3b80d8
2017-01-15 00:36:02 +00:00
David Majnemer
9b671f3db3 [PDB] Validate superblock addresses
- Validate the address of the block map.
- Validate the address of the free block map.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@290053 91177308-0d34-0410-b5e6-96231b3b80d8
2016-12-18 00:41:10 +00:00
Zachary Turner
f5898c18e7 [pdb] Fix unit test compilation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281560 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-14 23:17:08 +00:00
Zachary Turner
5e117855c3 [msf] Resubmit "Rename Msf -> MSF".
Previously this change was submitted from a Windows machine, so
changes made to the case of filenames and directory names did
not survive the commit, and as a result the CMake source file
names and the on-disk file names did not match on case-sensitive
file systems.

I'm resubmitting this patch from a Linux system, which hopefully
allows the case changes to make it through unfettered.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277213 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-29 20:56:36 +00:00
Zachary Turner
85c3e3ee9c Revert "[msf] Rename Msf to MSF."
This reverts commit 4d1557ffac41e079bcb1abbcf04f512474dcd6fe.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277194 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-29 18:38:47 +00:00
Zachary Turner
4d1557ffac [msf] Rename Msf to MSF.
In a previous patch, it was suggested to use all caps instead of
rolling caps for initialisms, so this patch changes everything
to do this.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277190 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-29 18:24:26 +00:00
Zachary Turner
a88b4402ae [pdb] Fix another narrowing conversion on x64 builds.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277026 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-28 19:47:04 +00:00
Zachary Turner
0c7a213a73 [pdb] Refactor library to more clearly separate reading/writing
Reviewed By: amccarth, ruiu
Differential Revision: https://reviews.llvm.org/D22693

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277019 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-28 19:12:28 +00:00
Zachary Turner
0c78d1d159 Get rid of IMsfStreamData class.
This was a pure virtual base class whose purpose was to abstract
away the notion of how you retrieve the layout of a discontiguous
stream of blocks in an Msf file.  This led to too many layers of
abstraction making it difficult to figure out what was going on
and extend things.  Ultimately, a stream's layout is decided by
its length and the array of block numbers that it lives on.  So
rather than have an abstract base class which can return this in
any number of ways, it's more straightforward to simply store them
as fields of a trivial struct, and also to give a more appropriate
name.

This patch does that.  It renames IMsfStreamData to MsfStreamLayout,
and deletes the 2 concrete implementations, DirectoryStreamData
and IndexedStreamData.  MsfStreamLayout is a trivial struct
with the necessary data.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277018 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-28 19:11:09 +00:00
Reid Kleckner
c39e246447 Fix dangling reference to temporary in use of ArrayRef
Fixes tests locally for me with MSVC 2015.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@277015 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-28 18:17:01 +00:00
Zachary Turner
c42c8e23a1 Make DebugInfoMsf a dependency of DebugInfoPDBTests.
For some reason this doesn't cause linker errors with MSVC or
clang-cl, but the bots seem to be failing with other compilers.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276463 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-22 20:05:30 +00:00
Zachary Turner
4ab13f5ec5 [msf] Create LLVMDebugInfoMsf
This provides a better layering of responsibilities among different
aspects of PDB writing code.  Some of the MSF related code was
contained in CodeView, and some was in PDB prior to this.  Further,
we were often saying PDB when we meant MSF, and the two are
actually independent of each other since in theory you can have
other types of data besides PDB data in an MSF.  So, this patch
separates the MSF specific code into its own library, with no
dependencies on anything else, and DebugInfoCodeView and
DebugInfoPDB take dependencies on DebugInfoMsf.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@276458 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-22 19:56:05 +00:00
Zachary Turner
9b30e73d48 [pdb] Teach MsfBuilder and other classes about the Free Page Map.
Block 1 and 2 of an MSF file are bit vectors that represent the
list of blocks allocated and free in the file.  We had been using
these blocks to write stream data and other data, so we mark them
as the free page map now.  We don't yet serialize these pages to
the disk, but at least we make a note of what it is, and avoid
writing random data to them.

Doing this also necessitated cleaning up some of the tests to be
more general and hardcode fewer values, which is nice.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275629 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-15 22:17:19 +00:00
Zachary Turner
6be3e7c0e8 [pdb] Use MsfBuilder to handle the writing PDBs.
Previously we would read a PDB, then write some of it back out,
but write the directory, super block, and other pertinent metadata
back out unchanged.  This generates incorrect PDBs since the amount
of data written was not always the same as the amount of data read.

This patch changes things to use the newly introduced `MsfBuilder`
class to write out a correct and accurate set of Msf metadata for
the data *actually* written, which opens up the door for adding and
removing type records, symbol records, and other types of data to
an existing PDB.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275627 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-15 22:16:56 +00:00
Zachary Turner
32cf292fd8 [pdb] Introduce MsfBuilder for laying out PDB files.
Reviewed by: ruiu
Differential Revision: https://reviews.llvm.org/D22308

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275611 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-15 20:43:38 +00:00
David Majnemer
7aa4746755 [llvm-pdbdump] Propagate errors a little more consistently
PDBFile::getBlockData didn't really return any indication that it
failed.  It merely returned an empty buffer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275009 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-10 03:34:47 +00:00
Zachary Turner
ed35fdadaf Try to fix compilation error in DebugInfoPDBTests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@274881 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-08 16:57:14 +00:00
NAKAMURA Takumi
76efceecfb DebugInfoPDBTests:MappedBlockStreamTest.TestWriteThenRead: Avoid assigning temporary object to ArrayRef.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272457 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-11 06:37:28 +00:00
Zachary Turner
b8e247d2f2 Try again to fix this endianness issue.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272440 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-10 22:12:18 +00:00
Zachary Turner
936d48972b [pdb] Fix issues with pdb writing.
This fixes an alignment issue by forcing all cached allocations
to be 8 byte aligned, and also fixes an issue arising on big
endian systems by writing ulittle32_t's instead of uint32_t's
in the test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272437 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-10 21:47:26 +00:00
Zachary Turner
579bef0ca2 Add support for writing through StreamInterface.
This adds method and tests for writing to a PDB stream.  With
this, even a PDB stream which is discontiguous can be treated
as a sequential stream of bytes for the purposes of writing.

Reviewed By: ruiu
Differential Revision: http://reviews.llvm.org/D21157

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272369 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-10 05:09:12 +00:00
Zachary Turner
47b89fc720 [pdb] Fix build errors in PDB unit tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272174 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-08 17:32:25 +00:00
Zachary Turner
ca5b485311 [pdb] Fix broken unit test compilation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272059 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-07 20:46:39 +00:00
Zachary Turner
804dc35d2d [pdb] Fix broken unit tests after r271982.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271983 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-07 05:32:48 +00:00
David Majnemer
81d9dfc70d [CodeView] Take the StreamRef::readBytes offset into account when validating
We only considered the length of the operation and the length of the
StreamRef without considered what it meant for the offset to be at a
non-zero position.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271496 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-02 06:21:44 +00:00
NAKAMURA Takumi
35ffaf1afa Rework r271439. I forgot to save the buffer for editing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271441 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 23:03:46 +00:00
NAKAMURA Takumi
d76edeb3ba MappedBlockStreamTest.cpp: Simplify array initializers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271439 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 22:59:06 +00:00
Zachary Turner
3ca9979447 [pdb] silence warnings about moving from a temporary.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271420 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 18:18:55 +00:00
David Majnemer
d5f9b7e57e [CodeView] Make sure StreamRef::readBytes doesn't read too much
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271418 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 18:13:06 +00:00
David Majnemer
2511caa18b [PDB] Silence sign comparison warnings in MappedBlockStreamTest
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271416 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 18:13:02 +00:00
NAKAMURA Takumi
cc0c9e0cf6 MappedBlockStreamTest.cpp: Appease msc18 to avoid initializer for std::vector.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271397 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 14:26:54 +00:00
NAKAMURA Takumi
249a797178 DebugInfoPDBTests: Update libdeps for r271346.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271355 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-01 00:30:15 +00:00
Zachary Turner
e781066bc2 [pdb] Add unit tests for PDB MappedBlockStream and zero copy
Differential Revision: http://reviews.llvm.org/D20837
Reviewed By: ruiu

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@271346 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-31 22:41:52 +00:00
Zachary Turner
41f8084128 Fix build of DebugInfoPDBTests.
Missing a using statement.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268552 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-04 21:06:12 +00:00