69 Commits

Author SHA1 Message Date
Nico Weber
4370387226 llvm-undame: Fix an assert-on-invalid
Found by oss-fuzz, fixes issue 12432 on os-fuzz.

Differential Revision: https://reviews.llvm.org/D60206

llvm-svn: 357648
2019-04-03 23:23:32 +00:00
Nico Weber
b6b1db8acb llvm-undname: Fix an assert-on-invalid
Found by oss-fuzz, fixes issues 12428 and 12429 on oss-fuzz.

Differential Revision: https://reviews.llvm.org/D60204

llvm-svn: 357647
2019-04-03 23:19:39 +00:00
Nico Weber
d503567673 llvm-undname: Fix a crash-on-invalid
Found by oss-fuzz, fixes issues 12435 and 12438 on oss-fuzz.

Differential Revision: https://reviews.llvm.org/D60202

llvm-svn: 357646
2019-04-03 23:15:56 +00:00
Chandler Carruth
43ee626c3c Update more file headers across all of the LLVM projects in the monorepo
to reflect the new license. These used slightly different spellings that
defeated my regular expressions.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351648
2019-01-19 10:56:40 +00:00
Erik Pilkington
bb4d660781 NFC: Make the copies of the demangler byte-for-byte identical
With this patch, the copies of the files ItaniumDemangle.h,
StringView.h, and Utility.h are kept byte-for-byte in sync between
libcxxabi and llvm. All differences (namespaces, fallthrough, and
unreachable macros) are defined in each copies' DemanglerConfig.h.

This patch also adds a script to copy changes from libcxxabi
(cp-to-llvm.sh), and a README.txt explaining the situation.

Differential revision: https://reviews.llvm.org/D53538

llvm-svn: 351474
2019-01-17 20:37:51 +00:00
Zachary Turner
3475d3dfe8 [llvm-undname] Add support for demangling msvc's noexcept types.
Starting in C++17, MSVC introduced a new mangling for function
parameters that are themselves noexcept functions.  This patch
makes llvm-undname properly demangle them.

Patch by Zachary Henkel
Differential Revision: https://reviews.llvm.org/D55769

llvm-svn: 350656
2019-01-08 21:05:51 +00:00
Zachary Turner
0a0be9b76e [MS Demangler] Fail gracefully on invalid pointer types.
Once we detect a 'P', we know we a pointer type is upcoming, so
we make some assumptions about the output that follows.  If those
assumptions didn't hold, we would assert.  Instead, we should
fail gracefully and propagate the error up.

llvm-svn: 349169
2018-12-14 18:10:13 +00:00
Zachary Turner
fd70d7c7f5 Fix a crash in llvm-undname with invalid types.
llvm-svn: 349165
2018-12-14 17:43:56 +00:00
Nico Weber
6b832b197b Make initializeOutputStream() return false on error and true on success.
As discussed in https://reviews.llvm.org/D52104

Differential Revision: https://reviews.llvm.org/D52143

llvm-svn: 346606
2018-11-11 10:04:00 +00:00
Zachary Turner
e39ca229b9 [MS Demangler] Expose the Demangler AST publicly.
LLDB would like to use this in order to build a clang AST from
a mangled name.

This is NFC otherwise.

llvm-svn: 345837
2018-11-01 15:07:32 +00:00
Benjamin Kramer
1fe8373aeb Move some helpers from the global namespace into anonymous ones.
llvm-svn: 344468
2018-10-13 22:18:22 +00:00
Nico Weber
c2ace82589 Update microsoftDemangle() to work more like itaniumDemangle().
* Use same method of initializing the output stream and its buffer
* Allow a nullptr Status pointer
* Don't print the mangled name on demangling error
* Write to N (if it is non-nullptr)

Differential Revision: https://reviews.llvm.org/D52104

llvm-svn: 342330
2018-09-15 18:24:20 +00:00
Zachary Turner
474ddb04c8 Remove some debugging code that was accidentally left in.
llvm-svn: 341122
2018-08-30 21:00:57 +00:00
Zachary Turner
5c0fec24c6 [MS Demangler] Add support for $$Z parameter pack separator.
$$Z appears between adjacent expanded parameter packs in the
same template instantiation.  We don't need to print it, it's
only there to disambiguate between manglings that would otherwise
be ambiguous.  So we just need to parse it and throw it away.

llvm-svn: 341119
2018-08-30 20:53:29 +00:00
Zachary Turner
696ab67afc [MS Demangler] Fix several crashes and demangling bugs.
These bugs were found by writing a Python script which spidered
the entire Chromium build directory tree demangling every symbol
in every object file.  At the start, the tool printed:

  Processed 27443 object files.
  2926377/2936108 symbols successfully demangled (99.6686%)
  9731 symbols could not be demangled (0.3314%)
  14589 files crashed while demangling (53.1611%)

After this patch, it prints:

  Processed 27443 object files.
  41295518/41295617 symbols successfully demangled (99.9998%)
  99 symbols could not be demangled (0.0002%)
  0 files crashed while demangling (0.0000%)

The issues fixed in this patch are:

  * Ignore empty parameter packs.  Previously we would encounter
    a mangling for an empty parameter pack and add a null node
    to the AST.  Since we don't print these anyway, we now just
    don't add anything to the AST and ignore it entirely.  This
    fixes some of the crashes.

  * Account for "incorrect" string literal demanglings.  Apparently
    an older version of clang would not truncate mangled string
    literals to 32 bytes of encoded character data.  The demangling
    code however would allocate a 32 byte buffer thinking that it
    would not encounter more than this, and overrun the buffer.
    We now demangle up to 128 bytes of data, since the buggy
    clang would encode up to 32 *characters* of data.

  * Extended support for demangling init-fini stubs.  If you had
    something like
      struct Foo {
        static vector<string> S;
      };
    this would generate a dynamic atexit initializer *for the
    variable*.  We didn't handle this, but now we print something
    nice.  This is actually an improvement over undname, which will
    fail to demangle this at all.

  * Fixed one case of static this adjustment.  We weren't handling
    several thunk codes so we didn't recognize the mangling.  These
    are now handled.

  * Fixed a back-referencing problem.  Member pointer templates
    should have their components considered for back-referencing

The remaining 99 symbols which can't be demangled are all symbols
which are compiler-generated and undname can't demangle either.

llvm-svn: 341000
2018-08-29 23:56:09 +00:00
Zachary Turner
e121f14e93 Add support for various C++14 demanglings.
Mostly this includes <auto> and <decltype-auto> return values.
Additionally, this fixes a fairly obscure back-referencing bug
that was encountered in one of the C++14 tests, which is that
if you have something like Foo<&bar, &bar> then the `bar`
forms a backreference.

llvm-svn: 340896
2018-08-29 04:12:44 +00:00
Zachary Turner
4217aad9fe [MS Demangler] Add output flags to all function calls.
Previously we had a FunctionSigFlags, but it's more flexible
to just have one set of output flags that apply to the entire
process and just pipe the entire set of flags through the
output process.

This will be useful when we start allowing the user to customize
the outputting behavior.

llvm-svn: 340894
2018-08-29 03:59:17 +00:00
Zachary Turner
b7710f8516 [MS Demangler] Re-write the Microsoft demangler.
This is a pretty large refactor / re-write of the Microsoft
demangler.  The previous one was a little hackish because it
evolved as I was learning about all the various edge cases,
exceptions, etc.  It didn't have a proper AST and so there was
lots of custom handling of things that should have been much
more clean.

Taking what was learned from that experience, it's now
re-written with a completely redesigned and much more sensible
AST.  It's probably still not perfect, but at least it's
comprehensible now to someone else who wants to come along
and make some modifications or read the code.

Incidentally, this fixed a couple of bugs, so I've enabled
the tests which now pass.

llvm-svn: 340710
2018-08-27 03:48:03 +00:00
Simon Pilgrim
6cb50cc2de Fix -Wunused-function warning. NFCI.
llvm-svn: 340687
2018-08-25 17:11:11 +00:00
Zachary Turner
9be9e38aa6 [MS Demangler] Print template constructor args.
Previously if you had something like this:

template<typename T>
struct Foo {
  template<typename U>
  Foo(U);
};

Foo F(3.7);

this would mangle as ??$?0N@?$Foo@H@@QEAA@N@Z

and this would be demangled as:

undname:      __cdecl Foo<int>::Foo<int><double>(double)
llvm-undname: __cdecl Foo<int>::Foo<int>(double)

Note the lack of the constructor template parameter in our
demangling.

This patch makes it so we print the constructor argument list.

llvm-svn: 340356
2018-08-21 22:52:52 +00:00
Zachary Turner
7a7fb4f525 [MS Demangler] Fix a few more edge cases.
I found these by running llvm-undname over a couple hundred
megabytes of object files generated as part of building chromium.
The issues fixed in this patch are:

  1) decltype-auto return types.
  2) Indirect vtables (e.g. const A::`vftable'{for `B'})
  3) Pointers, references, and rvalue-references to member pointers.

I have exactly one remaining symbol out of a few hundred MB of object
files that produces a name we can't demangle, and it's related to
back-referencing.

llvm-svn: 340341
2018-08-21 21:23:49 +00:00
Zachary Turner
234e63078e [MS Demangler] Demangle special operator 'dynamic initializer'.
This is encoded as __E and should print something like
"dynamic initializer for 'Foo'(void)"

This also adds support for dynamic atexit destructor, which is
basically identical but encoded as __F with slightly different
description.

llvm-svn: 340239
2018-08-20 23:59:21 +00:00
Zachary Turner
ac6c3f6394 [MS Demangler] Anonymous namespace hashes can be backreferenced.
Previously we were not remembering the key values of anonymous
namespaces, but we need to do this.

llvm-svn: 340238
2018-08-20 23:58:58 +00:00
Zachary Turner
41eec7abf4 [MS Demangler] Properly demangle anonymous namespaces.
llvm-svn: 340237
2018-08-20 23:58:35 +00:00
Richard Smith
933a42da48 Move Itanium demangler implementation into a header file and add visitation support.
Summary:
This transforms the Itanium demangler into a generic reusable library that can
be used to build, traverse, and transform Itanium mangled name trees.

This is in preparation for adding a canonicalizing demangler, which
cannot live in the Demangle library for layering reasons. In order to
keep the diffs simpler, this patch moves more code to the new header
than is strictly necessary: in particular, all of the printLeft /
printRight implementations can be moved to the implementation file.
(And indeed we could make them non-virtual now if we wished, and remove
the vptr from Node.)

All nodes are now included in the Kind enumeration, rather than omitting
some of the Expr nodes, and the three different floating-point literal
node types now have distinct Kind values.

As a proof of concept for the visitation / matching mechanism, this
patch implements a Node dumping facility on top of it, replacing the
prior mechanism that produced the pretty-printed output rather than a
tree dump. Sample dump output:

FunctionEncoding(
  NameType("int"),
  NameWithTemplateArgs(
    NestedName(
      NameWithTemplateArgs(
        NameType("A"),
        TemplateArgs(
          {NameType("B")})),
      NameType("f")),
    TemplateArgs(
      {NameType("int")})),
  {},
  <null>,
  QualConst, FunctionRefQual::FrefQualLValue)

As a next step, it would make sense to move the LLVM high-level interface to
the demangler (the itaniumDemangler function and ItaniumPartialDemangler class)
into the Support library, and implement them in terms of the Demangle library.
This would allow the libc++abi demangler implementation to be an identical copy
of the llvm Demangle library, and would allow the LLVM implementation to reuse
LLVM components such as llvm::BumpPtrAllocator, but we'll need to decide how to
coordinate that with the MS ABI demangler, so I'm not doing that in this patch.

No functionality change intended other than the behavior of dump().

Reviewers: erik.pilkington, zturner, chandlerc, dlj

Subscribers: aheejin, llvm-commits

Differential Revision: https://reviews.llvm.org/D50930

llvm-svn: 340203
2018-08-20 19:44:01 +00:00
Zachary Turner
08e75fdaa7 [MS Demangler] Demangle member pointer template parameters.
llvm-svn: 340199
2018-08-20 19:15:35 +00:00
Zachary Turner
5f102a9ff2 [MS Demangler] Resolve backreferences eagerly, not lazily.
A while back I submitted a patch to resolve backreferences
lazily, thinking this that it was not always possible to know
in advance what type you were looking at until you had completed
a full pass over the input, and therefore it would be impossible
to resolve backreferences eagerly.

This was mistaken though, and turned out to be an unrelated
problem.  In fact, the reverse is true.  You *must* resolve
backreferences eagerly.  This is because certain types of nested
mangled symbols do not share a backreference context with their
parent symbol, and as such, if you try to resolve them lazily
their backreference context will have been lost by the time you
finish demangling the entire input.  On the other hand, resolving
them eagerly appears to always work, and enables us to port
many more tests over.

llvm-svn: 340126
2018-08-18 18:49:48 +00:00
Zachary Turner
491ab30567 [MS Demangler] Properly print all thunk types.
We were only printing the vtordisp thunk before as the previous
patch was more aimed at getting special operators working, one
of which was a thunk.  This patch gets all thunk types to print
properly, and adds a test for each one.

llvm-svn: 340088
2018-08-17 21:32:07 +00:00
Zachary Turner
0387f750b1 [MS Demangler] Demangle all remaining types of operators.
This demangles all remaining special operators including thunks,
RTTI Descriptors, and local static guard variables.

llvm-svn: 340083
2018-08-17 21:18:05 +00:00
Zachary Turner
75b475a1c8 [MS Demangler] Rework the way operators are demangled.
Previously, some of the code for actually parsing mangled
operator names was more like formatting code in nature,
and was interspersed with the demangling code which builds
the AST.  This means that by the time we got to the printing
code, we had lost all information about what type of operator
we had, and all we were left with was a string that we just
had to print.  However, not all operators are actually even
operators.  it's basically just a catch-all mangling for
"special names", and for some of the other types it helps
to know when we're actually doing the printing what it is.

This patch changes the way things work by introducing an
OperatorInfo structure and corresponding enumeration.  When
we demangle we store the enumeration value and demangled
components separately.  This gives more flexibility during
printing.

In doing so, some demanglings of special names which we didn't
previously support come out of this for free, so we now demangle
those.

A few are more complex and are better left for a followup patch
though.

An exhaustive test of every possible operator code is included,
with the ones that don't yet work commented out.

llvm-svn: 340046
2018-08-17 16:14:05 +00:00
Zachary Turner
5c97e44a05 Fix memory leak in demangling of string literals.
llvm-svn: 339909
2018-08-16 17:48:32 +00:00
Zachary Turner
d8420c8c15 Fix -Wmicrosoft-goto warnings.
llvm-svn: 339894
2018-08-16 16:30:27 +00:00
Zachary Turner
98b577be85 [MS Demangler] Demangle string literals.
When demangling string literals, Microsoft's undname
simply prints 'string'.  This patch implements string
literal demangling while doing a bit better than this
by decoding as much of the string as possible and
trying to faithfully reproduce the original string
literal definition.

This is a bit tricky because the different character
types char, char16_t, and char32_t are not uniquely
identified by the mangling, so we have to use a
heuristic to try to guess the character type.  But
it works pretty well, and many tests are added to
illustrate the behavior.

Differential Revision: https://reviews.llvm.org/D50806

llvm-svn: 339892
2018-08-16 16:17:36 +00:00
Zachary Turner
8dd04a5968 [MS Demangler] Don't fail on MD5-mangled names.
When we have an MD5 mangled name, we shouldn't choke and say
that it's an invalid name.  Even though it's impossible to demangle,
we should just output the original name.

llvm-svn: 339891
2018-08-16 16:17:17 +00:00
Zachary Turner
609d36082b [MS Demangler] Fix some minor formatting bugs.
1) We print __restrict twice on member pointers.  This is fixed
   and relevant tests are re-enabled.

2) Several tests were disabled because of printing slightly
   different output than undname.  These were confirmed to be
   bugs in undname, so we just re-enable the tests.

3) The test for printing reference temporaries is re-enabled.  This
   is a clang mangling extension, so we have some flexibility with
   how we demangle it.  The output currently looks fine, so we just
   re-enable the test with no fixes.

llvm-svn: 339708
2018-08-14 18:54:28 +00:00
Zachary Turner
f5224914dd [MS Demangler] Support extern "C" functions.
There are two cases we need to support with extern "C"
functions.  The first is the case of a '9' indicating that
the function has no prototype.  This occurs when we mangle
a symbol inside of an extern "C" function, but not the
function itself.

The second case is when we have an overloaded extern "C"
functions.  In this case we emit $$J0 to indicate this.
This patch adds support for both of these cases.

llvm-svn: 339471
2018-08-10 21:09:05 +00:00
Zachary Turner
808fcfa26f [MS Demangler] Demangle cv qualifiers on template args.
Before we wouldn't properly demangle something like
Foo<const int>.  Template args have a special escape sequence
'$$C' that is optional, but if it is present contains
qualifiers.  So we need to check for this and only if it
present, demangle qualifiers before demangling the type.

With this fix, we re-enable some tests that were previously
marked FIXME.

llvm-svn: 339465
2018-08-10 19:57:36 +00:00
Zachary Turner
47a3242786 [MS Demangler] Properly demangle conversion operators.
These were completely broken before.  We need to handle
the 'B' operator tag.

llvm-svn: 339436
2018-08-10 15:04:56 +00:00
Zachary Turner
42c8ed3fec [MS Demangler] Fix several issues related to templates.
These were uncovered when porting the mangling tests in
ms-templates.cpp from clang/CodeGenCXX over to demangling
tests.  The main issues fixed here are surrounding integer
literal signed and unsignedness, empty array dimensions,
and pointer and reference non-type template parameters.

Differential Revision: https://reviews.llvm.org/D50512

llvm-svn: 339434
2018-08-10 14:31:04 +00:00
Zachary Turner
21da3ecbe2 [MS Demangler] Create a new backref context for template instantiations.
Template manglings use a fresh back-referencing context, so we
need to do the same.  This fixes several existing tests which are
marked as FIXME, so those are now actually run.

llvm-svn: 339275
2018-08-08 17:17:04 +00:00
Zachary Turner
5651b20005 [MS Demangler] Properly handle backreferencing of special names.
Function template names are not stored in the backref table,
but non-template function names are.  The general pattern seems
to be that when you are demangling a symbol name, if the name
starts with '?' it does not go into the backreference table,
otherwise it does.  Note that this even handles the general case
of operator names (template or otherwise) not going into the
back-reference table, anonymous namespaces not going into the
backreference table, etc.

It's important that we apply this check *only* for the
unqualified portion of a name, and only for symbol names.
For example, this does not apply to type names (such as class
templates) and we need to make sure that these still do go
into the backref table.

Differential Revision: https://reviews.llvm.org/D50394

llvm-svn: 339211
2018-08-08 00:43:31 +00:00
Zachary Turner
474432490e Fix one more warning.
llvm-svn: 338742
2018-08-02 17:33:33 +00:00
Zachary Turner
1fe2393239 Fix a couple of warnings.
llvm-svn: 338739
2018-08-02 17:18:01 +00:00
Zachary Turner
5f1a862f35 Use %.*s instead of %*s when formatting strings with explicit length.
llvm-svn: 338737
2018-08-02 17:08:24 +00:00
Zachary Turner
a1fabd04e3 [MS Demangler] Resolve back-references lazily.
We need to both record and resolve back-references lazily due to
not being able to know until a demangling is complete whether or
not a name should go into the back-reference table..  This patch
implements lazy resolution of back-references, but we still have
eager recording of back-references.  This will be fixed in a
subsequent patch.

llvm-svn: 338736
2018-08-02 17:08:03 +00:00
Zachary Turner
1a7296e0a4 Try to fix FreeBSD build.
It seems like perhaps because cstdio isn't directly included, the
compiler is accidentally picking up wprintf from somewhere else
and trying to call that.  Hopefully this fixes it.

llvm-svn: 338614
2018-08-01 18:44:12 +00:00
Zachary Turner
5d0ac84a37 [llvm-undname Add an option to dump back references.
This is useful for understanding how our demangler processes
back references and for investigating issues related to
back references.  But it's a feature only useful for debugging
the demangling process itself, so I'm marking it hidden.

llvm-svn: 338609
2018-08-01 18:33:04 +00:00
Zachary Turner
eb21f12ca7 [MS Demangler] Properly demangle templated operators.
After we detected the presence of a template via ?$ we would proceed by
only demangling a simple unqualified name. This means we would fail on
templated operators (and perhaps other yet-to-be-determined things)

This was discovered while doing some refactoring to store richer
semantic information about the demangled types to pave the way for
overhauling the way we handle backreferences. (Specifically, we need to
defer recording or resolving back-references until a symbol has been
completely demangled, because we need to use information that only
occurs later in the mangled string to decide whether a back-reference
should be recorded.)

Differential Revision: https://reviews.llvm.org/D50145

llvm-svn: 338608
2018-08-01 18:32:47 +00:00
Zachary Turner
87e9e4e392 [MS Demangler] Don't crash as often when demangling.
We crash a lot on unrecognized inputs.  This adds some error
handling so we early out when we get unrecognized names.

llvm-svn: 338607
2018-08-01 18:32:28 +00:00
Zachary Turner
b435cc5faa Resubmit r338340 "[MS Demangler] Better demangling of template arguments."
This broke the build with GCC, but has since been fixed.

llvm-svn: 338403
2018-07-31 17:16:44 +00:00