llvm-capstone/lld/ELF/DriverUtils.cpp
Ben Dunbobbin 1d16515fb4 [ELF] Implement Dependent Libraries Feature
This patch implements a limited form of autolinking primarily designed to allow
either the --dependent-library compiler option, or "comment lib" pragmas (
https://docs.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp?view=vs-2017) in
C/C++ e.g. #pragma comment(lib, "foo"), to cause an ELF linker to automatically
add the specified library to the link when processing the input file generated
by the compiler.

Currently this extension is unique to LLVM and LLD. However, care has been taken
to design this feature so that it could be supported by other ELF linkers.

The design goals were to provide:

- A simple linking model for developers to reason about.
- The ability to to override autolinking from the linker command line.
- Source code compatibility, where possible, with "comment lib" pragmas in other
  environments (MSVC in particular).

Dependent library support is implemented differently for ELF platforms than on
the other platforms. Primarily this difference is that on ELF we pass the
dependent library specifiers directly to the linker without manipulating them.
This is in contrast to other platforms where they are mapped to a specific
linker option by the compiler. This difference is a result of the greater
variety of ELF linkers and the fact that ELF linkers tend to handle libraries in
a more complicated fashion than on other platforms. This forces us to defer
handling the specifiers to the linker.

In order to achieve a level of source code compatibility with other platforms
we have restricted this feature to work with libraries that meet the following
"reasonable" requirements:

1. There are no competing defined symbols in a given set of libraries, or
   if they exist, the program owner doesn't care which is linked to their
   program.
2. There may be circular dependencies between libraries.

The binary representation is a mergeable string section (SHF_MERGE,
SHF_STRINGS), called .deplibs, with custom type SHT_LLVM_DEPENDENT_LIBRARIES
(0x6fff4c04). The compiler forms this section by concatenating the arguments of
the "comment lib" pragmas and --dependent-library options in the order they are
encountered. Partial (-r, -Ur) links are handled by concatenating .deplibs
sections with the normal mergeable string section rules. As an example, #pragma
comment(lib, "foo") would result in:

.section ".deplibs","MS",@llvm_dependent_libraries,1
         .asciz "foo"

For LTO, equivalent information to the contents of a the .deplibs section can be
retrieved by the LLD for bitcode input files.

LLD processes the dependent library specifiers in the following way:

1. Dependent libraries which are found from the specifiers in .deplibs sections
   of relocatable object files are added when the linker decides to include that
   file (which could itself be in a library) in the link. Dependent libraries
   behave as if they were appended to the command line after all other options. As
   a consequence the set of dependent libraries are searched last to resolve
   symbols.
2. It is an error if a file cannot be found for a given specifier.
3. Any command line options in effect at the end of the command line parsing apply
   to the dependent libraries, e.g. --whole-archive.
4. The linker tries to add a library or relocatable object file from each of the
   strings in a .deplibs section by; first, handling the string as if it was
   specified on the command line; second, by looking for the string in each of the
   library search paths in turn; third, by looking for a lib<string>.a or
   lib<string>.so (depending on the current mode of the linker) in each of the
   library search paths.
5. A new command line option --no-dependent-libraries tells LLD to ignore the
   dependent libraries.

Rationale for the above points:

1. Adding the dependent libraries last makes the process simple to understand
   from a developers perspective. All linkers are able to implement this scheme.
2. Error-ing for libraries that are not found seems like better behavior than
   failing the link during symbol resolution.
3. It seems useful for the user to be able to apply command line options which
   will affect all of the dependent libraries. There is a potential problem of
   surprise for developers, who might not realize that these options would apply
   to these "invisible" input files; however, despite the potential for surprise,
   this is easy for developers to reason about and gives developers the control
   that they may require.
4. This algorithm takes into account all of the different ways that ELF linkers
   find input files. The different search methods are tried by the linker in most
   obvious to least obvious order.
5. I considered adding finer grained control over which dependent libraries were
   ignored (e.g. MSVC has /nodefaultlib:<library>); however, I concluded that this
   is not necessary: if finer control is required developers can fall back to using
   the command line directly.

RFC thread: http://lists.llvm.org/pipermail/llvm-dev/2019-March/131004.html.

Differential Revision: https://reviews.llvm.org/D60274

llvm-svn: 360984
2019-05-17 03:44:15 +00:00

254 lines
8.7 KiB
C++

//===- DriverUtils.cpp ----------------------------------------------------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
//
// This file contains utility functions for the driver. Because there
// are so many small functions, we created this separate file to make
// Driver.cpp less cluttered.
//
//===----------------------------------------------------------------------===//
#include "Driver.h"
#include "lld/Common/ErrorHandler.h"
#include "lld/Common/Memory.h"
#include "lld/Common/Reproduce.h"
#include "lld/Common/Version.h"
#include "llvm/ADT/Optional.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/Triple.h"
#include "llvm/Option/Option.h"
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/Process.h"
using namespace llvm;
using namespace llvm::sys;
using namespace llvm::opt;
using namespace lld;
using namespace lld::elf;
// Create OptTable
// Create prefix string literals used in Options.td
#define PREFIX(NAME, VALUE) const char *const NAME[] = VALUE;
#include "Options.inc"
#undef PREFIX
// Create table mapping all options defined in Options.td
static const opt::OptTable::Info OptInfo[] = {
#define OPTION(X1, X2, ID, KIND, GROUP, ALIAS, X7, X8, X9, X10, X11, X12) \
{X1, X2, X10, X11, OPT_##ID, opt::Option::KIND##Class, \
X9, X8, OPT_##GROUP, OPT_##ALIAS, X7, X12},
#include "Options.inc"
#undef OPTION
};
ELFOptTable::ELFOptTable() : OptTable(OptInfo) {}
// Set color diagnostics according to -color-diagnostics={auto,always,never}
// or -no-color-diagnostics flags.
static void handleColorDiagnostics(opt::InputArgList &Args) {
auto *Arg = Args.getLastArg(OPT_color_diagnostics, OPT_color_diagnostics_eq,
OPT_no_color_diagnostics);
if (!Arg)
return;
if (Arg->getOption().getID() == OPT_color_diagnostics) {
errorHandler().ColorDiagnostics = true;
} else if (Arg->getOption().getID() == OPT_no_color_diagnostics) {
errorHandler().ColorDiagnostics = false;
} else {
StringRef S = Arg->getValue();
if (S == "always")
errorHandler().ColorDiagnostics = true;
else if (S == "never")
errorHandler().ColorDiagnostics = false;
else if (S != "auto")
error("unknown option: --color-diagnostics=" + S);
}
}
static cl::TokenizerCallback getQuotingStyle(opt::InputArgList &Args) {
if (auto *Arg = Args.getLastArg(OPT_rsp_quoting)) {
StringRef S = Arg->getValue();
if (S != "windows" && S != "posix")
error("invalid response file quoting: " + S);
if (S == "windows")
return cl::TokenizeWindowsCommandLine;
return cl::TokenizeGNUCommandLine;
}
if (Triple(sys::getProcessTriple()).getOS() == Triple::Win32)
return cl::TokenizeWindowsCommandLine;
return cl::TokenizeGNUCommandLine;
}
// Gold LTO plugin takes a `--plugin-opt foo=bar` option as an alias for
// `--plugin-opt=foo=bar`. We want to handle `--plugin-opt=foo=` as an
// option name and `bar` as a value. Unfortunately, OptParser cannot
// handle an option with a space in it.
//
// In this function, we concatenate command line arguments so that
// `--plugin-opt <foo>` is converted to `--plugin-opt=<foo>`. This is a
// bit hacky, but looks like it is still better than handling --plugin-opt
// options by hand.
static void concatLTOPluginOptions(SmallVectorImpl<const char *> &Args) {
SmallVector<const char *, 256> V;
for (size_t I = 0, E = Args.size(); I != E; ++I) {
StringRef S = Args[I];
if ((S == "-plugin-opt" || S == "--plugin-opt") && I + 1 != E) {
V.push_back(Saver.save(S + "=" + Args[I + 1]).data());
++I;
} else {
V.push_back(Args[I]);
}
}
Args = std::move(V);
}
// Parses a given list of options.
opt::InputArgList ELFOptTable::parse(ArrayRef<const char *> Argv) {
// Make InputArgList from string vectors.
unsigned MissingIndex;
unsigned MissingCount;
SmallVector<const char *, 256> Vec(Argv.data(), Argv.data() + Argv.size());
// We need to get the quoting style for response files before parsing all
// options so we parse here before and ignore all the options but
// --rsp-quoting.
opt::InputArgList Args = this->ParseArgs(Vec, MissingIndex, MissingCount);
// Expand response files (arguments in the form of @<filename>)
// and then parse the argument again.
cl::ExpandResponseFiles(Saver, getQuotingStyle(Args), Vec);
concatLTOPluginOptions(Vec);
Args = this->ParseArgs(Vec, MissingIndex, MissingCount);
handleColorDiagnostics(Args);
if (MissingCount)
error(Twine(Args.getArgString(MissingIndex)) + ": missing argument");
for (auto *Arg : Args.filtered(OPT_UNKNOWN)) {
std::string Nearest;
if (findNearest(Arg->getAsString(Args), Nearest) > 1)
error("unknown argument '" + Arg->getSpelling() + "'");
else
error("unknown argument '" + Arg->getSpelling() + "', did you mean '" +
Nearest + "'");
}
return Args;
}
void elf::printHelp() {
ELFOptTable().PrintHelp(
outs(), (Config->ProgName + " [options] file...").str().c_str(), "lld",
false /*ShowHidden*/, true /*ShowAllAliases*/);
outs() << "\n";
// Scripts generated by Libtool versions up to at least 2.4.6 (the most
// recent version as of March 2017) expect /: supported targets:.* elf/
// in a message for the -help option. If it doesn't match, the scripts
// assume that the linker doesn't support very basic features such as
// shared libraries. Therefore, we need to print out at least "elf".
outs() << Config->ProgName << ": supported targets: elf\n";
}
static std::string rewritePath(StringRef S) {
if (fs::exists(S))
return relativeToRoot(S);
return S;
}
// Reconstructs command line arguments so that so that you can re-run
// the same command with the same inputs. This is for --reproduce.
std::string elf::createResponseFile(const opt::InputArgList &Args) {
SmallString<0> Data;
raw_svector_ostream OS(Data);
OS << "--chroot .\n";
// Copy the command line to the output while rewriting paths.
for (auto *Arg : Args) {
switch (Arg->getOption().getUnaliasedOption().getID()) {
case OPT_reproduce:
break;
case OPT_INPUT:
OS << quote(rewritePath(Arg->getValue())) << "\n";
break;
case OPT_o:
// If -o path contains directories, "lld @response.txt" will likely
// fail because the archive we are creating doesn't contain empty
// directories for the output path (-o doesn't create directories).
// Strip directories to prevent the issue.
OS << "-o " << quote(sys::path::filename(Arg->getValue())) << "\n";
break;
case OPT_dynamic_list:
case OPT_library_path:
case OPT_rpath:
case OPT_script:
case OPT_symbol_ordering_file:
case OPT_sysroot:
case OPT_version_script:
OS << Arg->getSpelling() << " " << quote(rewritePath(Arg->getValue()))
<< "\n";
break;
default:
OS << toString(*Arg) << "\n";
}
}
return Data.str();
}
// Find a file by concatenating given paths. If a resulting path
// starts with "=", the character is replaced with a --sysroot value.
static Optional<std::string> findFile(StringRef Path1, const Twine &Path2) {
SmallString<128> S;
if (Path1.startswith("="))
path::append(S, Config->Sysroot, Path1.substr(1), Path2);
else
path::append(S, Path1, Path2);
if (fs::exists(S))
return S.str().str();
return None;
}
Optional<std::string> elf::findFromSearchPaths(StringRef Path) {
for (StringRef Dir : Config->SearchPaths)
if (Optional<std::string> S = findFile(Dir, Path))
return S;
return None;
}
// This is for -l<basename>. We'll look for lib<basename>.so or lib<basename>.a from
// search paths.
Optional<std::string> elf::searchLibraryBaseName(StringRef Name) {
for (StringRef Dir : Config->SearchPaths) {
if (!Config->Static)
if (Optional<std::string> S = findFile(Dir, "lib" + Name + ".so"))
return S;
if (Optional<std::string> S = findFile(Dir, "lib" + Name + ".a"))
return S;
}
return None;
}
// This is for -l<namespec>.
Optional<std::string> elf::searchLibrary(StringRef Name) {
if (Name.startswith(":"))
return findFromSearchPaths(Name.substr(1));
return searchLibraryBaseName (Name);
}
// If a linker/version script doesn't exist in the current directory, we also
// look for the script in the '-L' search paths. This matches the behaviour of
// '-T', --version-script=, and linker script INPUT() command in ld.bfd.
Optional<std::string> elf::searchScript(StringRef Name) {
if (fs::exists(Name))
return Name.str();
return findFromSearchPaths(Name);
}