This PR adds options to let CMake calculate the ninja job pools
depending on free memory and available cores.
You can provide memory requirements for each compile and link job which
is checked against CMake AVAILABLE_PHYSICAL_MEMORY and
NUMBER_OF_LOGICAL_CORES. [This information are available since CMake
3.0](https://cmake.org/cmake/help/v3.0/command/cmake_host_system_information.html).
This is very helpful in CI environments with multiple jobs per
environment or a VM with multiple users.
Its different to LLVM_PARALLEL_LINK_JOBS / LLVM_PARALLEL_COMPILE_JOBS
(or ninja -j 1) because it tries to use the resources more efficient
without being terminated. Only downside currently is that compile and
link jobs can run at the same time so there is an offset for link job
memory suggested which is added to the documentation.
The definitions aren't added as cache because if I understand it
correctly this would break it because values could be outdated.
If a CMake project doesn't enable the C language, then the CMake FFI and
Terminfo find modules will fail their checks for compilation and
linking.
This commit allows projects to enable only C++ by first checking if a C
compiler is set before testing C source compilation; if not, it checks
whether C++ compilation succeeds.
Fixes#53950
`CMAKE_{C/CXX}_FLAGS` affects all targets in LLVM. This can
be undesirable in situations, like the case of enabling thinLTO,
where `-flto` is added to every source file. In reality, we only
care about optimizing a select few of binaries, such as clang or lld,
that dominate the compilation pipeline. Auxiliary binaries in a
distribution and not on the critical path can be kept non-optimized.
This PR adds support of per-target linker flags, which can solve the
thinLTO problem by negating the effects of LTO via targeted linker
flags on the targets. The example of negating thinLTO
above can be done by doing the following:
```
set(LLVM_llvm-dwarfdump_LINKER_FLAGS "-Wl,--lto-O0" CACHE STRING "Custom linker flags to llvm-dwarfdump")
set(LLVM_lldb_LINKER_FLAGS "-Wl,--lto-O0" CACHE STRING "Custom linker flags to lldb")
```
There's other applications where this could be used (e.g. avoid
optimizing host tools for build speed improvement etc.).
I've generalized this so that users can apply their desired flags to
targets that are generated by `llvm_add_library` or
`add_llvm_executable`.
Internally, our toolchain builds were on average 1.4x faster when
selectively choosing the binaries that we want optimized.
...behind an experimental CMAKE option that's off by default.
This patch adds a new ilist-iterator-like class that can carry two extra bits
as well as the usual node pointer. This is part of the project to remove
debug-intrinsics from LLVM: see the rationale here [0], they're needed to
signal whether a "position" in a BasicBlock includes any debug-info before or
after the iterator.
This entirely duplicates ilist_iterator, attempting re-use showed it to be a
false economy. It's enable-able through the existing ilist_node options
interface, hence a few sites where the instruction-list type needs to be
updated. The actual main feature, the extra bits in the class, aren't part of
the class unless the cmake flag is given: this is because there's a
compile-time cost associated with it, and I'd like to get everything in-tree
but off-by-default so that we can do proper comparisons.
Nothing actually makes use of this yet, but will do soon, see the Phab patch
stack.
[0] https://discourse.llvm.org/t/rfc-instruction-api-changes-needed-to-eliminate-debug-intrinsics-from-ir/68939
Differential Revision: https://reviews.llvm.org/D153777
Currently, If the C or C++ compiler supports the `-gsplit-dwarf` option it is added to _all_ compilers.
If a project decides to use another language, such as Swift, this option will be sent to that compiler as well, regardless whether that compiler supports it or not (Swift doesnot).
This patch uses [generator expressions](https://cmake.org/cmake/help/latest/manual/cmake-generator-expressions.7.html) to limit the `-gsplit-dwarf` option to only those compilers that support it (C and C++).
It's no longer possible to submit bitcode apps to the Apple App Store.
The tools
used to create xar archived bitcode sections inside MachO files have
been
discontinued. Additionally, the xar APIs have been deprecated since
macOS 12,
so this change removes unnecessary code from objdump and all
dependencies on
libxar.
This fixes rdar://116600767
This is one of the most common issues new users face, especially as so
many reccomended CMake configurations include `-DLLVM_ENABLE_LLD=ON`.
I don't want the error message to get too long but let's at least say
that there are 2 main reasons for the failure. If it's not those then
maybe folks will find the actual problem while trying to discount them.
The new message looks like:
```
CMake Error at cmake/modules/HandleLLVMOptions.cmake:330 (message):
Host compiler does not support '-fuse-ld=not_a_linker'. Please make sure
that 'not_a_linker' is installed and that your host compiler can compile a
simple program when given the option '-fuse-ld=not_a_linker'.
Call Stack (most recent call first):
CMakeLists.txt:910 (include)
```
This patch adds a LLVM_FORCE_VC_REVISION option to force a custom
VC revision to be included instead of trying to fetch one from a
git command. This is helpful in environments where git is not
available or is non-functional but the vc revision is available
through some other means.
This cmake rule is used by external clients, who may or may not have
the LLVM_LIBRARY_OUTPUT_INTDIR variable set.
If it is not set, then we pass `-Wl,-rpath-link,` to the compiler. It
turns out that gcc and clang interpret this differently.
* gcc passes `-rpath-link ""` to the linker, which is what we want.
* clang passes `-rpath-link` to the linker. This is not what we want,
because then the linker gobbles the next command-line argument,
whatever it happens to be, and uses it as the -rpath-link target.
Fix this by passing -rpath-link only if we actually have a path we want.
This has been deprecated in favor of CMake's CMAKE_MSVC_RUNTIME_LIBRARY
in c6bd873403 .
Current release branch still contains it in deprecated status so no
existing end-users will be affected.
Otherwise CMake might throw and error:
CMake Error: try_run() invoked in cross-compiling mode, please set the
following cache variables appropriately:
Z3_RETURNCODE (advanced)
Z3_RETURNCODE__TRYRUN_OUTPUT (advanced)
This patch supports GNU ld on Solaris in addition to Solaris ld, the
default.
- Linker selection is dynamic: one can switch between Solaris ld and GNU ld
at runtime, with the default selectable with `-DCLANG_DEFAULT_LINKER`.
- Testcases have been adjusted to test both variants in case there are
differences.
- The `compiler-rt/cmake/config-ix.cmake` and
`llvm/cmake/modules/AddLLVM.cmake` changes to restrict the tests to
Solaris ld are necessary because GNU accepts unknown `-z` options, but
warns every time they are used, creating a lot of noise. Since there
seems to be no way to check for those warnings in
`llvm_check_compiler_linker_flag` or `llvm_check_compiler_linker_flag`, I
restrict the cmake tests to Solaris ld in the first place.
- The changes to `clang/test/Driver/hip-link-bundle-archive.hip` and
`flang/test/Driver/linker-flags.f90` are required when LLVM is built with
`-DCLANG_DEFAULT_LINKER=gld` on Solaris: `MSVC.cpp`
`visualstudio::Linker::ConstructJob` ultimately calls
`GetProgramPath("gld")`, resulting in a search for `gld`, which exists in
`/usr/bin/gld` on Solaris. With `-fuse-ld=`, this doesn't happen and the
expected `link` is returned.
- `compiler-rt/test/asan/TestCases/global-location-nodebug.cpp` needs to
enforce the Solaris ld, otherwise the test would `XPASS` with GNU ld
which has the `-S` semantics expected by the test.
Tested on `amd64-pc-solaris2.11` and `sparcv9-sun-solaris2.11` with both
`-DCLANG_DEFAULT_LINKER=gld` and the default, and `x86_64-pc-linux-gnu`.
No regressions in either case.
Differential Revision: https://reviews.llvm.org/D85309
Running lit tests on Windows can fail because its use of
`os.path.realpath` expands substitute drives, which are used to keep
paths short and avoid hitting MAX_PATH limitations.
Changes lit logic to:
Use `os.path.abspath` on Windows, where `MAX_PATH` is a concern that we
can work around using substitute drives, which `os.path.realpath` would
resolve.
Use `os.path.realpath` on Unix, where the current directory always has
symlinks resolved, so it is impossible to preserve symlinks in the
presence of relative paths, and so we must make sure that all code paths
use real paths.
Also updates clang's `FileManager::getCanonicalName` and `ExtractAPI`
code to avoid resolving substitute drives (i.e. resolving to a path
under a different root).
How tested: built with `-DLLVM_ENABLE_PROJECTS=clang` and built `check-all` on both Windows
Differential Revision: https://reviews.llvm.org/D154130
Reviewed By: @benlangmuir
Patch by Tristan Labelle <tristan@thebrowser.company>!
GCC 7.3 cannot build 16.x releases.
```
In file included from /tmp/llvm-16/llvm/lib/Transforms/IPO/AttributorAttributes.cpp:14:0:
/tmp/llvm-16/llvm/include/llvm/Transforms/IPO/Attributor.h:1137:32: error: duplicate initialization of ‘llvm::AnalysisGetter::HasLegacyWrapper<Analysis, std::void_t<typename Analysis::Lega
cyWrapper> >’
constexpr bool AnalysisGetter::HasLegacyWrapper<
^~~~~~~~~~~~~~~~~
Analysis, std::void_t<typename Analysis::LegacyWrapper>> = true;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/llvm-16/llvm/include/llvm/Transforms/IPO/Attributor.h:1137:32: error: got 1 template parameters for ‘constexpr const bool llvm::AnalysisGetter::HasLegacyWrapper< <template-parameter-1
-1>, <template-parameter-1-2> >’
/tmp/llvm-16/llvm/include/llvm/Transforms/IPO/Attributor.h:1137:32: error: but 2 required
```
The 17.x and main branches have more failures, e.g.
```
llvm/lib/Transforms/Vectorize/LoopVectorize.cpp: `error: cannot decompose class type ‘std::pair<llvm::Value*, const llvm::SCEV*>’: ...`
```
We probably should just give up 7.1 and say that GCC<=7.3 is unsupported.
There is evidence that GCC 7.4 works.
I have verified that GCC 7.5 is able to build `check-{llvm,clang,clang-tools,lldb,lld,polly,mlir,bolt}`,
but not flang due to at least `flang/Common/enum-class.h` and a `<charconv`> in a unittest.
Link: https://discourse.llvm.org/t/require-gcc-7-5-as-gcc-7-3-cannot-build-llvm/72310
Reviewed By: aaron.ballman
Differential Revision: https://reviews.llvm.org/D156286
This patch is the first part of https://llvm.org/OpenProjects.html#llvm_patch_coverage.
We have first define a new variable LLVM_TEST_COVERAGE which when set, pass --per-test-coverage option to
llvm-lit which will help in setting a unique value to LLVM_PROFILE_FILE for each RUN. So for example
coverage data for test case llvm/test/Analysis/AliasSet/memtransfer.ll will be emitted as
build/test/Analysis/AliasSet/memtransfer0.profraw
Reviewed By: hnrklssn
Differential Revision: https://reviews.llvm.org/D154280
I noticed during the build that GCC would emit a ton of nonnull
warnings.
Example:
```
/usr/local/google/home/fmzakari/code/github.com/llvm/llvm-project/clang/include/clang/AST/ExternalASTSource.h:378:54: warning: ‘this’ pointer is null [-Wnonnull]
378 | Ptr = reinterpret_cast<uint64_t>((Source->*Get)(Ptr >> 1));
| ~~~~~~~~~~~~~~^~~~~~~~~~
```
Reviewed By: MaskRay, rsmith
Differential Revision: https://reviews.llvm.org/D155857
This patch is the first part of https://llvm.org/OpenProjects.html#llvm_patch_coverage.
We have first define a new variable LLVM_TEST_COVERAGE which when set, pass --emit-coverage option to
llvm-lit which will help in setting a unique value to LLVM_PROFILE_FILE for each RUN. So for example
coverage data for test case llvm/test/Analysis/AliasSet/memtransfer.ll will be emitted as
build/test/Analysis/AliasSet/memtransfer.profraw
Reviewed By: hnrklssn
Differential Revision: https://reviews.llvm.org/D154280
This was left in place to reduce the risk of breaking anything,
and to keep the diff smaller, in D155233.
Differential Revision: https://reviews.llvm.org/D155431
With the new behaviour, the /MD or similar options aren't added to
e.g. CMAKE_CXX_FLAGS_RELEASE, but are added separately by CMake.
They can be changed by the cmake variable
CMAKE_MSVC_RUNTIME_LIBRARY or with the target property
MSVC_RUNTIME_LIBRARY.
LLVM has had its own custom CMake flags, e.g. LLVM_USE_CRT_RELEASE,
which affects which CRT is used for release mode builds. Deprecate
these and direct users to use CMAKE_MSVC_RUNTIME_LIBRARY directly
instead (and do a best effort attempt at setting CMAKE_MSVC_RUNTIME_LIBRARY
based on the existing LLVM_USE_CRT_ flags). This only handles the
simple cases, it doesn't handle multi-config generators with
different LLVM_USE_CRT_* variables for different configs though,
but that's probably fine - we should move over to the new upstream
CMake mechanism anyway, and push users towards that.
Change code in compiler-rt, that previously tried to override the
CRT choice to /MT, to set CMAKE_MSVC_RUNTIME_LIBRARY instead of
meddling in the old variables.
This resolves the policy issue in
https://github.com/llvm/llvm-project/issues/63286, and should
handle the issues that were observed originally when the
minimum CMake version was bumped, in
https://github.com/llvm/llvm-project/issues/62719 and
https://github.com/llvm/llvm-project/issues/62739.
Differential Revision: https://reviews.llvm.org/D155233
Add an option to specify additional linker flags for unit tests only.
For example, this allows using something like
-DLLVM_UNITTEST_LINK_FLAGS="-Wl,-plugin-opt=O0" if you're doing LTO
builds, or -DLLVM_UNITTEST_LINK_FLAGS="-fno-lto" if you're using
fat LTO objects.
The build system already does this itself if the LLVM_ENABLE_LTO
flag is used, but this does not cover all possible LTO configurations.
Differential Revision: https://reviews.llvm.org/D154212
This reverts commit f55fd19b6b.
As noted on the original thread, other uses of LLVM_LIBRARY_OUTPUT_INTDIR are optional. Will make a separate patch that makes this use optional as well.
It looks like MLIR is using the more modern CMAKE_LIBRARY_OUTPUT_DIRECTORY, but AddLLVM still uses this older LLVM specific alias.
In the specific case I was running into, the empty variable was causing `-Wl,-rpath-link,` on the command line, causing the following argument to be swallowed. This was maddening, because the following argument was the .o file containing `main` and I was getting `main` undefined errors when it was clearly there. This is egregious enough that I chose to guard it.
Differential Revision: https://reviews.llvm.org/D153373
Testing libc++ with CMake 3.27.0-rc1 fails. It seems some of the modules
used in CMake are not included.
The error before this change was
```
CMake Error at <mono-repo>/llvm/cmake/modules/HandleLLVMOptions.cmake:821 (CHECK_C_SOURCE_COMPILES):
Unknown CMake command "CHECK_C_SOURCE_COMPILES".
Call Stack (most recent call first):
CMakeLists.txt:156 (include)
```
Reviewed By: thesamesam
Differential Revision: https://reviews.llvm.org/D152619
Currently both ASAN and UBSAN are supported on Windows but only ASAN is enabled in LLVM build system. This patch enables the option to build LLVM with UBSAN on windows.
Tested with clang-cl released in LLVM16.
Submitting on behalf of @dwang.
Reviewed by: andrewng, pgousseau
Differential Revision: https://reviews.llvm.org/D151511
Building clang/llvm as a debug build results in TOC overflow even when
using large code model on AIX. This review updates the
llvm/cmake/modules/HandleLLVMOptions.cmake file to use large glink code
in addition to building as a large code model.
Differential Revision: https://reviews.llvm.org/D151733
This patch adds in CMake option LLVM_ENABLE_LLVM_LIBC which when set to
true automatically builds LLVM libc in overlay mode and links all
generated executables against the libc overlay. This is intended to
somewhat mirror the LLVM_ENABLE_LIBCXX flag.
Differential Revision: https://reviews.llvm.org/D151013
In CMake, "TARGET" is a special keyword. But it's also an LLVM component, which
means downstreams may request "target" or "TARGET" from CMake. Quote such input
so "TARGET" is interpreted as a string rather than a keyword.
This is a followup to 75a0502fe0 (D150884).
Fixes Meson's test suite and an issue which manifested identically to #61436
but appears to have been a slightly different problem.
Bug: https://github.com/mesonbuild/meson/issues/11642
Bug: https://github.com/llvm/llvm-project/issues/61436
Reviewed By: tstellar
Differential Revision: https://reviews.llvm.org/D152121
The existing BOLT install targets are broken on Windows becase they
don't properly handle the output extension. We cannot use the existing
LLVM macros since those make assumptions that don't hold for BOLT. This
change instead implements custom macros following the approach used by
Clang and LLD.
Differential Revision: https://reviews.llvm.org/D151595
The existing BOLT install targets are broken on Windows becase they
don't properly handle the output extension. We cannot use the existing
LLVM macros since those make assumptions that don't hold for BOLT. This
change instead implements custom macros following the approach used by
Clang and LLD.
Differential Revision: https://reviews.llvm.org/D151595
Fixes "dependent projects fail to build with cmake 3.25 and llvm 16-rc4".
When building [latest `iovisor/bcc`](1ea0390725), `cmake` complains about a change in the behavior of operator `IN_LIST`:
```console
CMake Warning (dev) at /usr/local/lib/cmake/llvm/LLVM-Config.cmake:230 (if):
Policy CMP0057 is not set: Support new IN_LIST if() operator. Run "cmake
--help-policy CMP0057" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
IN_LIST will be interpreted as an operator when the policy is set to NEW.
Since the policy is not set the OLD behavior will be used.
Call Stack (most recent call first):
cmake/clang_libs.cmake:32 (llvm_map_components_to_libnames)
src/cc/CMakeLists.txt:132 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Error at /usr/local/lib/cmake/llvm/LLVM-Config.cmake:230 (if):
if given arguments:
"engine" "IN_LIST" "link_components"
Unknown arguments specified
Call Stack (most recent call first):
cmake/clang_libs.cmake:32 (llvm_map_components_to_libnames)
src/cc/CMakeLists.txt:132 (include)
```
This patch addresses this issue by setting the aforementioned policy in `LLVM-Config.cmake`.
Reviewed By: mgorny, thesamesam
Differential Revision: https://reviews.llvm.org/D150884
Closes: https://github.com/llvm/llvm-project/issues/61436
As for now, 'extract_symbols.py' can use several tools to extract
symbols from object files and libraries and to guess if the target is
32-bit Windows. The tools are being found via PATH, so in most cases,
they are just system tools. This approach has a number of limitations,
in particular:
* System tools may not be able to handle the target format in case of
cross-platform builds,
* They cannot read symbols from LLVM bitcode files, so the staged LTO
build with plugins is not supported,
* The auto-selected tools may be suboptimal (see D113557),
* Support for multiple tools for a single task increases the complexity
of the script code.
The patch proposes using LLVM's own tools to solve these issues.
Specifically, 'llvm-readobj' detects the target platform, and 'llvm-nm'
reads symbols from all supported formats, including bitcode files. The
tools can be built in Release mode for the host platform or overridden
using CMake settings 'LLVM_READOBJ' and 'LLVM_NM' respectively. The
implementation also supports using precompiled tools via
'LLVM_NATIVE_TOOL_DIR'.
Differential Revision: https://reviews.llvm.org/D149119
