Revert "Revert #76246 and #76083"
This reverts commit 5c150e7eeba9db13cc65b329b3c3537b613ae61d.
Adds a small fix that should properly disable the tests on Windows.
Unfortunately the original poster has not provided feedback and the
original patch did not fail in the LLVM CI infrastructure.
Modules are known to fail on Windows due to non compliance of the
C library. Currently not having this patch prevents testing on other
platforms.
These cause test build failures on Windows.
This reverts the following commits:
57ca74843586c9a93c425036c5538aae0a2cfa60
d06ae33ec32122bb526fb35025c1f0cf979f1090
Previously there were two ways to override the verbose abort function
which gets called when a hardening assertion is triggered:
- compile-time: define the `_LIBCPP_VERBOSE_ABORT` macro;
- link-time: provide a definition of `__libcpp_verbose_abort` function.
This patch adds a new configure-time approach: the vendor can provide
a path to a custom header file which will get copied into the build by
CMake and included by the library. The header must provide a definition
of the
`_LIBCPP_ASSERTION_HANDLER` macro which is what will get called should
a hardening assertion fail. As of this patch, overriding
`_LIBCPP_VERBOSE_ABORT` will still work, but the previous mechanisms
will be effectively removed in a follow-up patch, making the
configure-time mechanism the sole way of overriding the default handler.
Note that `_LIBCPP_ASSERTION_HANDLER` only gets invoked when a hardening
assertion fails. It does not affect other cases where
`_LIBCPP_VERBOSE_ABORT` is currently used (e.g. when an exception is
thrown in the `-fno-exceptions` mode).
The library provides a default version of the custom header file that
will get used if it's not overridden by the vendor. That allows us to
always test the override mechanism and reduces the difference in
configuration between the pristine version of the library and
a platform-specific version.
As suggested in #71438 we should use
export import std;
in the std.compat module.
Testing this locally failed when building with the clang-tidy-17 plugin.
The std module was considered corrupt in the test
libcxx/test/libcxx/module_std_compat.gen.py
however the test
libcxx/test/libcxx/module_std.gen.py
passed. Both test generated identical std.pcm files. Using the
clang-tidy-18 plugin solves the issue.
This adds a new module test infrastructure. This requires tagging tests
using modules. The test runner uses this information to determine the
compiler flags needed to build and use the module.
Currently modules are build per test, which allows testing them for
tests with ADDITIONAL_COMPILE_FLAGS. At the moment only 4 tests use
modules. Therefore the performance penalty is not measurable. If in the
future more tests use modules it would be good to measure the overhead
and determine whether it's acceptable.
This removes the entire modules testing infrastructure.
The current infrastructure uses CMake to generate the std and std.compat
module. This requires quite a bit of plumbing and uses CMake. Since
CMake introduced module support in CMake 3.26, modules have a higher
CMake requirement than the rest of the LLVM project. (The LLVM project
requires 3.20.) The main motivation for this approach was how libc++
generated its modules. Every header had its own module partition. This
was changed to improve performance and now only two modules remain. The
code to build these can be manually crafted.
A followup patch will reenable testing modules, using a different
approach.
This simplifies the IWYU generation script by treating everything as a
file, instead of dealing with directories and files separately.
This has the downside that the `libcxx.imp` file is a lot larger than it
used to be, however we now have the flexibility of mapping files under
detail directories to different public headers. For example, this allows
us to map <__fwd/subrange.h> to <ranges> but <__fwd/pair.h> to
<utility>.
This patch also adds basic validation to ensure that we never map a
header to a public header that doesn't exist. We may still be missing
some mappings or we may be mapping to incorrect headers, but we won't be
mapping to headers that downright don't exist.
Fixes#63346
As suggested in #73262 this enable the stream printing on Apple
backdeployment targets. This omits the check whether the file is a
terminal. This is not entirely conforming, but the differences should be
minor and are typically not observable.
Fixes https://github.com/llvm/llvm-project/issues/75225
As a new contributor, I found it hard to find the documentation for the
meaning of the names of different tests and how those names translate to
Lit. This patch moves the documentation to the RST documentation we
publish on the website instead of leaving it in the source code only.
This patch adds a configuration of the libc++ test suite that enables
optimizations when building the tests. It also adds a new CI
configuration to exercise this on a regular basis. This is added in the
context of [1], which requires building with optimizations in order to
hit the bug.
[1]: https://github.com/llvm/llvm-project/issues/68552
Finishes implementation of
- P2093R14 Formatted output
- P2539R4 Should the output of std::print to a terminal be synchronized
with the underlying stream?
Differential Revision: https://reviews.llvm.org/D156609
This is the last PR that's needed (for now) to get libc++'s tests
working with MSVC's STL.
The ADDITIONAL_COMPILE_FLAGS machinery is very useful, but also very
problematic for MSVC, as it doesn't understand most of Clang's compiler
options. We've been dealing with this by simply marking anything that
uses ADDITIONAL_COMPILE_FLAGS as FAIL or SKIPPED, but that creates
significant gaps in test coverage.
Fortunately, ADDITIONAL_COMPILE_FLAGS also supports "features", which
can be slightly enhanced to send Clang-compatible and MSVC-compatible
options to the right compilers.
This patch adds the gcc-style-warnings and cl-style-warnings Lit features,
and uses that to pass the appropriate warning flags to tests. It also uses
TEST_MEOW_DIAGNOSTIC_IGNORED for a few local suppressions of MSVC
warnings.
The fix changes the way the validation script determines the qualified
name of a declaration. Inline namespaces without a reserved name are now
always part of the name. The Clang code only does this when the names
are ambigious. This method is often used for the operator""foo for UDLs.
Adjusted the newly flagged issue and removed a work-around in the test
code that is no longer required.
Fixes https://github.com/llvm/llvm-project/issues/72427
This adds the std.compat module. The patch contains a bit of refactoring
to avoid code duplication between the std and std.compat module.
Implements parts of
- P2465R3 Standard Library Modules std and std.compat
Found while running libc++'s test suite with MSVC's STL, where we use
both MSVC's compiler and Clang/LLVM.
MSVC's compiler rejects the non-Standard extension of zero-length
arrays. For conformance, I'm changing these occurrences to
`std::array<int, 0>`.
Many of these files already had `#include <array>`; I'm adding it to the
rest.
I wanted to add `-Wzero-length-array` to
`libcxx/utils/libcxx/test/params.py` to prevent future occurrences, but
it complained about product code 😿 :
```
In file included from /home/runner/_work/llvm-project/llvm-project/libcxx/test/std/input.output/iostream.format/input.streams/istream.formatted/istream.formatted.arithmetic/long.pass.cpp:18:
In file included from /home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/istream:170:
In file included from /home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/ostream:172:
In file included from /home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/__system_error/error_code.h:18:
In file included from /home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/__system_error/error_category.h:15:
/home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/string:811:25: error: zero size arrays are an extension [-Werror,-Wzero-length-array]
811 | char __padding_[sizeof(value_type) - 1];
| ^~~~~~~~~~~~~~~~~~~~~~
/home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/string:817:19: note: in instantiation of member class 'std::basic_string<char>::__short' requested here
817 | static_assert(sizeof(__short) == (sizeof(value_type) * (__min_cap + 1)), "__short has an unexpected size.");
| ^
/home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/string:2069:5: note: in instantiation of template class 'std::basic_string<char>' requested here
2069 | _LIBCPP_STRING_V1_EXTERN_TEMPLATE_LIST(_LIBCPP_DECLARE, char)
| ^
/home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/__string/extern_template_lists.h:31:60: note: expanded from macro '_LIBCPP_STRING_V1_EXTERN_TEMPLATE_LIST'
31 | _Func(_LIBCPP_EXPORTED_FROM_ABI basic_string<_CharType>& basic_string<_CharType>::replace(size_type, size_type, value_type const*, size_type)) \
| ^
```
I pushed a tiny commit to fix unrelated comment typos, in an attempt to
clear out spurious CI failures.
We now add -latomic whenever we detect that it's supported on the platform,
and we mark the tests as UNSUPPORTED on platforms where non-lockfree
atomics are not supported.
Picolibc is a C Standard Library that is commonly used in embedded
environments. This patch adds initial support for this configuration
along with pre-commit CI. As of this patch, the test suite only builds
the tests and nothing is run. A follow-up patch will make the test suite
actually run the tests.
Differential Revision: https://reviews.llvm.org/D154246
VSCode's Pylance extension informed me, and text searching confirmed,
that these imports are unused. I believe we should be able to remove
them harmlessly.
Lit features are evaluated in order. Some checks may require the
compiler detection to have run first in order to work properly, for
example some checks being added in https://reviews.llvm.org/D154246
which require GCC to have been detected. It is kind of brittle to rely
on such ordering, but in practice moving the compiler detection first
should never hurt.
Several experimental headers around std::pmr have been slated for
removal for a while now. This patch actually performs the removal and
cleanups from the code base.
I don't know when, but at some point we lost test coverage to ensue that
all the headers are in the modulemap. This adds a test to make sure all
the headers (excluding a few which shouldn't be part of the modulemap)
are at least mentioned. This also fixes a few headers which bit-rotted
while we were missing the coverage.
1. Instead of using individual "boolean" macros, have an "enum" macro
`_LIBCPP_HARDENING_MODE`. This avoids issues with macros being
mutually exclusive and makes overriding the hardening mode within a TU
more straightforward.
2. Rename the safe mode to debug-lite.
This brings the code in line with the RFC:
https://discourse.llvm.org/t/rfc-hardening-in-libc/73925Fixes#65101
This patch implements `std::basic_syncbuf` and `std::basic_osyncstream` as specified in paper p0053r7. ~~For ease of reviewing I am submitting this patch before submitting a patch for `std::basic_osyncstream`. ~~
~~Please note, this patch is not 100% complete. I plan on adding more tests (see comments), specifically I plan on adding tests for multithreading and synchronization.~~
Edit: I decided that it would be far easier for me to keep track of this and make changes that affect both `std::basic_syncbuf` and `std::basic_osyncstream` if both were in one patch.
The patch was originally written by @zoecarver
Implements
- P0053R7 - C++ Synchronized Buffered Ostream
- LWG-3127 basic_osyncstream::rdbuf needs a const_cast
- LWG-3334 basic_osyncstream move assignment and destruction calls basic_syncbuf::emit() twice
- LWG-3570 basic_osyncstream::emit should be an unformatted output function
- LWG-3867 Should std::basic_osyncstream's move assignment operator be noexcept?
Reviewed By: ldionne, #libc
Differential Revision: https://reviews.llvm.org/D67086
Add a new test mode that enables the hardened mode in combination with
ABI-breaking changes (only bounded iterators currently) and reenable the
bounded iterator tests for `span` and `string_view`.
According to https://developer.apple.com/support/xcode/, quite a few of
our availability macros don't do anything anymore, so we might as well
remove them to clean up the code a bit.
Without this patch, the functions `executeScriptInternal` and thus
`runOnce` in `llvm/utils/lit/lit/TestRunner.py` return either a tuple like
`(out, err, exitCode, timeoutInfo)` or a `lit.Test.Result` object. They
return the latter only when there's a lit internal shell parse error in
a RUN line. In my opinion, a more straight-forward way to handle
exceptional cases like that is to use python exceptions.
For that purpose, this patch introduces `ScriptFatal`. Thus, this patch
changes `executeScriptInternal` to always either return the tuple or
raise the `ScriptFatal` exception. It updates `runOnce` and
`libcxx/utils/libcxx/test/format.py` to catch the exception rather than
check for the special return type.
This patch also changes `runOnce` to convert the exception to a
`Test.UNRESOLVED` result instead of `TEST.FAIL`. The former is the
proper result for such a malformed test, for which a rerun (given an
`ALLOW_RETRIES:`) serves no purpose. There are at least two benefits
from this change. First, `_runShTest` no longer has to specially and
cryptically handle this case to avoid unnecessary reruns. Second, an
`XFAIL:` directive no longer hides such a failure [as we saw
previously](https://reviews.llvm.org/D154987#4501125).
To facilitate the `_runShTest` change, this patch inserts the internal
shell parse error diagnostic into the format of the test's normal debug
output rather than suppressing the latter entirely. That change is also
important for [D154987](https://reviews.llvm.org/D154987), which
proposes to reuse `ScriptFatal` for python compile errors in PYTHON
lines or in `config.prologue`. In that case, the diagnostic might follow
debugging output from the test's previous RUN or PYTHON lines, so
suppressing the normal debug output would lose information.
Mark tests as necessary to accommodate Android L (5.0 / API 21) and up.
Add three Android lit features:
- android
- android-device-api=(21,22,23,...)
- LIBCXX-ANDROID-FIXME (for failures that need follow-up work)
Enable an AIX workaround in filesystem_test_helper.h for the broken
chmod on older Android devices.
Mark failing test with XFAIL or UNSUPPORTED:
- Mark modules tests as UNSUPPORTED, matching other configurations.
- Mark a gdb test as UNSUPPORTED.
- XFAIL tests for old devices that lack an API (fmemopen).
- XFAIL various FS tests (because SELinux blocks FIFO and hard linking,
because fchmodat is broken on old devices).
- XFAIL various locale tests (because Bionic has limited locale
support). (Also XFAIL an re.traits test.)
- XFAIL some print.fun tests because the error exception has no system
error string.
- Mark std::{cin,wcin} tests UNSUPPORTED because they hang with
adb_run.py on old devices.
- Mark a few tests UNSUPPORTED because they allocate too much memory.
- notify_one.pass.cpp is flaky on Android.
- XFAIL libc++abi demangler test because of Android's special long
double on x86[-64].
N.B. The `__ANDROID_API__` macro specifies a minimum required API level
at build-time, whereas the android-device-api lit feature is the
detected API level of the device at run-time. The android-device-api
value will be >= `__ANDROID_API__`.
This commit was split out from https://reviews.llvm.org/D139147.
Fixes: https://github.com/llvm/llvm-project/issues/69270
I could probably break this commit into more pieces.
---
This patch adds libc++ support for Android L (Android 5.0+) and up,
tested using the Android team's current compiler, a recent version of
the AOSP sysroot, and the x86[-64] Android Emulator.
CMake and Lit Configuration:
Add runtimes/cmake/android/Arch-${ARCH}.cmake files that configure CMake
to cross-compile to Android without using CMake's built-in NDK support
(which only works with an actual packaged NDK).
Add libcxx/cmake/caches/AndroidNDK.cmake that builds and tests libc++
(and libc++abi) for Android. This file configures libc++ to match what
the NDK distributes, e.g.:
- libc++_shared.so (includes libc++abi objects, there is no
libc++abi.so). libunwind is linked statically but not exported.
- libc++_static.a (does not include libc++abi) and libc++abi.a
- `std::__ndk1` namespace
- All the libraries are built with `__ANDROID_API__=21`, even when they
are linked to something targeting a higher API level.
(However, when the Android LLVM team builds these components, they do
not use these CMake cache files. Instead they use Python scripts to
configure the builds. See
https://android.googlesource.com/toolchain/llvm_android/.)
Add llvm-libc++[abi].android-ndk.cfg.in files that test the Android
NDK's libc++_shared.so. These files can target old or new Android
devices. The Android LLVM team uses these test files to test libc++ for
both arm/arm64 and x86/x86_64 architectures.
The Android testing mode works by setting %{executor} to adb_run.py,
which uses `adb push` and `adb shell` to run tests remotely. adb_run.py
always runs tests as the "shell" user even on an old emulator where "adb
unroot" doesn't work. The script has workarounds for old Android
devices. The script uses a Unix domain socket on the host
(--job-limit-socket) to restrict concurrent adb invocations. Compiling
the tests is a major part of libc++ testing run-time, so it's desirable
to exploit all the host cores without overburdening the test devices,
which can have far fewer cores.
BuildKite CI:
Add a builder to run-buildbot, `android-ndk-*`, that uses Android Clang
and an Android sysroot to build libc++, then starts an Android emulator
container to run tests.
Run the emulator and an adb server in a separate Docker container
(libcxx-ci-android-emulator), and create a separate Docker image for
each emulator OS system image. Set ADB_SERVER_SOCKET to connect to the
container's adb server. Running the only adb server inside the container
makes cleanup more reliable between test runs, e.g. the adb client
doesn't create a `~/.android` directory and the adb server can be
restarted along with the emulator using docker stop/run. (N.B. The
emulator insists on connecting to an adb server and will start one
itself if it can't connect to one.)
The suffix to the android-ndk-* job is a label that concisely specifies
an Android SDK emulator image. e.g.:
- "system-images;android-21;default;x86" ==> 21-def-x86
- "system-images;android-33;google_apis;x86_64" ==> 33-goog-x86_64
Fixes: https://github.com/llvm/llvm-project/issues/69270
Differential Revision: https://reviews.llvm.org/D139147
Specifically, the test std/input.output/string.streams/stringstream/stringstream.members/gcount.pass.cpp
allocates a std::string with INT_MAX-1 elements, and then writes this to
a std::stringstream. On Linux, running this test consumes around 5.0 GB
of memory; on Windows, it ends up using up to 6.8 GB of memory.
This limits whether such tests can run on e.g. GitHub Actions runners,
where the free runners are limited to 8 GB of memory.
This is somewhat similar to, but still notably different, from the
existing test parameter long_tests.
This reapplies the change from
c218c80c730a14a1cbcebd588b18220a879702c6, which was lost in the
refactoring in 78d649a417b48cb8a2ba2e755f0e7c8fb8b1bb83.
On Windows, the Python interpreter is by default installed in a path
like "C:\Program Files\Python38\python.exe", which requires quoting when
included in a concatenated command string (as opposed to a command list,
where each argument is a separate element).
This doesn't show up in the CI environments as they use Python installed
in a different directory, like C:\Python.
In addition to the python executable, also quote the path to run.py, if
needed.
This is useful when trying to run multiple tests with different
arguments to the executor script. This is needed in my case since I
do not know the correct ssh connection arguments when building libc++.
The testing script I have spawns multiple QEMU instances that listen on
a given port on localhost and runs lit with the --num-shards/--run-shard
argument. In order to connect each shard to the right QEMU instances I
need to customize the arguments passed to ssh.py (--extra-ssh-args and
--extra-scp-args) but can't do this at configure time since the target
port is only known when running the tests but not when calling CMake.
This change allows me to pass `executor=ssh.py <args>` to lit once I
know
the right hostname/port for running the tests.
This also deprecates the `LIB{CXX,CXXABI,UNWIND}_EXECUTOR` CMake
variable
as the same can be achieved by adding `executor=...` to the
`LIB{CXX,CXXABI,UNWIND}_TEST_PARAMS` variable.
This patch and D154984 were discussed in
<https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839>.
Motivation
----------
D154984 removes the "Script:" section that lit prints along with a
test's output, and it makes -v and -a imply -vv. For example, after
D154984, the "Script:" section below is never shown, but -v is enough
to produce the execution trace following it:
```
Script:
--
: 'RUN: at line 1'; echo hello | FileCheck bogus.txt && echo success
--
Exit Code: 2
Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "echo" "hello"
# command output:
hello
$ "FileCheck" "bogus.txt"
# command stderr:
Could not open check file 'bogus.txt': No such file or directory
error: command failed with exit status: 2
--
```
In the D154984 review, some reviewers point out that they have been
using the "Script:" section for copying and pasting a test's shell
commands to a terminal window. The shell commands as printed in the
execution trace can be harder to copy and paste for the following
reasons:
- They drop redirections and break apart RUN lines at `&&`, `|`, etc.
- They add `$` at the start of every command, which makes it hard to
copy and paste multiple commands in bulk.
- Command stdout, stderr, etc. are interleaved with the commands and
are not clearly delineated.
- They don't always use proper shell quoting. Instead, they blindly
enclose all command-line arguments in double quotes.
Changes
-------
D154984 plus this patch converts the above example into:
```
Exit Code: 2
Command Output (stdout):
--
# RUN: at line 1
echo hello | FileCheck bogus-file.txt && echo success
# executed command: echo hello
# .---command stdout------------
# | hello
# `-----------------------------
# executed command: FileCheck bogus-file.txt
# .---command stderr------------
# | Could not open check file 'bogus-file.txt': No such file or directory
# `-----------------------------
# error: command failed with exit status: 2
--
```
Thus, this patch addresses the above issues as follows:
- The entire execution trace can be copied and pasted in bulk to a
terminal for correct execution of the RUN lines, which are printed
intact as they appeared in the original RUN lines except lit
substitutions are expanded. Everything else in the execution trace
appears in shell comments so it has no effect in a terminal.
- Each of the RUN line's commands is repeated (in shell comments) as
it executes to show (1) that the command actually executed (e.g.,
`echo success` above didn't) and (2) what stdout, stderr, non-zero
exit status, and output files are associated with the command, if
any. Shell quoting in the command is now correct and minimal but is
not necessarily the original shell quoting from the RUN line.
- The start and end of the contents of stdout, stderr, or an output
file is now delineated clearly in the trace.
To help produce some of the above output, this patch extends lit's
internal shell with a built-in `@echo` command. It's like `echo`
except lit suppresses the normal execution trace for `@echo` and just
prints its stdout directly. For now, `@echo` isn't documented for use
in lit tests.
Without this patch, libcxx's custom lit test format tries to parse the
stdout from `lit.TestRunner.executeScriptInternal` (which runs lit's
internal shell) to extract the stdout and stderr produced by shell
commands, and that parse no longer works after the above changes.
This patch makes a small adjustment to
`lit.TestRunner.executeScriptInternal` so libcxx can just request
stdout and stderr without an execution trace.
(As a minor drive-by fix that came up in testing: lit's internal `not`
command now always produces a numeric exit status and never `True`.)
Caveat
------
This patch only makes the above changes for lit's internal shell. In
most cases, we do not know how to force external shells (e.g., bash,
sh, window's `cmd`) to produce execution traces in the manner we want.
To configure a test suite to use lit's internal shell (which is
usually better for test portability than external shells anyway), add
this to the test suite's `lit.cfg` or other configuration file:
```
config.test_format = lit.formats.ShTest(execute_external=False)
```
Reviewed By: MaskRay, awarzynski
Differential Revision: https://reviews.llvm.org/D156954
When running multiple shards in parallel, one shard might write to the
cache while another one is reading this cache. Instead of updating the
file in place, write to a temporary file and swap the cache file using
os.replace(). This is an atomic operation and means shards will either
see the old state or the new one.
This gets rid of the separate parameter enable_modules_lsv in favor of
adding a named option to the enable_modules parameter. The patch also
removes the getModuleFlag helper, which was just a really complicated
way of hardcoding "none".
If an assertion fails during the configuration of the test suite because
the compiler doesn't support a flag (or the compiler configuration is
borked entirely), we will now print additional context to help debug the
issue instead of just saying "The compiler doesn't support the flag" and
bailing.
The safe mode is in-between the hardened and the debug modes, extending
the checks contained in the hardened mode with certain checks that are
relatively cheap and prevent common sources of errors but aren't
security-critical. Thus, the safe mode trades off some performance for
a wider set of checks, but unlike the debug mode, it can still be used
in production.
Differential Revision: https://reviews.llvm.org/D158823
