Revert "Revert #76246 and #76083"
This reverts commit 5c150e7eeb.
Adds a small fix that should properly disable the tests on Windows.
Unfortunately the original poster has not provided feedback and the
original patch did not fail in the LLVM CI infrastructure.
Modules are known to fail on Windows due to non compliance of the
C library. Currently not having this patch prevents testing on other
platforms.
Introduce a new `argument-within-domain` category that covers cases
where the given arguments make it impossible to produce a correct result
(or create a valid object in case of constructors). While the incorrect
result doesn't create an immediate problem within the library (like e.g.
a null pointer dereference would), it always indicates a logic error in
user code and is highly likely to lead to a bug in the program once the
value is used.
Follow up to #78862
These tests control hardening mode with `ADDITIONAL_COMPILE_FLAGS`, and
always set modes which use on TRAP. So we don't need to check
`libcpp-hardening-mode=fast`, and they must always fail with the current
HWASAN implementation.
...in attempt to fix the FreeBSD CI.
I noticed that suddenly some tests in the latest PRs fail to compile on
FreeBSD (`SIGILL` and `SIGTRAP` not defined). This tries to resolve
the issue.
Co-authored-by: Zingam <zingam@outlook.com>
After #77883, `fast` mode uses TRAP, and HWASAN
replaces TRAP with abort or error exit code.
On a quick looks it should be possible to avoid doing
that in HWASAN, but historically this is convention for all
sanitizers. Changing this behavior may break existing
users.
Other sanitizers are not affected because they don't
install TRAP handlers by default. But if they do, they also
replace TRAP with abort/exit.
When I implemented `condition_variable_any::wait`, I missed the most
important paragraph in the spec:
> The following wait functions will be notified when there is a stop
request on the passed stop_token.
> In that case the functions return immediately, returning false if the
predicate evaluates to false.
From
https://eel.is/c++draft/thread.condition#thread.condvarany.intwait-1.
Fixes#76807
CMake officially supports binary directory variable of installed
dependency using `FetchContent`. According to the current documentation,
it fetches `std` module and use its binary directory as hardcoded
string, `${CMAKE_BINARY_DIR}/_deps/std-build`, however it can be
replaced with `${std_BINARY_DIR}`.
Reference: https://cmake.org/cmake/help/latest/module/FetchContent.html
As @cpplearner explained in microsoft/STL#4328:
> libc++'s "ascii" mode (controlled by the `_LIBCPP_HAS_NO_UNICODE`
> macro) means "every code unit outside ASCII is treated as a valid
> printable character". AFAIK we \[MSVC's STL\] don't support such a mode.
Because these files are testing a non-Standard mode, they should be
moved from `libcxx/test/std` to `libcxx/test/libcxx`.
In the hardening modes that can be used in production (`fast` and
`extensive`), make a failed assertion invoke a trap instruction rather
than calling verbose abort. In the debug mode, still keep calling
verbose abort to provide a better user experience and to allow us to
keep our existing testing infrastructure for verifying assertion
messages. Since the debug mode by definition enables all assertions, we
can be sure that we still check all the assertion messages in the
library when running the test suite in the debug mode.
The main motivation to use trapping in production is to achieve better
code generation and reduce the binary size penalty. This way, the
assertion handler can compile to a single instruction, whereas the
existing mechanism with verbose abort results in generating a function
call that in general cannot be optimized away (made worse by the fact
that it's a variadic function, imposing an additional penalty). See the
[RFC](https://discourse.llvm.org/t/rfc-hardening-in-libc/73925) for more
details. Note that this mechanism can now be completely [overridden at
CMake configuration
time](https://github.com/llvm/llvm-project/pull/77883).
This patch also significantly refactors `check_assertion.h` and expands
its test coverage. The main changes:
- when overriding `verbose_abort`, don't do matching inside the function
-- just print the error message to `stderr`. This removes the need to
set a global matcher and allows to do matching in the parent process
after the child finishes;
- remove unused logic for matching source locations and for using
wildcards;
- make matchers simple functors;
- introduce `DeathTestResult` that keeps data about the test run,
primarily to make it easier to test.
In addition to the refactoring, `check_assertion.h` can now recognize
when a process exits due to a trap.
Closes#77638, #24186
Rebased from <https://reviews.llvm.org/D156032>, see there for more
information.
Implements wording change in [CWG2137](https://wg21.link/CWG2137) in the
first commit.
This also implements an approach to [CWG2311](https://wg21.link/CWG2311)
in the second commit, because too much code that relies on `T{ T_prvalue}`
being an elision would break. Because that issue is still open and
the CWG issue doesn't provide wording to fix the issue, there may be
different behaviours on other compilers.
The tag name was long for an ABI tag. The name was misleading too, the
tag is first introduced in LLVM 18 in 2024 and not in 2023.
---------
Co-authored-by: Louis Dionne <ldionne.2@gmail.com>
This commit turns on ASan annotations in `std::basic_string` for short
stings (SSO case).
Originally suggested here: https://reviews.llvm.org/D147680
String annotations added here:
https://github.com/llvm/llvm-project/pull/72677
Requires to pass CI without fails:
- https://github.com/llvm/llvm-project/pull/75845
- https://github.com/llvm/llvm-project/pull/75858
Annotating `std::basic_string` with default allocator is implemented in
https://github.com/llvm/llvm-project/pull/72677 but annotations for
short strings (SSO - Short String Optimization) are turned off there.
This commit turns them on. This also removes
`_LIBCPP_SHORT_STRING_ANNOTATIONS_ALLOWED`, because we do not plan to
support turning on and off short string annotations.
Support in ASan API exists since
dd1b7b797a.
You can turn off annotations for a specific allocator based on changes
from
2fa1bec7a2.
This PR is a part of a series of patches extending AddressSanitizer C++
container overflow detection capabilities by adding annotations, similar
to those existing in `std::vector` and `std::deque` collections. These
enhancements empower ASan to effectively detect instances where the
instrumented program attempts to access memory within a collection's
internal allocation that remains unused. This includes cases where
access occurs before or after the stored elements in `std::deque`, or
between the `std::basic_string`'s size (including the null terminator)
and capacity bounds.
The introduction of these annotations was spurred by a real-world
software bug discovered by Trail of Bits, involving an out-of-bounds
memory access during the comparison of two strings using the
`std::equals` function. This function was taking iterators
(`iter1_begin`, `iter1_end`, `iter2_begin`) to perform the comparison,
using a custom comparison function. When the `iter1` object exceeded the
length of `iter2`, an out-of-bounds read could occur on the `iter2`
object. Container sanitization, upon enabling these annotations, would
effectively identify and flag this potential vulnerability.
If you have any questions, please email:
advenam.tacet@trailofbits.comdisconnect3d@trailofbits.com
Previously there were two ways to override the verbose abort function
which gets called when a hardening assertion is triggered:
- compile-time: define the `_LIBCPP_VERBOSE_ABORT` macro;
- link-time: provide a definition of `__libcpp_verbose_abort` function.
This patch adds a new configure-time approach: the vendor can provide
a path to a custom header file which will get copied into the build by
CMake and included by the library. The header must provide a definition
of the
`_LIBCPP_ASSERTION_HANDLER` macro which is what will get called should
a hardening assertion fail. As of this patch, overriding
`_LIBCPP_VERBOSE_ABORT` will still work, but the previous mechanisms
will be effectively removed in a follow-up patch, making the
configure-time mechanism the sole way of overriding the default handler.
Note that `_LIBCPP_ASSERTION_HANDLER` only gets invoked when a hardening
assertion fails. It does not affect other cases where
`_LIBCPP_VERBOSE_ABORT` is currently used (e.g. when an exception is
thrown in the `-fno-exceptions` mode).
The library provides a default version of the custom header file that
will get used if it's not overridden by the vendor. That allows us to
always test the override mechanism and reduces the difference in
configuration between the pristine version of the library and
a platform-specific version.
The behavior of `std::regex_search` for patterns anchored both to the
start and to the end of the input went wrong after merging #77256 .
Patterns like `"^b*$"` started matching the strings such as `"a"`, which
is not expected.
Reverts the PR: #77256
This commit simplifies ASan helper functions in `std::vector` by
removing arguments which can be calculated later.
Short term it improves readability of helper functions in `std::vector`.
Long term it aims to help with a bigger refactor of container
annotations.
As suggested in #71438 we should use
export import std;
in the std.compat module.
Using this exports some named declarations from functions and records,
adding them to the global namespace. Clang correctly does not export
these and it's an issue in the declaration filtering. Declarations in
function or record context are not considered a global named
declaration.
Clang-tidy 18 no longer has false positives with the spaceship operator.
Note that I'm quite sure there are more occurrences in our headers that
are not caught.
As suggested in #71438 we should use
export import std;
in the std.compat module.
Testing this locally failed when building with the clang-tidy-17 plugin.
The std module was considered corrupt in the test
libcxx/test/libcxx/module_std_compat.gen.py
however the test
libcxx/test/libcxx/module_std.gen.py
passed. Both test generated identical std.pcm files. Using the
clang-tidy-18 plugin solves the issue.
This adds a new module test infrastructure. This requires tagging tests
using modules. The test runner uses this information to determine the
compiler flags needed to build and use the module.
Currently modules are build per test, which allows testing them for
tests with ADDITIONAL_COMPILE_FLAGS. At the moment only 4 tests use
modules. Therefore the performance penalty is not measurable. If in the
future more tests use modules it would be good to measure the overhead
and determine whether it's acceptable.
This removes the entire modules testing infrastructure.
The current infrastructure uses CMake to generate the std and std.compat
module. This requires quite a bit of plumbing and uses CMake. Since
CMake introduced module support in CMake 3.26, modules have a higher
CMake requirement than the rest of the LLVM project. (The LLVM project
requires 3.20.) The main motivation for this approach was how libc++
generated its modules. Every header had its own module partition. This
was changed to improve performance and now only two modules remain. The
code to build these can be manually crafted.
A followup patch will reenable testing modules, using a different
approach.
This simplifies the IWYU generation script by treating everything as a
file, instead of dealing with directories and files separately.
This has the downside that the `libcxx.imp` file is a lot larger than it
used to be, however we now have the flexibility of mapping files under
detail directories to different public headers. For example, this allows
us to map <__fwd/subrange.h> to <ranges> but <__fwd/pair.h> to
<utility>.
This patch also adds basic validation to ensure that we never map a
header to a public header that doesn't exist. We may still be missing
some mappings or we may be mapping to incorrect headers, but we won't be
mapping to headers that downright don't exist.
Fixes#63346
As suggested in #73262 this enable the stream printing on Apple
backdeployment targets. This omits the check whether the file is a
terminal. This is not entirely conforming, but the differences should be
minor and are typically not observable.
Fixes https://github.com/llvm/llvm-project/issues/75225
Previously the header included several headers, possibly granularized
threading headers. This could lead to build errors when these headers
were incompatible with threading disabled.
Now test the guard before inclusion. This matches the pattern used for
no localization and no wide characters.
Fixes: https://github.com/llvm/llvm-project/issues/76620
We discussed the removal of these enable-all macros in the libc++
monthly meeting and we agreed that we should deprecate these macros in
LLVM 18, and then remove them in LLVM 19 since they can silently enable
deprecated features that are implemented after the first release of the
macro.
This patch does the first part of this -- it deprecates the macro.
Note that the file
test/libcxx/depr/enable_removed_cpp20_features.compile.pass.cpp
does not exist so this file is not adapted. Since the feature is
deprecated and slated for removal soon the missing test is not
implemented.
Partly addresses: https://github.com/llvm/llvm-project/issues/75976
---------
Co-authored-by: Louis Dionne <ldionne.2@gmail.com>
The overloads of `println` are specified in terms of `format`. The
function `format` is specified to work with ranges.
The implementations for `println` do not include `<format>`, but
libc++'s granularized header. This means the following example does not
work
#include <vector>
#include <print>
int main() {
std::vector<int> v{1, 2, 3};
std::println("{}", v);
}
(The other print functions also require this to work, they are specified
in terms of other format functions.)
Fixes: https://github.com/llvm/llvm-project/issues/71925
The old notes are kept to make it easier to backport changes to the
release branch. There are no LLVM-17 releases expected and this
documentation should not be available in the LLVM-18 release.
Note after branching LLVM-18 both LLVM-18 and LLVM-19 release notes
should be available.
GCC 13 is the latest GCC release and tested in the libc++ CI for several
month. According to our policy we only support the latest version,
update the documentation to the latest version.
This commit turns on ASan annotations in `std::basic_string` for all
allocators by default.
Originally suggested here: https://reviews.llvm.org/D146214
String annotations added here:
https://github.com/llvm/llvm-project/pull/72677
This commit is part of our efforts to support container annotations with
(almost) every allocator. Annotating `std::basic_string` with default
allocator is implemented in
https://github.com/llvm/llvm-project/pull/72677.
Additionally it removes `__begin != nullptr` because `data()` should
never return a nullptr.
Support in ASan API exists since
1c5ad6d2c0.
This patch removes the check in std::basic_string annotation member
function (__annotate_contiguous_container) to support different
allocators.
You can turn off annotations for a specific allocator based on changes
from
2fa1bec7a2.
The motivation for a research and those changes was a bug, found by
Trail of Bits, in a real code where an out-of-bounds read could happen
as two strings were compared via a call to `std::equal` that took
`iter1_begin`, `iter1_end`, `iter2_begin` iterators (with a custom
comparison function). When object `iter1` was longer than `iter2`, read
out-of-bounds on `iter2` could happen. Container sanitization would
detect it.
If you have any questions, please email:
- advenam.tacet@trailofbits.com
- disconnect3d@trailofbits.com
The updated picolibc version has "isblank" function with external
linkage. This is required for C++ modules support.
This should solve all the problems reported in #76980, but
we'll wait to validate this with the modules build without
closing that issue.
Rather than allow for a message to be considered a match for the actual
assertion if it is anywhere in the assertion text, make sure that the
expected and the actual assertion are identical.
Addresses #77701
A logical mistake is made in #77058, we should try to find a new file
path for socket creation when the path's length generated is bigger than
the socket length limit.
As described in #69994, using the escape hatch makes us non-conforming
in C++20 due to incorrect constexpr-ness. It also leads to bad
diagnostics as reported by #63900. We discussed the issue in the libc++
monthly meeting and we agreed that we should deprecate the macro in LLVM
18, and then remove it in LLVM 19 since it causes too many problems.
This patch does the first part of this -- it deprecates the macro.
Fixes#69994Fixes#63900
Partially addresses #75975
When using LIBCXX_CXX_ABI=system-libcxxabi on Apple platforms, we would not
re-export the libc++abi symbols unlike when LIBCXX_CXX_ABI=libcxxabi. This
was caused by overly strict string matching in CMake.
https://github.com/NixOS/nixpkgs/issues/269548
This commit is a refactor (increases readability) and optimization fix.
This is a fixed commit of
https://github.com/llvm/llvm-project/pull/76200 First reverthed here:
1ea7a56057
Please, check original PR for details.
The difference is a return type of the lambda.
Original description:
This commit addresses optimization and instrumentation challenges
encountered within comma constructors.
1) _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS does not work in comma
constructors.
2) Code inside comma constructors is not always correctly optimized.
Problematic code examples:
- `: __r_(((__str.__is_long() ? 0 : (__str.__annotate_delete(), 0)),
std::move(__str.__r_))) {`
- `: __r_(__r_([&](){ if(!__s.__is_long()) __s.__annotate_delete();
return std::move(__s.__r_);}())) {`
However, lambda with argument seems to be correctly optimized. This
patch uses that fact.
Use of lambda based on idea from @ldionne.
I recently came across LIBCXXABI_USE_LLVM_UNWINDER and was surprised to
notice it was disabled by default. Since we build libunwind by default
and ship it in the LLVM toolchain, it would seem to make sense that
libc++ and libc++abi rely on libunwind for unwinding instead of using
the system-provided unwinding library (if any).
Most importantly, using the system unwinder implies that libc++abi is
ABI compatible with that system unwinder, which is not necessarily the
case. Hence, it makes a lot more sense to instead default to using the
known-to-be-compatible LLVM unwinder, and let vendors manually select a
different unwinder if desired.
As a follow-up change, we should probably apply the same default to
compiler-rt.
Differential Revision: https://reviews.llvm.org/D150897Fixes#77662
rdar://120801778
Since we use _LIBCPP_USING_IF_EXISTS to handle missing C library functions
now, _LIBCPP_C_HAS_NO_GETS shouldn't be necessary anymore.
See the discussion thread in #77242 for more details.
As a new contributor, I found it hard to find the documentation for the
meaning of the names of different tests and how those names translate to
Lit. This patch moves the documentation to the RST documentation we
publish on the website instead of leaving it in the source code only.
Instead of using a concept defined in the internal implementation, use a
definition of the simple_view ranges concept separately defined and
included in test code.
This was causing compilation errors when attempting to compare a
`shared_ptr<T[]>` with `nullptr`, as `get()` returns `T*` rather than `T
(*)[]`. `unique_ptr` did not have this issue, but I've added tests to
make sure.
Using `regex_search` with the regex_constant `match_default` and a
simple regex pattern `$` is expected to match general strings such as
_"a", "ab", "abc"..._ at `[last, last)` positions. But, the current
implementation fails to do so.
Fixes#75042
If TMP is set to a folder which path is too long, the current libcxx
test helper function `create_socket()` will fail because of the test
temp folder `test_root`'s path is too long to be used in socket
creation.
In such case, this patch will try to create the socket file directly in
`/tmp` folder.
This patch also add an assertion for `bind()`.
The current CI badge is currently in libc++ documentation. This does not
seem the right place:
- The typical location on GitHub is on the main README.
- The documentation is shipped as part of the release:
- This link does not work in off-line mode. Currently our documentation
works in off-line mode.
- The status in the release documentation does not reflect the status of
the shipped library. So users looking at it may see a red status and get
confused.
This moves the badge to the README.
As pointed out by @Zingam the paper was implemented in libc++ as an
extension. This patch does the bookkeeping. The inital release version
is based on historical release dates.
Completes:
- Add a conditional noexcept specification to std::apply
This patch adds a configuration of the libc++ test suite that enables
optimizations when building the tests. It also adds a new CI
configuration to exercise this on a regular basis. This is added in the
context of [1], which requires building with optimizations in order to
hit the bug.
[1]: https://github.com/llvm/llvm-project/issues/68552
This function replaces a call to `__move_assign` (internal function)
with two calls to public member functions (`resize` and `erase`). The
order of calls is chosen for the best performance.
This change is required to [turn on ASan string annotations for short
strings](https://github.com/llvm/llvm-project/pull/75882) (Short String
Optimization - SSO).
The `std::basic_string` class's `void __move_assign(basic_string&&
__str, size_type __pos, size_type __len)` function operates on
uninitialized strings, where it is reasonable to assume that the memory
is not poisoned. However, in `sstream` this function is applied to
existing strings that already have poisoned memory.
String ASan annotations turned on here:
https://github.com/llvm/llvm-project/pull/72677
This commit addresses optimization and instrumentation challenges
encountered within comma constructors.
1) _LIBCPP_STRING_INTERNAL_MEMORY_ACCESS does not work in comma
constructors.
2) Code inside comma constructors is not always correctly optimized.
Problematic code examples:
- `: __r_(((__str.__is_long() ? 0 : (__str.__annotate_delete(), 0)),
std::move(__str.__r_))) {`
- `: __r_(__r_([&](){ if(!__s.__is_long()) __s.__annotate_delete();
return std::move(__s.__r_);}())) {`
However, lambda with argument seems to be correctly optimized. The patch employs this.
Use of lambda based on an idea from @ldionne.
The macros were already updated
- __cpp_lib_string_view in 466df1718e
- __cpp_lib_array_constexpr in 77b9abfc8e
Based on the dates of the commit and that
P0858 "Constexpr iterator requirements"
was completed in LLVM 12, set this issue as completed in the same
version.
Completes
- LWG3257 Missing feature testing macro update from P0858
And add a check in the python script that the binary given to `--qemu`
actually exists. Otherwise you get a generic Python error:
```
# .---command stderr------------
# | Traceback (most recent call last):
# | File "/home/david.spickett/modules-llvm-project/libcxx/utils/qemu_baremetal.py", line 70, in <module>
# | exit(main())
# | File "/home/david.spickett/modules-llvm-project/libcxx/utils/qemu_baremetal.py", line 66, in main
# | os.execvp(qemu_commandline[0], qemu_commandline)
# | File "/usr/lib/python3.8/os.py", line 568, in execvp
# | _execvpe(file, args)
# | File "/usr/lib/python3.8/os.py", line 610, in _execvpe
# | raise last_exc
# | File "/usr/lib/python3.8/os.py", line 601, in _execvpe
# | exec_func(fullname, *argrest)
# | FileNotFoundError: [Errno 2] No such file or directory
# `-----------------------------
# error: command failed with exit status: 1
```
When it tries to run the entire command later.
For the builder, it's only ever going to use qemu-system-arm so error at
config time if it's not there.
Also introduce `_LIBCPP_ASSERT_PEDANTIC` for assertions violating which
results in a no-op or other benign behavior, but which may nevertheless
indicate a bug in the invoking code.
…te initialization" (#76272)""
With updates the libc++ tests.
This reverts commit 2205d23 and relands
86dc6e1 and
7ab16fb.
Original commit was reverted because of failing libc++ tests, see #76232
for the discussion.
The errors in the tests are spurious in the first place (coming from
initialization of invalid classes), so update the tests to match new
behavior that does not show those errors.
The original patch was written by @ilya-biryukov
To fix the CI two libc++ tests are temporary disabled for clang-18.
With updates the libc++ tests.
This reverts commit 2205d2334f and relands
86dc6e15f2 and
7ab16fb520.
Original commit was reverted because of failing libc++ tests, see #76232 for
the discussion.
The errors in the tests are spurious in the first place (coming from initialization
of invalid classes), so update the tests to match new behavior that does
not show those errors.
fixes#70506
The detailed problem description is in #70506
The original proposed fix was to remove `[[no_unique_address]]` except
when `_Tp` is empty.
Edit:
After the discussion in the comments below, the new fix here is to
remove the `[[no_unique_address]]` from `movable_box` in the cases where
we need to add our own assignment operator, which has contains the
problematic `construct_at`
The status table incorrectly marks P0521R0 as nothing to do. This is not
correct the function should be deprecated.
During our latest monthly meeting we argreed to remove the
_LIBCPP_ENABLE_CXXyy_REMOVED_FEATURES macros, therefore the new macro is
not
added to that global list.
Implements
- P0521R0 Proposed Resolution for CA 14 (shared_ptr use_count/unique)
Implements parts of
- P0619R4 Reviewing Deprecated Facilities of C++17 for C++20
---------
Co-authored-by: Nikolas Klauser <nikolasklauser@berlin.de>
As pointed out in #72883, the implementation only needs to return the
value of ranges::next and does not need to obtain the value through
ranges::advance, which causes it to have O(n) complexity in the case
of random-access-sized but non-common range.
Fixes#72883
Recent CI changes have disabled testing modules in different
configurations. This broke building the std and std.compat module in
C++20. This was found by the CI in #76246.
This commit implements conditional compilation for ASan helper code.
As convey to me by @EricWF, string benchmarks with UBSan have been
experiencing significant performance hit after the commit with ASan
string annotations. This is likely due to the fact that no-op ASan code
is not optimized out with UBSan. To address this issue, this commit
conditionalizes the inclusion of ASan helper function bodies using
`#ifdef` directives. This approach allows us to selectively include only
the ASan code when it's actually required, thereby enhancing
optimizations and improving performance.
While issue was noticed in string benchmarks, I expect same overhead
(just less noticeable) in other containers, therefore `std::vector` and
`std::deque` have same changes.
To see impact of that change run `string.libcxx.out` with UBSan and
`--benchmark_filter=BM_StringAssign` or
`--benchmark_filter=BM_StringConstruct`.
