258 Commits

Author SHA1 Message Date
Nikolas Klauser
68b1035965 [libc++][PSTL] Add a __parallel_sort implementation to libdispatch
Reviewed By: #libc, ldionne

Spies: ldionne, libcxx-commits

Differential Revision: https://reviews.llvm.org/D155136
2023-08-15 12:20:40 -07:00
Nikolas Klauser
475bd19ee8 [libc++][NFC] Refactor return type enable_ifs to defaulted template arguments
This brings most of the enable_ifs in libc++ to the same style. It also has the nice side-effect of reducing the size of names of these symbols, since the depedent return type is shorter.

Reviewed By: #libc, ldionne

Spies: ldionne, libcxx-commits

Differential Revision: https://reviews.llvm.org/D157736
2023-08-15 12:19:21 -07:00
Nikolas Klauser
051c863012 [libc++][PSTL] Simplify the partitioning algorithm until we have better data to know how to chunk better
The current chunking strategy is very bad for sorting, and we don't really know how to chunk in general. This fixes the performance problem for sorting.

Reviewed By: ldionne, #libc

Spies: libcxx-commits, krytarowski

Differential Revision: https://reviews.llvm.org/D155531
2023-08-14 16:44:30 -07:00
Gonzalo Brito Gadeschi
0e2de665f3 [libc++][PSTL] Parallelize random_access_iterator
P2408 requires this for C++23, but implementing it in C++20 is safe
because the only code impacted would be code that violated a
precondition of the parallel algorithm. It was P2408 intent to
enable implementations to backport this to C++20.

Closes #63447 .

Reviewed By: philnik, #libc

Differential Revision: https://reviews.llvm.org/D154305
2023-08-07 17:58:27 +02:00
Mark de Wever
f0e9cd2b1c [libc++][PSTL] Adds a missing include.
This should fix an error in the Apple CI.
2023-08-02 18:10:58 +02:00
Nikolas Klauser
8670b53e11 [libc++] Optimize ranges::find for vector<bool>
Benchmark results:
```
----------------------------------------------------------------
Benchmark                                    old             new
----------------------------------------------------------------
bm_vector_bool_ranges_find/1             5.64 ns         6.08 ns
bm_vector_bool_ranges_find/2             16.5 ns         6.03 ns
bm_vector_bool_ranges_find/3             20.3 ns         6.07 ns
bm_vector_bool_ranges_find/4             22.2 ns         6.08 ns
bm_vector_bool_ranges_find/5             23.5 ns         6.05 ns
bm_vector_bool_ranges_find/6             24.4 ns         6.10 ns
bm_vector_bool_ranges_find/7             26.7 ns         6.10 ns
bm_vector_bool_ranges_find/8             25.0 ns         6.08 ns
bm_vector_bool_ranges_find/16            27.9 ns         6.07 ns
bm_vector_bool_ranges_find/64            44.5 ns         5.35 ns
bm_vector_bool_ranges_find/512            243 ns         25.7 ns
bm_vector_bool_ranges_find/4096          1858 ns         35.6 ns
bm_vector_bool_ranges_find/32768        15461 ns         93.5 ns
bm_vector_bool_ranges_find/262144      126462 ns          571 ns
bm_vector_bool_ranges_find/1048576     497736 ns         2272 ns
```

Reviewed By: #libc, Mordante

Spies: var-const, Mordante, libcxx-commits

Differential Revision: https://reviews.llvm.org/D156039
2023-08-01 10:28:25 -07:00
Nikolas Klauser
9391330293 [libc++][PSTL] Fix std::copy frontend dispatching
Reviewed By: #libc, Mordante

Spies: Mordante, libcxx-commits

Differential Revision: https://reviews.llvm.org/D155325
2023-07-31 18:39:02 -07:00
Ian Anderson
165841b681 [libc++][Modules] Fix a few module related warnings
I'm getting a few -Wundefined-inline warnings, and a -Wnon-modular-include-in-module too. Fix all of those.

Reviewed By: Mordante, #libc

Differential Revision: https://reviews.llvm.org/D156508
2023-07-29 07:33:35 -07:00
Nikolas Klauser
37e5baf318 [libc++][PSTL] Implement std::sort
Reviewed By: #libc, ldionne

Spies: ldionne, libcxx-commits, mgrang

Differential Revision: https://reviews.llvm.org/D152860
2023-07-20 14:45:36 -07:00
Louis Dionne
503f2ee4a8 [libc++] Make sure we use the libdispatch backend on Apple platforms
The Apple.cmake cache wasn't set up properly, so we wouldn't enable
the libdispatch backend by default on Apple platforms. This patch
fixes the issue and adds a test.

We also need to make various drive-by fixes:
- Drop the usage of std::vector in libdispatch.h to avoid changing
  the transitive includes only on Apple platforms.
- Fix includes
- Use __construct at since construct_at is unavailable in C++17
- Get rid of the (unused) __get_memory_resource function since that
  adds a back-deployment requirement and we don't use it right now.
- Fix bugs in the chunking logic around boundary conditions.

Differential Revision: https://reviews.llvm.org/D155649
2023-07-20 15:53:27 -04:00
Nikolas Klauser
2b2e7f6e57 [libc++][PSTL] Add a GCD backend
Reviewed By: ldionne, #libc

Spies: arichardson, mgrang, krytarowski, libcxx-commits, h-vetinari

Differential Revision: https://reviews.llvm.org/D151717
2023-07-12 13:27:33 -07:00
varconst
d1367ca46e [libc++][hardening][NFC] Add macros to enable hardened mode.
This patch only adds new configuration knobs -- the actual assertions
will be added in follow-up patches.

Differential Revision: https://reviews.llvm.org/D153902
2023-07-12 10:12:58 -07:00
Nikolas Klauser
a70ce8cb0e [libc++][PSTL] Fix double-move in std::transform_reduce
Reviewed By: #libc, ldionne

Spies: h-vetinari, libcxx-commits

Differential Revision: https://reviews.llvm.org/D154913
2023-07-12 08:56:10 -07:00
Louis Dionne
9101479420 [libc++][NFC] Add 'const' to some operator()
This is NFC because the function object is stateless anyway. This is
done solely for consistency with surrounding code and this was probably
an oversight in https://reviews.llvm.org/D132505.

Differential Revision: https://reviews.llvm.org/D154612
2023-07-06 16:46:34 -04:00
Nicole Rabjohn
92e4d6791f Fixing conflicting macro definitions between curses.h and the standard library.
POSIX allows certain macros to exist with generic names (i.e. refresh(), move(), and erase()) to exist in `curses.h` which conflict with functions found in std::filesystem, among others. This patch undefs the macros in question and adds them to LIBCPP_PUSH_MACROS and LIBCPP_POP_MACROS.

Reviewed By: #libc, philnik, ldionne

Differential Revision: https://reviews.llvm.org/D147356
2023-07-06 17:21:08 +00:00
Ian Anderson
6f05da6cf8 [libc++][Modules] Add missing submodules
Several headers that are included by the modular headers are missing from the module map, add those in.
The either/or implementation headers `<__algorithm/pstl_backends/cpu_backends/serial.h>`/`<__algorithm/pstl_backends/cpu_backends/thread.h>` need to be textual, as does `<__undef_macros>`.

Reviewed By: ldionne, #libc

Differential Revision: https://reviews.llvm.org/D153213
2023-07-05 09:51:11 -07:00
Louis Dionne
60a6a0d17a [libc++] Add missing includes
Those were found while trying to enable configurations like no-threads
and no-localization with Clang modules enabled.

Differential Revision: https://reviews.llvm.org/D153977
2023-07-03 09:40:09 -04:00
Amirreza Ashouri
f4ec54b51b Fix typo "__is_parititioned_impl" to "__is_partitioned_impl"
A typo has been fixed

Reviewed By: #libc, Mordante

Differential Revision: https://reviews.llvm.org/D154296
2023-07-02 16:56:32 +02:00
Nikolas Klauser
c4e98722ca [libc++] Fix std::copy and std::move for ranges with potentially overlapping tail padding
This fixes thr bug reported in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108846.

Reviewed By: ldionne, #libc

Spies: mstorsjo, libcxx-commits

Differential Revision: https://reviews.llvm.org/D151953
2023-06-30 13:48:16 -07:00
Louis Dionne
f13e1a65ca [libc++] Revert __uninitialized_buffer changes
This patch reverts the following commits:

    015cd317eaed28a923d14a33c9d6739012a688be (add missing HIDE_FROM_ABI)
    420a204d52205f1277a8d5df3dbafac6082e02e2 (add _LIBCPP_NO_CFI)
    31eeba3f7c0e2ef4a21c07da9326a4ae1a8de7e2 (add __uninitialized_buffer)

It also reverts a small part of b935ab8e747cf52ff12471879460206a9f433eea
which is required to make the stable_partition.pass.cpp test pass on GCC.

Some issues were pointed out in https://reviews.llvm.org/D152208 and
in https://reviews.llvm.org/D154017, so I am reverting this patch
until we have time to weigh the various solutions and get consensus
on the design of the API.

Differential Revision: https://reviews.llvm.org/D154161
2023-06-30 09:17:24 -04:00
varconst
b5270ba20d [libc++] Remove the legacy debug mode.
See https://discourse.llvm.org/t/rfc-removing-the-legacy-debug-mode-from-libc/71026

Reviewed By: #libc, Mordante, ldionne

Differential Revision: https://reviews.llvm.org/D153672
2023-06-29 14:49:51 -07:00
varconst
cd0ad4216c [libc++][hardening][NFC] Introduce _LIBCPP_ASSERT_UNCATEGORIZED.
Replace most uses of `_LIBCPP_ASSERT` with
`_LIBCPP_ASSERT_UNCATEGORIZED`.

This is done as a prerequisite to introducing hardened mode to libc++.
The idea is to make enabling assertions an opt-in with (somewhat)
fine-grained controls over which categories of assertions are enabled.
The vast majority of assertions are currently uncategorized; the new
macro will allow turning on `_LIBCPP_ASSERT` (the underlying mechanism
for all kinds of assertions) without enabling all the uncategorized
assertions (in the future; this patch preserves the current behavior).

Differential Revision: https://reviews.llvm.org/D153816
2023-06-28 15:10:31 -07:00
Ian Anderson
d5ce68afdf [libc++] __iterator/readable_traits.h isn't standalone
`__iterator/readable_traits.h` can't be used by itself, intantiating `iter_value_t` requires `__iterator/iterator_traits.h`. `readable_traits.h` can't include `iterator_traits.h` though because `iterator_traits.h` requires `readable_traits.h`.

Move `iter_value_t` to `__iterator/iterator_traits.h` so that both headers can work standalone.

Reviewed By: Mordante, #libc

Differential Revision: https://reviews.llvm.org/D153828
2023-06-27 10:52:08 -07:00
Louis Dionne
5aa03b648b [libc++][NFC] Apply clang-format on large parts of the code base
This commit does a pass of clang-format over files in libc++ that
don't require major changes to conform to our style guide, or for
which we're not overly concerned about conflicting with in-flight
patches or hindering the git blame.

This roughly covers:
- benchmarks
- range algorithms
- concepts
- type traits

I did a manual verification of all the changes, and in particular I
applied clang-format on/off annotations in a few places where the
result was less readable after than before. This was not necessary
in a lot of places, however I did find that clang-format had pretty
bad taste when it comes to formatting concepts.

Differential Revision: https://reviews.llvm.org/D153140
2023-06-19 11:19:51 -04:00
Louis Dionne
d8655fb365 [libc++][NFC] Rename __lower_bound_impl to __lower_bound
For consistency with other algorithms.

Differential Revision: https://reviews.llvm.org/D153141
2023-06-19 09:05:14 -04:00
Nikolas Klauser
31eeba3f7c [libc++] Introduce __make_uninitialized_buffer and use it instead of get_temporary_buffer
This will also be used in some PSTL backends.

Reviewed By: ldionne, #libc, Mordante

Spies: arichardson, mstorsjo, Mordante, sstefan1, jplehr, libcxx-commits

Differential Revision: https://reviews.llvm.org/D152208
2023-06-16 07:53:56 -07:00
Nikolas Klauser
3a7876f6e2 [libc++][PSTL] Implement std::is_partitioned
Reviewed By: #libc, ldionne

Spies: ldionne, libcxx-commits

Differential Revision: https://reviews.llvm.org/D152853
2023-06-15 11:23:07 -07:00
Nikolas Klauser
b935ab8e74 [libc++] Add tests to make sure that stable algorithms work without memory available
Reviewed By: #libc, ldionne

Spies: power-llvm-team, ldionne, libcxx-commits, arichardson, mstorsjo

Differential Revision: https://reviews.llvm.org/D152939
2023-06-15 10:59:34 -07:00
Nikolas Klauser
f1ea0b11ca [libc++] Merge _LIBCPP_FUNC_VIS, _LIBCPP_TYPE_VIS and _LIBCPP_EXCEPTION_ABI into _LIBCPP_EXPORTED_FROM_ABI
These macros are always defined identically, so we can simplify the code a bit by merging them.

Reviewed By: ldionne, #libc

Spies: libcxx-commits, krytarowski, smeenai

Differential Revision: https://reviews.llvm.org/D152652
2023-06-15 08:56:45 -07:00
Louis Dionne
3456b2f60a [libc++] Refactor __debug_three_way_comp
This makes __debug_three_way_comp consistent with __debug_less and
in particular gets rid of a potential use-after-move caused by the
use of std::forward. In the previous version of the code, we would
call `__do_compare_assert` after forwarding the arguments into the
comparator, which could end up using the arguments after they've been
moved from.

This also simplifies how we call `__do_compare_assert` by using
`if constexpr` and adds a missing test for proxy iterators in
lexicographical_compare_three_way, which could have found this
issue.

Differential Revision: https://reviews.llvm.org/D152753
2023-06-13 14:24:56 -07:00
Nikolas Klauser
cd916108b4 [libc++][PSTL] Implement std::generate{,_n}
Reviewed By: #libc, ldionne

Spies: ldionne, libcxx-commits

Differential Revision: https://reviews.llvm.org/D152581
2023-06-13 07:55:06 -07:00
Hristo Hristov
8fe609cb3a [libc++][spaceship] Fixed __debug_three_way_comp's operator() for vector<bool>'s operator<=>`
An issue with `operator()` was found during the implementation of https://reviews.llvm.org/D132268.

This patch aims to resolve the issues by updating the operator to use perfect forwarding.

The original motivation for `three_way_comp_ref_type` is given in: https://reviews.llvm.org/D131395

`three_way_comp_ref_type`'s implementation is inspired by `comp_ref_type`, which has two overloads:

```
    template <class _Tp, class _Up>
    bool operator()(const _Tp& __x,  const _Up& __y);

    template <class _Tp, class _Up>
    bool operator()(_Tp& __x,  _Up& __y);
```

`__debug_three_way_comp` is missing the first overload and also declares the typealias`_three_way_comp_ref_type ` incorrectly.

Reviewed By: #libc, philnik

Differential Revision: https://reviews.llvm.org/D150188
2023-06-10 06:48:01 +03:00
Nikolas Klauser
6adb1ca555 [libc++] Add concepts that ensure a given iterator meets the syntactic requirements
These concepts are used to ensure valid iterators are passed to PSTL algorithms, but can also be used for other interfaces.

Reviewed By: ldionne, #libc

Spies: EricWF, libcxx-commits

Differential Revision: https://reviews.llvm.org/D150493
2023-06-09 08:38:22 -07:00
Nikolas Klauser
88632e4806 [libc++] Refactor __less
This simplifies the usage of `__less` by making the class not depend on the types compared, but instead the `operator()`. We can't remove the template completely because we explicitly instantiate `std::__sort` with `__less<T>`.

Reviewed By: ldionne, #libc

Spies: arichardson, EricWF, libcxx-commits, mgrang

Differential Revision: https://reviews.llvm.org/D145285
2023-06-06 13:58:52 -07:00
Nikolas Klauser
51131edf83 [libc++][PSTL] Implement std::replace{,_if,_copy,_copy_if}
Reviewed By: #libc, ldionne

Spies: ldionne, libcxx-commits

Differential Revision: https://reviews.llvm.org/D151841
2023-06-06 08:44:53 -07:00
Nikolas Klauser
7a3b528e1b [libc++][PSTL] Implement std::count{,_if}
Reviewed By: ldionne, #libc

Spies: libcxx-commits

Differential Revision: https://reviews.llvm.org/D150128
2023-06-06 08:42:59 -07:00
Nikolas Klauser
355f466744 [libc++][NFC] Add __element_count and use it in the constexpr C functions
This makes it less ambiguous what the parameter is meant to get.

Reviewed By: #libc, ldionne

Spies: ldionne, libcxx-commits

Differential Revision: https://reviews.llvm.org/D152040
2023-06-05 15:08:01 -07:00
Nikolas Klauser
d965960fcf Revert "[libc++] Optimize for_each for segmented iterators"
This reverts commit b1dc43aa3a05c2f14725e2e6428544208ccbe161.
2023-06-05 10:00:02 -07:00
Danila Kutenin
7e1ee1e10d [libcxx] Add strict weak ordering checks to sorting algorithms
This is the implementation of the first proposal of strict weak ordering checks described in https://discourse.llvm.org/t/rfc-strict-weak-ordering-checks-in-the-debug-libc/70217

This targets the most vulnerable algorithms like std::sort

Reviewed By: philnik, #libc

Differential Revision: https://reviews.llvm.org/D150264
2023-06-04 19:27:31 +02:00
Nikolas Klauser
d51a84b405 [libc++][PSTL] Implement std::stable_sort
Reviewed By: #libc, ldionne

Spies: ldionne, libcxx-commits

Differential Revision: https://reviews.llvm.org/D151573
2023-06-01 15:07:28 -07:00
Nikolas Klauser
ee6ec2c5f1 [libc++][PSTL] Implement std::reduce and std::transform_reduce
Reviewed By: ldionne, #libc

Spies: libcxx-commits, miyuki

Differential Revision: https://reviews.llvm.org/D150736
2023-06-01 08:52:08 -07:00
Nikolas Klauser
b1dc43aa3a [libc++] Optimize for_each for segmented iterators
```
---------------------------------------------------
Benchmark                       old             new
---------------------------------------------------
bm_for_each/1               3.00 ns         2.98 ns
bm_for_each/2               4.53 ns         4.57 ns
bm_for_each/3               5.82 ns         5.82 ns
bm_for_each/4               6.94 ns         6.91 ns
bm_for_each/5               7.55 ns         7.75 ns
bm_for_each/6               7.06 ns         7.45 ns
bm_for_each/7               6.69 ns         7.14 ns
bm_for_each/8               6.86 ns         4.06 ns
bm_for_each/16              11.5 ns         5.73 ns
bm_for_each/64              43.7 ns         4.06 ns
bm_for_each/512              356 ns         7.98 ns
bm_for_each/4096            2787 ns         53.6 ns
bm_for_each/32768          20836 ns          438 ns
bm_for_each/262144        195362 ns         4945 ns
bm_for_each/1048576       685482 ns        19822 ns
```

Reviewed By: ldionne, Mordante, #libc

Spies: arichardson, libcxx-commits

Differential Revision: https://reviews.llvm.org/D151274
2023-05-31 18:15:25 -07:00
Nikolas Klauser
dc124cda7c [libc++] Introduce __for_each_segment and use it in copy/move
This simplifies the code inside copy/move and makes it easier to apply the optimization to other algorithms.

Reviewed By: ldionne, Mordante, #libc

Spies: arichardson, libcxx-commits

Differential Revision: https://reviews.llvm.org/D151265
2023-05-31 18:15:20 -07:00
Nikolas Klauser
bf63b15bd4 [libc++][PSTL] Implement std::merge
Reviewed By: ldionne, #libc

Spies: pcwang-thead, libcxx-commits

Differential Revision: https://reviews.llvm.org/D151375
2023-05-30 15:07:06 -07:00
Nikolas Klauser
b1cf54f12e [libc++][PSTL] Add a test to make sure that customization points work properly
Reviewed By: #libc, ldionne

Spies: ldionne, libcxx-commits

Differential Revision: https://reviews.llvm.org/D151257
2023-05-26 10:53:08 -07:00
Nikolas Klauser
1fd08edd58 [libc++] Forward to std::{,w}memchr in std::find
Reviewed By: #libc, ldionne

Spies: Mordante, libcxx-commits, ldionne, mikhail.ramalho

Differential Revision: https://reviews.llvm.org/D144394
2023-05-25 07:59:50 -07:00
Nikolas Klauser
e837f4b7db [libc++][PSTL] Add a simple std::thread backend
This is just to test that the PSTL works with parallelization. This is not supposed to be a production-ready backend.

Reviewed By: ldionne, #libc

Spies: EricWF, arichardson, libcxx-commits

Differential Revision: https://reviews.llvm.org/D150284
2023-05-24 15:33:58 -07:00
Nikolas Klauser
80643d9366 [libc++][NFC] Rename iterator category checks to make it obvious that they check //only// the iterator category
We plan to add concepts for checking that iterators actually provide what they claim to. This is to avoid people thinking that these type traits actually check the iterator requirements in more detail.

Reviewed By: ldionne, #libc

Spies: Mordante, libcxx-commits, wenlei

Differential Revision: https://reviews.llvm.org/D150801
2023-05-18 15:37:28 -07:00
Louis Dionne
35f9fd6435 [libc++] Avoid dereferencing a const iterator in std::sort
This is a workaround to provide a grace period for folks that were
broken by D147089. As a fly-by, also apply comments by Mark I had
somehow missed in the review.

Differential Revision: https://reviews.llvm.org/D150779
2023-05-18 08:24:17 -07:00
Nikolas Klauser
7ace54e64b [libc++][PSTL] Implement std::copy{,_n}
Reviewed By: ldionne, #libc

Spies: jloser, libcxx-commits

Differential Revision: https://reviews.llvm.org/D149706
2023-05-15 14:46:59 -07:00