darling-libcxx/benchmarks/ContainerBenchmarks.h

141 lines
4.1 KiB
C
Raw Permalink Normal View History

// -*- C++ -*-
//===----------------------------------------------------------------------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
#ifndef BENCHMARK_CONTAINER_BENCHMARKS_H
#define BENCHMARK_CONTAINER_BENCHMARKS_H
[libcxx] Add support for benchmark tests using Google Benchmark. Summary: This patch does the following: 1. Checks in a copy of the Google Benchmark library into the libc++ repo under `utils/google-benchmark`. 2. Teaches libc++ how to build Google Benchmark against both (A) in-tree libc++ and (B) the platforms native STL. 3. Allows performance benchmarks to be built as part of the libc++ build. Building the benchmarks (and Google Benchmark) is off by default. It must be enabled using the CMake option `-DLIBCXX_INCLUDE_BENCHMARKS=ON`. When this option is enabled the tests under `libcxx/benchmarks` can be built using the `libcxx-benchmarks` target. On Linux platforms where libstdc++ is the default STL the CMake option `-DLIBCXX_BUILD_BENCHMARKS_NATIVE_STDLIB=ON` can be used to build each benchmark test against libstdc++ as well. This is useful for comparing performance between standard libraries. Support for benchmarks is currently very minimal. They must be manually run by the user and there is no mechanism for detecting performance regressions. Known Issues: * `-DLIBCXX_INCLUDE_BENCHMARKS=ON` is only supported for Clang, and not GCC, since the `-stdlib=libc++` option is needed to build Google Benchmark. Reviewers: danalbert, dberlin, chandlerc, mclow.lists, jroelofs Subscribers: chandlerc, dberlin, tberghammer, danalbert, srhines, hfinkel Differential Revision: https://reviews.llvm.org/D22240 git-svn-id: https://llvm.org/svn/llvm-project/libcxx/trunk@276049 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-19 23:07:03 +00:00
#include <cassert>
#include "Utilities.h"
#include "benchmark/benchmark.h"
[libcxx] Add support for benchmark tests using Google Benchmark. Summary: This patch does the following: 1. Checks in a copy of the Google Benchmark library into the libc++ repo under `utils/google-benchmark`. 2. Teaches libc++ how to build Google Benchmark against both (A) in-tree libc++ and (B) the platforms native STL. 3. Allows performance benchmarks to be built as part of the libc++ build. Building the benchmarks (and Google Benchmark) is off by default. It must be enabled using the CMake option `-DLIBCXX_INCLUDE_BENCHMARKS=ON`. When this option is enabled the tests under `libcxx/benchmarks` can be built using the `libcxx-benchmarks` target. On Linux platforms where libstdc++ is the default STL the CMake option `-DLIBCXX_BUILD_BENCHMARKS_NATIVE_STDLIB=ON` can be used to build each benchmark test against libstdc++ as well. This is useful for comparing performance between standard libraries. Support for benchmarks is currently very minimal. They must be manually run by the user and there is no mechanism for detecting performance regressions. Known Issues: * `-DLIBCXX_INCLUDE_BENCHMARKS=ON` is only supported for Clang, and not GCC, since the `-stdlib=libc++` option is needed to build Google Benchmark. Reviewers: danalbert, dberlin, chandlerc, mclow.lists, jroelofs Subscribers: chandlerc, dberlin, tberghammer, danalbert, srhines, hfinkel Differential Revision: https://reviews.llvm.org/D22240 git-svn-id: https://llvm.org/svn/llvm-project/libcxx/trunk@276049 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-19 23:07:03 +00:00
namespace ContainerBenchmarks {
Fix PR35637: suboptimal codegen for `vector<unsigned char>`. The optimizer is petulant and temperamental. In this case LLVM failed to lower the the "insert at end" loop used by`vector<unsigned char>` to a `memset` despite `memset` being substantially faster over a range of bytes. LLVM has the ability to lower loops to `memset` whet appropriate, but the odd nature of libc++'s loops prevented the optimization from taking places. This patch addresses the issue by rewriting the loops from the form `do [ ... --__n; } while (__n > 0);` to instead use a for loop over a pointer range (For example: `for (auto *__i = ...; __i < __e; ++__i)`). This patch also rewrites the asan annotations to unposion all additional memory at the start of the loop instead of once per iterations. This could potentially permit false negatives where the constructor of element N attempts to access element N + 1 during its construction. The before and after results for the `BM_ConstructSize/vector_byte/5140480_mean` benchmark (run 5 times) are: -------------------------------------------------------------------------------------------- Benchmark Time CPU Iterations -------------------------------------------------------------------------------------------- Before ------ BM_ConstructSize/vector_byte/5140480_mean 12530140 ns 12469693 ns N/A BM_ConstructSize/vector_byte/5140480_median 12512818 ns 12445571 ns N/A BM_ConstructSize/vector_byte/5140480_stddev 106224 ns 107907 ns 5 ----- After ----- BM_ConstructSize/vector_byte/5140480_mean 167285 ns 166500 ns N/A BM_ConstructSize/vector_byte/5140480_median 166749 ns 166069 ns N/A BM_ConstructSize/vector_byte/5140480_stddev 3242 ns 3184 ns 5 git-svn-id: https://llvm.org/svn/llvm-project/libcxx/trunk@367183 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-28 04:37:02 +00:00
template <class Container>
void BM_ConstructSize(benchmark::State& st, Container) {
auto size = st.range(0);
for (auto _ : st) {
Container c(size);
DoNotOptimizeData(c);
Fix PR35637: suboptimal codegen for `vector<unsigned char>`. The optimizer is petulant and temperamental. In this case LLVM failed to lower the the "insert at end" loop used by`vector<unsigned char>` to a `memset` despite `memset` being substantially faster over a range of bytes. LLVM has the ability to lower loops to `memset` whet appropriate, but the odd nature of libc++'s loops prevented the optimization from taking places. This patch addresses the issue by rewriting the loops from the form `do [ ... --__n; } while (__n > 0);` to instead use a for loop over a pointer range (For example: `for (auto *__i = ...; __i < __e; ++__i)`). This patch also rewrites the asan annotations to unposion all additional memory at the start of the loop instead of once per iterations. This could potentially permit false negatives where the constructor of element N attempts to access element N + 1 during its construction. The before and after results for the `BM_ConstructSize/vector_byte/5140480_mean` benchmark (run 5 times) are: -------------------------------------------------------------------------------------------- Benchmark Time CPU Iterations -------------------------------------------------------------------------------------------- Before ------ BM_ConstructSize/vector_byte/5140480_mean 12530140 ns 12469693 ns N/A BM_ConstructSize/vector_byte/5140480_median 12512818 ns 12445571 ns N/A BM_ConstructSize/vector_byte/5140480_stddev 106224 ns 107907 ns 5 ----- After ----- BM_ConstructSize/vector_byte/5140480_mean 167285 ns 166500 ns N/A BM_ConstructSize/vector_byte/5140480_median 166749 ns 166069 ns N/A BM_ConstructSize/vector_byte/5140480_stddev 3242 ns 3184 ns 5 git-svn-id: https://llvm.org/svn/llvm-project/libcxx/trunk@367183 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-28 04:37:02 +00:00
}
}
template <class Container>
void BM_ConstructSizeValue(benchmark::State& st, Container, typename Container::value_type const& val) {
const auto size = st.range(0);
for (auto _ : st) {
Container c(size, val);
DoNotOptimizeData(c);
Fix PR35637: suboptimal codegen for `vector<unsigned char>`. The optimizer is petulant and temperamental. In this case LLVM failed to lower the the "insert at end" loop used by`vector<unsigned char>` to a `memset` despite `memset` being substantially faster over a range of bytes. LLVM has the ability to lower loops to `memset` whet appropriate, but the odd nature of libc++'s loops prevented the optimization from taking places. This patch addresses the issue by rewriting the loops from the form `do [ ... --__n; } while (__n > 0);` to instead use a for loop over a pointer range (For example: `for (auto *__i = ...; __i < __e; ++__i)`). This patch also rewrites the asan annotations to unposion all additional memory at the start of the loop instead of once per iterations. This could potentially permit false negatives where the constructor of element N attempts to access element N + 1 during its construction. The before and after results for the `BM_ConstructSize/vector_byte/5140480_mean` benchmark (run 5 times) are: -------------------------------------------------------------------------------------------- Benchmark Time CPU Iterations -------------------------------------------------------------------------------------------- Before ------ BM_ConstructSize/vector_byte/5140480_mean 12530140 ns 12469693 ns N/A BM_ConstructSize/vector_byte/5140480_median 12512818 ns 12445571 ns N/A BM_ConstructSize/vector_byte/5140480_stddev 106224 ns 107907 ns 5 ----- After ----- BM_ConstructSize/vector_byte/5140480_mean 167285 ns 166500 ns N/A BM_ConstructSize/vector_byte/5140480_median 166749 ns 166069 ns N/A BM_ConstructSize/vector_byte/5140480_stddev 3242 ns 3184 ns 5 git-svn-id: https://llvm.org/svn/llvm-project/libcxx/trunk@367183 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-28 04:37:02 +00:00
}
}
template <class Container, class GenInputs>
void BM_ConstructIterIter(benchmark::State& st, Container, GenInputs gen) {
auto in = gen(st.range(0));
const auto begin = in.begin();
const auto end = in.end();
benchmark::DoNotOptimize(&in);
while (st.KeepRunning()) {
Container c(begin, end);
DoNotOptimizeData(c);
}
}
[libcxx] Add support for benchmark tests using Google Benchmark. Summary: This patch does the following: 1. Checks in a copy of the Google Benchmark library into the libc++ repo under `utils/google-benchmark`. 2. Teaches libc++ how to build Google Benchmark against both (A) in-tree libc++ and (B) the platforms native STL. 3. Allows performance benchmarks to be built as part of the libc++ build. Building the benchmarks (and Google Benchmark) is off by default. It must be enabled using the CMake option `-DLIBCXX_INCLUDE_BENCHMARKS=ON`. When this option is enabled the tests under `libcxx/benchmarks` can be built using the `libcxx-benchmarks` target. On Linux platforms where libstdc++ is the default STL the CMake option `-DLIBCXX_BUILD_BENCHMARKS_NATIVE_STDLIB=ON` can be used to build each benchmark test against libstdc++ as well. This is useful for comparing performance between standard libraries. Support for benchmarks is currently very minimal. They must be manually run by the user and there is no mechanism for detecting performance regressions. Known Issues: * `-DLIBCXX_INCLUDE_BENCHMARKS=ON` is only supported for Clang, and not GCC, since the `-stdlib=libc++` option is needed to build Google Benchmark. Reviewers: danalbert, dberlin, chandlerc, mclow.lists, jroelofs Subscribers: chandlerc, dberlin, tberghammer, danalbert, srhines, hfinkel Differential Revision: https://reviews.llvm.org/D22240 git-svn-id: https://llvm.org/svn/llvm-project/libcxx/trunk@276049 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-19 23:07:03 +00:00
template <class Container, class GenInputs>
void BM_InsertValue(benchmark::State& st, Container c, GenInputs gen) {
auto in = gen(st.range(0));
[libcxx] Add support for benchmark tests using Google Benchmark. Summary: This patch does the following: 1. Checks in a copy of the Google Benchmark library into the libc++ repo under `utils/google-benchmark`. 2. Teaches libc++ how to build Google Benchmark against both (A) in-tree libc++ and (B) the platforms native STL. 3. Allows performance benchmarks to be built as part of the libc++ build. Building the benchmarks (and Google Benchmark) is off by default. It must be enabled using the CMake option `-DLIBCXX_INCLUDE_BENCHMARKS=ON`. When this option is enabled the tests under `libcxx/benchmarks` can be built using the `libcxx-benchmarks` target. On Linux platforms where libstdc++ is the default STL the CMake option `-DLIBCXX_BUILD_BENCHMARKS_NATIVE_STDLIB=ON` can be used to build each benchmark test against libstdc++ as well. This is useful for comparing performance between standard libraries. Support for benchmarks is currently very minimal. They must be manually run by the user and there is no mechanism for detecting performance regressions. Known Issues: * `-DLIBCXX_INCLUDE_BENCHMARKS=ON` is only supported for Clang, and not GCC, since the `-stdlib=libc++` option is needed to build Google Benchmark. Reviewers: danalbert, dberlin, chandlerc, mclow.lists, jroelofs Subscribers: chandlerc, dberlin, tberghammer, danalbert, srhines, hfinkel Differential Revision: https://reviews.llvm.org/D22240 git-svn-id: https://llvm.org/svn/llvm-project/libcxx/trunk@276049 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-19 23:07:03 +00:00
const auto end = in.end();
while (st.KeepRunning()) {
c.clear();
for (auto it = in.begin(); it != end; ++it) {
benchmark::DoNotOptimize(&(*c.insert(*it).first));
}
benchmark::ClobberMemory();
}
}
template <class Container, class GenInputs>
void BM_InsertValueRehash(benchmark::State& st, Container c, GenInputs gen) {
auto in = gen(st.range(0));
[libcxx] Add support for benchmark tests using Google Benchmark. Summary: This patch does the following: 1. Checks in a copy of the Google Benchmark library into the libc++ repo under `utils/google-benchmark`. 2. Teaches libc++ how to build Google Benchmark against both (A) in-tree libc++ and (B) the platforms native STL. 3. Allows performance benchmarks to be built as part of the libc++ build. Building the benchmarks (and Google Benchmark) is off by default. It must be enabled using the CMake option `-DLIBCXX_INCLUDE_BENCHMARKS=ON`. When this option is enabled the tests under `libcxx/benchmarks` can be built using the `libcxx-benchmarks` target. On Linux platforms where libstdc++ is the default STL the CMake option `-DLIBCXX_BUILD_BENCHMARKS_NATIVE_STDLIB=ON` can be used to build each benchmark test against libstdc++ as well. This is useful for comparing performance between standard libraries. Support for benchmarks is currently very minimal. They must be manually run by the user and there is no mechanism for detecting performance regressions. Known Issues: * `-DLIBCXX_INCLUDE_BENCHMARKS=ON` is only supported for Clang, and not GCC, since the `-stdlib=libc++` option is needed to build Google Benchmark. Reviewers: danalbert, dberlin, chandlerc, mclow.lists, jroelofs Subscribers: chandlerc, dberlin, tberghammer, danalbert, srhines, hfinkel Differential Revision: https://reviews.llvm.org/D22240 git-svn-id: https://llvm.org/svn/llvm-project/libcxx/trunk@276049 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-19 23:07:03 +00:00
const auto end = in.end();
while (st.KeepRunning()) {
c.clear();
c.rehash(16);
for (auto it = in.begin(); it != end; ++it) {
benchmark::DoNotOptimize(&(*c.insert(*it).first));
}
benchmark::ClobberMemory();
}
}
template <class Container, class GenInputs>
void BM_InsertDuplicate(benchmark::State& st, Container c, GenInputs gen) {
auto in = gen(st.range(0));
const auto end = in.end();
c.insert(in.begin(), in.end());
benchmark::DoNotOptimize(&c);
benchmark::DoNotOptimize(&in);
while (st.KeepRunning()) {
for (auto it = in.begin(); it != end; ++it) {
benchmark::DoNotOptimize(&(*c.insert(*it).first));
}
benchmark::ClobberMemory();
}
}
template <class Container, class GenInputs>
void BM_EmplaceDuplicate(benchmark::State& st, Container c, GenInputs gen) {
auto in = gen(st.range(0));
const auto end = in.end();
c.insert(in.begin(), in.end());
benchmark::DoNotOptimize(&c);
benchmark::DoNotOptimize(&in);
while (st.KeepRunning()) {
for (auto it = in.begin(); it != end; ++it) {
benchmark::DoNotOptimize(&(*c.emplace(*it).first));
}
benchmark::ClobberMemory();
}
}
[libcxx] Add support for benchmark tests using Google Benchmark. Summary: This patch does the following: 1. Checks in a copy of the Google Benchmark library into the libc++ repo under `utils/google-benchmark`. 2. Teaches libc++ how to build Google Benchmark against both (A) in-tree libc++ and (B) the platforms native STL. 3. Allows performance benchmarks to be built as part of the libc++ build. Building the benchmarks (and Google Benchmark) is off by default. It must be enabled using the CMake option `-DLIBCXX_INCLUDE_BENCHMARKS=ON`. When this option is enabled the tests under `libcxx/benchmarks` can be built using the `libcxx-benchmarks` target. On Linux platforms where libstdc++ is the default STL the CMake option `-DLIBCXX_BUILD_BENCHMARKS_NATIVE_STDLIB=ON` can be used to build each benchmark test against libstdc++ as well. This is useful for comparing performance between standard libraries. Support for benchmarks is currently very minimal. They must be manually run by the user and there is no mechanism for detecting performance regressions. Known Issues: * `-DLIBCXX_INCLUDE_BENCHMARKS=ON` is only supported for Clang, and not GCC, since the `-stdlib=libc++` option is needed to build Google Benchmark. Reviewers: danalbert, dberlin, chandlerc, mclow.lists, jroelofs Subscribers: chandlerc, dberlin, tberghammer, danalbert, srhines, hfinkel Differential Revision: https://reviews.llvm.org/D22240 git-svn-id: https://llvm.org/svn/llvm-project/libcxx/trunk@276049 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-19 23:07:03 +00:00
template <class Container, class GenInputs>
static void BM_Find(benchmark::State& st, Container c, GenInputs gen) {
auto in = gen(st.range(0));
[libcxx] Add support for benchmark tests using Google Benchmark. Summary: This patch does the following: 1. Checks in a copy of the Google Benchmark library into the libc++ repo under `utils/google-benchmark`. 2. Teaches libc++ how to build Google Benchmark against both (A) in-tree libc++ and (B) the platforms native STL. 3. Allows performance benchmarks to be built as part of the libc++ build. Building the benchmarks (and Google Benchmark) is off by default. It must be enabled using the CMake option `-DLIBCXX_INCLUDE_BENCHMARKS=ON`. When this option is enabled the tests under `libcxx/benchmarks` can be built using the `libcxx-benchmarks` target. On Linux platforms where libstdc++ is the default STL the CMake option `-DLIBCXX_BUILD_BENCHMARKS_NATIVE_STDLIB=ON` can be used to build each benchmark test against libstdc++ as well. This is useful for comparing performance between standard libraries. Support for benchmarks is currently very minimal. They must be manually run by the user and there is no mechanism for detecting performance regressions. Known Issues: * `-DLIBCXX_INCLUDE_BENCHMARKS=ON` is only supported for Clang, and not GCC, since the `-stdlib=libc++` option is needed to build Google Benchmark. Reviewers: danalbert, dberlin, chandlerc, mclow.lists, jroelofs Subscribers: chandlerc, dberlin, tberghammer, danalbert, srhines, hfinkel Differential Revision: https://reviews.llvm.org/D22240 git-svn-id: https://llvm.org/svn/llvm-project/libcxx/trunk@276049 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-19 23:07:03 +00:00
c.insert(in.begin(), in.end());
benchmark::DoNotOptimize(&(*c.begin()));
const auto end = in.data() + in.size();
while (st.KeepRunning()) {
for (auto it = in.data(); it != end; ++it) {
benchmark::DoNotOptimize(&(*c.find(*it)));
}
benchmark::ClobberMemory();
}
}
template <class Container, class GenInputs>
static void BM_FindRehash(benchmark::State& st, Container c, GenInputs gen) {
c.rehash(8);
auto in = gen(st.range(0));
[libcxx] Add support for benchmark tests using Google Benchmark. Summary: This patch does the following: 1. Checks in a copy of the Google Benchmark library into the libc++ repo under `utils/google-benchmark`. 2. Teaches libc++ how to build Google Benchmark against both (A) in-tree libc++ and (B) the platforms native STL. 3. Allows performance benchmarks to be built as part of the libc++ build. Building the benchmarks (and Google Benchmark) is off by default. It must be enabled using the CMake option `-DLIBCXX_INCLUDE_BENCHMARKS=ON`. When this option is enabled the tests under `libcxx/benchmarks` can be built using the `libcxx-benchmarks` target. On Linux platforms where libstdc++ is the default STL the CMake option `-DLIBCXX_BUILD_BENCHMARKS_NATIVE_STDLIB=ON` can be used to build each benchmark test against libstdc++ as well. This is useful for comparing performance between standard libraries. Support for benchmarks is currently very minimal. They must be manually run by the user and there is no mechanism for detecting performance regressions. Known Issues: * `-DLIBCXX_INCLUDE_BENCHMARKS=ON` is only supported for Clang, and not GCC, since the `-stdlib=libc++` option is needed to build Google Benchmark. Reviewers: danalbert, dberlin, chandlerc, mclow.lists, jroelofs Subscribers: chandlerc, dberlin, tberghammer, danalbert, srhines, hfinkel Differential Revision: https://reviews.llvm.org/D22240 git-svn-id: https://llvm.org/svn/llvm-project/libcxx/trunk@276049 91177308-0d34-0410-b5e6-96231b3b80d8
2016-07-19 23:07:03 +00:00
c.insert(in.begin(), in.end());
benchmark::DoNotOptimize(&(*c.begin()));
const auto end = in.data() + in.size();
while (st.KeepRunning()) {
for (auto it = in.data(); it != end; ++it) {
benchmark::DoNotOptimize(&(*c.find(*it)));
}
benchmark::ClobberMemory();
}
}
} // end namespace ContainerBenchmarks
#endif // BENCHMARK_CONTAINER_BENCHMARKS_H