reactos/CMake
1
0
Fork 0
mirror of https://github.com/reactos/CMake.git synced 2024-12-04 09:54:15 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge topic 'ctest-hardware-allocation'

Browse Source 
e9500271a3 Help: Add documentation for CTest hardware allocation
d1f100a415 CTest: Add Json output for PROCESSES property
b741ee820d Tests: Add test for CTest hardware allocation feature
3c8a5aad46 Tests: Write tests for cthwalloc helper tool
2d74e54661 Tests: Write cthwalloc helper tool
e34de0691b CTest: Allocate hardware to tests
aee0964851 CTest: Add bin-packing algorithm
c494b2973a CTest: Add cmCTestHardwareAllocator class
...

Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !3858
This commit is contained in:
Brad King 2019-10-03 18:07:37 +00:00 committed by Kitware Robot
commit 3247506662
122 changed files with 6343 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
Help/command/ctest_test.rst
View File

@ -17,6 +17,7 @@ Perform the :ref:`CTest Test Step` as a :ref:`Dashboard Client`.
[EXCLUDE_FIXTURE_SETUP <regex>]
[EXCLUDE_FIXTURE_CLEANUP <regex>]
[PARALLEL_LEVEL <level>]
[HARDWARE_SPEC_FILE <file>]
[TEST_LOAD <threshold>]
[SCHEDULE_RANDOM <ON|OFF>]
[STOP_TIME <time-of-day>]
@ -82,6 +83,11 @@ The options are:
Specify a positive number representing the number of tests to
be run in parallel.
``HARDWARE_SPEC_FILE <file>``
Specify a
:ref:`hardware specification file <ctest-hardware-specification-file>`. See
:ref:`ctest-hardware-allocation` for more information.
``TEST_LOAD <threshold>``
While running tests in parallel, try not to start tests when they
may cause the CPU load to pass above a given threshold. If not

1
Help/manual/cmake-properties.7.rst
View File

@ -414,6 +414,7 @@ Properties on Tests
/prop_test/LABELS
/prop_test/MEASUREMENT
/prop_test/PASS_REGULAR_EXPRESSION
/prop_test/PROCESSES
/prop_test/PROCESSOR_AFFINITY
/prop_test/PROCESSORS
/prop_test/REQUIRED_FILES

229
Help/manual/ctest.1.rst
View File

@ -90,6 +90,15 @@ Options
See `Label and Subproject Summary`_.
``--hardware-spec-file <file>``
Run CTest with :ref:`hardware allocation <ctest-hardware-allocation>` enabled,
using the
:ref:`hardware specification file <ctest-hardware-specification-file>`
specified in ``<file>``.
When ``ctest`` is run as a `Dashboard Client`_ this sets the
``HardwareSpecFile`` option of the `CTest Test Step`_.
``--test-load <level>``
While running tests in parallel (e.g. with ``-j``), try not to start
tests when they may cause the CPU load to pass above a given threshold.
@ -958,6 +967,11 @@ Arguments to the command may specify some of the step settings.
Configuration settings include:
``HardwareSpecFile``
Specify a
:ref:`hardware specification file <ctest-hardware-specification-file>`. See
:ref:`ctest-hardware-allocation` for more information.
``LabelsForSubprojects``
Specify a semicolon-separated list of labels that will be treated as
subprojects. This mapping will be passed on to CDash when configure, test or
@ -1267,6 +1281,221 @@ model is defined as follows:
Test properties.
Can contain keys for each of the supported test properties.
.. _`ctest-hardware-allocation`:
Hardware Allocation
===================
CTest provides a mechanism for tests to specify the hardware that they need and
how much of it they need, and for users to specify the hardware availiable on
the running machine. This allows CTest to internally keep track of which
hardware is in use and which is free, scheduling tests in a way that prevents
them from trying to claim hardware that is not available.
A common use case for this feature is for tests that require the use of a GPU.
Multiple tests can simultaneously allocate memory from a GPU, but if too many
tests try to do this at once, some of them will fail to allocate, resulting in
a failed test, even though the test would have succeeded if it had the memory
it needed. By using the hardware allocation feature, each test can specify how
much memory it requires from a GPU, allowing CTest to schedule tests in a way
that running several of these tests at once does not exhaust the GPU's memory
pool.
Please note that CTest has no concept of what a GPU is or how much memory it
has, nor does it have any way of communicating with a GPU to retrieve this
information or perform any memory management. CTest simply keeps track of a
list of abstract resource types, each of which has a certain number of slots
available for tests to use. Each test specifies the number of slots that it
requires from a certain resource, and CTest then schedules them in a way that
prevents the total number of slots in use from exceeding the listed capacity.
When a test is executed, and slots from a resource are allocated to that test,
tests may assume that they have exclusive use of those slots for the duration
of the test's process.
The CTest hardware allocation feature consists of two inputs:
* The :ref:`hardware specification file <ctest-hardware-specification-file>`,
described below, which describes the hardware resources available on the
system, and
* The :prop_test:`PROCESSES` property of tests, which describes the resources
required by the test
When CTest runs a test, the hardware allocated to that test is passed in the
form of a set of
:ref:`environment variables <ctest-hardware-environment-variables>` as
described below. Using this information to decide which resource to connect to
is left to the test writer.
Please note that these processes are not spawned by CTest. The ``PROCESSES``
property merely tells CTest what processes the test expects to launch. It is up
to the test itself to do this process spawning, and read the :ref:`environment
variables <ctest-hardware-environment-variables>` to determine which resources
each process has been allocated.
.. _`ctest-hardware-specification-file`:
Hardware Specification File
---------------------------
The hardware specification file is a JSON file which is passed to CTest, either
on the :manual:`ctest(1)` command line as ``--hardware-spec-file``, or as the
``HARDWARE_SPEC_FILE`` argument of :command:`ctest_test`. The hardware
specification file must be a JSON object. All examples in this document assume
the following hardware specification file:
.. code-block:: json
{
"local": [
{
"gpus": [
{
"id": "0",
"slots": 2
},
{
"id": "1",
"slots": 4
},
{
"id": "2",
"slots": 2
},
{
"id": "3"
}
],
"crypto_chips": [
{
"id": "card0",
"slots": 4
}
]
}
]
}
The members are:
``local``
A JSON array consisting of CPU sockets present on the system. Currently, only
one socket is supported.
Each socket is a JSON object with members whose names are equal to the
desired resource types, such as ``gpu``. These names must start with a
lowercase letter or an underscore, and subsequent characters can be a
lowercase letter, a digit, or an underscore. Uppercase letters are not
allowed, because certain platforms have case-insensitive environment
variables. See the `Environment Variables`_ section below for
more information. It is recommended that the resource type name be the plural
of a noun, such as ``gpus`` or ``crypto_chips`` (and not ``gpu`` or
``crypto_chip``.)
Please note that the names ``gpus`` and ``crypto_chips`` are just examples,
and CTest does not interpret them in any way. You are free to make up any
resource type you want to meet your own requirements.
The value for each resource type is a JSON array consisting of JSON objects,
each of which describe a specific instance of the specified resource. These
objects have the following members:
``id``
A string consisting of an identifier for the resource. Each character in
the identifier can be a lowercase letter, a digit, or an underscore.
Uppercase letters are not allowed.
Identifiers must be unique within a resource type. However, they do not
have to be unique across resource types. For example, it is valid to have a
``gpus`` resource named ``0`` and a ``crypto_chips`` resource named ``0``,
but not two ``gpus`` resources both named ``0``.
Please note that the IDs ``0``, ``1``, ``2``, ``3``, and ``card0`` are just
examples, and CTest does not interpret them in any way. You are free to
make up any IDs you want to meet your own requirements.
``slots``
An optional unsigned number specifying the number of slots available on the
resource. For example, this could be megabytes of RAM on a GPU, or
cryptography units available on a cryptography chip. If ``slots`` is not
specified, a default value of ``1`` is assumed.
In the example file above, there are four GPUs with ID's 0 through 3. GPU 0 has
2 slots, GPU 1 has 4, GPU 2 has 2, and GPU 3 has a default of 1 slot. There is
also one cryptography chip with 4 slots.
``PROCESSES`` Property
----------------------
See :prop_test:`PROCESSES` for a description of this property.
.. _`ctest-hardware-environment-variables`:
Environment Variables
---------------------
Once CTest has decided which resources to allocate to a test, it passes this
information to the test executable as a series of environment variables. For
each example below, we will assume that the test in question has a
:prop_test:`PROCESSES` property of ``2,gpus:2;gpus:4,gpus:1,crypto_chips:2``.
The following variables are passed to the test process:
.. envvar:: CTEST_PROCESS_COUNT
The total number of processes specified by the :prop_test:`PROCESSES`
property. For example:
* ``CTEST_PROCESS_COUNT=3``
This variable will only be defined if :manual:`ctest(1)` has been given a
``--hardware-spec-file``, or if :command:`ctest_test` has been given a
``HARDWARE_SPEC_FILE``. If no hardware specification file has been given,
this variable will not be defined.
.. envvar:: CTEST_PROCESS_<num>
The list of resource types allocated to each process, with each item
separated by a comma. ``<num>`` is a number from zero to
``CTEST_PROCESS_COUNT`` minus one. ``CTEST_PROCESS_<num>`` is defined for
each ``<num>`` in this range. For example:
* ``CTEST_PROCESS_0=gpus``
* ``CTEST_PROCESS_1=gpus``
* ``CTEST_PROCESS_2=crypto_chips,gpus``
.. envvar:: CTEST_PROCESS_<num>_<resource-type>
The list of resource IDs and number of slots from each ID allocated to each
process for a given resource type. This variable consists of a series of
pairs, each pair separated by a semicolon, and with the two items in the pair
separated by a comma. The first item in each pair is ``id:`` followed by the
ID of a resource of type ``<resource-type>``, and the second item is
``slots:`` followed by the number of slots from that resource allocated to
the given process. For example:
* ``CTEST_PROCESS_0_GPUS=id:0,slots:2``
* ``CTEST_PROCESS_1_GPUS=id:2,slots:2``
* ``CTEST_PROCESS_2_GPUS=id:1,slots:4;id:3,slots:1``
* ``CTEST_PROCESS_2_CRYPTO_CHIPS=id:card0,slots:2``
In this example, process 0 gets 2 slots from GPU ``0``, process 1 gets 2 slots
from GPU ``2``, and process 2 gets 4 slots from GPU ``1`` and 2 slots from
cryptography chip ``card0``.
``<num>`` is a number from zero to ``CTEST_PROCESS_COUNT`` minus one.
``<resource-type>`` is the name of a resource type, converted to uppercase.
``CTEST_PROCESS_<num>_<resource-type>`` is defined for the product of each
``<num>`` in the range listed above and each resource type listed in
``CTEST_PROCESS_<num>``.
Because some platforms have case-insensitive names for environment variables,
the names of resource types may not clash in a case-insensitive environment.
Because of this, for the sake of simplicity, all resource types must be
listed in all lowercase in the
:ref:`hardware specification file <ctest-hardware-specification-file>` and in
the :prop_test:`PROCESSES` property, and they are converted to all uppercase
in the ``CTEST_PROCESS_<num>_<resource-type>`` environment variable.
See Also
========

54
Help/prop_test/PROCESSES.rst Normal file
View File

@ -0,0 +1,54 @@
PROCESSES
----------
Set to specify the number of processes spawned by a test, and the resources
that they require. See :ref:`hardware allocation <ctest-hardware-allocation>`
for more information on how this property integrates into the CTest hardware
allocation feature.
The ``PROCESSES`` property is a :ref:`semicolon-separated list <CMake Language
Lists>` of process descriptions. Each process description consists of an
optional number of processes for the description followed by a series of
resource requirements for those processes. These requirements (and the number
of processes) are separated by commas. The resource requirements consist of the
name of a resource type, followed by a colon, followed by an unsigned integer
specifying the number of slots required on one resource of the given type.
Please note that these processes are not spawned by CTest. The ``PROCESSES``
property merely tells CTest what processes the test expects to launch. It is up
to the test itself to do this process spawning, and read the :ref:`environment
variables <ctest-hardware-environment-variables>` to determine which resources
each process has been allocated.
Consider the following example:
.. code-block:: cmake
add_test(NAME MyTest COMMAND MyExe)
set_property(TEST MyTest PROPERTY PROCESSES
"2,gpus:2"
"gpus:4,crypto_chips:2")
In this example, there are two process descriptions (implicitly separated by a
semicolon.) The content of the first description is ``2,gpus:2``. This
description spawns 2 processes, each of which requires 2 slots from a single
GPU. The content of the second description is ``gpus:4,crypto_chips:2``. This
description does not specify a process count, so a default of 1 is assumed.
This single process requires 4 slots from a single GPU and 2 slots from a
single cryptography chip. In total, 3 processes are spawned from this test,
each with their own unique requirements.
When CTest sets the :ref:`environment variables
<ctest-hardware-environment-variables>` for a test, it assigns a process number
based on the process description, starting at 0 on the left and the number of
processes minus 1 on the right. For example, in the example above, the two
processes in the first description would have IDs of 0 and 1, and the single
process in the second description would have an ID of 2.
Both the ``PROCESSES`` and :prop_test:`RESOURCE_LOCK` properties serve similar
purposes, but they are distinct and orthogonal. Resources specified by
``PROCESSES`` do not affect :prop_test:`RESOURCE_LOCK`, and vice versa. Whereas
:prop_test:`RESOURCE_LOCK` is a simpler property that is used for locking one
global resource, ``PROCESSES`` is a more advanced property that allows multiple
tests to simultaneously use multiple resources of the same type, specifying
their requirements in a fine-grained manner.

8
Help/prop_test/RESOURCE_LOCK.rst
View File

@ -8,3 +8,11 @@ not to run concurrently.
See also :prop_test:`FIXTURES_REQUIRED` if the resource requires any setup or
cleanup steps.
Both the :prop_test:`PROCESSES` and ``RESOURCE_LOCK`` properties serve similar
purposes, but they are distinct and orthogonal. Resources specified by
:prop_test:`PROCESSES` do not affect ``RESOURCE_LOCK``, and vice versa. Whereas
``RESOURCE_LOCK`` is a simpler property that is used for locking one global
resource, :prop_test:`PROCESSES` is a more advanced property that allows
multiple tests to simultaneously use multiple resources of the same type,
specifying their requirements in a fine-grained manner.

6
Help/release/dev/ctest-hardware-allocation.rst Normal file
View File

@ -0,0 +1,6 @@
ctest-hardware-allocation
-------------------------
* :manual:`ctest(1)` now has the ability to serialize tests based on hardware
requirements for each test. See :ref:`ctest-hardware-allocation` for
details.

8
Source/CMakeLists.txt
View File

@ -899,6 +899,7 @@ include_directories(
#
set(CTEST_SRCS cmCTest.cxx
CTest/cmProcess.cxx
CTest/cmCTestBinPacker.cxx
CTest/cmCTestBuildAndTestHandler.cxx
CTest/cmCTestBuildCommand.cxx
CTest/cmCTestBuildHandler.cxx
@ -918,10 +919,13 @@ set(CTEST_SRCS cmCTest.cxx
CTest/cmCTestEmptyBinaryDirectoryCommand.cxx
CTest/cmCTestGenericHandler.cxx
CTest/cmCTestHandlerCommand.cxx
CTest/cmCTestHardwareAllocator.cxx
CTest/cmCTestHardwareSpec.cxx
CTest/cmCTestLaunch.cxx
CTest/cmCTestMemCheckCommand.cxx
CTest/cmCTestMemCheckHandler.cxx
CTest/cmCTestMultiProcessHandler.cxx
CTest/cmCTestProcessesLexerHelper.cxx
CTest/cmCTestReadCustomFilesCommand.cxx
CTest/cmCTestRunScriptCommand.cxx
CTest/cmCTestRunTest.cxx
@ -953,6 +957,10 @@ set(CTEST_SRCS cmCTest.cxx
CTest/cmCTestHG.h
CTest/cmCTestP4.cxx
CTest/cmCTestP4.h
LexerParser/cmCTestProcessesLexer.cxx
LexerParser/cmCTestProcessesLexer.h
LexerParser/cmCTestProcessesLexer.in.l
)
# Build CTestLib

201
Source/CTest/cmCTestBinPacker.cxx Normal file
View File

@ -0,0 +1,201 @@
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
file Copyright.txt or https://cmake.org/licensing for details. */
#include "cmCTestBinPacker.h"
#include <algorithm>
#include <utility>
bool cmCTestBinPackerAllocation::operator==(
const cmCTestBinPackerAllocation& other) const
{
return this->ProcessIndex == other.ProcessIndex &&
this->SlotsNeeded == other.SlotsNeeded && this->Id == other.Id;
}
bool cmCTestBinPackerAllocation::operator!=(
const cmCTestBinPackerAllocation& other) const
{
return !(*this == other);
}
namespace {
/*
* The following algorithm is used to do two things:
*
* 1) Determine if a test's hardware requirements can fit within the hardware
* present on the system, and
* 2) Do the actual allocation
*
* This algorithm performs a recursive search, looking for a bin pack that will
* fit the specified requirements. It has a template to specify different
* optimization strategies. If it ever runs out of room, it backtracks as far
* down the stack as it needs to and tries a different combination until no
* more combinations can be tried.
*/
template <typename AllocationStrategy>
static bool AllocateCTestHardware(
const std::map<std::string, cmCTestHardwareAllocator::Resource>& hardware,
const std::vector<std::string>& hardwareSorted, std::size_t currentIndex,
std::vector<cmCTestBinPackerAllocation*>& allocations)
{
// Iterate through all large enough resources until we find a solution
std::size_t hardwareIndex = 0;
while (hardwareIndex < hardwareSorted.size()) {
auto const& resource = hardware.at(hardwareSorted[hardwareIndex]);
if (resource.Free() >=
static_cast<unsigned int>(allocations[currentIndex]->SlotsNeeded)) {
// Preemptively allocate the resource
allocations[currentIndex]->Id = hardwareSorted[hardwareIndex];
if (currentIndex + 1 >= allocations.size()) {
// We have a solution
return true;
}
// Move the resource up the list until it is sorted again
auto hardware2 = hardware;
auto hardwareSorted2 = hardwareSorted;
hardware2[hardwareSorted2[hardwareIndex]].Locked +=
allocations[currentIndex]->SlotsNeeded;
AllocationStrategy::IncrementalSort(hardware2, hardwareSorted2,
hardwareIndex);
// Recurse one level deeper
if (AllocateCTestHardware<AllocationStrategy>(
hardware2, hardwareSorted2, currentIndex + 1, allocations)) {
return true;
}
}
// No solution found here, deallocate the resource and try the next one
allocations[currentIndex]->Id.clear();
auto freeSlots = hardware.at(hardwareSorted.at(hardwareIndex)).Free();
do {
++hardwareIndex;
} while (hardwareIndex < hardwareSorted.size() &&
hardware.at(hardwareSorted.at(hardwareIndex)).Free() ==
freeSlots);
}
// No solution was found
return false;
}
template <typename AllocationStrategy>
static bool AllocateCTestHardware(
const std::map<std::string, cmCTestHardwareAllocator::Resource>& hardware,
std::vector<cmCTestBinPackerAllocation>& allocations)
{
// Sort the resource requirements in descending order by slots needed
std::vector<cmCTestBinPackerAllocation*> allocationsPtr;
allocationsPtr.reserve(allocations.size());
for (auto& allocation : allocations) {
allocationsPtr.push_back(&allocation);
}
std::stable_sort(
allocationsPtr.rbegin(), allocationsPtr.rend(),
[](cmCTestBinPackerAllocation* a1, cmCTestBinPackerAllocation* a2) {
return a1->SlotsNeeded < a2->SlotsNeeded;
});
// Sort the resources according to sort strategy
std::vector<std::string> hardwareSorted;
hardwareSorted.reserve(hardware.size());
for (auto const& hw : hardware) {
hardwareSorted.push_back(hw.first);
}
AllocationStrategy::InitialSort(hardware, hardwareSorted);
// Do the actual allocation
return AllocateCTestHardware<AllocationStrategy>(
hardware, hardwareSorted, std::size_t(0), allocationsPtr);
}
class RoundRobinAllocationStrategy
{
public:
static void InitialSort(
const std::map<std::string, cmCTestHardwareAllocator::Resource>& hardware,
std::vector<std::string>& hardwareSorted);
static void IncrementalSort(
const std::map<std::string, cmCTestHardwareAllocator::Resource>& hardware,
std::vector<std::string>& hardwareSorted, std::size_t lastAllocatedIndex);
};
void RoundRobinAllocationStrategy::InitialSort(
const std::map<std::string, cmCTestHardwareAllocator::Resource>& hardware,
std::vector<std::string>& hardwareSorted)
{
std::stable_sort(
hardwareSorted.rbegin(), hardwareSorted.rend(),
[&hardware](const std::string& id1, const std::string& id2) {
return hardware.at(id1).Free() < hardware.at(id2).Free();
});
}
void RoundRobinAllocationStrategy::IncrementalSort(
const std::map<std::string, cmCTestHardwareAllocator::Resource>& hardware,
std::vector<std::string>& hardwareSorted, std::size_t lastAllocatedIndex)
{
auto tmp = hardwareSorted[lastAllocatedIndex];
std::size_t i = lastAllocatedIndex;
while (i < hardwareSorted.size() - 1 &&
hardware.at(hardwareSorted[i + 1]).Free() > hardware.at(tmp).Free()) {
hardwareSorted[i] = hardwareSorted[i + 1];
++i;
}
hardwareSorted[i] = tmp;
}
class BlockAllocationStrategy
{
public:
static void InitialSort(
const std::map<std::string, cmCTestHardwareAllocator::Resource>& hardware,
std::vector<std::string>& hardwareSorted);
static void IncrementalSort(
const std::map<std::string, cmCTestHardwareAllocator::Resource>& hardware,
std::vector<std::string>& hardwareSorted, std::size_t lastAllocatedIndex);
};
void BlockAllocationStrategy::InitialSort(
const std::map<std::string, cmCTestHardwareAllocator::Resource>& hardware,
std::vector<std::string>& hardwareSorted)
{
std::stable_sort(
hardwareSorted.rbegin(), hardwareSorted.rend(),
[&hardware](const std::string& id1, const std::string& id2) {
return hardware.at(id1).Free() < hardware.at(id2).Free();
});
}
void BlockAllocationStrategy::IncrementalSort(
const std::map<std::string, cmCTestHardwareAllocator::Resource>&,
std::vector<std::string>& hardwareSorted, std::size_t lastAllocatedIndex)
{
auto tmp = hardwareSorted[lastAllocatedIndex];
std::size_t i = lastAllocatedIndex;
while (i > 0) {
hardwareSorted[i] = hardwareSorted[i - 1];
--i;
}
hardwareSorted[i] = tmp;
}
}
bool cmAllocateCTestHardwareRoundRobin(
const std::map<std::string, cmCTestHardwareAllocator::Resource>& hardware,
std::vector<cmCTestBinPackerAllocation>& allocations)
{
return AllocateCTestHardware<RoundRobinAllocationStrategy>(hardware,
allocations);
}
bool cmAllocateCTestHardwareBlock(
const std::map<std::string, cmCTestHardwareAllocator::Resource>& hardware,
std::vector<cmCTestBinPackerAllocation>& allocations)
{
return AllocateCTestHardware<BlockAllocationStrategy>(hardware, allocations);
}

31
Source/CTest/cmCTestBinPacker.h Normal file
View File

@ -0,0 +1,31 @@
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
file Copyright.txt or https://cmake.org/licensing for details. */
#ifndef cmCTestBinPacker_h
#define cmCTestBinPacker_h
#include <cstddef>
#include <map>
#include <string>
#include <vector>
#include "cmCTestHardwareAllocator.h"
struct cmCTestBinPackerAllocation
{
std::size_t ProcessIndex;
int SlotsNeeded;
std::string Id;
bool operator==(const cmCTestBinPackerAllocation& other) const;
bool operator!=(const cmCTestBinPackerAllocation& other) const;
};
bool cmAllocateCTestHardwareRoundRobin(
const std::map<std::string, cmCTestHardwareAllocator::Resource>& hardware,
std::vector<cmCTestBinPackerAllocation>& allocations);
bool cmAllocateCTestHardwareBlock(
const std::map<std::string, cmCTestHardwareAllocator::Resource>& hardware,
std::vector<cmCTestBinPackerAllocation>& allocations);
#endif

86
Source/CTest/cmCTestHardwareAllocator.cxx Normal file
View File

@ -0,0 +1,86 @@
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
file Copyright.txt or https://cmake.org/licensing for details. */
#include "cmCTestHardwareAllocator.h"
#include <utility>
#include <vector>
#include "cmCTestHardwareSpec.h"
void cmCTestHardwareAllocator::InitializeFromHardwareSpec(
const cmCTestHardwareSpec& spec)
{
this->Resources.clear();
for (auto const& it : spec.LocalSocket.Resources) {
auto& res = this->Resources[it.first];
for (auto const& specRes : it.second) {
res[specRes.Id].Total = specRes.Capacity;
res[specRes.Id].Locked = 0;
}
}
}
const std::map<std::string,
std::map<std::string, cmCTestHardwareAllocator::Resource>>&
cmCTestHardwareAllocator::GetResources() const
{
return this->Resources;
}
bool cmCTestHardwareAllocator::AllocateResource(const std::string& name,
const std::string& id,
unsigned int slots)
{
auto it = this->Resources.find(name);
if (it == this->Resources.end()) {
return false;
}
auto resIt = it->second.find(id);
if (resIt == it->second.end()) {
return false;
}
if (resIt->second.Total < resIt->second.Locked + slots) {
return false;
}
resIt->second.Locked += slots;
return true;
}
bool cmCTestHardwareAllocator::DeallocateResource(const std::string& name,
const std::string& id,
unsigned int slots)
{
auto it = this->Resources.find(name);
if (it == this->Resources.end()) {
return false;
}
auto resIt = it->second.find(id);
if (resIt == it->second.end()) {
return false;
}
if (resIt->second.Locked < slots) {
return false;
}
resIt->second.Locked -= slots;
return true;
}
bool cmCTestHardwareAllocator::Resource::operator==(
const Resource& other) const
{
return this->Total == other.Total && this->Locked == other.Locked;
}
bool cmCTestHardwareAllocator::Resource::operator!=(
const Resource& other) const
{
return !(*this == other);
}

39
Source/CTest/cmCTestHardwareAllocator.h Normal file
View File

@ -0,0 +1,39 @@
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
file Copyright.txt or https://cmake.org/licensing for details. */
#ifndef cmCTestHardwareAllocator_h
#define cmCTestHardwareAllocator_h
#include <map>
#include <string>
class cmCTestHardwareSpec;
class cmCTestHardwareAllocator
{
public:
struct Resource
{
unsigned int Total;
unsigned int Locked;
unsigned int Free() const { return this->Total - this->Locked; }
bool operator==(const Resource& other) const;
bool operator!=(const Resource& other) const;
};
void InitializeFromHardwareSpec(const cmCTestHardwareSpec& spec);
const std::map<std::string, std::map<std::string, Resource>>& GetResources()
const;
bool AllocateResource(const std::string& name, const std::string& id,
unsigned int slots);
bool DeallocateResource(const std::string& name, const std::string& id,
unsigned int slots);
private:
std::map<std::string, std::map<std::string, Resource>> Resources;
};
#endif

133
Source/CTest/cmCTestHardwareSpec.cxx Normal file
View File

@ -0,0 +1,133 @@
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
file Copyright.txt or https://cmake.org/licensing for details. */
#include "cmCTestHardwareSpec.h"
#include <map>
#include <string>
#include <utility>
#include <vector>
#include "cmsys/FStream.hxx"
#include "cmsys/RegularExpression.hxx"
#include "cm_jsoncpp_reader.h"
#include "cm_jsoncpp_value.h"
static const cmsys::RegularExpression IdentifierRegex{ "^[a-z_][a-z0-9_]*$" };
static const cmsys::RegularExpression IdRegex{ "^[a-z0-9_]+$" };
bool cmCTestHardwareSpec::ReadFromJSONFile(const std::string& filename)
{
cmsys::ifstream fin(filename.c_str());
if (!fin) {
return false;
}
Json::Value root;
Json::CharReaderBuilder builder;
if (!Json::parseFromStream(builder, fin, &root, nullptr)) {
return false;
}
if (!root.isObject()) {
return false;
}
auto const& local = root["local"];
if (!local.isArray()) {
return false;
}
if (local.size() > 1) {
return false;
}
if (local.empty()) {
this->LocalSocket.Resources.clear();
return true;
}
auto const& localSocket = local[0];
if (!localSocket.isObject()) {
return false;
}
std::map<std::string, std::vector<cmCTestHardwareSpec::Resource>> resources;
cmsys::RegularExpressionMatch match;
for (auto const& key : localSocket.getMemberNames()) {
if (IdentifierRegex.find(key.c_str(), match)) {
auto const& value = localSocket[key];
auto& r = resources[key];
if (value.isArray()) {
for (auto const& item : value) {
if (item.isObject()) {
cmCTestHardwareSpec::Resource resource;
if (!item.isMember("id")) {
return false;
}
auto const& id = item["id"];
if (!id.isString()) {
return false;
}
resource.Id = id.asString();
if (!IdRegex.find(resource.Id.c_str(), match)) {
return false;
}
if (item.isMember("slots")) {
auto const& capacity = item["slots"];
if (!capacity.isConvertibleTo(Json::uintValue)) {
return false;
}
resource.Capacity = capacity.asUInt();
} else {
resource.Capacity = 1;
}
r.push_back(resource);
} else {
return false;
}
}
} else {
return false;
}
}
}
this->LocalSocket.Resources = std::move(resources);
return true;
}
bool cmCTestHardwareSpec::operator==(const cmCTestHardwareSpec& other) const
{
return this->LocalSocket == other.LocalSocket;
}
bool cmCTestHardwareSpec::operator!=(const cmCTestHardwareSpec& other) const
{
return !(*this == other);
}
bool cmCTestHardwareSpec::Socket::operator==(
const cmCTestHardwareSpec::Socket& other) const
{
return this->Resources == other.Resources;
}
bool cmCTestHardwareSpec::Socket::operator!=(
const cmCTestHardwareSpec::Socket& other) const
{
return !(*this == other);
}
bool cmCTestHardwareSpec::Resource::operator==(
const cmCTestHardwareSpec::Resource& other) const
{
return this->Id == other.Id && this->Capacity == other.Capacity;
}
bool cmCTestHardwareSpec::Resource::operator!=(
const cmCTestHardwareSpec::Resource& other) const
{
return !(*this == other);
}

40
Source/CTest/cmCTestHardwareSpec.h Normal file
View File

@ -0,0 +1,40 @@
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
file Copyright.txt or https://cmake.org/licensing for details. */
#ifndef cmCTestHardwareSpec_h
#define cmCTestHardwareSpec_h
#include <map>
#include <string>
#include <vector>
class cmCTestHardwareSpec
{
public:
class Resource
{
public:
std::string Id;
unsigned int Capacity;
bool operator==(const Resource& other) const;
bool operator!=(const Resource& other) const;
};
class Socket
{
public:
std::map<std::string, std::vector<Resource>> Resources;
bool operator==(const Socket& other) const;
bool operator!=(const Socket& other) const;
};
Socket LocalSocket;
bool ReadFromJSONFile(const std::string& filename);
bool operator==(const cmCTestHardwareSpec& other) const;
bool operator!=(const cmCTestHardwareSpec& other) const;
};
#endif

166
Source/CTest/cmCTestMultiProcessHandler.cxx
View File

@ -3,8 +3,10 @@
#include "cmCTestMultiProcessHandler.h"
#include <algorithm>
#include <cassert>
#include <chrono>
#include <cmath>
#include <cstddef>
#include <cstdlib>
#include <cstring>
#include <iomanip>
@ -27,6 +29,7 @@
#include "cmAffinity.h"
#include "cmAlgorithms.h"
#include "cmCTest.h"
#include "cmCTestBinPacker.h"
#include "cmCTestRunTest.h"
#include "cmCTestTestHandler.h"
#include "cmDuration.h"
@ -133,6 +136,12 @@ void cmCTestMultiProcessHandler::RunTests()
uv_run(&this->Loop, UV_RUN_DEFAULT);
uv_loop_close(&this->Loop);
if (!this->StopTimePassed) {
assert(this->Completed == this->Total);
assert(this->Tests.empty());
}
assert(this->AllHardwareAvailable());
this->MarkFinished();
this->UpdateCostData();
}
@ -168,6 +177,10 @@ bool cmCTestMultiProcessHandler::StartTestProcess(int test)
}
testRun->SetIndex(test);
testRun->SetTestProperties(this->Properties[test]);
if (this->TestHandler->UseHardwareSpec) {
testRun->SetUseAllocatedHardware(true);
testRun->SetAllocatedHardware(this->AllocatedHardware[test]);
}
// Find any failed dependencies for this test. We assume the more common
// scenario has no failed tests, so make it the outer loop.
@ -179,7 +192,13 @@ bool cmCTestMultiProcessHandler::StartTestProcess(int test)
// Always lock the resources we'll be using, even if we fail to set the
// working directory because FinishTestProcess() will try to unlock them
this->LockResources(test);
this->AllocateResources(test);
if (!this->TestsHaveSufficientHardware[test]) {
testRun->StartFailure("Insufficient hardware");
this->FinishTestProcess(testRun, false);
return false;
}
cmWorkingDirectory workdir(this->Properties[test]->Directory);
if (workdir.Failed()) {
@ -199,6 +218,110 @@ bool cmCTestMultiProcessHandler::StartTestProcess(int test)
return false;
}
bool cmCTestMultiProcessHandler::AllocateHardware(int index)
{
if (!this->TestHandler->UseHardwareSpec) {
return true;
}
std::map<std::string, std::vector<cmCTestBinPackerAllocation>> allocations;
if (!this->TryAllocateHardware(index, allocations)) {
return false;
}
auto& allocatedHardware = this->AllocatedHardware[index];
allocatedHardware.resize(this->Properties[index]->Processes.size());
for (auto const& it : allocations) {
for (auto const& alloc : it.second) {
bool result = this->HardwareAllocator.AllocateResource(
it.first, alloc.Id, alloc.SlotsNeeded);
(void)result;
assert(result);
allocatedHardware[alloc.ProcessIndex][it.first].push_back(
{ alloc.Id, static_cast<unsigned int>(alloc.SlotsNeeded) });
}
}
return true;
}
bool cmCTestMultiProcessHandler::TryAllocateHardware(
int index,
std::map<std::string, std::vector<cmCTestBinPackerAllocation>>& allocations)
{
allocations.clear();
std::size_t processIndex = 0;
for (auto const& process : this->Properties[index]->Processes) {
for (auto const& requirement : process) {
for (int i = 0; i < requirement.UnitsNeeded; ++i) {
allocations[requirement.ResourceType].push_back(
{ processIndex, requirement.SlotsNeeded, "" });
}
}
++processIndex;
}
auto const& availableHardware = this->HardwareAllocator.GetResources();
for (auto& it : allocations) {
if (!availableHardware.count(it.first)) {
return false;
}
if (!cmAllocateCTestHardwareRoundRobin(availableHardware.at(it.first),
it.second)) {
return false;
}
}
return true;
}
void cmCTestMultiProcessHandler::DeallocateHardware(int index)
{
if (!this->TestHandler->UseHardwareSpec) {
return;
}
{
auto& allocatedHardware = this->AllocatedHardware[index];
for (auto const& processAlloc : allocatedHardware) {
for (auto const& it : processAlloc) {
auto resourceType = it.first;
for (auto const& it2 : it.second) {
bool success = this->HardwareAllocator.DeallocateResource(
resourceType, it2.Id, it2.Slots);
(void)success;
assert(success);
}
}
}
}
this->AllocatedHardware.erase(index);
}
bool cmCTestMultiProcessHandler::AllHardwareAvailable()
{
for (auto const& it : this->HardwareAllocator.GetResources()) {
for (auto const& it2 : it.second) {
if (it2.second.Locked != 0) {
return false;
}
}
}
return true;
}
void cmCTestMultiProcessHandler::CheckHardwareAvailable()
{
for (auto test : this->SortedTests) {
std::map<std::string, std::vector<cmCTestBinPackerAllocation>> allocations;
this->TestsHaveSufficientHardware[test] =
!this->TestHandler->UseHardwareSpec ||
this->TryAllocateHardware(test, allocations);
}
}
bool cmCTestMultiProcessHandler::CheckStopTimePassed()
{
if (!this->StopTimePassed) {
@ -223,7 +346,7 @@ void cmCTestMultiProcessHandler::SetStopTimePassed()
}
}
void cmCTestMultiProcessHandler::LockResources(int index)
void cmCTestMultiProcessHandler::AllocateResources(int index)
{
this->LockedResources.insert(
this->Properties[index]->LockedResources.begin(),
@ -234,7 +357,7 @@ void cmCTestMultiProcessHandler::LockResources(int index)
}
}
void cmCTestMultiProcessHandler::UnlockResources(int index)
void cmCTestMultiProcessHandler::DeallocateResources(int index)
{
for (std::string const& i : this->Properties[index]->LockedResources) {
this->LockedResources.erase(i);
@ -281,12 +404,20 @@ bool cmCTestMultiProcessHandler::StartTest(int test)
}
}
// Allocate hardware
if (this->TestsHaveSufficientHardware[test] &&
!this->AllocateHardware(test)) {
this->DeallocateHardware(test);
return false;
}
// if there are no depends left then run this test
if (this->Tests[test].empty()) {
return this->StartTestProcess(test);
}
// This test was not able to start because it is waiting
// on depends to run
this->DeallocateHardware(test);
return false;
}
@ -471,7 +602,8 @@ void cmCTestMultiProcessHandler::FinishTestProcess(cmCTestRunTest* runner,
this->TestFinishMap[test] = true;
this->TestRunningMap[test] = false;
this->WriteCheckpoint(test);
this->UnlockResources(test);
this->DeallocateHardware(test);
this->DeallocateResources(test);
this->RunningCount -= GetProcessorsUsed(test);
for (auto p : properties->Affinity) {
@ -780,6 +912,28 @@ static Json::Value DumpTimeoutAfterMatch(
return timeoutAfterMatch;
}
static Json::Value DumpProcessesToJsonArray(
const std::vector<
std::vector<cmCTestTestHandler::cmCTestTestResourceRequirement>>&
processes)
{
Json::Value jsonProcesses = Json::arrayValue;
for (auto const& it : processes) {
Json::Value jsonProcess = Json::objectValue;
Json::Value requirements = Json::arrayValue;
for (auto const& it2 : it) {
Json::Value res = Json::objectValue;
res[".type"] = it2.ResourceType;
// res[".units"] = it2.UnitsNeeded; // Intentionally commented out
res["slots"] = it2.SlotsNeeded;
requirements.append(res);
}
jsonProcess["requirements"] = requirements;
jsonProcesses.append(jsonProcess);
}
return jsonProcesses;
}
static Json::Value DumpCTestProperty(std::string const& name,
Json::Value value)
{
@ -851,6 +1005,10 @@ static Json::Value DumpCTestProperties(
"PASS_REGULAR_EXPRESSION",
DumpRegExToJsonArray(testProperties.RequiredRegularExpressions)));
}
if (!testProperties.Processes.empty()) {
properties.append(DumpCTestProperty(
"PROCESSES", DumpProcessesToJsonArray(testProperties.Processes)));
}
if (testProperties.WantAffinity) {
properties.append(
DumpCTestProperty("PROCESSOR_AFFINITY", testProperties.WantAffinity));

33
Source/CTest/cmCTestMultiProcessHandler.h
View File

@ -14,10 +14,13 @@
#include "cm_uv.h"
#include "cmCTestHardwareAllocator.h"
#include "cmCTestTestHandler.h"
#include "cmUVHandlePtr.h"
class cmCTest;
struct cmCTestBinPackerAllocation;
class cmCTestHardwareSpec;
class cmCTestRunTest;
/** \class cmCTestMultiProcessHandler
@ -44,6 +47,11 @@ public:
: public std::map<int, cmCTestTestHandler::cmCTestTestProperties*>
{
};
struct HardwareAllocation
{
std::string Id;
unsigned int Slots;
};
cmCTestMultiProcessHandler();
virtual ~cmCTestMultiProcessHandler();
@ -79,6 +87,13 @@ public:
void SetQuiet(bool b) { this->Quiet = b; }
void InitHardwareAllocator(const cmCTestHardwareSpec& spec)
{
this->HardwareAllocator.InitializeFromHardwareSpec(spec);
}
void CheckHardwareAvailable();
protected:
// Start the next test or tests as many as are allowed by
// ParallelLevel
@ -119,8 +134,17 @@ protected:
bool CheckStopTimePassed();
void SetStopTimePassed();
void LockResources(int index);
void UnlockResources(int index);
void AllocateResources(int index);
void DeallocateResources(int index);
bool AllocateHardware(int index);
bool TryAllocateHardware(
int index,
std::map<std::string, std::vector<cmCTestBinPackerAllocation>>&
allocations);
void DeallocateHardware(int index);
bool AllHardwareAvailable();
// map from test number to set of depend tests
TestMap Tests;
TestList SortedTests;
@ -141,6 +165,11 @@ protected:
std::vector<std::string>* Failed;
std::vector<std::string> LastTestsFailed;
std::set<std::string> LockedResources;
std::map<int,
std::vector<std::map<std::string, std::vector<HardwareAllocation>>>>
AllocatedHardware;
std::map<int, bool> TestsHaveSufficientHardware;
cmCTestHardwareAllocator HardwareAllocator;
std::vector<cmCTestTestHandler::cmCTestTestResult>* TestResults;
size_t ParallelLevel; // max number of process that can be run at once
unsigned long TestLoad;

55
Source/CTest/cmCTestProcessesLexerHelper.cxx Normal file
View File

@ -0,0 +1,55 @@
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
file Copyright.txt or https://cmake.org/licensing for details. */
#include "cmCTestProcessesLexerHelper.h"
#include "cmCTestProcessesLexer.h"
#include "cmCTestTestHandler.h"
cmCTestProcessesLexerHelper::cmCTestProcessesLexerHelper(
std::vector<std::vector<cmCTestTestHandler::cmCTestTestResourceRequirement>>&
output)
: Output(output)
{
}
bool cmCTestProcessesLexerHelper::ParseString(const std::string& value)
{
yyscan_t lexer;
cmCTestProcesses_yylex_init_extra(this, &lexer);
auto state = cmCTestProcesses_yy_scan_string(value.c_str(), lexer);
int retval = cmCTestProcesses_yylex(lexer);
cmCTestProcesses_yy_delete_buffer(state, lexer);
cmCTestProcesses_yylex_destroy(lexer);
return retval == 0;
}
void cmCTestProcessesLexerHelper::SetProcessCount(unsigned int count)
{
this->ProcessCount = count;
}
void cmCTestProcessesLexerHelper::SetResourceType(const std::string& type)
{
this->ResourceType = type;
}
void cmCTestProcessesLexerHelper::SetNeededSlots(int count)
{
this->NeededSlots = count;
}
void cmCTestProcessesLexerHelper::WriteRequirement()
{
this->Process.push_back({ this->ResourceType, this->NeededSlots, 1 });
}
void cmCTestProcessesLexerHelper::WriteProcess()
{
for (unsigned int i = 0; i < this->ProcessCount; ++i) {
this->Output.push_back(this->Process);
}
this->Process.clear();
this->ProcessCount = 1;
}

44
Source/CTest/cmCTestProcessesLexerHelper.h Normal file
View File

@ -0,0 +1,44 @@
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
file Copyright.txt or https://cmake.org/licensing for details. */
#ifndef cmCTestProcessesLexerHelper_h
#define cmCTestProcessesLexerHelper_h
#include <string>
#include <vector>
#include "cmCTestTestHandler.h"
class cmCTestProcessesLexerHelper
{
public:
struct ParserType
{
};
cmCTestProcessesLexerHelper(
std::vector<
std::vector<cmCTestTestHandler::cmCTestTestResourceRequirement>>&
output);
~cmCTestProcessesLexerHelper() = default;
bool ParseString(const std::string& value);
void SetProcessCount(unsigned int count);
void SetResourceType(const std::string& type);
void SetNeededSlots(int count);
void WriteRequirement();
void WriteProcess();
private:
std::vector<std::vector<cmCTestTestHandler::cmCTestTestResourceRequirement>>&
Output;
unsigned int ProcessCount = 1;
std::string ResourceType;
int NeededSlots;
std::vector<cmCTestTestHandler::cmCTestTestResourceRequirement> Process;
};
#define YY_EXTRA_TYPE cmCTestProcessesLexerHelper*
#endif

43
Source/CTest/cmCTestRunTest.cxx
View File

@ -3,6 +3,7 @@
#include "cmCTestRunTest.h"
#include <chrono>
#include <cstddef>
#include <cstdint>
#include <cstdio>
#include <cstring>
@ -689,10 +690,52 @@ bool cmCTestRunTest::ForkProcess(cmDuration testTimeOut, bool explicitTimeout,
cmSystemTools::AppendEnv(*environment);
}
if (this->UseAllocatedHardware) {
this->SetupHardwareEnvironment();
} else {
cmSystemTools::UnsetEnv("CTEST_PROCESS_COUNT");
}
return this->TestProcess->StartProcess(this->MultiTestHandler.Loop,
affinity);
}
void cmCTestRunTest::SetupHardwareEnvironment()
{
std::string processCount = "CTEST_PROCESS_COUNT=";
processCount += std::to_string(this->AllocatedHardware.size());
cmSystemTools::PutEnv(processCount);
std::size_t i = 0;
for (auto const& process : this->AllocatedHardware) {
std::string prefix = "CTEST_PROCESS_";
prefix += std::to_string(i);
std::string resourceList = prefix + '=';
prefix += '_';
bool firstType = true;
for (auto const& it : process) {
if (!firstType) {
resourceList += ',';
}
firstType = false;
auto resourceType = it.first;
resourceList += resourceType;
std::string var = prefix + cmSystemTools::UpperCase(resourceType) + '=';
bool firstName = true;
for (auto const& it2 : it.second) {
if (!firstName) {
var += ';';
}
firstName = false;
var += "id:" + it2.Id + ",slots:" + std::to_string(it2.Slots);
}
cmSystemTools::PutEnv(var);
}
cmSystemTools::PutEnv(resourceList);
++i;
}
}
void cmCTestRunTest::WriteLogOutputTop(size_t completed, size_t total)
{
std::ostringstream outputStream;

19
Source/CTest/cmCTestRunTest.h
View File

@ -5,6 +5,7 @@
#include "cmConfigure.h" // IWYU pragma: keep
#include <map>
#include <memory>
#include <set>
#include <string>
@ -12,12 +13,12 @@
#include <stddef.h>
#include "cmCTestMultiProcessHandler.h"
#include "cmCTestTestHandler.h"
#include "cmDuration.h"
#include "cmProcess.h"
class cmCTest;
class cmCTestMultiProcessHandler;
/** \class cmRunTest
* \brief represents a single test to be run
@ -83,6 +84,16 @@ public:
bool TimedOutForStopTime() const { return this->TimeoutIsForStopTime; }
void SetUseAllocatedHardware(bool use) { this->UseAllocatedHardware = use; }
void SetAllocatedHardware(
const std::vector<
std::map<std::string,
std::vector<cmCTestMultiProcessHandler::HardwareAllocation>>>&
hardware)
{
this->AllocatedHardware = hardware;
}
private:
bool NeedsToRerun();
void DartProcessing();
@ -94,6 +105,8 @@ private:
// Run post processing of the process output for MemCheck
void MemCheckPostProcess();
void SetupHardwareEnvironment();
// Returns "completed/total Test #Index: "
std::string GetTestPrefix(size_t completed, size_t total) const;
@ -112,6 +125,10 @@ private:
std::string StartTime;
std::string ActualCommand;
std::vector<std::string> Arguments;
bool UseAllocatedHardware = false;
std::vector<std::map<
std::string, std::vector<cmCTestMultiProcessHandler::HardwareAllocation>>>
AllocatedHardware;
bool RunUntilFail;
int NumberOfRunsLeft;
bool RunAgain;

4
Source/CTest/cmCTestTestCommand.cxx
View File

@ -32,6 +32,7 @@ void cmCTestTestCommand::BindArguments()
this->Bind("SCHEDULE_RANDOM"_s, this->ScheduleRandom);
this->Bind("STOP_TIME"_s, this->StopTime);
this->Bind("TEST_LOAD"_s, this->TestLoad);
this->Bind("HARDWARE_SPEC_FILE"_s, this->HardwareSpecFile);
}
cmCTestGenericHandler* cmCTestTestCommand::InitializeHandler()
@ -87,6 +88,9 @@ cmCTestGenericHandler* cmCTestTestCommand::InitializeHandler()
if (!this->ScheduleRandom.empty()) {
handler->SetOption("ScheduleRandom", this->ScheduleRandom.c_str());
}
if (!this->HardwareSpecFile.empty()) {
handler->SetOption("HardwareSpecFile", this->HardwareSpecFile.c_str());
}
if (!this->StopTime.empty()) {
this->CTest->SetStopTime(this->StopTime);
}

1
Source/CTest/cmCTestTestCommand.h
View File

@ -58,6 +58,7 @@ protected:
std::string ScheduleRandom;
std::string StopTime;
std::string TestLoad;
std::string HardwareSpecFile;
};
#endif

43
Source/CTest/cmCTestTestHandler.cxx
View File

@ -29,6 +29,7 @@
#include "cmAlgorithms.h"
#include "cmCTest.h"
#include "cmCTestMultiProcessHandler.h"
#include "cmCTestProcessesLexerHelper.h"
#include "cmDuration.h"
#include "cmExecutionStatus.h"
#include "cmGeneratedFileStream.h"
@ -288,6 +289,7 @@ cmCTestTestHandler::cmCTestTestHandler()
this->UseIncludeRegExpFlag = false;
this->UseExcludeRegExpFlag = false;
this->UseExcludeRegExpFirst = false;
this->UseHardwareSpec = false;
this->CustomMaximumPassedTestOutputSize = 1 * 1024;
this->CustomMaximumFailedTestOutputSize = 300 * 1024;
@ -508,6 +510,16 @@ bool cmCTestTestHandler::ProcessOptions()
}
this->SetRerunFailed(cmIsOn(this->GetOption("RerunFailed")));
val = this->GetOption("HardwareSpecFile");
if (val) {
this->UseHardwareSpec = true;
if (!this->HardwareSpec.ReadFromJSONFile(val)) {
cmCTestLog(this->CTest, ERROR_MESSAGE,
"Could not read hardware spec file: " << val << std::endl);
return false;
}
}
return true;
}
@ -1225,6 +1237,9 @@ void cmCTestTestHandler::ProcessDirectory(std::vector<std::string>& passed,
} else {
parallel->SetTestLoad(this->CTest->GetTestLoad());
}
if (this->UseHardwareSpec) {
parallel->InitHardwareAllocator(this->HardwareSpec);
}
*this->LogFile
<< "Start testing: " << this->CTest->CurrentTime() << std::endl
@ -1268,6 +1283,7 @@ void cmCTestTestHandler::ProcessDirectory(std::vector<std::string>& passed,
parallel->SetPassFailVectors(&passed, &failed);
this->TestResults.clear();
parallel->SetTestResults(&this->TestResults);
parallel->CheckHardwareAvailable();
if (this->CTest->ShouldPrintLabels()) {
parallel->PrintLabels();
@ -1610,6 +1626,14 @@ std::string cmCTestTestHandler::FindExecutable(
return fullPath;
}
bool cmCTestTestHandler::ParseProcessesProperty(
const std::string& val,
std::vector<std::vector<cmCTestTestResourceRequirement>>& processes)
{
cmCTestProcessesLexerHelper lexer(processes);
return lexer.ParseString(val);
}
void cmCTestTestHandler::GetListOfTests()
{
if (!this->IncludeLabelRegExp.empty()) {
@ -2179,6 +2203,11 @@ bool cmCTestTestHandler::SetTestsProperties(
if (key == "PROCESSOR_AFFINITY") {
rt.WantAffinity = cmIsOn(val);
}
if (key == "PROCESSES") {
if (!ParseProcessesProperty(val, rt.Processes)) {
return false;
}
}
if (key == "SKIP_RETURN_CODE") {
rt.SkipReturnCode = atoi(val.c_str());
if (rt.SkipReturnCode < 0 || rt.SkipReturnCode > 255) {
@ -2356,3 +2385,17 @@ bool cmCTestTestHandler::AddTest(const std::vector<std::string>& args)
this->TestList.push_back(test);
return true;
}
bool cmCTestTestHandler::cmCTestTestResourceRequirement::operator==(
const cmCTestTestResourceRequirement& other) const
{
return this->ResourceType == other.ResourceType &&
this->SlotsNeeded == other.SlotsNeeded &&
this->UnitsNeeded == other.UnitsNeeded;
}
bool cmCTestTestHandler::cmCTestTestResourceRequirement::operator!=(
const cmCTestTestResourceRequirement& other) const
{
return !(*this == other);
}

19
Source/CTest/cmCTestTestHandler.h
View File

@ -19,6 +19,7 @@
#include "cmsys/RegularExpression.hxx"
#include "cmCTestGenericHandler.h"
#include "cmCTestHardwareSpec.h"
#include "cmDuration.h"
#include "cmListFileCache.h"
@ -102,6 +103,16 @@ public:
void Initialize() override;
struct cmCTestTestResourceRequirement
{
std::string ResourceType;
int SlotsNeeded;
int UnitsNeeded;
bool operator==(const cmCTestTestResourceRequirement& other) const;
bool operator!=(const cmCTestTestResourceRequirement& other) const;
};
// NOTE: This struct is Saved/Restored
// in cmCTestTestHandler, if you add to this class
// then you must add the new members to that code or
@ -147,6 +158,7 @@ public:
std::set<std::string> FixturesCleanup;
std::set<std::string> FixturesRequired;
std::set<std::string> RequireSuccessDepends;
std::vector<std::vector<cmCTestTestResourceRequirement>> Processes;
// Private test generator properties used to track backtraces
cmListFileBacktrace Backtrace;
};
@ -190,6 +202,10 @@ public:
std::vector<std::string>& extraPaths,
std::vector<std::string>& failed);
static bool ParseProcessesProperty(
const std::string& val,
std::vector<std::vector<cmCTestTestResourceRequirement>>& processes);
using ListOfTests = std::vector<cmCTestTestProperties>;
protected:
@ -320,6 +336,9 @@ private:
cmsys::RegularExpression IncludeTestsRegularExpression;
cmsys::RegularExpression ExcludeTestsRegularExpression;
bool UseHardwareSpec;
cmCTestHardwareSpec HardwareSpec;
void GenerateRegressionImages(cmXMLWriter& xml, const std::string& dart);
cmsys::RegularExpression DartStuff1;
void CheckLabelFilter(cmCTestTestProperties& it);

2
Source/LexerParser/.gitattributes vendored
View File

@ -2,6 +2,8 @@
/cmCommandArgumentLexer.h generated
/cmCommandArgumentParser.cxx generated
/cmCommandArgumentParserTokens.h generated
/cmCTestProcessesLexer.cxx generated
/cmCTestProcessesLexer.h generated
/cmDependsJavaLexer.cxx generated
/cmDependsJavaLexer.h generated
/cmDependsJavaParser.cxx generated

2224
Source/LexerParser/cmCTestProcessesLexer.cxx Normal file
View File

File diff suppressed because it is too large Load Diff

692
Source/LexerParser/cmCTestProcessesLexer.h Normal file
View File

@ -0,0 +1,692 @@
#ifndef cmCTestProcesses_yyHEADER_H
#define cmCTestProcesses_yyHEADER_H 1
#define cmCTestProcesses_yyIN_HEADER 1
#define FLEXINT_H 1
#define YY_INT_ALIGNED short int
/* A lexical scanner generated by flex */
#define FLEX_SCANNER
#define YY_FLEX_MAJOR_VERSION 2
#define YY_FLEX_MINOR_VERSION 6
#define YY_FLEX_SUBMINOR_VERSION 4
#if YY_FLEX_SUBMINOR_VERSION > 0
#define FLEX_BETA
#endif
#ifdef yy_create_buffer
#define cmCTestProcesses_yy_create_buffer_ALREADY_DEFINED
#else
#define yy_create_buffer cmCTestProcesses_yy_create_buffer
#endif
#ifdef yy_delete_buffer
#define cmCTestProcesses_yy_delete_buffer_ALREADY_DEFINED
#else
#define yy_delete_buffer cmCTestProcesses_yy_delete_buffer
#endif
#ifdef yy_scan_buffer
#define cmCTestProcesses_yy_scan_buffer_ALREADY_DEFINED
#else
#define yy_scan_buffer cmCTestProcesses_yy_scan_buffer
#endif
#ifdef yy_scan_string
#define cmCTestProcesses_yy_scan_string_ALREADY_DEFINED
#else
#define yy_scan_string cmCTestProcesses_yy_scan_string
#endif
#ifdef yy_scan_bytes
#define cmCTestProcesses_yy_scan_bytes_ALREADY_DEFINED
#else
#define yy_scan_bytes cmCTestProcesses_yy_scan_bytes
#endif
#ifdef yy_init_buffer
#define cmCTestProcesses_yy_init_buffer_ALREADY_DEFINED
#else
#define yy_init_buffer cmCTestProcesses_yy_init_buffer
#endif
#ifdef yy_flush_buffer
#define cmCTestProcesses_yy_flush_buffer_ALREADY_DEFINED
#else
#define yy_flush_buffer cmCTestProcesses_yy_flush_buffer
#endif
#ifdef yy_load_buffer_state
#define cmCTestProcesses_yy_load_buffer_state_ALREADY_DEFINED
#else
#define yy_load_buffer_state cmCTestProcesses_yy_load_buffer_state
#endif
#ifdef yy_switch_to_buffer
#define cmCTestProcesses_yy_switch_to_buffer_ALREADY_DEFINED
#else
#define yy_switch_to_buffer cmCTestProcesses_yy_switch_to_buffer
#endif
#ifdef yypush_buffer_state
#define cmCTestProcesses_yypush_buffer_state_ALREADY_DEFINED
#else
#define yypush_buffer_state cmCTestProcesses_yypush_buffer_state
#endif
#ifdef yypop_buffer_state
#define cmCTestProcesses_yypop_buffer_state_ALREADY_DEFINED
#else
#define yypop_buffer_state cmCTestProcesses_yypop_buffer_state
#endif
#ifdef yyensure_buffer_stack
#define cmCTestProcesses_yyensure_buffer_stack_ALREADY_DEFINED
#else
#define yyensure_buffer_stack cmCTestProcesses_yyensure_buffer_stack
#endif
#ifdef yylex
#define cmCTestProcesses_yylex_ALREADY_DEFINED
#else
#define yylex cmCTestProcesses_yylex
#endif
#ifdef yyrestart
#define cmCTestProcesses_yyrestart_ALREADY_DEFINED
#else
#define yyrestart cmCTestProcesses_yyrestart
#endif
#ifdef yylex_init
#define cmCTestProcesses_yylex_init_ALREADY_DEFINED
#else
#define yylex_init cmCTestProcesses_yylex_init
#endif
#ifdef yylex_init_extra
#define cmCTestProcesses_yylex_init_extra_ALREADY_DEFINED
#else
#define yylex_init_extra cmCTestProcesses_yylex_init_extra
#endif
#ifdef yylex_destroy
#define cmCTestProcesses_yylex_destroy_ALREADY_DEFINED
#else
#define yylex_destroy cmCTestProcesses_yylex_destroy
#endif
#ifdef yyget_debug
#define cmCTestProcesses_yyget_debug_ALREADY_DEFINED
#else
#define yyget_debug cmCTestProcesses_yyget_debug
#endif
#ifdef yyset_debug
#define cmCTestProcesses_yyset_debug_ALREADY_DEFINED
#else
#define yyset_debug cmCTestProcesses_yyset_debug
#endif
#ifdef yyget_extra
#define cmCTestProcesses_yyget_extra_ALREADY_DEFINED
#else
#define yyget_extra cmCTestProcesses_yyget_extra
#endif
#ifdef yyset_extra
#define cmCTestProcesses_yyset_extra_ALREADY_DEFINED
#else
#define yyset_extra cmCTestProcesses_yyset_extra
#endif
#ifdef yyget_in
#define cmCTestProcesses_yyget_in_ALREADY_DEFINED
#else
#define yyget_in cmCTestProcesses_yyget_in
#endif
#ifdef yyset_in
#define cmCTestProcesses_yyset_in_ALREADY_DEFINED
#else
#define yyset_in cmCTestProcesses_yyset_in
#endif
#ifdef yyget_out
#define cmCTestProcesses_yyget_out_ALREADY_DEFINED
#else
#define yyget_out cmCTestProcesses_yyget_out
#endif
#ifdef yyset_out
#define cmCTestProcesses_yyset_out_ALREADY_DEFINED
#else
#define yyset_out cmCTestProcesses_yyset_out
#endif
#ifdef yyget_leng
#define cmCTestProcesses_yyget_leng_ALREADY_DEFINED
#else
#define yyget_leng cmCTestProcesses_yyget_leng
#endif
#ifdef yyget_text
#define cmCTestProcesses_yyget_text_ALREADY_DEFINED
#else
#define yyget_text cmCTestProcesses_yyget_text
#endif
#ifdef yyget_lineno
#define cmCTestProcesses_yyget_lineno_ALREADY_DEFINED
#else
#define yyget_lineno cmCTestProcesses_yyget_lineno
#endif
#ifdef yyset_lineno
#define cmCTestProcesses_yyset_lineno_ALREADY_DEFINED
#else
#define yyset_lineno cmCTestProcesses_yyset_lineno
#endif
#ifdef yyget_column
#define cmCTestProcesses_yyget_column_ALREADY_DEFINED
#else
#define yyget_column cmCTestProcesses_yyget_column
#endif
#ifdef yyset_column
#define cmCTestProcesses_yyset_column_ALREADY_DEFINED
#else
#define yyset_column cmCTestProcesses_yyset_column
#endif
#ifdef yywrap
#define cmCTestProcesses_yywrap_ALREADY_DEFINED
#else
#define yywrap cmCTestProcesses_yywrap
#endif
#ifdef yyalloc
#define cmCTestProcesses_yyalloc_ALREADY_DEFINED
#else
#define yyalloc cmCTestProcesses_yyalloc
#endif
#ifdef yyrealloc
#define cmCTestProcesses_yyrealloc_ALREADY_DEFINED
#else
#define yyrealloc cmCTestProcesses_yyrealloc
#endif
#ifdef yyfree
#define cmCTestProcesses_yyfree_ALREADY_DEFINED
#else
#define yyfree cmCTestProcesses_yyfree
#endif
/* First, we deal with platform-specific or compiler-specific issues. */
/* begin standard C headers. */
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <stdlib.h>
/* end standard C headers. */
/* flex integer type definitions */
#ifndef FLEXINT_H
#define FLEXINT_H
/* C99 systems have <inttypes.h>. Non-C99 systems may or may not. */
#if defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L
/* C99 says to define __STDC_LIMIT_MACROS before including stdint.h,
* if you want the limit (max/min) macros for int types.
*/
#ifndef __STDC_LIMIT_MACROS
#define __STDC_LIMIT_MACROS 1
#endif
#include <inttypes.h>
typedef int8_t flex_int8_t;
typedef uint8_t flex_uint8_t;
typedef int16_t flex_int16_t;
typedef uint16_t flex_uint16_t;
typedef int32_t flex_int32_t;
typedef uint32_t flex_uint32_t;
#else
typedef signed char flex_int8_t;
typedef short int flex_int16_t;
typedef int flex_int32_t;
typedef unsigned char flex_uint8_t;
typedef unsigned short int flex_uint16_t;
typedef unsigned int flex_uint32_t;
/* Limits of integral types. */
#ifndef INT8_MIN
#define INT8_MIN (-128)
#endif
#ifndef INT16_MIN
#define INT16_MIN (-32767-1)
#endif
#ifndef INT32_MIN
#define INT32_MIN (-2147483647-1)
#endif
#ifndef INT8_MAX
#define INT8_MAX (127)
#endif
#ifndef INT16_MAX
#define INT16_MAX (32767)
#endif
#ifndef INT32_MAX
#define INT32_MAX (2147483647)
#endif
#ifndef UINT8_MAX
#define UINT8_MAX (255U)
#endif
#ifndef UINT16_MAX
#define UINT16_MAX (65535U)
#endif
#ifndef UINT32_MAX
#define UINT32_MAX (4294967295U)
#endif
#ifndef SIZE_MAX
#define SIZE_MAX (~(size_t)0)
#endif
#endif /* ! C99 */
#endif /* ! FLEXINT_H */
/* begin standard C++ headers. */
/* TODO: this is always defined, so inline it */
#define yyconst const
#if defined(__GNUC__) && __GNUC__ >= 3
#define yynoreturn __attribute__((__noreturn__))
#else
#define yynoreturn
#endif
/* An opaque pointer. */
#ifndef YY_TYPEDEF_YY_SCANNER_T
#define YY_TYPEDEF_YY_SCANNER_T
typedef void* yyscan_t;
#endif
/* For convenience, these vars (plus the bison vars far below)
are macros in the reentrant scanner. */
#define yyin yyg->yyin_r
#define yyout yyg->yyout_r
#define yyextra yyg->yyextra_r
#define yyleng yyg->yyleng_r
#define yytext yyg->yytext_r
#define yylineno (YY_CURRENT_BUFFER_LVALUE->yy_bs_lineno)
#define yycolumn (YY_CURRENT_BUFFER_LVALUE->yy_bs_column)
#define yy_flex_debug yyg->yy_flex_debug_r
/* Size of default input buffer. */
#ifndef YY_BUF_SIZE
#ifdef __ia64__
/* On IA-64, the buffer size is 16k, not 8k.
* Moreover, YY_BUF_SIZE is 2*YY_READ_BUF_SIZE in the general case.
* Ditto for the __ia64__ case accordingly.
*/
#define YY_BUF_SIZE 32768
#else
#define YY_BUF_SIZE 16384
#endif /* __ia64__ */
#endif
#ifndef YY_TYPEDEF_YY_BUFFER_STATE
#define YY_TYPEDEF_YY_BUFFER_STATE
typedef struct yy_buffer_state *YY_BUFFER_STATE;
#endif
#ifndef YY_TYPEDEF_YY_SIZE_T
#define YY_TYPEDEF_YY_SIZE_T
typedef size_t yy_size_t;
#endif
#ifndef YY_STRUCT_YY_BUFFER_STATE
#define YY_STRUCT_YY_BUFFER_STATE
struct yy_buffer_state
{
FILE *yy_input_file;
char *yy_ch_buf; /* input buffer */
char *yy_buf_pos; /* current position in input buffer */
/* Size of input buffer in bytes, not including room for EOB
* characters.
*/
int yy_buf_size;
/* Number of characters read into yy_ch_buf, not including EOB
* characters.
*/
int yy_n_chars;
/* Whether we "own" the buffer - i.e., we know we created it,
* and can realloc() it to grow it, and should free() it to
* delete it.
*/
int yy_is_our_buffer;
/* Whether this is an "interactive" input source; if so, and
* if we're using stdio for input, then we want to use getc()
* instead of fread(), to make sure we stop fetching input after
* each newline.
*/
int yy_is_interactive;
/* Whether we're considered to be at the beginning of a line.
* If so, '^' rules will be active on the next match, otherwise
* not.
*/
int yy_at_bol;
int yy_bs_lineno; /**< The line count. */
int yy_bs_column; /**< The column count. */
/* Whether to try to fill the input buffer when we reach the
* end of it.
*/
int yy_fill_buffer;
int yy_buffer_status;
};
#endif /* !YY_STRUCT_YY_BUFFER_STATE */
void yyrestart ( FILE *input_file , yyscan_t yyscanner );
void yy_switch_to_buffer ( YY_BUFFER_STATE new_buffer , yyscan_t yyscanner );
YY_BUFFER_STATE yy_create_buffer ( FILE *file, int size , yyscan_t yyscanner );
void yy_delete_buffer ( YY_BUFFER_STATE b , yyscan_t yyscanner );
void yy_flush_buffer ( YY_BUFFER_STATE b , yyscan_t yyscanner );
void yypush_buffer_state ( YY_BUFFER_STATE new_buffer , yyscan_t yyscanner );
void yypop_buffer_state ( yyscan_t yyscanner );
YY_BUFFER_STATE yy_scan_buffer ( char *base, yy_size_t size , yyscan_t yyscanner );
YY_BUFFER_STATE yy_scan_string ( const char *yy_str , yyscan_t yyscanner );
YY_BUFFER_STATE yy_scan_bytes ( const char *bytes, int len , yyscan_t yyscanner );
void *yyalloc ( yy_size_t , yyscan_t yyscanner );
void *yyrealloc ( void *, yy_size_t , yyscan_t yyscanner );
void yyfree ( void * , yyscan_t yyscanner );
/* Begin user sect3 */
#define cmCTestProcesses_yywrap(yyscanner) (/*CONSTCOND*/1)
#define YY_SKIP_YYWRAP
#define yytext_ptr yytext_r
#ifdef YY_HEADER_EXPORT_START_CONDITIONS
#define INITIAL 0
#define PROCESSES_START 1
#define PROCESSES_END 2
#define RESOURCE_START 3
#define RESOURCE_COUNT 4
#define RESOURCE_END 5
#endif
#ifndef YY_EXTRA_TYPE
#define YY_EXTRA_TYPE void *
#endif
int yylex_init (yyscan_t* scanner);
int yylex_init_extra ( YY_EXTRA_TYPE user_defined, yyscan_t* scanner);
/* Accessor methods to globals.
These are made visible to non-reentrant scanners for convenience. */
int yylex_destroy ( yyscan_t yyscanner );
int yyget_debug ( yyscan_t yyscanner );
void yyset_debug ( int debug_flag , yyscan_t yyscanner );
YY_EXTRA_TYPE yyget_extra ( yyscan_t yyscanner );
void yyset_extra ( YY_EXTRA_TYPE user_defined , yyscan_t yyscanner );
FILE *yyget_in ( yyscan_t yyscanner );
void yyset_in ( FILE * _in_str , yyscan_t yyscanner );
FILE *yyget_out ( yyscan_t yyscanner );
void yyset_out ( FILE * _out_str , yyscan_t yyscanner );
int yyget_leng ( yyscan_t yyscanner );
char *yyget_text ( yyscan_t yyscanner );
int yyget_lineno ( yyscan_t yyscanner );
void yyset_lineno ( int _line_number , yyscan_t yyscanner );
int yyget_column ( yyscan_t yyscanner );
void yyset_column ( int _column_no , yyscan_t yyscanner );
/* Macros after this point can all be overridden by user definitions in
* section 1.
*/
#ifndef YY_SKIP_YYWRAP
#ifdef __cplusplus
extern "C" int yywrap ( yyscan_t yyscanner );
#else
extern int yywrap ( yyscan_t yyscanner );
#endif
#endif
#ifndef yytext_ptr
static void yy_flex_strncpy ( char *, const char *, int , yyscan_t yyscanner);
#endif
#ifdef YY_NEED_STRLEN
static int yy_flex_strlen ( const char * , yyscan_t yyscanner);
#endif
#ifndef YY_NO_INPUT
#endif
/* Amount of stuff to slurp up with each read. */
#ifndef YY_READ_BUF_SIZE
#ifdef __ia64__
/* On IA-64, the buffer size is 16k, not 8k */
#define YY_READ_BUF_SIZE 16384
#else
#define YY_READ_BUF_SIZE 8192
#endif /* __ia64__ */
#endif
/* Number of entries by which start-condition stack grows. */
#ifndef YY_START_STACK_INCR
#define YY_START_STACK_INCR 25
#endif
/* Default declaration of generated scanner - a define so the user can
* easily add parameters.
*/
#ifndef YY_DECL
#define YY_DECL_IS_OURS 1
extern int yylex (yyscan_t yyscanner);
#define YY_DECL int yylex (yyscan_t yyscanner)
#endif /* !YY_DECL */
/* yy_get_previous_state - get the state just before the EOB char was reached */
#undef YY_NEW_FILE
#undef YY_FLUSH_BUFFER
#undef yy_set_bol
#undef yy_new_buffer
#undef yy_set_interactive
#undef YY_DO_BEFORE_ACTION
#ifdef YY_DECL_IS_OURS
#undef YY_DECL_IS_OURS
#undef YY_DECL
#endif
#ifndef cmCTestProcesses_yy_create_buffer_ALREADY_DEFINED
#undef yy_create_buffer
#endif
#ifndef cmCTestProcesses_yy_delete_buffer_ALREADY_DEFINED
#undef yy_delete_buffer
#endif
#ifndef cmCTestProcesses_yy_scan_buffer_ALREADY_DEFINED
#undef yy_scan_buffer
#endif
#ifndef cmCTestProcesses_yy_scan_string_ALREADY_DEFINED
#undef yy_scan_string
#endif
#ifndef cmCTestProcesses_yy_scan_bytes_ALREADY_DEFINED
#undef yy_scan_bytes
#endif
#ifndef cmCTestProcesses_yy_init_buffer_ALREADY_DEFINED
#undef yy_init_buffer
#endif
#ifndef cmCTestProcesses_yy_flush_buffer_ALREADY_DEFINED
#undef yy_flush_buffer
#endif
#ifndef cmCTestProcesses_yy_load_buffer_state_ALREADY_DEFINED
#undef yy_load_buffer_state
#endif
#ifndef cmCTestProcesses_yy_switch_to_buffer_ALREADY_DEFINED
#undef yy_switch_to_buffer
#endif
#ifndef cmCTestProcesses_yypush_buffer_state_ALREADY_DEFINED
#undef yypush_buffer_state
#endif
#ifndef cmCTestProcesses_yypop_buffer_state_ALREADY_DEFINED
#undef yypop_buffer_state
#endif
#ifndef cmCTestProcesses_yyensure_buffer_stack_ALREADY_DEFINED
#undef yyensure_buffer_stack
#endif
#ifndef cmCTestProcesses_yylex_ALREADY_DEFINED
#undef yylex
#endif
#ifndef cmCTestProcesses_yyrestart_ALREADY_DEFINED
#undef yyrestart
#endif
#ifndef cmCTestProcesses_yylex_init_ALREADY_DEFINED
#undef yylex_init
#endif
#ifndef cmCTestProcesses_yylex_init_extra_ALREADY_DEFINED
#undef yylex_init_extra
#endif
#ifndef cmCTestProcesses_yylex_destroy_ALREADY_DEFINED
#undef yylex_destroy
#endif
#ifndef cmCTestProcesses_yyget_debug_ALREADY_DEFINED
#undef yyget_debug
#endif
#ifndef cmCTestProcesses_yyset_debug_ALREADY_DEFINED
#undef yyset_debug
#endif
#ifndef cmCTestProcesses_yyget_extra_ALREADY_DEFINED
#undef yyget_extra
#endif
#ifndef cmCTestProcesses_yyset_extra_ALREADY_DEFINED
#undef yyset_extra
#endif
#ifndef cmCTestProcesses_yyget_in_ALREADY_DEFINED
#undef yyget_in
#endif
#ifndef cmCTestProcesses_yyset_in_ALREADY_DEFINED
#undef yyset_in
#endif
#ifndef cmCTestProcesses_yyget_out_ALREADY_DEFINED
#undef yyget_out
#endif
#ifndef cmCTestProcesses_yyset_out_ALREADY_DEFINED
#undef yyset_out
#endif
#ifndef cmCTestProcesses_yyget_leng_ALREADY_DEFINED
#undef yyget_leng
#endif
#ifndef cmCTestProcesses_yyget_text_ALREADY_DEFINED
#undef yyget_text
#endif
#ifndef cmCTestProcesses_yyget_lineno_ALREADY_DEFINED
#undef yyget_lineno
#endif
#ifndef cmCTestProcesses_yyset_lineno_ALREADY_DEFINED
#undef yyset_lineno
#endif
#ifndef cmCTestProcesses_yyget_column_ALREADY_DEFINED
#undef yyget_column
#endif
#ifndef cmCTestProcesses_yyset_column_ALREADY_DEFINED
#undef yyset_column
#endif
#ifndef cmCTestProcesses_yywrap_ALREADY_DEFINED
#undef yywrap
#endif
#ifndef cmCTestProcesses_yyget_lval_ALREADY_DEFINED
#undef yyget_lval
#endif
#ifndef cmCTestProcesses_yyset_lval_ALREADY_DEFINED
#undef yyset_lval
#endif
#ifndef cmCTestProcesses_yyget_lloc_ALREADY_DEFINED
#undef yyget_lloc
#endif
#ifndef cmCTestProcesses_yyset_lloc_ALREADY_DEFINED
#undef yyset_lloc
#endif
#ifndef cmCTestProcesses_yyalloc_ALREADY_DEFINED
#undef yyalloc
#endif
#ifndef cmCTestProcesses_yyrealloc_ALREADY_DEFINED
#undef yyrealloc
#endif
#ifndef cmCTestProcesses_yyfree_ALREADY_DEFINED
#undef yyfree
#endif
#ifndef cmCTestProcesses_yytext_ALREADY_DEFINED
#undef yytext
#endif
#ifndef cmCTestProcesses_yyleng_ALREADY_DEFINED
#undef yyleng
#endif
#ifndef cmCTestProcesses_yyin_ALREADY_DEFINED
#undef yyin
#endif
#ifndef cmCTestProcesses_yyout_ALREADY_DEFINED
#undef yyout
#endif
#ifndef cmCTestProcesses_yy_flex_debug_ALREADY_DEFINED
#undef yy_flex_debug
#endif
#ifndef cmCTestProcesses_yylineno_ALREADY_DEFINED
#undef yylineno
#endif
#ifndef cmCTestProcesses_yytables_fload_ALREADY_DEFINED
#undef yytables_fload
#endif
#ifndef cmCTestProcesses_yytables_destroy_ALREADY_DEFINED
#undef yytables_destroy
#endif
#ifndef cmCTestProcesses_yyTABLES_NAME_ALREADY_DEFINED
#undef yyTABLES_NAME
#endif
#undef cmCTestProcesses_yyIN_HEADER
#endif /* cmCTestProcesses_yyHEADER_H */

102
Source/LexerParser/cmCTestProcessesLexer.in.l Normal file
View File

@ -0,0 +1,102 @@
%{
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
file Copyright.txt or https://cmake.org/licensing for details. */
/*
This file must be translated to C++ and modified to build everywhere.
Run flex >= 2.6 like this:
flex --nounistd -DFLEXINT_H --noline --header-file=cmCTestProcessesLexer.h -ocmCTestProcessesLexer.cxx cmCTestProcessesLexer.in.l
Modify cmCTestProcessesLexer.cxx:
- remove trailing whitespace: sed -i 's/\s*$//' cmCTestProcessesLexer.h cmCTestProcessesLexer.cxx
- remove blank lines at end of file: sed -i '${/^$/d;}' cmCTestProcessesLexer.h cmCTestProcessesLexer.cxx
- #include "cmStandardLexer.h" at the top: sed -i '1i#include "cmStandardLexer.h"' cmCTestProcessesLexer.cxx
*/
/* IWYU pragma: no_forward_declare yyguts_t */
#ifndef __clang_analyzer__ /* Suppress clang scan-build warnings */
#include "cmCTestProcessesLexerHelper.h"
#include <string>
#include <cstddef>
/*--------------------------------------------------------------------------*/
%}
%option prefix="cmCTestProcesses_yy"
%option reentrant
%option noyywrap
%option nodefault
%pointer
%s PROCESSES_START
%s PROCESSES_END
%s RESOURCE_START
%s RESOURCE_COUNT
%s RESOURCE_END
NUMBER [0-9]+
IDENTIFIER [a-z_][a-z0-9_]*
%%
<INITIAL,PROCESSES_START,RESOURCE_START>{IDENTIFIER}: {
BEGIN(RESOURCE_COUNT);
yyextra->SetResourceType(std::string(yytext, yyleng - 1));
}
<INITIAL,PROCESSES_START>{NUMBER} {
BEGIN(PROCESSES_END);
std::size_t len = yyleng;
yyextra->SetProcessCount(std::stoll(yytext, &len, 10));
}
<RESOURCE_COUNT>{NUMBER} {
BEGIN(RESOURCE_END);
std::size_t len = yyleng;
yyextra->SetNeededSlots(std::stoll(yytext, &len, 10));
yyextra->WriteRequirement();
}
<PROCESSES_END,RESOURCE_END>,+ {
BEGIN(RESOURCE_START);
}
<INITIAL,PROCESSES_START,RESOURCE_START>;+ {
BEGIN(PROCESSES_START);
}
<PROCESSES_END,RESOURCE_END>;+ {
BEGIN(PROCESSES_START);
yyextra->WriteProcess();
}
<RESOURCE_START,PROCESSES_END,RESOURCE_END><<EOF>> {
yyextra->WriteProcess();
return 0;
}
<INITIAL,PROCESSES_START><<EOF>> {
return 0;
}
<<EOF>> {
return 1;
}
.|\n {
return 1;
}
%%
/*--------------------------------------------------------------------------*/
#endif /* __clang_analyzer__ */

9
Source/cmCTest.cxx
View File

@ -2090,6 +2090,15 @@ bool cmCTest::HandleCommandLineArguments(size_t& i,
"ExcludeFixtureCleanupRegularExpression", args[i].c_str());
}
if (this->CheckArgument(arg, "--hardware-spec-file") &&
i < args.size() - 1) {
i++;
this->GetTestHandler()->SetPersistentOption("HardwareSpecFile",
args[i].c_str());
this->GetMemCheckHandler()->SetPersistentOption("HardwareSpecFile",
args[i].c_str());
}
if (this->CheckArgument(arg, "--rerun-failed")) {
this->GetTestHandler()->SetPersistentOption("RerunFailed", "true");
this->GetMemCheckHandler()->SetPersistentOption("RerunFailed", "true");

1
Source/ctest.cxx
View File

@ -103,6 +103,7 @@ static const char* cmDocumentationOptions[][2] = {
"times without failing in order to pass" },
{ "--max-width <width>", "Set the max width for a test name to output" },
{ "--interactive-debug-mode [0|1]", "Set the interactive mode to 0 or 1." },
{ "--hardware-spec-file <file>", "Set the hardware spec file to use." },
{ "--no-label-summary", "Disable timing summary information for labels." },
{ "--no-subproject-summary",
"Disable timing summary information for "

8
Tests/CMakeLib/CMakeLists.txt
View File

@ -2,10 +2,15 @@ include_directories(
${CMAKE_CURRENT_BINARY_DIR}
${CMake_BINARY_DIR}/Source
${CMake_SOURCE_DIR}/Source
${CMake_SOURCE_DIR}/Source/CTest
)
set(CMakeLib_TESTS
testArgumentParser.cxx
testCTestBinPacker.cxx
testCTestProcesses.cxx
testCTestHardwareAllocator.cxx
testCTestHardwareSpec.cxx
testGeneratedFileStream.cxx
testRST.cxx
testRange.cxx
@ -27,6 +32,7 @@ add_executable(testUVProcessChainHelper testUVProcessChainHelper.cxx)
set(testRST_ARGS ${CMAKE_CURRENT_SOURCE_DIR})
set(testUVProcessChain_ARGS $<TARGET_FILE:testUVProcessChainHelper>)
set(testUVStreambuf_ARGS $<TARGET_FILE:cmake>)
set(testCTestHardwareSpec_ARGS ${CMAKE_CURRENT_SOURCE_DIR})
if(WIN32)
list(APPEND CMakeLib_TESTS
@ -41,7 +47,7 @@ configure_file(${CMAKE_CURRENT_SOURCE_DIR}/testXMLParser.h.in
create_test_sourcelist(CMakeLib_TEST_SRCS CMakeLibTests.cxx ${CMakeLib_TESTS})
add_executable(CMakeLibTests ${CMakeLib_TEST_SRCS})
target_link_libraries(CMakeLibTests CMakeLib)
target_link_libraries(CMakeLibTests CMakeLib CTestLib)
set_property(TARGET CMakeLibTests PROPERTY C_CLANG_TIDY "")
set_property(TARGET CMakeLibTests PROPERTY CXX_CLANG_TIDY "")

300
Tests/CMakeLib/testCTestBinPacker.cxx Normal file
View File

@ -0,0 +1,300 @@
#include <cstddef>
#include <iostream>
#include <map>
#include <string>
#include <vector>
#include "cmCTestBinPacker.h"
#include "cmCTestHardwareAllocator.h"
struct ExpectedPackResult
{
std::vector<int> SlotsNeeded;
std::map<std::string, cmCTestHardwareAllocator::Resource> Hardware;
bool ExpectedReturnValue;
std::vector<cmCTestBinPackerAllocation> ExpectedRoundRobinAllocations;
std::vector<cmCTestBinPackerAllocation> ExpectedBlockAllocations;
};
static const std::vector<ExpectedPackResult> expectedResults
{
/* clang-format off */
{
{ 2, 2, 2, 2 },
{ { "0", { 4, 0 } }, { "1", { 4, 0 } }, { "2", { 4, 0 } },
{ "3", { 4, 0 } } },
true,
{
{ 0, 2, "0" },
{ 1, 2, "1" },
{ 2, 2, "2" },
{ 3, 2, "3" },
},
{
{ 0, 2, "0" },
{ 1, 2, "0" },
{ 2, 2, "1" },
{ 3, 2, "1" },
},
},
{
{ 2, 3, 2 },
{ { "0", { 5, 0 } }, { "1", { 2, 0 } } },
true,
{
{ 0, 2, "0" },
{ 1, 3, "0" },
{ 2, 2, "1" },
},
{
{ 0, 2, "0" },
{ 1, 3, "0" },
{ 2, 2, "1" },
},
},
{
{ 1, 2, 3 },
{ { "0", { 1, 0 } }, { "1", { 2, 0 } }, { "2", { 2, 0 } } },
false,
{ },
{ },
},
{
{ 48, 21, 31, 10, 40 },
{ { "0", { 81, 0 } }, { "1", { 68, 0 } }, { "2", { 20, 0 } },
{ "3", { 13, 0 } } },
true,
{
{ 0, 48, "0" },
{ 1, 21, "1" },
{ 2, 31, "0" },
{ 3, 10, "2" },
{ 4, 40, "1" },
},
{
{ 0, 48, "0" },
{ 1, 21, "1" },
{ 2, 31, "0" },
{ 3, 10, "2" },
{ 4, 40, "1" },
},
},
{
{ 30, 31, 39, 67 },
{ { "0", { 16, 0 } }, { "1", { 81, 0 } }, { "2", { 97, 0 } } },
true,
{
{ 0, 30, "2" },
{ 1, 31, "1" },
{ 2, 39, "1" },
{ 3, 67, "2" },
},
{
{ 0, 30, "2" },
{ 1, 31, "1" },
{ 2, 39, "1" },
{ 3, 67, "2" },
},
},
{
{ 63, 47, 1, 9 },
{ { "0", { 18, 0 } }, { "1", { 29, 0 } }, { "2", { 9, 0 } },
{ "3", { 52, 0 } } },
false,
{ },
{ },
},
{
{ 22, 29, 46, 85 },
{ { "0", { 65, 0 } }, { "1", { 85, 0 } }, { "2", { 65, 0 } },
{ "3", { 78, 0 } } },
true,
{
{ 0, 22, "2" },
{ 1, 29, "0" },
{ 2, 46, "3" },
{ 3, 85, "1" },
},
{
{ 0, 22, "0" },
{ 1, 29, "3" },
{ 2, 46, "3" },
{ 3, 85, "1" },
},
},
{
{ 66, 11, 34, 21 },
{ { "0", { 24, 0 } }, { "1", { 57, 0 } }, { "2", { 61, 0 } },
{ "3", { 51, 0 } } },
false,
{ },
{ },
},
{
{ 72, 65, 67, 45 },
{ { "0", { 29, 0 } }, { "1", { 77, 0 } }, { "2", { 98, 0 } },
{ "3", { 58, 0 } } },
false,
{ },
{ },
},
/*
* The following is a contrived attack on the bin-packing algorithm that
* causes it to execute with n! complexity, where n is the number of
* resources. This case is very unrepresentative of real-world usage, and
* has been documented but disabled. The bin-packing problem is NP-hard, and
* we may not be able to fix this case at all.
*/
#if 0
{
{ 1000, 999, 998, 997, 996, 995, 994, 993, 992, 991, 19 },
{ { "0", { 1000, 0 } }, { "1", { 1001, 0 } }, { "2", { 1002, 0 } },
{ "3", { 1003, 0 } }, { "4", { 1004, 0 } }, { "5", { 1005, 0 } },
{ "6", { 1006, 0 } }, { "7", { 1007, 0 } }, { "8", { 1008, 0 } },
{ "9", { 1009, 0 } } },
false,
{ },
{ },
},
#endif
/*
* These cases are more representative of real-world usage (the resource
* sizes are all the same.)
*/
{
{ 1000, 999, 998, 997, 996, 995, 994, 993, 992, 991, 10 },
{ { "0", { 1000, 0 } }, { "1", { 1000, 0 } }, { "2", { 1000, 0 } },
{ "3", { 1000, 0 } }, { "4", { 1000, 0 } }, { "5", { 1000, 0 } },
{ "6", { 1000, 0 } }, { "7", { 1000, 0 } }, { "8", { 1000, 0 } },
{ "9", { 1000, 0 } } },
false,
{ },
{ },
},
{
{ 1000, 999, 998, 997, 996, 995, 994, 993, 992, 991, 9 },
{ { "0", { 1000, 0 } }, { "1", { 1000, 0 } }, { "2", { 1000, 0 } },
{ "3", { 1000, 0 } }, { "4", { 1000, 0 } }, { "5", { 1000, 0 } },
{ "6", { 1000, 0 } }, { "7", { 1000, 0 } }, { "8", { 1000, 0 } },
{ "9", { 1000, 0 } } },
true,
{
{ 0, 1000, "0" },
{ 1, 999, "1" },
{ 2, 998, "2" },
{ 3, 997, "3" },
{ 4, 996, "4" },
{ 5, 995, "5" },
{ 6, 994, "6" },
{ 7, 993, "7" },
{ 8, 992, "8" },
{ 9, 991, "9" },
{ 10, 9, "9" },
},
{
{ 0, 1000, "0" },
{ 1, 999, "1" },
{ 2, 998, "2" },
{ 3, 997, "3" },
{ 4, 996, "4" },
{ 5, 995, "5" },
{ 6, 994, "6" },
{ 7, 993, "7" },
{ 8, 992, "8" },
{ 9, 991, "9" },
{ 10, 9, "9" },
},
},
/* clang-format on */
};
struct AllocationComparison
{
cmCTestBinPackerAllocation First;
cmCTestBinPackerAllocation Second;
bool Equal;
};
static const std::vector<AllocationComparison> comparisons{
/* clang-format off */
{ { 0, 1, "0" }, { 0, 1, "0" }, true },
{ { 0, 1, "0" }, { 1, 1, "0" }, false },
{ { 0, 1, "0" }, { 0, 2, "0" }, false },
{ { 0, 1, "0" }, { 0, 1, "1" }, false },
/* clang-format on */
};
bool TestExpectedPackResult(const ExpectedPackResult& expected)
{
std::vector<cmCTestBinPackerAllocation> roundRobinAllocations;
roundRobinAllocations.reserve(expected.SlotsNeeded.size());
std::size_t index = 0;
for (auto const& n : expected.SlotsNeeded) {
roundRobinAllocations.push_back({ index++, n, "" });
}
bool roundRobinResult = cmAllocateCTestHardwareRoundRobin(
expected.Hardware, roundRobinAllocations);
if (roundRobinResult != expected.ExpectedReturnValue) {
std::cout
<< "cmAllocateCTestHardwareRoundRobin did not return expected value"
<< std::endl;
return false;
}
if (roundRobinResult &&
roundRobinAllocations != expected.ExpectedRoundRobinAllocations) {
std::cout << "cmAllocateCTestHardwareRoundRobin did not return expected "
"allocations"
<< std::endl;
return false;
}
std::vector<cmCTestBinPackerAllocation> blockAllocations;
blockAllocations.reserve(expected.SlotsNeeded.size());
index = 0;
for (auto const& n : expected.SlotsNeeded) {
blockAllocations.push_back({ index++, n, "" });
}
bool blockResult =
cmAllocateCTestHardwareBlock(expected.Hardware, blockAllocations);
if (blockResult != expected.ExpectedReturnValue) {
std::cout << "cmAllocateCTestHardwareBlock did not return expected value"
<< std::endl;
return false;
}
if (blockResult && blockAllocations != expected.ExpectedBlockAllocations) {
std::cout << "cmAllocateCTestHardwareBlock did not return expected"
" allocations"
<< std::endl;
return false;
}
return true;
}
int testCTestBinPacker(int /*unused*/, char* /*unused*/ [])
{
int retval = 0;
for (auto const& comparison : comparisons) {
if ((comparison.First == comparison.Second) != comparison.Equal) {
std::cout << "Comparison did not match expected" << std::endl;
retval = 1;
}
if ((comparison.First != comparison.Second) == comparison.Equal) {
std::cout << "Comparison did not match expected" << std::endl;
retval = 1;
}
}
for (auto const& expected : expectedResults) {
if (!TestExpectedPackResult(expected)) {
retval = 1;
}
}
return retval;
}

426
Tests/CMakeLib/testCTestHardwareAllocator.cxx Normal file
View File

@ -0,0 +1,426 @@
#include <iostream>
#include <map>
#include <string>
#include <vector>
#include "cmCTestHardwareAllocator.h"
#include "cmCTestHardwareSpec.h"
static const cmCTestHardwareSpec spec{ { {
/* clang-format off */
{ "gpus", { { "0", 4 }, { "1", 8 }, { "2", 0 }, { "3", 8 } } },
/* clang-format on */
} } };
bool testInitializeFromHardwareSpec()
{
bool retval = true;
cmCTestHardwareAllocator allocator;
allocator.InitializeFromHardwareSpec(spec);
static const std::map<
std::string, std::map<std::string, cmCTestHardwareAllocator::Resource>>
expected{
/* clang-format off */
{ "gpus", {
{ "0", { 4, 0 } },
{ "1", { 8, 0 } },
{ "2", { 0, 0 } },
{ "3", { 8, 0 } },
} },
/* clang-format on */
};
if (allocator.GetResources() != expected) {
std::cout << "GetResources() did not return expected value\n";
retval = false;
}
return retval;
}
bool testAllocateResource()
{
bool retval = true;
cmCTestHardwareAllocator allocator;
allocator.InitializeFromHardwareSpec(spec);
static const std::map<
std::string, std::map<std::string, cmCTestHardwareAllocator::Resource>>
expected1{
/* clang-format off */
{ "gpus", {
{ "0", { 4, 2 } },
{ "1", { 8, 0 } },
{ "2", { 0, 0 } },
{ "3", { 8, 0 } },
} },
/* clang-format on */
};
if (!allocator.AllocateResource("gpus", "0", 2)) {
std::cout
<< "AllocateResource(\"gpus\", \"0\", 2) returned false, should be "
"true\n";
retval = false;
}
if (allocator.GetResources() != expected1) {
std::cout << "GetResources() did not return expected value\n";
retval = false;
}
static const std::map<
std::string, std::map<std::string, cmCTestHardwareAllocator::Resource>>
expected2{
/* clang-format off */
{ "gpus", {
{ "0", { 4, 4 } },
{ "1", { 8, 0 } },
{ "2", { 0, 0 } },
{ "3", { 8, 0 } },
} },
/* clang-format on */
};
if (!allocator.AllocateResource("gpus", "0", 2)) {
std::cout
<< "AllocateResource(\"gpus\", \"0\", 2) returned false, should be "
"true\n";
retval = false;
}
if (allocator.GetResources() != expected2) {
std::cout << "GetResources() did not return expected value\n";
retval = false;
}
static const std::map<
std::string, std::map<std::string, cmCTestHardwareAllocator::Resource>>
expected3{
/* clang-format off */
{ "gpus", {
{ "0", { 4, 4 } },
{ "1", { 8, 0 } },
{ "2", { 0, 0 } },
{ "3", { 8, 0 } },
} },
/* clang-format on */
};
if (allocator.AllocateResource("gpus", "0", 1)) {
std::cout
<< "AllocateResource(\"gpus\", \"0\", 1) returned true, should be "
"false\n";
retval = false;
}
if (allocator.GetResources() != expected3) {
std::cout << "GetResources() did not return expected value\n";
retval = false;
}
static const std::map<
std::string, std::map<std::string, cmCTestHardwareAllocator::Resource>>
expected4{
/* clang-format off */
{ "gpus", {
{ "0", { 4, 4 } },
{ "1", { 8, 7 } },
{ "2", { 0, 0 } },
{ "3", { 8, 0 } },
} },
/* clang-format on */
};
if (!allocator.AllocateResource("gpus", "1", 7)) {
std::cout
<< "AllocateResource(\"gpus\", \"1\", 7) returned false, should be "
"true\n";
retval = false;
}
if (allocator.AllocateResource("gpus", "1", 2)) {
std::cout
<< "AllocateResource(\"gpus\", \"1\", 2) returned true, should be "
"false\n";
retval = false;
}
if (allocator.GetResources() != expected4) {
std::cout << "GetResources() did not return expected value\n";
retval = false;
}
static const std::map<
std::string, std::map<std::string, cmCTestHardwareAllocator::Resource>>
expected5{
/* clang-format off */
{ "gpus", {
{ "0", { 4, 4 } },
{ "1", { 8, 7 } },
{ "2", { 0, 0 } },
{ "3", { 8, 0 } },
} },
/* clang-format on */
};
if (allocator.AllocateResource("gpus", "2", 1)) {
std::cout
<< "AllocateResource(\"gpus\", \"2\", 1) returned true, should be "
"false\n";
retval = false;
}
if (allocator.GetResources() != expected5) {
std::cout << "GetResources() did not return expected value\n";
retval = false;
}
static const std::map<
std::string, std::map<std::string, cmCTestHardwareAllocator::Resource>>
expected6{
/* clang-format off */
{ "gpus", {
{ "0", { 4, 4 } },
{ "1", { 8, 7 } },
{ "2", { 0, 0 } },
{ "3", { 8, 0 } },
} },
/* clang-format on */
};
if (allocator.AllocateResource("gpus", "4", 1)) {
std::cout
<< "AllocateResource(\"gpus\", \"4\", 1) returned true, should be "
"false\n";
retval = false;
}
if (allocator.GetResources() != expected6) {
std::cout << "GetResources() did not return expected value\n";
retval = false;
}
static const std::map<
std::string, std::map<std::string, cmCTestHardwareAllocator::Resource>>
expected7{
/* clang-format off */
{ "gpus", {
{ "0", { 4, 4 } },
{ "1", { 8, 7 } },
{ "2", { 0, 0 } },
{ "3", { 8, 0 } },
} },
/* clang-format on */
};
if (allocator.AllocateResource("threads", "0", 1)) {
std::cout
<< "AllocateResource(\"threads\", \"0\", 1) returned true, should be"
" false\n";
retval = false;
}
if (allocator.GetResources() != expected7) {
std::cout << "GetResources() did not return expected value\n";
retval = false;
}
return retval;
}
bool testDeallocateResource()
{
bool retval = true;
cmCTestHardwareAllocator allocator;
allocator.InitializeFromHardwareSpec(spec);
static const std::map<
std::string, std::map<std::string, cmCTestHardwareAllocator::Resource>>
expected1{
/* clang-format off */
{ "gpus", {
{ "0", { 4, 1 } },
{ "1", { 8, 0 } },
{ "2", { 0, 0 } },
{ "3", { 8, 0 } },
} },
/* clang-format on */
};
if (!allocator.AllocateResource("gpus", "0", 2)) {
std::cout
<< "AllocateResource(\"gpus\", \"0\", 2) returned false, should be "
"true\n";
retval = false;
}
if (!allocator.DeallocateResource("gpus", "0", 1)) {
std::cout
<< "DeallocateResource(\"gpus\", \"0\", 1) returned false, should be"
" true\n";
retval = false;
}
if (allocator.GetResources() != expected1) {
std::cout << "GetResources() did not return expected value\n";
retval = false;
}
static const std::map<
std::string, std::map<std::string, cmCTestHardwareAllocator::Resource>>
expected2{
/* clang-format off */
{ "gpus", {
{ "0", { 4, 1 } },
{ "1", { 8, 0 } },
{ "2", { 0, 0 } },
{ "3", { 8, 0 } },
} },
/* clang-format on */
};
if (allocator.DeallocateResource("gpus", "0", 2)) {
std::cout
<< "DeallocateResource(\"gpus\", \"0\", 2) returned true, should be"
" false\n";
retval = false;
}
if (allocator.GetResources() != expected2) {
std::cout << "GetResources() did not return expected value\n";
retval = false;
}
static const std::map<
std::string, std::map<std::string, cmCTestHardwareAllocator::Resource>>
expected3{
/* clang-format off */
{ "gpus", {
{ "0", { 4, 0 } },
{ "1", { 8, 0 } },
{ "2", { 0, 0 } },
{ "3", { 8, 0 } },
} },
/* clang-format on */
};
if (!allocator.DeallocateResource("gpus", "0", 1)) {
std::cout
<< "DeallocateResource(\"gpus\", \"0\", 1) returned false, should be"
" true\n";
retval = false;
}
if (allocator.GetResources() != expected3) {
std::cout << "GetResources() did not return expected value\n";
retval = false;
}
static const std::map<
std::string, std::map<std::string, cmCTestHardwareAllocator::Resource>>
expected4{
/* clang-format off */
{ "gpus", {
{ "0", { 4, 0 } },
{ "1", { 8, 0 } },
{ "2", { 0, 0 } },
{ "3", { 8, 0 } },
} },
/* clang-format on */
};
if (allocator.DeallocateResource("gpus", "0", 1)) {
std::cout
<< "DeallocateResource(\"gpus\", \"0\", 1) returned true, should be"
" false\n";
retval = false;
}
if (allocator.GetResources() != expected4) {
std::cout << "GetResources() did not return expected value\n";
retval = false;
}
static const std::map<
std::string, std::map<std::string, cmCTestHardwareAllocator::Resource>>
expected5{
/* clang-format off */
{ "gpus", {
{ "0", { 4, 0 } },
{ "1", { 8, 0 } },
{ "2", { 0, 0 } },
{ "3", { 8, 0 } },
} },
/* clang-format on */
};
if (allocator.DeallocateResource("gpus", "4", 1)) {
std::cout
<< "DeallocateResource(\"gpus\", \"4\", 1) returned true, should be"
" false\n";
retval = false;
}
if (allocator.GetResources() != expected5) {
std::cout << "GetResources() did not return expected value\n";
retval = false;
}
static const std::map<
std::string, std::map<std::string, cmCTestHardwareAllocator::Resource>>
expected6{
/* clang-format off */
{ "gpus", {
{ "0", { 4, 0 } },
{ "1", { 8, 0 } },
{ "2", { 0, 0 } },
{ "3", { 8, 0 } },
} },
/* clang-format on */
};
if (allocator.DeallocateResource("threads", "0", 1)) {
std::cout
<< "DeallocateResource(\"threads\", \"0\", 1) returned true, should be"
" false\n";
retval = false;
}
if (allocator.GetResources() != expected6) {
std::cout << "GetResources() did not return expected value\n";
retval = false;
}
return retval;
}
bool testResourceFree()
{
bool retval = true;
const cmCTestHardwareAllocator::Resource r1{ 5, 0 };
if (r1.Free() != 5) {
std::cout << "cmCTestHardwareAllocator::Resource::Free() did not return "
"expected value for { 5, 0 }\n";
retval = false;
}
const cmCTestHardwareAllocator::Resource r2{ 3, 2 };
if (r2.Free() != 1) {
std::cout << "cmCTestHardwareAllocator::Resource::Free() did not return "
"expected value for { 3, 2 }\n";
retval = false;
}
const cmCTestHardwareAllocator::Resource r3{ 4, 4 };
if (r3.Free() != 0) {
std::cout << "cmCTestHardwareAllocator::Resource::Free() did not return "
"expected value for { 4, 4 }\n";
retval = false;
}
return retval;
}
int testCTestHardwareAllocator(int, char** const)
{
int retval = 0;
if (!testInitializeFromHardwareSpec()) {
std::cout << "in testInitializeFromHardwareSpec()\n";
retval = -1;
}
if (!testAllocateResource()) {
std::cout << "in testAllocateResource()\n";
retval = -1;
}
if (!testDeallocateResource()) {
std::cout << "in testDeallocateResource()\n";
retval = -1;
}
if (!testResourceFree()) {
std::cout << "in testResourceFree()\n";
retval = -1;
}
return retval;
}

84
Tests/CMakeLib/testCTestHardwareSpec.cxx Normal file
View File

@ -0,0 +1,84 @@
#include <iostream>
#include <string>
#include <vector>
#include "cmCTestHardwareSpec.h"
struct ExpectedSpec
{
std::string Path;
bool ParseResult;
cmCTestHardwareSpec Expected;
};
static const std::vector<ExpectedSpec> expectedHardwareSpecs = {
/* clang-format off */
{"spec1.json", true, {{{
{"gpus", {
{"2", 4},
{"e", 1},
}},
{"threads", {
}},
}}}},
{"spec2.json", true, {{{
}}}},
{"spec3.json", false, {{{}}}},
{"spec4.json", false, {{{}}}},
{"spec5.json", false, {{{}}}},
{"spec6.json", false, {{{}}}},
{"spec7.json", false, {{{}}}},
{"spec8.json", false, {{{}}}},
{"spec9.json", false, {{{}}}},
{"spec10.json", false, {{{}}}},
{"spec11.json", false, {{{}}}},
{"spec12.json", false, {{{}}}},
{"spec13.json", false, {{{}}}},
{"spec14.json", true, {{{}}}},
{"spec15.json", true, {{{}}}},
{"spec16.json", true, {{{}}}},
{"spec17.json", false, {{{}}}},
{"spec18.json", false, {{{}}}},
{"noexist.json", false, {{{}}}},
/* clang-format on */
};
static bool testSpec(const std::string& path, bool expectedResult,
const cmCTestHardwareSpec& expected)
{
cmCTestHardwareSpec actual;
bool result = actual.ReadFromJSONFile(path);
if (result != expectedResult) {
std::cout << "ReadFromJSONFile(\"" << path << "\") returned " << result
<< ", should be " << expectedResult << std::endl;
return false;
}
if (result && actual != expected) {
std::cout << "ReadFromJSONFile(\"" << path
<< "\") did not give expected spec" << std::endl;
return false;
}
return true;
}
int testCTestHardwareSpec(int argc, char** const argv)
{
if (argc < 2) {
std::cout << "Invalid arguments.\n";
return -1;
}
int retval = 0;
for (auto const& spec : expectedHardwareSpecs) {
std::string path = argv[1];
path += "/testCTestHardwareSpec_data/";
path += spec.Path;
if (!testSpec(path, spec.ParseResult, spec.Expected)) {
retval = -1;
}
}
return retval;
}

23
Tests/CMakeLib/testCTestHardwareSpec_data/spec1.json Normal file
View File

@ -0,0 +1,23 @@
{
"local": [
{
"gpus": [
{
"id": "2",
"slots": 4
},
{
"id": "e"
}
],
".reserved": [
{
"id": "a",
"slots": 3
}
],
"threads": [
]
}
]
}

11
Tests/CMakeLib/testCTestHardwareSpec_data/spec10.json Normal file
View File

@ -0,0 +1,11 @@
{
"local": [
{
"gpus": [
{
"id": 4
}
]
}
]
}

12
Tests/CMakeLib/testCTestHardwareSpec_data/spec11.json Normal file
View File

@ -0,0 +1,12 @@
{
"local": [
{
"gpus": [
{
"id": "4",
"slots": "giraffe"
}
]
}
]
}

1
Tests/CMakeLib/testCTestHardwareSpec_data/spec12.json Normal file
View File

@ -0,0 +1 @@
[]

1
Tests/CMakeLib/testCTestHardwareSpec_data/spec13.json Normal file
View File

@ -0,0 +1 @@
not json

8
Tests/CMakeLib/testCTestHardwareSpec_data/spec14.json Normal file
View File

@ -0,0 +1,8 @@
{
"local": [
{
"0": [
]
}
]
}

8
Tests/CMakeLib/testCTestHardwareSpec_data/spec15.json Normal file
View File

@ -0,0 +1,8 @@
{
"local": [
{
"-": [
]
}
]
}

8
Tests/CMakeLib/testCTestHardwareSpec_data/spec16.json Normal file
View File

@ -0,0 +1,8 @@
{
"local": [
{
"A": [
]
}
]
}

11
Tests/CMakeLib/testCTestHardwareSpec_data/spec17.json Normal file
View File

@ -0,0 +1,11 @@
{
"local": [
{
"gpus": [
{
"id": "A"
}
]
}
]
}

11
Tests/CMakeLib/testCTestHardwareSpec_data/spec18.json Normal file
View File

@ -0,0 +1,11 @@
{
"local": [
{
"gpus": [
{
"id": "-"
}
]
}
]
}

4
Tests/CMakeLib/testCTestHardwareSpec_data/spec2.json Normal file
View File

@ -0,0 +1,4 @@
{
"local": [
]
}

8
Tests/CMakeLib/testCTestHardwareSpec_data/spec3.json Normal file
View File

@ -0,0 +1,8 @@
{
"local": [
{
},
{
}
]
}

4
Tests/CMakeLib/testCTestHardwareSpec_data/spec4.json Normal file
View File

@ -0,0 +1,4 @@
{
"local": {
}
}

2
Tests/CMakeLib/testCTestHardwareSpec_data/spec5.json Normal file
View File

@ -0,0 +1,2 @@
{
}

5
Tests/CMakeLib/testCTestHardwareSpec_data/spec6.json Normal file
View File

@ -0,0 +1,5 @@
{
"local": [
[]
]
}

8
Tests/CMakeLib/testCTestHardwareSpec_data/spec7.json Normal file
View File

@ -0,0 +1,8 @@
{
"local": [
{
"gpus": {
}
}
]
}

9
Tests/CMakeLib/testCTestHardwareSpec_data/spec8.json Normal file
View File

@ -0,0 +1,9 @@
{
"local": [
{
"gpus": [
[]
]
}
]
}

10
Tests/CMakeLib/testCTestHardwareSpec_data/spec9.json Normal file
View File

@ -0,0 +1,10 @@
{
"local": [
{
"gpus": [
{
}
]
}
]
}

137
Tests/CMakeLib/testCTestProcesses.cxx Normal file
View File

@ -0,0 +1,137 @@
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
file Copyright.txt or https://cmake.org/licensing for details. */
#include <iostream>
#include <string>
#include <vector>
#include "cmCTestTestHandler.h"
struct ExpectedParseResult
{
std::string String;
bool ExpectedReturnValue;
std::vector<std::vector<cmCTestTestHandler::cmCTestTestResourceRequirement>>
ExpectedValue;
};
static const std::vector<ExpectedParseResult> expectedResults{
/* clang-format off */
{ "threads:2", true, {
{ { "threads", 2, 1 } },
} },
{ "3,threads:2", true, {
{ { "threads", 2, 1 } },
{ { "threads", 2, 1 } },
{ { "threads", 2, 1 } },
} },
{ "3,threads:2,gpus:4", true, {
{ { "threads", 2, 1 }, { "gpus", 4, 1 } },
{ { "threads", 2, 1 }, { "gpus", 4, 1 } },
{ { "threads", 2, 1 }, { "gpus", 4, 1 } },
} },
{ "2,threads:2;gpus:4", true, {
{ { "threads", 2, 1 } },
{ { "threads", 2, 1 } },
{ { "gpus", 4, 1 } },
} },
{ "threads:2;2,gpus:4", true, {
{ { "threads", 2, 1 } },
{ { "gpus", 4, 1 } },
{ { "gpus", 4, 1 } },
} },
{ "threads:2;gpus:4", true, {
{ { "threads", 2, 1 } },
{ { "gpus", 4, 1 } },
} },
{ "1,threads:2;0,gpus:4", true, {
{ { "threads", 2, 1 } },
} },
{ "1,_:1", true, {
{ { "_", 1, 1 } },
} },
{ "1,a:1", true, {
{ { "a", 1, 1 } },
} },
{ "2", true, {
{},
{},
} },
{ "1;2,threads:1", true, {
{},
{ { "threads", 1, 1 } },
{ { "threads", 1, 1 } },
} },
{ "1,,threads:1", true, {
{ { "threads", 1, 1 } },
} },
{ ";1,threads:1", true, {
{ { "threads", 1, 1 } },
} },
{ "1,threads:1;", true, {
{ { "threads", 1, 1 } },
} },
{ "1,threads:1,", true, {
{ { "threads", 1, 1 } },
} },
{ "threads:1;;threads:2", true, {
{ { "threads", 1, 1 } },
{ { "threads", 2, 1 } },
} },
{ "1,", true, {
{},
} },
{ ";", true, {} },
{ "", true, {} },
{ ",", false, {} },
{ "1,0:1", false, {} },
{ "1,A:1", false, {} },
{ "1,a-b:1", false, {} },
{ "invalid", false, {} },
{ ",1,invalid:1", false, {} },
{ "1,1", false, {} },
{ "-1,invalid:1", false, {} },
{ "1,invalid:*", false, {} },
{ "1,invalid:-1", false, {} },
{ "1,invalid:-", false, {} },
{ "1,invalid:ab2", false, {} },
{ "1,invalid :2", false, {} },
{ "1, invalid:2", false, {} },
{ "1,invalid:ab", false, {} },
/* clang-format on */
};
bool TestExpectedParseResult(const ExpectedParseResult& expected)
{
std::vector<std::vector<cmCTestTestHandler::cmCTestTestResourceRequirement>>
result;
bool retval;
if ((retval = cmCTestTestHandler::ParseProcessesProperty(
expected.String, result)) != expected.ExpectedReturnValue) {
std::cout << "ParseProcessesProperty(\"" << expected.String
<< "\") returned " << retval << ", should be "
<< expected.ExpectedReturnValue << std::endl;
return false;
}
if (result != expected.ExpectedValue) {
std::cout << "ParseProcessesProperty(\"" << expected.String
<< "\") did not yield expected set of processes" << std::endl;
return false;
}
return true;
}
int testCTestProcesses(int /*unused*/, char* /*unused*/ [])
{
int retval = 0;
for (auto const& expected : expectedResults) {
if (!TestExpectedParseResult(expected)) {
retval = 1;
}
}
return retval;
}

38
Tests/RunCMake/CMakeLists.txt
View File

@ -335,6 +335,44 @@ add_RunCMake_test(no_install_prefix)
add_RunCMake_test(configure_file)
add_RunCMake_test(CTestTimeoutAfterMatch)
# cthwalloc links against CMakeLib and CTestLib, which means it can't be built
# if CMake_TEST_EXTERNAL_CMAKE is activated (the compiler might be different.)
# So, it has to be provided in the original build tree.
if(CMake_TEST_EXTERNAL_CMAKE)
set(no_package_root_path)
if(NOT CMAKE_VERSION VERSION_LESS 3.12)
set(no_package_root_path NO_PACKAGE_ROOT_PATH)
endif()
find_program(cthwalloc cthwalloc PATHS ${CMake_TEST_EXTERNAL_CMAKE}
NO_DEFAULT_PATH
${no_package_root_path}
NO_CMAKE_PATH
NO_CMAKE_ENVIRONMENT_PATH
NO_SYSTEM_ENVIRONMENT_PATH
NO_CMAKE_SYSTEM_PATH
NO_CMAKE_FIND_ROOT_PATH
)
if(cthwalloc)
add_executable(cthwalloc IMPORTED)
set_property(TARGET cthwalloc PROPERTY IMPORTED_LOCATION ${cthwalloc})
endif()
else()
add_executable(cthwalloc CTestHardwareAllocation/cthwalloc.cxx)
target_link_libraries(cthwalloc CTestLib)
target_include_directories(cthwalloc PRIVATE
${CMake_BINARY_DIR}/Source
${CMake_SOURCE_DIR}/Source
${CMake_SOURCE_DIR}/Source/CTest
)
set_property(TARGET cthwalloc PROPERTY RUNTIME_OUTPUT_DIRECTORY ${CMake_BIN_DIR})
endif()
if(TARGET cthwalloc)
add_RunCMake_test(CTestHardwareAllocation -DCTHWALLOC_COMMAND=$<TARGET_FILE:cthwalloc>)
else()
message(WARNING "Could not find or build cthwalloc")
endif()
find_package(Qt4 QUIET)
find_package(Qt5Core QUIET)
if (QT4_FOUND AND Qt5Core_FOUND AND NOT Qt5Core_VERSION VERSION_LESS 5.1.0)

1
Tests/RunCMake/CTestCommandLine/RunCMakeTest.cmake
View File

@ -258,6 +258,7 @@ function(run_ShowOnly)
add_test(ShowOnly \"${CMAKE_COMMAND}\" -E echo)
set_tests_properties(ShowOnly PROPERTIES
WILL_FAIL true
PROCESSES \"2,threads:2,gpus:4;gpus:2,threads:4\"
REQUIRED_FILES RequiredFileDoesNotExist
_BACKTRACE_TRIPLES \"file1;1;add_test;file0;;\"
)

65
Tests/RunCMake/CTestCommandLine/show-only_json-v1_check.py
View File

@ -80,6 +80,62 @@ def check_willfail_property(p):
assert p["name"] == "WILL_FAIL"
assert p["value"] == True
def check_processes_property(p):
assert is_dict(p)
assert sorted(p.keys()) == ["name", "value"]
assert is_string(p["name"])
assert is_list(p["value"])
assert p["name"] == "PROCESSES"
assert len(p["value"]) == 3
assert is_dict(p["value"][0])
assert sorted(p["value"][0].keys()) == ["requirements"]
assert is_list(p["value"][0]["requirements"])
assert len(p["value"][0]["requirements"]) == 2
assert is_dict(p["value"][0]["requirements"][0])
assert sorted(p["value"][0]["requirements"][0].keys()) == \
[".type", "slots"]
assert is_string(p["value"][0]["requirements"][0][".type"])
assert p["value"][0]["requirements"][0][".type"] == "threads"
assert is_int(p["value"][0]["requirements"][0]["slots"])
assert p["value"][0]["requirements"][0]["slots"] == 2
assert is_string(p["value"][0]["requirements"][1][".type"])
assert p["value"][0]["requirements"][1][".type"] == "gpus"
assert is_int(p["value"][0]["requirements"][1]["slots"])
assert p["value"][0]["requirements"][1]["slots"] == 4
assert is_dict(p["value"][1])
assert sorted(p["value"][1].keys()) == ["requirements"]
assert is_list(p["value"][1]["requirements"])
assert len(p["value"][1]["requirements"]) == 2
assert is_dict(p["value"][1]["requirements"][0])
assert sorted(p["value"][1]["requirements"][0].keys()) == \
[".type", "slots"]
assert is_string(p["value"][1]["requirements"][0][".type"])
assert p["value"][1]["requirements"][0][".type"] == "threads"
assert is_int(p["value"][1]["requirements"][0]["slots"])
assert p["value"][1]["requirements"][0]["slots"] == 2
assert is_string(p["value"][1]["requirements"][1][".type"])
assert p["value"][1]["requirements"][1][".type"] == "gpus"
assert is_int(p["value"][1]["requirements"][1]["slots"])
assert p["value"][1]["requirements"][1]["slots"] == 4
assert is_dict(p["value"][2])
assert sorted(p["value"][2].keys()) == ["requirements"]
assert is_list(p["value"][2]["requirements"])
assert len(p["value"][2]["requirements"]) == 2
assert is_dict(p["value"][2]["requirements"][0])
assert sorted(p["value"][2]["requirements"][0].keys()) == \
[".type", "slots"]
assert is_string(p["value"][2]["requirements"][0][".type"])
assert p["value"][2]["requirements"][0][".type"] == "gpus"
assert is_int(p["value"][2]["requirements"][0]["slots"])
assert p["value"][2]["requirements"][0]["slots"] == 2
assert is_string(p["value"][2]["requirements"][1][".type"])
assert p["value"][2]["requirements"][1][".type"] == "threads"
assert is_int(p["value"][2]["requirements"][1]["slots"])
assert p["value"][2]["requirements"][1]["slots"] == 4
def check_workingdir_property(p):
assert is_dict(p)
assert sorted(p.keys()) == ["name", "value"]
@ -90,10 +146,11 @@ def check_workingdir_property(p):
def check_properties(p):
assert is_list(p)
assert len(p) == 3
check_reqfiles_property(p[0])
check_willfail_property(p[1])
check_workingdir_property(p[2])
assert len(p) == 4
check_processes_property(p[0])
check_reqfiles_property(p[1])
check_willfail_property(p[2])
check_workingdir_property(p[3])
def check_tests(t):
assert is_list(t)

9
Tests/RunCMake/CTestHardwareAllocation/CMakeLists.txt.in Normal file
View File

@ -0,0 +1,9 @@
cmake_minimum_required(VERSION 3.15)
set(CASE_NAME "@CASE_NAME@")
if(CASE_NAME MATCHES "^(.*)-ctest-s")
set(projname "${CMAKE_MATCH_1}")
project(${projname} NONE)
include(CTest)
include("@RunCMake_SOURCE_DIR@/HardwareCommon.cmake")
include("@RunCMake_SOURCE_DIR@/${projname}.cmake")
endif()

23
Tests/RunCMake/CTestHardwareAllocation/HardwareCommon.cmake Normal file
View File

@ -0,0 +1,23 @@
function(setup_hardware_tests)
if(CTEST_HARDWARE_ALLOC_ENABLED)
add_test(NAME HardwareSetup COMMAND "${CMAKE_COMMAND}" -E remove -f "${CMAKE_BINARY_DIR}/cthwalloc.log")
endif()
endfunction()
function(add_hardware_test name sleep_time proc)
if(CTEST_HARDWARE_ALLOC_ENABLED)
add_test(NAME "${name}" COMMAND "${CTHWALLOC_COMMAND}" write "${CMAKE_BINARY_DIR}/cthwalloc.log" "${name}" "${sleep_time}" "${proc}")
set_property(TEST "${name}" PROPERTY DEPENDS HardwareSetup)
else()
add_test(NAME "${name}" COMMAND "${CTHWALLOC_COMMAND}" write "${CMAKE_BINARY_DIR}/cthwalloc.log" "${name}" "${sleep_time}")
endif()
set_property(TEST "${name}" PROPERTY PROCESSES "${proc}")
list(APPEND HARDWARE_TESTS "${name}")
set(HARDWARE_TESTS "${HARDWARE_TESTS}" PARENT_SCOPE)
endfunction()
function(cleanup_hardware_tests)
if(CTEST_HARDWARE_ALLOC_ENABLED)
file(WRITE "${CMAKE_BINARY_DIR}/hwtests.txt" "${HARDWARE_TESTS}")
endif()
endfunction()

167
Tests/RunCMake/CTestHardwareAllocation/RunCMakeTest.cmake Normal file
View File

@ -0,0 +1,167 @@
include(RunCMake)
include(RunCTest)
###############################################################################
# Test cthwalloc itself - we want to make sure it's not just rubber-stamping
# the test results
###############################################################################
function(cthwalloc_verify_log expected_contents)
if(NOT EXISTS "${RunCMake_TEST_BINARY_DIR}/cthwalloc.log")
string(APPEND RunCMake_TEST_FAILED "Log file was not written\n")
set(RunCMake_TEST_FAILED "${RunCMake_TEST_FAILED}" PARENT_SCOPE)
return()
endif()
file(READ "${RunCMake_TEST_BINARY_DIR}/cthwalloc.log" actual_contents)
if(NOT actual_contents STREQUAL expected_contents)
string(APPEND RunCMake_TEST_FAILED "Actual log did not match expected log\n")
set(RunCMake_TEST_FAILED "${RunCMake_TEST_FAILED}" PARENT_SCOPE)
endif()
endfunction()
function(run_cthwalloc_write_proc name proc)
file(REMOVE "${RunCMake_BINARY_DIR}/${name}-build/cthwalloc.log")
run_cthwalloc_write_proc_nodel("${name}" "${proc}" "${ARGN}")
endfunction()
function(run_cthwalloc_write_proc_nodel name proc)
string(REPLACE ";" "\\;" proc "${proc}")
run_cmake_command(${name} "${CMAKE_COMMAND}" -E env "${ARGN}" "${CTHWALLOC_COMMAND}" write "${RunCMake_BINARY_DIR}/${name}-build/cthwalloc.log" "${name}" 0 "${proc}")
endfunction()
function(run_cthwalloc_write_noproc name)
run_cmake_command(${name} "${CMAKE_COMMAND}" -E env "${ARGN}" "${CTHWALLOC_COMMAND}" write "${RunCMake_BINARY_DIR}/${name}-build/cthwalloc.log" "${name}" 0)
endfunction()
function(run_cthwalloc_verify name tests)
string(REPLACE ";" "\\;" tests "${tests}")
run_cmake_command(${name} "${CTHWALLOC_COMMAND}" verify "${RunCMake_SOURCE_DIR}/${name}.log" "${CMAKE_CURRENT_LIST_DIR}/hwspec.json" "${tests}")
endfunction()
unset(ENV{CTEST_PROCESS_COUNT})
set(RunCMake_TEST_NO_CLEAN 1)
file(REMOVE_RECURSE "${RunCMake_BINARY_DIR}/cthwalloc-write-proc-good1-build")
file(MAKE_DIRECTORY "${RunCMake_BINARY_DIR}/cthwalloc-write-proc-good1-build")
file(WRITE "${RunCMake_BINARY_DIR}/cthwalloc-write-proc-good1-build/cthwalloc.log"
[[begin test1
alloc widgets 0 1
dealloc widgets 0 1
end test1
]])
run_cthwalloc_write_proc_nodel(cthwalloc-write-proc-good1 "1,widgets:2,transmogrifiers:1;2,widgets:1,widgets:2"
CTEST_PROCESS_COUNT=3
CTEST_PROCESS_0=widgets,transmogrifiers
CTEST_PROCESS_0_WIDGETS=id:0,slots:2
CTEST_PROCESS_0_TRANSMOGRIFIERS=id:calvin,slots:1
CTEST_PROCESS_1=widgets
"CTEST_PROCESS_1_WIDGETS=id:0,slots:1\\;id:2,slots:2"
CTEST_PROCESS_2=widgets
"CTEST_PROCESS_2_WIDGETS=id:0,slots:1\\;id:2,slots:2"
)
set(RunCMake_TEST_NO_CLEAN 0)
run_cthwalloc_write_proc(cthwalloc-write-proc-good2 "widgets:8"
CTEST_PROCESS_COUNT=1
CTEST_PROCESS_0=widgets
CTEST_PROCESS_0_WIDGETS=id:3,slots:8
)
run_cthwalloc_write_proc(cthwalloc-write-proc-nocount "widgets:8")
run_cthwalloc_write_proc(cthwalloc-write-proc-badcount "widgets:8"
CTEST_PROCESS_COUNT=2
)
run_cthwalloc_write_proc(cthwalloc-write-proc-nores "widgets:8"
CTEST_PROCESS_COUNT=1
)
run_cthwalloc_write_proc(cthwalloc-write-proc-badres "widgets:8"
CTEST_PROCESS_COUNT=1
CTEST_PROCESS_0=widgets,transmogrifiers
)
run_cthwalloc_write_proc(cthwalloc-write-proc-nowidgets "widgets:8"
CTEST_PROCESS_COUNT=1
CTEST_PROCESS_0=widgets
)
run_cthwalloc_write_proc(cthwalloc-write-proc-badwidgets1 "widgets:8"
CTEST_PROCESS_COUNT=1
CTEST_PROCESS_0=widgets
CTEST_PROCESS_0_WIDGETS=
)
run_cthwalloc_write_proc(cthwalloc-write-proc-badwidgets2 "widgets:8"
CTEST_PROCESS_COUNT=1
CTEST_PROCESS_0=widgets
"CTEST_PROCESS_0_WIDGETS=id:3,slots:8\\;id:0,slots:1"
)
run_cthwalloc_write_proc(cthwalloc-write-proc-badwidgets3 "widgets:8"
CTEST_PROCESS_COUNT=1
CTEST_PROCESS_0=widgets
CTEST_PROCESS_0_WIDGETS=id:3,slots:7
)
run_cthwalloc_write_proc(cthwalloc-write-proc-badwidgets4 "widgets:8"
CTEST_PROCESS_COUNT=1
CTEST_PROCESS_0=widgets
CTEST_PROCESS_0_WIDGETS=invalid
)
run_cthwalloc_write_proc(cthwalloc-write-proc-badwidgets5 "widgets:2,widgets:2"
CTEST_PROCESS_COUNT=1
CTEST_PROCESS_0=widgets
"CTEST_PROCESS_0_WIDGETS=id:0,slots:2\\;id:0,slots:1"
)
run_cthwalloc_write_proc(cthwalloc-write-proc-badwidgets6 "widgets:2"
CTEST_PROCESS_COUNT=1
CTEST_PROCESS_0=widgets
"CTEST_PROCESS_0_WIDGETS=id:0,slots:2\\;id:0,slots:1"
)
run_cthwalloc_write_proc(cthwalloc-write-proc-badwidgets7 "widgets:2,widgets:2"
CTEST_PROCESS_COUNT=1
CTEST_PROCESS_0=widgets
CTEST_PROCESS_0_WIDGETS=id:0,slots:2
)
run_cthwalloc_write_noproc(cthwalloc-write-noproc-good1)
run_cthwalloc_write_noproc(cthwalloc-write-noproc-count
CTEST_PROCESS_COUNT=1
)
run_cthwalloc_verify(cthwalloc-verify-good1 "test1;test2")
run_cthwalloc_verify(cthwalloc-verify-good2 "")
run_cthwalloc_verify(cthwalloc-verify-nolog "")
run_cthwalloc_verify(cthwalloc-verify-nores "")
run_cthwalloc_verify(cthwalloc-verify-noid "")
run_cthwalloc_verify(cthwalloc-verify-notenough "")
run_cthwalloc_verify(cthwalloc-verify-baddealloc "")
run_cthwalloc_verify(cthwalloc-verify-leak "")
run_cthwalloc_verify(cthwalloc-verify-badtest1 "")
run_cthwalloc_verify(cthwalloc-verify-badtest2 "test1")
run_cthwalloc_verify(cthwalloc-verify-badtest3 "test1")
run_cthwalloc_verify(cthwalloc-verify-badtest4 "test1")
run_cthwalloc_verify(cthwalloc-verify-badtest5 "test1")
run_cthwalloc_verify(cthwalloc-verify-nobegin "test1")
run_cthwalloc_verify(cthwalloc-verify-noend "test1")
###############################################################################
# Now test the hardware allocation feature of CTest
###############################################################################
function(run_ctest_hardware name parallel random)
run_ctest("${name}-ctest-s-hw" "-DCTEST_HARDWARE_ALLOC_ENABLED=1" "-DCTHWALLOC_COMMAND=${CTHWALLOC_COMMAND}" "-DCTEST_PARALLEL=${parallel}" "-DCTEST_RANDOM=${random}")
run_ctest("${name}-ctest-s-nohw" "-DCTEST_HARDWARE_ALLOC_ENABLED=0" "-DCTHWALLOC_COMMAND=${CTHWALLOC_COMMAND}" "-DCTEST_PARALLEL=${parallel}" "-DCTEST_RANDOM=${random}")
endfunction()
function(verify_ctest_hardware)
file(READ "${RunCMake_TEST_BINARY_DIR}/hwtests.txt" hwtests)
execute_process(COMMAND "${CTHWALLOC_COMMAND}" verify "${RunCMake_TEST_BINARY_DIR}/cthwalloc.log" "${CMAKE_CURRENT_LIST_DIR}/hwspec.json" "${hwtests}"
OUTPUT_VARIABLE output ERROR_QUIET RESULT_VARIABLE result)
if(result)
string(APPEND RunCMake_TEST_FAILED "${output}")
set(RunCMake_TEST_FAILED "${RunCMake_TEST_FAILED}" PARENT_SCOPE)
endif()
endfunction()
run_ctest_hardware(lotsoftests 10 1)
run_ctest_hardware(checkfree1 2 0)
run_ctest_hardware(checkfree2 1 0)
run_ctest_hardware(notenough1 1 0)
run_ctest_hardware(notenough2 1 0)
run_ctest_hardware(ensure_parallel 2 0)
set(ENV{CTEST_PROCESS_COUNT} 2)
run_ctest_hardware(process_count 1 0)
unset(ENV{CTEST_PROCESS_COUNT})

1
Tests/RunCMake/CTestHardwareAllocation/checkfree1-ctest-s-hw-check.cmake Normal file
View File

@ -0,0 +1 @@
verify_ctest_hardware()

7
Tests/RunCMake/CTestHardwareAllocation/checkfree1.cmake Normal file
View File

@ -0,0 +1,7 @@
setup_hardware_tests()
add_hardware_test(Test1 1 "widgets:8")
add_hardware_test(Test2 1 "fluxcapacitors:50;fluxcapacitors:50,widgets:8")
add_hardware_test(Test3 1 "fluxcapacitors:121")
cleanup_hardware_tests()

1
Tests/RunCMake/CTestHardwareAllocation/checkfree2-ctest-s-hw-check.cmake Normal file
View File

@ -0,0 +1 @@
verify_ctest_hardware()

8
Tests/RunCMake/CTestHardwareAllocation/checkfree2.cmake Normal file
View File

@ -0,0 +1,8 @@
setup_hardware_tests()
# This test is an attack on the hardware scheduling algorithm. It has been
# carefully crafted to fool the algorithm into thinking there isn't sufficient
# hardware for it.
add_hardware_test(Test1 1 "widgets:2;4,widgets:4")
cleanup_hardware_tests()

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-baddealloc-result.txt Normal file
View File

@ -0,0 +1 @@
1

2
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-baddealloc.log Normal file
View File

@ -0,0 +1,2 @@
alloc widgets 0 1
dealloc widgets 0 2

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-badtest1-result.txt Normal file
View File

@ -0,0 +1 @@
1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-badtest1.log Normal file
View File

@ -0,0 +1 @@
begin test1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-badtest2-result.txt Normal file
View File

@ -0,0 +1 @@
1

2
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-badtest2.log Normal file
View File

@ -0,0 +1,2 @@
begin test1
begin test1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-badtest3-result.txt Normal file
View File

@ -0,0 +1 @@
1

3
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-badtest3.log Normal file
View File

@ -0,0 +1,3 @@
begin test1
end test1
begin test1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-badtest4-result.txt Normal file
View File

@ -0,0 +1 @@
1

3
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-badtest4.log Normal file
View File

@ -0,0 +1,3 @@
begin test1
end test1
end test1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-badtest5-result.txt Normal file
View File

@ -0,0 +1 @@
1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-badtest5.log Normal file
View File

@ -0,0 +1 @@
end test1

14
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-good1.log Normal file
View File

@ -0,0 +1,14 @@
begin test1
alloc widgets 3 4
alloc widgets 4 1
alloc transmogrifiers calvin 2
alloc fluxcapacitors outatime 121
begin test2
alloc widgets 3 4
dealloc widgets 3 4
dealloc widgets 4 1
dealloc transmogrifiers calvin 2
dealloc fluxcapacitors outatime 121
end test1
dealloc widgets 3 4
end test2

0
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-good2.log Normal file
View File

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-leak-result.txt Normal file
View File

@ -0,0 +1 @@
1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-leak.log Normal file
View File

@ -0,0 +1 @@
alloc widgets 0 1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-nobegin-result.txt Normal file
View File

@ -0,0 +1 @@
1

0
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-nobegin.log Normal file
View File

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-noend-result.txt Normal file
View File

@ -0,0 +1 @@
1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-noend.log Normal file
View File

@ -0,0 +1 @@
begin test1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-noid-result.txt Normal file
View File

@ -0,0 +1 @@
1

2
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-noid.log Normal file
View File

@ -0,0 +1,2 @@
alloc fluxcapacitors train 1
dealloc fluxcapacitors train 1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-nolog-result.txt Normal file
View File

@ -0,0 +1 @@
1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-nores-result.txt Normal file
View File

@ -0,0 +1 @@
1

2
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-nores.log Normal file
View File

@ -0,0 +1,2 @@
alloc gpus 0 1
dealloc gpus 0 1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-notenough-result.txt Normal file
View File

@ -0,0 +1 @@
1

2
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-verify-notenough.log Normal file
View File

@ -0,0 +1,2 @@
alloc widgets 0 8
dealloc widgets 0 8

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-write-noproc-count-result.txt Normal file
View File

@ -0,0 +1 @@
1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-write-proc-badcount-result.txt Normal file
View File

@ -0,0 +1 @@
1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-write-proc-badres-result.txt Normal file
View File

@ -0,0 +1 @@
1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-write-proc-badwidgets1-result.txt Normal file
View File

@ -0,0 +1 @@
1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-write-proc-badwidgets2-result.txt Normal file
View File

@ -0,0 +1 @@
1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-write-proc-badwidgets3-result.txt Normal file
View File

@ -0,0 +1 @@
1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-write-proc-badwidgets4-result.txt Normal file
View File

@ -0,0 +1 @@
1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-write-proc-badwidgets5-result.txt Normal file
View File

@ -0,0 +1 @@
1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-write-proc-badwidgets6-result.txt Normal file
View File

@ -0,0 +1 @@
1

1
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-write-proc-badwidgets7-result.txt Normal file
View File

@ -0,0 +1 @@
1

20
Tests/RunCMake/CTestHardwareAllocation/cthwalloc-write-proc-good1-check.cmake Normal file
View File

@ -0,0 +1,20 @@
cthwalloc_verify_log(
[[begin test1
alloc widgets 0 1
dealloc widgets 0 1
end test1
begin cthwalloc-write-proc-good1
alloc transmogrifiers calvin 1
alloc widgets 0 2
alloc widgets 0 1
alloc widgets 2 2
alloc widgets 0 1
alloc widgets 2 2
dealloc transmogrifiers calvin 1
dealloc widgets 0 2
dealloc widgets 0 1
dealloc widgets 2 2
dealloc widgets 0 1
dealloc widgets 2 2
end cthwalloc-write-proc-good1
]])

Some files were not shown because too many files have changed in this diff Show More