My #73541 added lines to
`llvm/utils/lit/tests/Inputs/testrunner-custom-parsers/test.txt` so what
was previously on line 7 is now on line 12.
Before:
28412d1800/llvm/utils/lit/tests/Inputs/testrunner-custom-parsers/test.txt (L7-L8)
After:
6fb7c2d713/llvm/utils/lit/tests/Inputs/testrunner-custom-parsers/test.txt (L12-L13)
This didn't show up in the PR checks, but caused a buildbot failure
after merging, https://lab.llvm.org/buildbot/#/builders/139/builds/54597
:
```
# | ======================================================================
# | FAIL: test_commands (__main__.TestIntegratedTestKeywordParser)
# | ----------------------------------------------------------------------
# | Traceback (most recent call last):
# | File "/home/buildbot/buildbot-root/llvm-clang-x86_64-sie-ubuntu-fast/build/utils/lit/tests/unit/TestRunner.py", line 135, in test_commands
# | self.assertEqual(value[1].command.strip(), "%dbg(MY_RUN: at line 7) foo bar")
# | AssertionError: '%dbg(MY_RUN: at line 12) foo bar' != '%dbg(MY_RUN: at line 7) foo bar'
# | - %dbg(MY_RUN: at line 12) foo bar
# | ? ^^
# | + %dbg(MY_RUN: at line 7) foo bar
# | ? ^
```
Apologies for the build break 🙀
This patch adds "nice-to-have" feature in lit.
it prints the total number of discovered tests at the beginning. It is
covenient to see the total number of tests and avoid scrolling up to the
beginning of log.
Further, this patch also prints %ge of tests.
This patch fixes tests pointed by previous attempt of landing this
patch.
Reviewed By: RoboTux, jdenny-ornl
Co-authored-by: Madhur A <madhura@nvidia.com>
Without this patch, the functions `executeScriptInternal` and thus
`runOnce` in `llvm/utils/lit/lit/TestRunner.py` return either a tuple like
`(out, err, exitCode, timeoutInfo)` or a `lit.Test.Result` object. They
return the latter only when there's a lit internal shell parse error in
a RUN line. In my opinion, a more straight-forward way to handle
exceptional cases like that is to use python exceptions.
For that purpose, this patch introduces `ScriptFatal`. Thus, this patch
changes `executeScriptInternal` to always either return the tuple or
raise the `ScriptFatal` exception. It updates `runOnce` and
`libcxx/utils/libcxx/test/format.py` to catch the exception rather than
check for the special return type.
This patch also changes `runOnce` to convert the exception to a
`Test.UNRESOLVED` result instead of `TEST.FAIL`. The former is the
proper result for such a malformed test, for which a rerun (given an
`ALLOW_RETRIES:`) serves no purpose. There are at least two benefits
from this change. First, `_runShTest` no longer has to specially and
cryptically handle this case to avoid unnecessary reruns. Second, an
`XFAIL:` directive no longer hides such a failure [as we saw
previously](https://reviews.llvm.org/D154987#4501125).
To facilitate the `_runShTest` change, this patch inserts the internal
shell parse error diagnostic into the format of the test's normal debug
output rather than suppressing the latter entirely. That change is also
important for [D154987](https://reviews.llvm.org/D154987), which
proposes to reuse `ScriptFatal` for python compile errors in PYTHON
lines or in `config.prologue`. In that case, the diagnostic might follow
debugging output from the test's previous RUN or PYTHON lines, so
suppressing the normal debug output would lose information.
This patch adds "nice-to-have" feature in lit.
it prints the total number of discovered tests at the beginning. It is
covenient to see the total number of tests and avoid scrolling up to the
beginning of log.
Further, this patch also prints %ge of tests.
Reviewed By: RoboTux, jdenny-ornl
Co-authored-by: Madhur A <madhura@nvidia.com>
This patch addresses the missed review comment from PR #67063. It
renames LIT flag "--disable-gtest-sharding" to "--no-gtest-sharding"
and corrects the code style issue.
[unittest] Add lit option to allow disabling sharding in unittest
By default, googletest based unit tests uses sharding to speed up the
testing. However, when these unit tests are run through wrapper program
on a remote platform with large round trip time, the sharding will increase
the time cost dramatically. This patch adds a LLVM LIT option
"--disable-gtest-sharding" to allow sharding on gtest based unittest to
be disabled.
In PR #65242 (landed as 9e739fdb85ac672f3e25e971d96e71823e07ebda), I
claimed that RUN lines cannot contain newlines. Actually, they can after
substitution expansion. More generally, a lit config file can define
substitutions or preamble commands containing newlines. While both of
those cases seem unlikely in practice,
[D154987](https://reviews.llvm.org/D154987) proposes PYTHON directives
where it seems very likely.
Regardless of the use case, without this patch, such newlines break
expansion of `%dbg(RUN: at line N)`, and the fix is simple.
This patch and D154984 were discussed in
<https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839>.
Motivation
----------
D154984 removes the "Script:" section that lit prints along with a
test's output, and it makes -v and -a imply -vv. For example, after
D154984, the "Script:" section below is never shown, but -v is enough
to produce the execution trace following it:
```
Script:
--
: 'RUN: at line 1'; echo hello | FileCheck bogus.txt && echo success
--
Exit Code: 2
Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "echo" "hello"
# command output:
hello
$ "FileCheck" "bogus.txt"
# command stderr:
Could not open check file 'bogus.txt': No such file or directory
error: command failed with exit status: 2
--
```
In the D154984 review, some reviewers point out that they have been
using the "Script:" section for copying and pasting a test's shell
commands to a terminal window. The shell commands as printed in the
execution trace can be harder to copy and paste for the following
reasons:
- They drop redirections and break apart RUN lines at `&&`, `|`, etc.
- They add `$` at the start of every command, which makes it hard to
copy and paste multiple commands in bulk.
- Command stdout, stderr, etc. are interleaved with the commands and
are not clearly delineated.
- They don't always use proper shell quoting. Instead, they blindly
enclose all command-line arguments in double quotes.
Changes
-------
D154984 plus this patch converts the above example into:
```
Exit Code: 2
Command Output (stdout):
--
# RUN: at line 1
echo hello | FileCheck bogus-file.txt && echo success
# executed command: echo hello
# .---command stdout------------
# | hello
# `-----------------------------
# executed command: FileCheck bogus-file.txt
# .---command stderr------------
# | Could not open check file 'bogus-file.txt': No such file or directory
# `-----------------------------
# error: command failed with exit status: 2
--
```
Thus, this patch addresses the above issues as follows:
- The entire execution trace can be copied and pasted in bulk to a
terminal for correct execution of the RUN lines, which are printed
intact as they appeared in the original RUN lines except lit
substitutions are expanded. Everything else in the execution trace
appears in shell comments so it has no effect in a terminal.
- Each of the RUN line's commands is repeated (in shell comments) as
it executes to show (1) that the command actually executed (e.g.,
`echo success` above didn't) and (2) what stdout, stderr, non-zero
exit status, and output files are associated with the command, if
any. Shell quoting in the command is now correct and minimal but is
not necessarily the original shell quoting from the RUN line.
- The start and end of the contents of stdout, stderr, or an output
file is now delineated clearly in the trace.
To help produce some of the above output, this patch extends lit's
internal shell with a built-in `@echo` command. It's like `echo`
except lit suppresses the normal execution trace for `@echo` and just
prints its stdout directly. For now, `@echo` isn't documented for use
in lit tests.
Without this patch, libcxx's custom lit test format tries to parse the
stdout from `lit.TestRunner.executeScriptInternal` (which runs lit's
internal shell) to extract the stdout and stderr produced by shell
commands, and that parse no longer works after the above changes.
This patch makes a small adjustment to
`lit.TestRunner.executeScriptInternal` so libcxx can just request
stdout and stderr without an execution trace.
(As a minor drive-by fix that came up in testing: lit's internal `not`
command now always produces a numeric exit status and never `True`.)
Caveat
------
This patch only makes the above changes for lit's internal shell. In
most cases, we do not know how to force external shells (e.g., bash,
sh, window's `cmd`) to produce execution traces in the manner we want.
To configure a test suite to use lit's internal shell (which is
usually better for test portability than external shells anyway), add
this to the test suite's `lit.cfg` or other configuration file:
```
config.test_format = lit.formats.ShTest(execute_external=False)
```
Reviewed By: MaskRay, awarzynski
Differential Revision: https://reviews.llvm.org/D156954
This patch and D156954 were discussed in
<https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839>.
**Motivation**: -a shows output from all tests, and -v shows output
from just failed tests. Without this patch, that output from each
test includes a section called "Script:", which includes all shell
commands that lit has computed from RUN directives and will attempt to
run for that test. The effect of -vv (which also implies -v if
neither -a or -v is specified) is to extend that output with shell
commands as they are executing so you can easily see which one failed.
For example, when using lit's internal shell and -vv:
```
Script:
--
: 'RUN: at line 1'; echo hello world
: 'RUN: at line 2'; 3c40 hello world
: 'RUN: at line 3'; echo hello world
--
Exit Code: 127
Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "echo" "hello" "world"
hello world
$ ":" "RUN: at line 2"
$ "3c40" "hello" "world"
'3c40': command not found
error: command failed with exit status: 127
--
```
Notice that all shell commands that actually execute appear in the
output twice, once for "Script:" and once for -vv. Especially for
tests with many RUN directives, the result is noisy. When searching
through the output for a particular shell command, it is easy to get
lost and mistake shell commands under "Script:" for shell commands
that actually executed.
**Change**: With this patch, a test's output changes in two ways.
First, the "Script:" section is never shown. Second, omitting -vv no
longer disables printing of shell commands as they execute. That is,
-a and -v imply -vv, and so -vv is deprecated as it is just an alias
for -v.
**Secondary motivation**: We are also working to introduce a PYTHON
directive, which can appear between RUN directives. How should PYTHON
directives be represented in the "Script:" section, which has
previously been just a shell script? We could probably think of
something, but adding info about PYTHON directive execution in the -vv
trace seems more straight-forward and more useful.
(This patch also removes a confusing point in the -vv documentation:
at least when using bash as an external shell, -vv echoes commands to
the shell's stderr not stdout.)
Reviewed By: awarzynski, Endill, ldionne, MaskRay
Differential Revision: https://reviews.llvm.org/D154984
D154280 (landed in 64d19542e78a in July, 2023) implements
`--per-test-coverage` (which can also be specified via
`lit_config.per_test_coverage`). However, it has a few issues, which
the current patch addresses:
1. D154280 implements `--per-test-coverage` only for the case that lit
is configured to use an external shell. The current patch extends
the implementation to lit's internal shell.
2. In the case that lit is configured to use an external shell,
regardless of whether `--per-test-coverage` is actually specified,
D154280 causes `%dbg(RUN: at line N)` to be expanded in RUN lines
early and in a manner that is specific to sh-like shells. As a
result, later code in lit that expands it in a shell-specific
manner is useless as there's nothing left to expand. The current
patch cleans up the implementation to avoid useless code.
3. Because of issue 2, D154280 corrupts support for windows `cmd` as
an external shell (effectively comments out all RUN lines with
`:`). The current patch happens to fix that particular corruption
by addressing issue 2. However, D122569 (landed in 1041a9642ba0 in
April, 2022) had already broken support for windows `cmd` as an
external shell (discards RUN lines when expanding `%dbg(RUN: at
line N)`). The current patch does not attempt to fix that bug.
For further details, see the PR discussion of the current patch.
The current patch addresses the above issues by implementing
`--per-test-coverage` before selecting the shell (internal or
external) and by leaving `%dbg(RUN: at line N)` unexpanded there.
Thus, it is expanded later in a shell-specific manner, as before
D154280.
This patch introduces `buildPdbgCommand` into lit's implementation to
encapsulate the process of building (or rebuilding in the case of the
`--per-test-coverage` implementation) a full `%dbg(RUN: at line N)
cmd` line and asserting that the result matches `kPdbgRegex`. It also
cleans up that and all other uses of `kPdbgRegex` to operate on the
full line with `re.fullmatch` not `re.match`. This change better
reflects the intention in every case, but it is expected to be NFC
because `kPdbgRegex` ends in `.*` and thus avoids the difference
between `re.fullmatch` and `re.match`. The only caveat is that `.*`
does not match newlines, but RUN lines cannot contain newlines
currently, so this caveat currently shouldn't matter in practice.
The original `--per-test-coverage` implementation avoided accumulating
`export LLVM_PROFILE_FILE={profile}` insertions across retries (due to
`ALLOW_RETRIES`) by skipping the insertion if `%dbg(RUN: at line N)`
was not present and thus had already been expanded. However, the
current patch makes sure the insertions also happen for commands
without `%dbg(RUN: at line N)`, such as preamble commands or some
commands from other lit test formats. Thus, the current patch
implements a different mechanism to avoid accumulating those
insertions (see code comments).
For `llvm/utils/lit/tests/shtest-output-printing.py`, the executable
in `%{python}` wasn't properly shell-quoted for windows. This caused
the 127 exit code mentioned in f254bbf23374. Fix quoting and expect
exit code 1 again.
Fix shell-quoting issue in a few more file names in
`llvm/utils/lit/tests/shtest-shell.py`, missed in f254bbf23374.
Test failures seen in
<https://lab.llvm.org/buildbot/#/builders/216/builds/26436>.
Failures were seen at
<https://lab.llvm.org/buildbot/#/builders/216/builds/26431>.
All but one failure is due to different shell-quoting of file names
because they contain special characters under windows. Generalize
associated FileCheck patterns.
`llvm/utils/lit/tests/shtest-output-printing.py` fails because the
exit code was 127 instead of the expected 1. Unfortunately, this CI
config doesn't pass `-dump-input-filter=all` to FileCheck, so we
cannot see the rest of the lit execution trace. For now, generalize
the FileCheck pattern to accept any non-zero exit code to get past
this error.
This patch and D154984 were discussed in
<https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839>.
Motivation
----------
D154984 removes the "Script:" section that lit prints along with a
test's output, and it makes -v and -a imply -vv. For example, after
D154984, the "Script:" section below is never shown, but -v is enough
to produce the execution trace following it:
```
Script:
--
: 'RUN: at line 1'; echo hello | FileCheck bogus.txt && echo success
--
Exit Code: 2
Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "echo" "hello"
# command output:
hello
$ "FileCheck" "bogus.txt"
# command stderr:
Could not open check file 'bogus.txt': No such file or directory
error: command failed with exit status: 2
--
```
In the D154984 review, some reviewers point out that they have been
using the "Script:" section for copying and pasting a test's shell
commands to a terminal window. The shell commands as printed in the
execution trace can be harder to copy and paste for the following
reasons:
- They drop redirections and break apart RUN lines at `&&`, `|`, etc.
- They add `$` at the start of every command, which makes it hard to
copy and paste multiple commands in bulk.
- Command stdout, stderr, etc. are interleaved with the commands and
are not clearly delineated.
- They don't always use proper shell quoting. Instead, they blindly
enclose all command-line arguments in double quotes.
Changes
-------
D154984 plus this patch converts the above example into:
```
Exit Code: 2
Command Output (stdout):
--
# RUN: at line 1
echo hello | FileCheck bogus-file.txt && echo success
# executed command: echo hello
# .---command stdout------------
# | hello
# `-----------------------------
# executed command: FileCheck bogus-file.txt
# .---command stderr------------
# | Could not open check file 'bogus-file.txt': No such file or directory
# `-----------------------------
# error: command failed with exit status: 2
--
```
Thus, this patch addresses the above issues as follows:
- The entire execution trace can be copied and pasted in bulk to a
terminal for correct execution of the RUN lines, which are printed
intact as they appeared in the original RUN lines except lit
substitutions are expanded. Everything else in the execution trace
appears in shell comments so it has no effect in a terminal.
- Each of the RUN line's commands is repeated (in shell comments) as
it executes to show (1) that the command actually executed (e.g.,
`echo success` above didn't) and (2) what stdout, stderr, non-zero
exit status, and output files are associated with the command, if
any. Shell quoting in the command is now correct and minimal but is
not necessarily the original shell quoting from the RUN line.
- The start and end of the contents of stdout, stderr, or an output
file is now delineated clearly in the trace.
To help produce some of the above output, this patch extends lit's
internal shell with a built-in `@echo` command. It's like `echo`
except lit suppresses the normal execution trace for `@echo` and just
prints its stdout directly. For now, `@echo` isn't documented for use
in lit tests.
Without this patch, libcxx's custom lit test format tries to parse the
stdout from `lit.TestRunner.executeScriptInternal` (which runs lit's
internal shell) to extract the stdout and stderr produced by shell
commands, and that parse no longer works after the above changes.
This patch makes a small adjustment to
`lit.TestRunner.executeScriptInternal` so libcxx can just request
stdout and stderr without an execution trace.
(As a minor drive-by fix that came up in testing: lit's internal `not`
command now always produces a numeric exit status and never `True`.)
Caveat
------
This patch only makes the above changes for lit's internal shell. In
most cases, we do not know how to force external shells (e.g., bash,
sh, window's `cmd`) to produce execution traces in the manner we want.
To configure a test suite to use lit's internal shell (which is
usually better for test portability than external shells anyway), add
this to the test suite's `lit.cfg` or other configuration file:
```
config.test_format = lit.formats.ShTest(execute_external=False)
```
Reviewed By: MaskRay, awarzynski
Differential Revision: https://reviews.llvm.org/D156954
This patch and D156954 were discussed in
<https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839>.
**Motivation**: -a shows output from all tests, and -v shows output
from just failed tests. Without this patch, that output from each
test includes a section called "Script:", which includes all shell
commands that lit has computed from RUN directives and will attempt to
run for that test. The effect of -vv (which also implies -v if
neither -a or -v is specified) is to extend that output with shell
commands as they are executing so you can easily see which one failed.
For example, when using lit's internal shell and -vv:
```
Script:
--
: 'RUN: at line 1'; echo hello world
: 'RUN: at line 2'; 3c40 hello world
: 'RUN: at line 3'; echo hello world
--
Exit Code: 127
Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "echo" "hello" "world"
hello world
$ ":" "RUN: at line 2"
$ "3c40" "hello" "world"
'3c40': command not found
error: command failed with exit status: 127
--
```
Notice that all shell commands that actually execute appear in the
output twice, once for "Script:" and once for -vv. Especially for
tests with many RUN directives, the result is noisy. When searching
through the output for a particular shell command, it is easy to get
lost and mistake shell commands under "Script:" for shell commands
that actually executed.
**Change**: With this patch, a test's output changes in two ways.
First, the "Script:" section is never shown. Second, omitting -vv no
longer disables printing of shell commands as they execute. That is,
-a and -v imply -vv, and so -vv is deprecated as it is just an alias
for -v.
**Secondary motivation**: We are also working to introduce a PYTHON
directive, which can appear between RUN directives. How should PYTHON
directives be represented in the "Script:" section, which has
previously been just a shell script? We could probably think of
something, but adding info about PYTHON directive execution in the -vv
trace seems more straight-forward and more useful.
(This patch also removes a confusing point in the -vv documentation:
at least when using bash as an external shell, -vv echoes commands to
the shell's stderr not stdout.)
Reviewed By: awarzynski, Endill, ldionne, MaskRay
Differential Revision: https://reviews.llvm.org/D154984
Running lit tests on Windows can fail because its use of
`os.path.realpath` expands substitute drives, which are used to keep
paths short and avoid hitting MAX_PATH limitations.
Changes lit logic to:
Use `os.path.abspath` on Windows, where `MAX_PATH` is a concern that we
can work around using substitute drives, which `os.path.realpath` would
resolve.
Use `os.path.realpath` on Unix, where the current directory always has
symlinks resolved, so it is impossible to preserve symlinks in the
presence of relative paths, and so we must make sure that all code paths
use real paths.
Also updates clang's `FileManager::getCanonicalName` and `ExtractAPI`
code to avoid resolving substitute drives (i.e. resolving to a path
under a different root).
How tested: built with `-DLLVM_ENABLE_PROJECTS=clang` and built `check-all` on both Windows
Differential Revision: https://reviews.llvm.org/D154130
Reviewed By: @benlangmuir
Patch by Tristan Labelle <tristan@thebrowser.company>!
This patch is the first part of https://llvm.org/OpenProjects.html#llvm_patch_coverage.
We have first define a new variable LLVM_TEST_COVERAGE which when set, pass --per-test-coverage option to
llvm-lit which will help in setting a unique value to LLVM_PROFILE_FILE for each RUN. So for example
coverage data for test case llvm/test/Analysis/AliasSet/memtransfer.ll will be emitted as
build/test/Analysis/AliasSet/memtransfer0.profraw
Reviewed By: hnrklssn
Differential Revision: https://reviews.llvm.org/D154280
This patch is the first part of https://llvm.org/OpenProjects.html#llvm_patch_coverage.
We have first define a new variable LLVM_TEST_COVERAGE which when set, pass --emit-coverage option to
llvm-lit which will help in setting a unique value to LLVM_PROFILE_FILE for each RUN. So for example
coverage data for test case llvm/test/Analysis/AliasSet/memtransfer.ll will be emitted as
build/test/Analysis/AliasSet/memtransfer.profraw
Reviewed By: hnrklssn
Differential Revision: https://reviews.llvm.org/D154280
This option had originally been added in D83069 to allow disabling the
check that something is going to get run at all when a specific test name
is used on the command-line. Since we now use getTestsForPath() (from D151664)
to get the tests to run for a specific path, we don't need a specific check
for this anymore -- Lit will produce the same complaint it would produce if
you provided a directory with no tests.
If one needs to run a specific test on the command-line and the Lit
configuration would normally not include that test, the configuration
should be set up as a "standalone" configuration or it should be fixed
to allow for that test to be found (i.e. probably fix the allowed test
suffixes).
Differential Revision: https://reviews.llvm.org/D153967
Lit TestFormat classes already needed to implement the getTestsInDirectory
method. However, some test formats may want to expand a single test path
to multiple Lit tests, for example in the case of a test that actually
generates other Lit tests.
To accommodate that, this commit adds the getTestsForPath method to
TestFormat. This method can be used to turn a single path in a Lit
test suite into a list of tests associated to that path.
Differential Revision: https://reviews.llvm.org/D151664
This reverts commit c1cf459cbd79cc7d6ca834390649fb9185a4b237.
Reverting to permit time to explore the underlying issue. This change
regressed the clang-PPC64-AIX and m68k-linux-cross builders.
Differential Revision: https://reviews.llvm.org/D153138
Reviewed By: compnerd
lit.py uses os.path.realpath on file paths. Somewhere between Python 3.7
and 3.9, os.path.realpath was updated to resolve substitute drives on
Windows (subst S: C:\Long\Path\To\My\Code). This is a problem because it
prevents using substitute drives to work around MAX_PATH path length
limitations on Windows.
We run into this while building & testing, the Swift compiler on
Windows, which uses a substitute drive in CI to shorten the workspace
directory. cmake builds without resolving the substitute drive and can
apply its logic to avoid output files exceeding MAX_PATH. However, when
running tests, lit.py's use of os.path.realpath will resolve the
substitute drive (with newer Python versions), resulting in some paths
being longer than MAX_PATH, which cause all kinds of failures (for
example rd in tests fails, or link.exe fails, etc).
How tested: Ran check-all, and lit tests, saw no failures
```
> ninja -C build check-all
Testing Time: 262.63s
Skipped : 24
Unsupported : 2074
Passed : 51812
Expectedly Failed: 167
> python utils\lit\lit.py --path ..\build\bin utils\lit\tests
Testing Time: 12.17s
Unsupported: 6
Passed : 47
```
Patch by Tristan Labelle!
Differential Revision: https://reviews.llvm.org/D152709
Reviewed By: rnk, compnerd
This is a follow-up to b71edfaa4ec3c998aadb35255ce2f60bba2940b0
since I forgot the lit.local.cfg files in that one.
Reformatting is done with `black`.
If you end up having problems merging this commit because you
have made changes to a python file, the best way to handle that
is to run git checkout --ours <yourfile> and then reformat it
with black.
If you run into any problems, post to discourse about it and
we will try to help.
RFC Thread below:
https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Reviewed By: barannikov88, kwk
Differential Revision: https://reviews.llvm.org/D150762
This is flicky for buildbots (for example, https://lab.llvm.org/buildbot/#/builders/188/builds/21560)
because of the short timeout. The test coverage is not reduced because
the stdout check is performed in other googtest-* unit tests.
We were seeing an intermittent local test failure of
utils\lit\tests\test-output.py in which the elapsed time output was
being given in scientific notation. Python automatically represents
small floating-point values in scientific notation so I have altered
these tests regex to capture output in that format.
Differential Revision: https://reviews.llvm.org/D136469
Due to CMake mis-configurations, some gtest binaries may be added to the test
list more than once. This patch makes lit avoid such cases and issues a
warning when it happens.
