Many svn-based buildbots seem to be getting stuck continually
in tree conflicts due to the output of pyc files. I'm disabling
these as a temporary measure in an attempt to get everything
stable again.
I'll try to remove this code once I understand the problem
better.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313698 91177308-0d34-0410-b5e6-96231b3b80d8
Since the path a user specifies to the llvm-lit script might be
different than the source tree they built from (since they could
be behind different symlinks), we need to use realpath to make
sure that path comparisons work as expected.
Even better would be to use a custom dictionary comparison with
actual file equivalence comparison semantics, but this is the
least friction to unbreak things for now.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313594 91177308-0d34-0410-b5e6-96231b3b80d8
NFC intended here, this only updates clang and lld's lit configs
to use some helper functionality in the lit.llvm submodule.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313579 91177308-0d34-0410-b5e6-96231b3b80d8
It doesn't make sense to me why these bots are failing as the
traceback does not agree with the source code. It's possible
something is stale or there is some other mysterious error,
but in any case hopefully this fixes it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313469 91177308-0d34-0410-b5e6-96231b3b80d8
There were some issues surrounding Py2 / Py3 compatibility, but
I've now tested with both Py2 and Py3 and everything seems to
work.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313467 91177308-0d34-0410-b5e6-96231b3b80d8
A few tests were manually constructing a LitConfig object, since
I added a new argument to it this was triggering some failures
I didn't detect. `ninja check-lit` passes now.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313461 91177308-0d34-0410-b5e6-96231b3b80d8
This is helpful for debugging test failures since it removes
the multiprocessing pool from the picture. This will obviously
slow down the test suite by a few orders of magnitude, so it
should only be used for debugging specific failures.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313460 91177308-0d34-0410-b5e6-96231b3b80d8
It looks like this is going to be non-trivial to get working
in both Py2 and Py3, so for now I'm reverting until I have time
to fully test it under Python 3.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313429 91177308-0d34-0410-b5e6-96231b3b80d8
This was reverted alongside the revert of the lit/llvm-lit refactor,
but now that that has re-landed, I'm relanding this as well.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313426 91177308-0d34-0410-b5e6-96231b3b80d8
This is a resubmission of r313270. It broke standalone builds of
compiler-rt because we were not correctly generating the llvm-lit
script in the standalone build directory.
The fixes incorporated here attempt to find llvm/utils/llvm-lit
from the source tree returned by llvm-config. If present, it
will generate llvm-lit into the output directory. Regardless,
the user can specify -DLLVM_EXTERNAL_LIT to point to a specific
lit.py on their file system. This supports the use case of
someone installing lit via a package manager. If it cannot find
a source tree, and -DLLVM_EXTERNAL_LIT is either unspecified or
invalid, then we print a warning that tests will not be able
to run.
Differential Revision: https://reviews.llvm.org/D37756
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313407 91177308-0d34-0410-b5e6-96231b3b80d8
This patch is still breaking several multi-stage compiler-rt bots.
I already know what the fix is, but I want to get the bots green
for now and then try re-applying in the morning.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313335 91177308-0d34-0410-b5e6-96231b3b80d8
This is breaking due to some changes I forgot to merge in, so I'm
temporarily reverting them until I can re-test that this works.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313328 91177308-0d34-0410-b5e6-96231b3b80d8
To further reduce duplicate code, this patch introduces a module
that configs can simply import and get access to a lot of useful
functionality such as setting up paths, adding features that are
useful across all projects, and other utility-type functions.
For now this only updates llvm's suite to use this new library,
but subsequent patches will update other projects.
Differential Revision: https://reviews.llvm.org/D37778
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313325 91177308-0d34-0410-b5e6-96231b3b80d8
This patch simplifies LLVM's lit infrastructure by enforcing an ordering
that a site config is always run before a source-tree config.
A significant amount of the complexity from lit config files arises from
the fact that inside of a source-tree config file, we don't yet know if
the site config has been run. However it is *always* required to run
a site config first, because it passes various variables down through
CMake that the main config depends on. As a result, every config
file has to do a bunch of magic to try to reverse-engineer the location
of the site config file if they detect (heuristically) that the site
config file has not yet been run.
This patch solves the problem by emitting a mapping from source tree
config file to binary tree site config file in llvm-lit.py. Then, during
discovery when we find a config file, we check to see if we have a
target mapping for it, and if so we use that instead.
This mechanism is generic enough that it does not affect external users
of lit. They will just not have a config mapping defined, and everything
will work as normal.
On the other hand, for us it allows us to make many simplifications:
* We are guaranteed that a site config will be executed first
* Inside of a main config, we no longer have to assume that attributes
might not be present and use getattr everywhere.
* We no longer have to pass parameters such as --param llvm_site_config=<path>
on the command line.
* It is future-proof, meaning you don't have to edit llvm-lit.in to add
support for new projects.
* All of the duplicated logic of trying various fallback mechanisms of
finding a site config from the main config are now gone.
One potentially noteworthy thing that was required to implement this
change is that whereas the ninja check targets previously used the first
method to spawn lit, they now use the second. In particular, you can no
longer run lit.py against the source tree while specifying the various
`foo_site_config=<path>` parameters. Instead, you need to run
llvm-lit.py.
Differential Revision: https://reviews.llvm.org/D37756
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313270 91177308-0d34-0410-b5e6-96231b3b80d8
Use os.path.normpath instead of realpath to collapse '..' and '.' path
components. Use realpath when caching search results about a path for
good measure.
I considered rigging up a test involving symlinks for this, but I doubt
I can check a symlink into SVN. The test would have to conditionally
create a symlink at runtime if the host OS supports it. This sounds too
fragile and complicated to me to be worth it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312254 91177308-0d34-0410-b5e6-96231b3b80d8
This preserves symlinks in paths, so that someone can symlink more tests
into a larger test suite. For example, debuginfo-tests is currently
designed to be checked out into clang/test. With this change, it can be
symlinked into place instead, which works better with the monorepo.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@312250 91177308-0d34-0410-b5e6-96231b3b80d8
It was marked as unsupported on Windows in r311230 because on some Win10
machines it failed or caused hang. The problem was that on these machines
system bash (C:\Windows\System32\bash.exe) was used which requires paths to be
passed like '/mnt/c/path/to/my/script' instead of 'C:\path\to\my\script'.
TODO: we should make lit detect if system bash is used instead of msys and set
appropriate path format.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@311558 91177308-0d34-0410-b5e6-96231b3b80d8
Multi-configuration CMake generators such as those for Visual Studio or Xcode do not
specify a build config at configure time, but let the user choose at build
time. In these cases binaries go into build/${Configuration}/bin rather than
build/bin. Prior to this commit, check-lit would fail when using multi-configuration
generators as it did not know how to resolve ${Configuration} in order
to find tools such as FileCheck. This commit teaches it to resolve
llvm_tools_dir within lit using the value specified with --param
build_mode.
Differential Revision: https://reviews.llvm.org/D36263
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309967 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This is an alternative solution to running the lit test suite on bots
without polluting the source directory. Each input test suite gets an
auto-generated site config in the build directory that points back to
the test input source directory.
This adds some cmake comlexity, but now we don't need to remove and
re-copy the test input directory before every test.
Reviewers: delcypher, modocache
Subscribers: mgorny, llvm-commits
Differential Revision: https://reviews.llvm.org/D36026
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309602 91177308-0d34-0410-b5e6-96231b3b80d8
Traceback (most recent call last):
File "llvm/utils/lit/tests/Inputs/shtest-format/external_shell/write-bad-encoding.py", line 5, in <module>
sys.stdout.write(b"a line with bad encoding: \xc2.")
sys.stdout.write doesn't accept bytes but sys.stdout.buffer.write accepts.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309473 91177308-0d34-0410-b5e6-96231b3b80d8
This should fix googletest-format test failures on the clang modules
buildbots, which have a stale copy of the OneTest script in the build
directory.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309432 91177308-0d34-0410-b5e6-96231b3b80d8
When using win32 cmd.exe, turn off command echoing at the beginning of
the script (@echo off).
Replace a bash shell script with a python script for the
fail_with_bad_encoding test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309399 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
The technique of directly calling subprocess.Popen on a python script
doesn't work on Windows. The executable path of the command must refer
to a valid win32 executable.
Instead, rename all the python scripts masquerading as gtest executables
to have .py extensions, so we can easily detect then and call the python
executable for them. Do this on Linux as well as Windows for
consistency.
The test suite directory names also come out in lower-case on Windows.
We can consider removing that in a later patch. This change just updates
the FileCheck lines to match on Windows.
Fixes PR33933
Reviewers: modocache, mgorny
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D35909
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309347 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Normally Python converts all newline characters, Windows or Unix,
to Unix newlines when opening a file. However, lit opens files in
binary mode, which does not perform this conversion. As a result,
trailing Windows newlines are not stripped from test input, which
caused a failure in the TestRunner unit test:
```
FAIL: test_custom (__main__.TestIntegratedTestKeywordParser)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Users\bgesiak\src\llvm\llvm\utils\lit\tests\unit\TestRunner.py", line 109, in test_custom
self.assertItemsEqual(value, ['a', 'b', 'c'])
AssertionError: Element counts were not equal:
First has 1, Second has 0: 'c\r'
First has 0, Second has 1: 'c'
```
Fix the discrepancy in behavior across the two platforms by
manually stripping Windows newlines before yielding each line in
the test file.
Reviewers: echristo, beanz, ddunbar, delcypher, rnk
Reviewed By: rnk
Subscribers: mehdi_amini, llvm-commits
Differential Revision: https://reviews.llvm.org/D27746
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309312 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
An expectation in `utils/lit/tests/Inputs/shtest-shell/redirects.txt`
expects that first a string printed to stdout is seen, and then a
string printed to stderr. Add `flush()` calls to ensure that stdout is
printed before stderr, as expected.
Reviewers: rnk, mgorny, jroelofs
Reviewed By: rnk
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D35947
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309292 91177308-0d34-0410-b5e6-96231b3b80d8
Previously these tests would use the same Output directory leading to
flaky non-deterministic failures.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309227 91177308-0d34-0410-b5e6-96231b3b80d8
Rewrite the write-to-stderr.sh and write-to-stdout-and-stderr.sh shell
scripts as python scripts and call python on them.
Fixes PR33940
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309200 91177308-0d34-0410-b5e6-96231b3b80d8
This passes locally for me, which fails the overall lit test suite. I
can't debug a passing test, but I will try to help debug the test when
we get some failing logs.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309190 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
rL257221 attempted to run lit's own test suite continuously, but that
commit was reverted because lit's test suite does not pass on Windows.
Because lit's tests do not run continuously, they often regress.
In order to un-revert rL257221, mark lit tests that fail as XFAIL for
Windows platforms.
Test Plan:
On a Windows development environment, follow the instructions in
utils/lit/README.txt to run lit's test suite:
```
utils/lit/lit.py \
--path /path/to/your/llvm/build/bin \
utils/lit/tests
```
Verify that the test suite is run and a successful exit code is
returned.
Reviewers: mgorny, rnk, delcypher, beanz
Reviewed By: rnk
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D35879
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309123 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Whereas rL299560 and rL309071 call `parallelism_groups.items()`, under the
assumption that `parallelism_groups` is a `dict` type, the default
parameter for that attribute is a `list`. Change the default to a
`dict` for type correctness.
This regression in the unit tests would have been caught if the
unit tests were being run continously. It also would have been caught
if the lit project used a Python type checker such as `mypy`.
Test Plan:
As per the instructions in `utils/lit/README.txt`, run the lit unit
test suite:
```
utils/lit/lit.py \
--path /path/to/your/llvm/build/bin \
utils/lit/tests
```
Verify that the test `lit :: unit/TestRunner.py` fails before applying this
patch, but passes once this patch is applied.
Reviewers: mgorny, rnk, rafael
Reviewed By: mgorny
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D35878
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309122 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This reverts rL306623, which removed `FileBasedTest`, an abstract base class,
but did not also remove the usages of that class in the lit unit tests.
The revert fixes four test failures in the lit unit test suite.
Test plan:
As per the instructions in `utils/lit/README.txt`, run the lit unit
test suite:
```
utils/lit/lit.py \
--path /path/to/your/llvm/build/bin \
utils/lit/tests
```
Verify that the following tests fail before applying this patch, and
pass once the patch is applied:
```
lit :: test-data.py
lit :: test-output.py
lit :: xunit-output.py
```
In addition, run `check-llvm` to make sure the existing LLVM test suite
executes normally.
Reviewers: george.karpenkov, mgorny, dlj
Reviewed By: mgorny
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D35877
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309120 91177308-0d34-0410-b5e6-96231b3b80d8
