Commit Graph

33 Commits

Author SHA1 Message Date
Mark Young 88c33c9e63 Changes to additive env var based on code review 2022-03-04 15:38:46 -07:00
Mark Young 759d5b4c94 Add "additive" environment variables
Add "additive" environment variables for adding additional layer
paths or driver JSON files instead of replacing default ones.

Also, rename VK_ICD_FILENAMES to VK_DRIVER_FILES since we're trying
to remove references to ICDs because software driver implementations
of Vulkan aren't ICDs (but continue to support the old name as well).

Added documentation around these changes to reflect the new name and
the new variables.
2022-03-04 15:38:46 -07:00
Charles Giessen 02609516e8 Guard building of live_verification tests
Don't build live verification tests by default. These are meant as manual tests for
developers to run to diagnose issues or verify correctness on a real system.
2022-02-15 16:51:46 -07:00
Charles Giessen 5f7588d136 Add live_verification tests for runnign with real drivers
Some things need to be verified against real drivers. The supporting code in test_util and
test_environment make it easy to write application code which works against real drivers and
layers. This allows quickly verifying loader behavior out in the wild.
2022-02-15 13:56:21 -07:00
Charles Giessen f2d1b6e536 Update tests function loading implementation
Remove the need for macros and use well crafted helper loading functions which
can figure out the type to convert to based on the type being assigned.
2022-02-15 10:48:53 -07:00
Charles Giessen ef44cdd106 Clarify behavior of override_paths + VK_LAYER_PATH
If a meta layer which contains `override_paths` is active, all of the paths in
VK_LAYER_PATH are ignored when searchign for explicit layers. This may be changed
changed in the future but for now is better tested and documented.
2022-02-11 14:16:05 -07:00
Mark Young 12d70af385 loader log error if a requested layer not loaded
This should throw an error message if a requested layer didn't load.
This could happen if the JSON file is present, but only a library
exists on the system for the wrong target (like 32 vs 64-bit).

If it did fail because of being the wrong bit-type, we report an info
message only, unless it was directly requested by the application.

Modify the improper bit-depth layer loading test so that it is now 4
tests:
 - Verify bad explicit layer generates VK_ERROR_LAYER_NOT_PRESENT error
 - Verify bad implicit layer generates only an info in the loader output
 - Test case with both a bad implicit and explicit layer with INFO logging
   we should see both messages.
 - Test case with both a bad implicit and explicit layer with ERROR logging
   we should see only the explicit failure message.

Report info failure for ICD wrong bit-type as well.

Finally, fixed a small compilation warning for disabled code.
2022-01-31 14:35:07 -07:00
Charles Giessen 9850f714d2 Add test for implicit layer with 0.1 API version
The loader disables implicit layers if their API version isn't the same or
greater than the application. Make sure this still works even with pre 1.0
versions.
2022-01-28 16:24:27 -07:00
Charles Giessen 99e5c4c542 Add TestLayerDetails helper struct
Increasingly it is common to need more control over what the framework
environment does when adding layers. While only a refactor, this change
makes it easier to extend the functionality by adding data to the
TestLayerDetails struct.

The code to check whether the lib_path specified in add_layer_impl was
improved by using the `.stem()` functionality of the fs::path abstraction.

size_t was replaced with uint32_t where the compiler gave warnings.
2022-01-25 12:32:56 -07:00
Charles Giessen 6e5c65d627 Simplify test framework FolderManager write function
Condense the overloaded write function from taking either an icd manifest or a layer manifest
and only take a string. This reduces the amount of copy-paste code.
2022-01-25 12:32:56 -07:00
Mark Young c77cf061a5 Fix a few code review snippets
Messages weren't clear on some errors, and rework the library ending
code based on @charles-lunarg suggestions
2022-01-07 16:56:38 -07:00
Mark Young 08677c866f Add device extension layer tests
Need to make sure the loader behaves appropriately for layers which report
one or more device extensions.
2022-01-07 16:56:38 -07:00
Charles Giessen 7cc308cd92 Tests to use unique layer names
While the logic to copy TestLayer binaries was present, it didn't rename them
to a unique string to prevent collisions when using the same binary.
2021-12-10 17:08:23 -07:00
Charles Giessen ade4922867 Refactor TestLayer framework interface
By adding builder style usage to most of the layer interface, it is easier to
create and use layers in tests. This commit refactors the layer interface and
updates most of the tests to use this style.
2021-12-10 17:08:23 -07:00
Charles Giessen 9661a8c019 Refactor tests FrameworkEnvironment class
Move all the behavior found in child classes of FrameworkEnvironment to the
FrameworkEnvironment itself, allowing easier composibility of behavior. The
original idea was to allow special case behavior for specific test cases but
this resulted in a lot of code duplication and difficulty combining different
framework behavior, such as adding a fake ICD and setting VK_ICD_FILENAMES at
the same time.
2021-12-10 17:08:23 -07:00
Charles Giessen a92b915ee5 Add Builder Pattern helpers to test framework
Allows quick creation of builders in test code.
2021-12-10 17:08:23 -07:00
Charles Giessen dfe050353a Fix InstWrapper not checking instance creation
The CheckCreate function was not correctly checking that the VkResult matched what was
expected. The old code required the client to wrap the call in EXPECT_TRUE().
2021-12-10 17:08:23 -07:00
Charles Giessen 9068ac580c Fix test framework leaking registry entries
Previously, Death tests would unknowningly create multiple registry overrides
and subsequently leak them when the death test dies. This is fixed by only
calling override once in main and setting an environment variable to prevent
subprocesses (death tests) from calling override again.

This change also moves the override setup from the shim setup to main, since
it had no depencies on the shim library itself.
2021-12-02 22:15:32 -07:00
Eleni Maria Stea 91d68b7849 loader: include FreeBSD in supported platforms
Added FreeBSD in supported platforms in vk_loader_platform.h, and tests,
defined BSD_VISIBLE and included some necessary header files for
Vulkan-Loader to compile on FreeBSD.

v2: did the same in loader/loader.c (Charles Giessen)
v3: did the same in the tests (Charles Giessen)
v4: applied this downstream patch from Jan Beich because there's no /proc
on FreeBSD: https://github.com/freebsd/freebsd-ports/blob/7caf52e37ba2174dc50a7440d63eec70d7d58ebc/graphics/vulkan-loader/files/patch-loader_vk__loader__platform.h (Jan Beich)
v4: rebased to master and modified tests/loader_regression_tests.cpp,
tests/loader_testing_main.cpp
v5: moved the definition of __BSD_VISIBLE 1 (required for alloca to
work) in CMakeLists.txt from header files (Charles Giessen)
v6: replaced ${CMAKE_SYSTEM_NAME} with CMAKE_SYSTEM_NAME in
loader/CMakeLists.txt to match the style of previous checks (Jan Beich)
v7: replaced add_compile_definitions with target_compile_definitions to
prevent the macro from being applied unnecessarily (Charles Giessen)
2021-10-20 10:29:26 -06:00
Charles Giessen 936a4c7a57 test: Update WrapObjects test to new framework
Moved over the WrapObjects test logic into the new framework, removing all
the old tests infrastructure that is no longer needed.
2021-10-07 10:00:08 -06:00
Charles Giessen 5f96256afb test: Add test which calls GetPDProps inside CreateInstance
This test makes sure that any layer calling GetPDProps inside of CreateInstance
succeeds and doesn't crash.
2021-09-10 13:59:45 -06:00
Charles Giessen fea09c7783 test: Add callback to layers create instance and device
This allows some code to be written inside the test which then gets run
during the call to vkCreateInstance and vkCreateDevice.
2021-09-10 13:59:45 -06:00
Charles Giessen d76cef07ce test: Improved Test Wrapper interface
Made InstanceWrapper and DeviceWrapper simpler to use by moving them to test_environment
then making the Create() member functions and doing the gtest assertions in the function
itself.

Additionally:
Removed the `detail` namespace in the test_environment.
Renamed get_new_test_icd/layer to just `reset_icd/layer`. This is clearer about what it
is doing and makes the interface more expressive with code such as
`env->reset_icd().SetMinInterfaceVersion(5);`
2021-09-06 18:19:14 -06:00
Charles Giessen 5d448542f2 test: Add test for PR #639
This test tries to recreate an out of memory condition which causes
a binary of the wrong type to be confused with a valid binary, which
causes VK_ERROR_INCOMPATIBLE_DRIVER to be returned instead of VK_SUCCESS
(Or OOM if the case may be)
2021-08-16 15:18:43 -06:00
Charles Giessen 601c0f9f9a test: Add linux x64 and x86 for wrong binaries
Test only looks for ICD binaries.
2021-08-16 11:38:17 -06:00
Charles Giessen bad1aabe90 test: Clear TestICDs before each test
Calling `dlclose` doesn't necessarily mean that the static variables will
be reset nor the libraries destructor gets called. This caused cascading
errors in tests on linux with GCC. By always manually resetting the ICD
or Layer before each test, we prevent any funny business from occuring.
2021-08-04 09:57:16 -06:00
Charles Giessen b1bd463f49 test: Add noexcept in test_environment 2021-08-04 09:57:16 -06:00
Charles Giessen 266e0783a7 ci: Enable regression tests on MacOS
Change-Id: I85b08bfd2914e03840a027240470e2596415f4dc
2021-06-22 12:03:05 -06:00
Charles Giessen 820fc25d1d test: Fix framework to work on MacOS
Main issue was getting function redireciton to work, dyld-interposer accomplishes that.
It required making the shim-library a dylib, but otherwise only required minor changes.

Change-Id: I7b7e2c86cc0c0b082f58241a7e146b74af72e792
2021-06-22 12:03:05 -06:00
Charles Giessen db6662c3f8 test: Refactor tests that use VK_ICD_FILENAMES
Change-Id: I34117fd500f7110669ae19d32ff1e1be920238a9
2021-06-07 14:07:38 -07:00
Charles Giessen f6fab4352d test: Control process elevation on Windows
Since the loader only uses environment variables if the process is
not running with elevated priviledges, the test harness needs to be
able to control that aspect to ensure consistency as CI systems may
run the tests in an 'elevated' context.

Change-Id: I8b15a1ecd4ab4960049ba198af8199943e872bba
2021-06-07 14:07:38 -07:00
Charles Giessen b9f4dd01e2 test: Delete files properly on windows
Move windows specific error reporting to test_util.h
This allows for its use anywhere its required.

Change-Id: I14a0da85dc0d3f4e651b99aa005469b47d98d940
2021-06-07 14:07:38 -07:00
Charles Giessen 6176c3a9d2 test: Add framework test_environment
The test environment is the code which glues the platform shim,
TestICD, and TestLayers together so that test code can be easily
written. The main components tests use are the SingleICDShim and
MultipleICDShim classes. Both take care of copying the ICD binaries,
setting up the manifests, and redirecting the platform specific
systems to point at those manifests.

Change-Id: I7cbb76c9555fe13b323173c3838cbb3616b77a18
2021-05-27 16:59:28 -06:00