The pixel shader is built with rslc's builder instead of printed as text
and parsed again, on Vulkan, D3D11, D3D12 and Metal. Built and printed
stages compile to the same SPIR-V and DXBC words: 15,619 random pixel
uids and host configs, with and without subgroup reduction, and every
uid the text refuses the builder refuses too.
The builder's flags say what the text's header said of the device:
subgroup operations, an inverted gl_HelperInvocation and subgroup
operations broken by discard. Vulkan and Metal pass them; Metal builds
at MSL 2.3, as its text compiles.
OpenGL still prints the stage as text until rslc prints framebuffer
fetch, conservative depth and subgroups in GLSL.
A generator can now build its stage through rslc's builder instead of
writing GLSL for the backend to parse: AbstractGfx::CreateShaderFromBuild
takes a build function, and each backend asks rslc for its own
destination from the built stage - SPIR-V for Vulkan, DXBC at the
device's feature level for D3D11 and D3D12, GLSL or GLSL ES at the
context's version for OpenGL, MSL for Metal. VideoCommon/ShaderBuild
carries the per-backend resource layout the shader headers' macros gave,
and builds a stage in the worker's shader arena, again from the start
where the arena has to grow.
The framebuffer utility stages move first: the screen quad, texture
copy, clear and EFB poke vertex stages, the passthrough geometry stages,
and the texture copy, color and multisample resolve pixel stages. Each
compiles to the same SPIR-V and DXBC words as its GLSL did; the text
writers are gone.
rslc prints OpenGL's GLSL without bindings, so the OpenGL backend binds
blocks, samplers, attributes and outputs by name for every program.
A thread kept one freed shader text block, but a pipeline has its
stages' texts alive at once, so all but the first took a new block:
each thread now keeps up to four, handing out the smallest that fits.
rslc's result blobs come from the same kind of per-thread store, through
its allocator hooks, and ShaderBinary gives them back with its release
function. Without the disk shader cache a Vulkan shader gives its SPIR-V
back once its module is made, for the thread's next compile.
Play allocations on the compile workers: 226 to 172, 833 KB to 44 KB.
Each thread that compiles hands rslc one block as its arena, doubled
and compiled again where a shader needed more, so the parse, the IR and
the emit allocate nothing once the block has grown; only the blob the
shader keeps is allocated. The blocks go as the backend shuts down. C89.
Allocations after boot over the corpus fall from 1702 to 1356, and the
bytes allocated from about 28 MB to 2 MB.
The stats build counts the cycles generating GX shader text and in
rslc, and the words rslc emits. The gate holds the words against the
baseline and the parent, and reports each state's cycles in text, rslc,
the workers and the driver.
A backend is handed the ShaderCode: Vulkan and D3D write their header
into the room kept in front of the text and compile the two as one span,
and GL hands the header, any prefix and the text to the driver as they
are and keeps the text's block for its link messages.
SPIR-V and DXBC stay in the block rslc returns instead of being copied
into vectors, and the disk cache is lent the code to write. Code read
back from the cache is copied once into a block of its own.
Every file the core touches goes through libretro-common's filestream,
path and dirent layers, taken from RetroArch: vfs_hybrid opens local
paths with libretro-common's own implementation and hands URI paths
(and, on Android, anything a local open cannot reach) to the
frontend's VFS. File::IOFile and File::DirectIOFile sit on RFILE, and
FileUtil, FileSearch and the directory walks on path_stat, path_mkdir
and retro_opendir. Dolphin's fopen, POSIX and Win32 file code,
std::filesystem and std::fstream are gone, with the dual VFS paths and
the core option that chose between them.
Files opened for reading alone are memory-mapped where the platform
can: reads copy straight out of the page cache, DirectIOFile's offset
reads on one mapped object are thread safe, and GetMappedPtr(), View()
and Prefetch() lend and warm without a copy. Text files are read a
line at a time through File::LineReader, lent from the mapping or a
window refilled a block at a time, never a byte at a time; JSON is
parsed straight from the mapping. Ini files and the cheat file are
built in memory and written once, an ini only when it changed and
then atomically.
The vendored rslc prints subgroup operations, so a device that runs them
in the pixel stage gets the subgroup header back: each subgroup reduces
its bounding box and one invocation writes it. Vulkan offers it where
the device has the basic, arithmetic, ballot and shuffle operations in
the fragment stage; Metal on Mac2 and Apple7 GPUs, whose
simd_is_helper_thread is probed at init for the inverted answer some AMD
drivers give. The driver bug that breaks the reduction with discard
still keeps it off.
Nothing ever made one: every backend was handed a null includer, and the
OpenGL backend's #include resolver returned the source untouched. The
parameter goes from CreateShaderFromSource and everything beneath it,
and ShaderCompileUtils with it.
The compute header comes back for the decoders, and compute shaders and
GPU texture decoding are reported as supported once more. The core
option still decides whether textures are decoded on the GPU.
Vendors rslc from RetroArch's libretro-common (gfx/shaders, built with
RSLC_EXTENDED) and has the Vulkan backend hand every vertex, geometry
and pixel stage to it for SPIR-V: no glslang on that path, no global
lock, any number of compiles in parallel. D3D and Metal still go
through glslang and SPIRV-Cross until rslc writes their code.
A stage rslc refuses is logged and written, with the error, to the
dump directory as bad_<stage>_Vulkan_<n>.txt.
rslc has no compute stage, storage buffers, atomics or subgroup
operations yet, so for now the backend does not offer what needs them:
GPU texture decoding (decoding stays on the CPU), the bounding box,
the dynamic vertex loader and the vertex-shader line and point
expansion (the geometry shader expands them).
While the NV extension is totally fine, the KHR extension should be able to support more hardware.
For NVIDIA, the hardware either supports both or neither, it just needs a driver from the last two years.
For AMD, the drivers from late 2022-12 seems to bring support for the KHR extension.
For Intel, the KHR is also supported for some years.
Fixes a crash that could occur if the static constructor function for
the MainSettings.cpp TU happened to run before the variables in
Common/Version.cpp are initialised. (This is known as the static
initialisation order fiasco.)
By using wrapper functions, those variables are now guaranteed to be
constructed on first use.
SPDX standardizes how source code conveys its copyright and licensing
information. See https://spdx.github.io/spdx-spec/1-rationale/ . SPDX
tags are adopted in many large projects, including things like the Linux
kernel.
Now that we utilize C++17, we can simply return an optional containing
the code instead of using an out variable and a boolean result,
essentially combining them into one.
This provides a much more straightforward interface.
Greatly simplifies the overall interface when it comes to compiling
shaders. Also allows getting rid of a std::string overload of the same
name. Now std::string and const char* both go through the same function.
This reverts commit d23fd17e1a.
Dynamic sampler indexing is broken in VK_NV_glsl as of 385.41. The
performance gap doesn't seem to be as wide with the updated driver, so
to save maintaining two code paths, it's easier to just drop the
extension support completely.
LibretroAdmin