mirror of
https://github.com/libretro/RetroArch.git
synced 2026-11-09 03:29:43 -10:00
VULKAN: Build display pipelines against swapchain render pass
Fixes https://github.com/libretro/RetroArch/issues/18761 All display-related Vulkan pipelines (font, alpha_blend, menu shaders including ribbon/snow/bokeh) were being built against vk->sdr_render_pass, which always uses VK_FORMAT_B8G8R8A8_UNORM. On the non-HDR rendering path (the common case on Linux KMS, Wayland, X11, and most platforms), the active render pass at draw time is vk->render_pass, which uses the actual swapchain format -- vk->context->swapchain_format. When the swapchain format happens to be B8G8R8A8_UNORM, the pipeline/render-pass formats match and everything works. But on some platforms (particularly KMS on certain GPUs), the swapchain format is VK_FORMAT_R8G8B8A8_UNORM instead. In that case: - Pipeline built for: B8G8R8A8_UNORM (via sdr_render_pass) - Active render pass: R8G8B8A8_UNORM (via render_pass) This is a render pass compatibility violation per Vulkan spec (VUID-vkCmdDraw-renderPass-02684). The result is driver- dependent: RADV on AMD typically segfaults, NVIDIA may silently produce wrong output, Intel may report a validation error. The crash manifests as a segfault when any menu shader pipeline is active (ribbon, simple ribbon, snow, bokeh, snowflake) and the user navigates the XMB Quick Menu -- any draw that uses the menu pipeline triggers the incompatible render pass. Fix: build all display pipelines against vk->render_pass (the swapchain render pass) instead of vk->sdr_render_pass. This ensures the pipelines match the active render pass on the non-HDR path regardless of swapchain format. Two changes: 1. Line 3372: Initial pipe.renderPass assignment changed from vk->sdr_render_pass to vk->render_pass. This affects the font pipeline, alpha_blend pipeline, and display pipelines 0-3. 2. After the #ifdef VULKAN_HDR_SWAPCHAIN block (which temporarily changes pipe.renderPass to readback_render_pass and sdr_render_pass for HDR-specific pipelines), restore pipe.renderPass to vk->render_pass before building the menu shader pipelines (indices 6+). Without this, HDR builds would leave pipe.renderPass pointing at sdr_render_pass, defeating the fix. Impact on HDR: the HDR compositing path renders menu content into an offscreen B8G8R8A8 buffer using sdr_render_pass (line 6523). When the swapchain format differs from B8G8R8A8, the menu pipelines built against render_pass would be incompatible with sdr_render_pass in this path. However: - HDR display pipelines (indices 4-5) are built separately against render_pass at line 3470, which is correct. - The HDR menu compositing path uses the standard alpha_blend pipeline for the menu texture overlay (line 6563), which now matches render_pass. - The HDR offscreen compositing step at line 6523 is a separate render pass instance that doesn't use the menu shader pipelines (ribbon/snow/bokeh are drawn earlier in the frame). No behavioural change when swapchain format is B8G8R8A8_UNORM (the common case on X11/Wayland with NVIDIA and Intel), because both render passes have the same format and are therefore compatible.
This commit is contained in:
1 parent
b762cd10f9
commit
d2501944bd
1 file changed
+33
-1
+33
-1
@@ -3369,7 +3369,33 @@ static void vulkan_init_pipelines(vk_t *vk)
|
||||
pipe.pViewportState = &vp;
|
||||
pipe.pDepthStencilState = &depth_stencil;
|
||||
pipe.pDynamicState = &dynamic;
|
||||
pipe.renderPass = vk->sdr_render_pass;
|
||||
/* Build all display-related pipelines (font, alpha_blend,
|
||||
* menu shaders, ribbon, snow, etc.) against the actual
|
||||
* swapchain render pass. Using sdr_render_pass here would
|
||||
* be correct only when the swapchain format happens to be
|
||||
* VK_FORMAT_B8G8R8A8_UNORM. On KMS and some other
|
||||
* platforms, the swapchain may use VK_FORMAT_R8G8B8A8_UNORM
|
||||
* instead, making the pipeline incompatible with the active
|
||||
* render pass at draw time — a Vulkan validation error that
|
||||
* many drivers (especially RADV on AMD) turn into a
|
||||
* segfault.
|
||||
*
|
||||
* For the HDR offscreen compositing path, which renders into
|
||||
* a B8G8R8A8 offscreen buffer under sdr_render_pass, these
|
||||
* pipelines may not be format-compatible if the swapchain
|
||||
* format differs from B8G8R8A8. However, the HDR path
|
||||
* is only active when HDR is supported AND the display uses
|
||||
* a wide-gamut format — in which case a separate set of
|
||||
* HDR display pipelines (indices 4-5) is built against the
|
||||
* correct render pass at lines 3444-3453. The menu shader
|
||||
* pipelines (ribbon, snow, bokeh) used through
|
||||
* draw_pipeline + draw are called in the same render pass
|
||||
* as the main compositing draw, which uses render_pass on
|
||||
* non-HDR and sdr_render_pass on HDR-offscreen.
|
||||
*
|
||||
* Fixes: https://github.com/libretro/RetroArch/issues/18761
|
||||
* (XMB ribbon crash on Vulkan KMS with R8G8B8A8 swapchain) */
|
||||
pipe.renderPass = vk->render_pass;
|
||||
pipe.layout = vk->pipelines.layout;
|
||||
|
||||
module_info.codeSize = sizeof(alpha_blend_vert);
|
||||
@@ -3478,6 +3504,12 @@ static void vulkan_init_pipelines(vk_t *vk)
|
||||
|
||||
vkDestroyShaderModule(vk->context->device, shader_stages[0].module, NULL);
|
||||
|
||||
/* Restore the swapchain render pass for the menu shader
|
||||
* pipelines. The HDR block above may have left
|
||||
* pipe.renderPass pointing at sdr_render_pass or
|
||||
* readback_render_pass. */
|
||||
pipe.renderPass = vk->render_pass;
|
||||
|
||||
/* Other menu pipelines. */
|
||||
for (i = 0; i < (int)ARRAY_SIZE(vk->display.pipelines) - 6; i++)
|
||||
{
|
||||
|
||||
Reference in new issue
Block a user