Files
fs2open.github.com/code/ddsutils/ddsutils.cpp
T
b26b9cf743 Add Vulkan rendering backend, with HDR10 output and raytraced shadows (#7554)
* Shadow overhaul round 2: PCSS, dynamic cascades, cockpit/instanced shadows

Squashed snapshot of lafiel/shadow_overhaul_2's continuation work
(d3922aa87..010f9d2d9), which has no separate upstream PR of its own.

* Add Vulkan rendering backend

Squashed snapshot of the Vulkan backend as vendored into this branch
(ce2d1e85f..6ce539463, authored by Mara van der Laan with build/CI
fixes by Taylor Richards). This work has its own upstream PR #7553
(notimaginative:vulkan-pr-new) and is not otherwise original to true-hdr.

* Add true HDR output, RT shadows, and PCSS shadows for Vulkan

Squashed snapshot of true-hdr's own integration work: true HDR
framebuffer output, wiring the shadow overhaul into the Vulkan
backend (raytraced + cascaded shadow maps), logging refactors, log
filter UI, and CI/build fixes.

* refactor: update Vulkan logging calls to use `nprintf` for scoped logging

* fix: correct shadow cascade count in Vulkan logging calls

* fix: fall back to PATH-resolved ccache when CCACHE_PATH is stale

CI hardcodes /usr/local/bin/ccache, but macOS runners on Apple Silicon
hosts install Homebrew (and ccache) under /opt/homebrew instead,
causing all four Mac configs to fail with "No such file or directory".

* Fix vulkan texture readback to correctly handle transparent pipelines

* Put the pixel swizzles into a util file

* OOB safety check

* Fix fog handling

* Fix MSAA in Vulkan

* fix: enable SMAA for vulkan

* adapt the post-processing AA shaders to hdr

* fix clang-tidy warnings: concatenate nested namespaces, static methods

pixel_swizzle.{h,cpp}: collapse `namespace graphics { namespace util {`
into `namespace graphics::util {` (modernize-concat-nested-namespaces).

VulkanState.h/VulkanDraw.h: getClearColor()/setClearColor() only touch
gr_screen and other globals, no instance state, so make them static
(readability-convert-member-functions-to-static) and drop the
top-level const on getClearColor()'s return type
(readability-const-return-type).

* fix clang-tidy warning

* Extend raytraced shadows to all directional lights

RT shadows previously only shadowed the first directional light in the
scene, even though multi-sun missions push more than one directional
light into the engine. The TLAS raytraced shadows query against is
already light-agnostic, so this extends both the deferred lighting
pass and the forward MODEL shader to shadow every directional light
(capped by a new Max Raytraced Shadow Lights option, configurable via
the in-game Options menu or the Lab's Scene rendering options).

Cascaded shadow maps are untouched: they're built for a single light
only, so the deferred pass still limits itself to one shadowed
directional light whenever CSM (not raytracing) is the active method.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_016DA6JHJ3twz2oHGQcZ1Udc

* refactor: improve shader flag handling and render target finalization in Vulkan backend

- Consolidate shader flag setup logic in VulkanPostProcessingLighting to simplify configuration.
- Fix dead field usage in VulkanDraw by ensuring correct image view selection for environment maps.
- Add mip chain regeneration in VulkanRenderer::endRenderTarget to ensure correct mip levels for cubemaps.

* fix: ensure correct mip chain initialization in cubemap rendering

- Add mip chain generation to VulkanDraw for cubemaps with multiple mip levels.
- Fix potential issue with uninitialized mip levels above level 0 in deferred-f.sdr sampling paths.

* fix: don't crash on startup when environment locale is unavailable

std::locale("") pulls in the process's environment locale categories
(LC_CTYPE, LC_MONETARY, LC_PAPER, etc.), and throws std::runtime_error
if any of them names a locale not installed on the host (e.g.
mismatched regional-format overrides). Since SCP_default_locale is
constructed during static initialization, before main() and before any
of our own exception handling exists, an uncaught throw here aborted
the whole process before the game even started. Fall back to the
classic "C" locale instead, which is sufficient since this locale is
only ever used for ASCII case-folding (SCP_toupper/tolower/isdigit).

Co-Authored-By: Claude Sonnet 5 <[email protected]>

* `feat: add raytraced shadows for local lights and optimize TLAS management`

* Make the Commandline authoritative for -res and -vulkan / -opengl

* address BMagnu's comments on the original PR

* update prebuilt version hash to 62c89ef6

* refactor: standardize graphics API handling and document Vulkan limitations in qtFRED

- Updated `AboutDialogModel.cpp` to use `GraphicsAPI` enum for better clarity.
- Ensured qtFRED falls back to OpenGL when Vulkan is requested.
- Added detailed documentation in `qtfred/README.md` regarding Vulkan backend limitations and requirements.

* chore: include cmdline library in main and mission management modules

* docs: add known issue for blank render viewport under Wayland in README.md

* fix CI issues

* chore: add `GR_DEBUG_SCOPE` markers across Vulkan post-processing and rendering paths for improved debugging

* feat: improve Vulkan HDR pipeline with separate encode paths for SDR and HDR10

- Added an HDR10-specific output encode pass in `VulkanRenderer` and `VulkanPostProcessor` that applies PQ/BT.2020 transfer.
- Introduced an optimized SDR fallback path with direct blit support for devices that can skip shader draws.
- Updated `m_sceneLdrCompressed` to provide [0,1]-clamped proxies for HDR edge detection.
- Refactored `copyImageToImage` and added `blitImageToImage` to allow format conversions and optimized transfers.
- Replaced `hdr_mode` with explicit HDR10 flags and data structures for clearer state handling.

* refactor: encapsulate SMAA processing and refactor VulkanPostProcessor

- Moved SMAA-specific logic into `VulkanSMAA` class for better separation of concerns.
- Simplified `initSMAA`, `shutdownSMAA`, and `executeSMAA` logic by leveraging `VulkanLDR` and `PostProcessContext` abstractions.
- Improved initialization and cleanup code with clearer resource lifecycle management.
- Updated debug markers and messages to reference `VulkanSMAA`.
- Enhanced flexibility of fullscreen triangle rendering API with additional parameters (`sampleCount`, `bindGlobalSet`).

* Update Vulkan porting credits in AboutDialogModel

* fix CI issues again

* fix: correct Y-flip in viewport setup for Vulkan light volumes

- Adjusted viewport height to negative value to account for G-buffer render pass's negative Y-axis convention.
- Ensured proper alignment of point and tube light volumes with actual light positions.

* feat: add user-adjustable gamma correction to SDR and HDR10 output paths

- Added gamma correction to HDR10 path, applying user settings before PQ/BT.2020 encode.
- Replaced the SDR blit-only path with an encode pass that applies gamma correction, ensuring consistent behavior across devices.
- Refactored VulkanPostProcessor and associated shaders to support gamma slider adjustments for both SDR and HDR output.

* fix: ensure proper uniform and cull mode setup for Vulkan sphere rendering

- Added uniform buffer setup and default material uniforms in `vulkan_draw_sphere`.
- Disabled face culling in material definition for sphere rendering to align behavior with OpenGL.

* Ensure memory flushes after UBO updates and add explicit pipeline barrier for MoltenVK compatibility.

* fix: add clear pass for SMAA edge detection to prevent persistent artifacts

* feat: add Vulkan debug-utils messenger, opt-in sync validation, frame diagnostics

Groundwork for the renderer-review fix series (.agents/plans/
vulkan-renderer-review-fixes.md, Phase 0):

- Register a VK_EXT_debug_utils messenger (preferred) instead of only the
  deprecated VK_EXT_debug_report callback, which is kept as a fallback for
  loaders without debug-utils. The messenger config is also chained into
  instance creation so vkCreateInstance-time messages are captured.
  Validation errors/warnings now go to the main log via mprintf.
- New -gr_sync_validation cmdline flag (implies -gr_debug): chains
  VkValidationFeaturesEXT with synchronization validation into instance
  creation when VK_LAYER_KHRONOS_validation is available. This is the
  regression net for the synchronization fixes in the following commits.
- Frame stats now report descriptor sets/writes allocated per frame and the
  total pipeline count (descriptor-manager counters; DescriptorWriter::flush
  moved out-of-line to report into the manager).

Co-Authored-By: Claude Fable 5 <[email protected]>

* refactor: unify Vulkan render-pass begins, clamp in-pass clears, add per-frame UBO ring

Phase 1 of the renderer-review fix series: shared helpers that the
correctness fixes build on.

- New VulkanRenderer::beginTrackedRenderPass(PassBeginDesc): every render-pass
  begin on the frame command buffer now records vkCmdBeginRenderPass and
  updates the state tracker (pass, color attachment count, sample count,
  render area, viewport) in one place. Converted: setupFrame,
  beginSceneRendering (both branches), resumeSceneRendering,
  endSceneRendering, beginRenderTarget (viewport kept for gr_set_viewport),
  resumeSwapChainPass, plus new resumeScenePassAfterCopy() shared by
  copyEffectTexture/copySceneDepthForParticles and
  rebeginSwapChainPassAfterReadback() shared by both readbackFramebuffer
  exits (~200 lines of duplicated begin/clear/viewport blocks removed).
- The state tracker records the active render area; all four
  vkCmdClearAttachments sites clamp their rects against it and skip empty
  results. Previously gr_screen-sized clear rects could exceed a smaller
  off-screen render target's render area, which is invalid API usage.
- New PerFrameUboRing utility (VulkanPerFrameUbo.h/.cpp): a persistently
  mapped uniform buffer with one slot region per frame in flight and a
  self-flushing alloc(). Used by the next commit to fix the cross-frame
  UBO races.

Deliberate behavior deltas (all safe-direction): the helper always sets
colorAttachmentCount/sampleCount (some sites previously inherited stale
values), and resumeSwapChainPass/resumeSceneRendering now also reset the
standard flipped viewport instead of leaving the previous one.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix: eliminate Vulkan cross-frame UBO/buffer races, make gr_sync honest

Phase 2 of the renderer-review fix series: with two frames in flight, several
host-visible resources were rewritten by the CPU while the previous frame's
GPU work could still be reading them.

- The post-processing scratch UBO is now a PerFrameUboRing (per-frame-in-
  flight slot regions, persistently mapped, self-flushing alloc). All the
  per-pass map/unmap/cursor bookkeeping in bloom/LDR/fog/SMAA is deleted;
  the cursor resets exactly once per frame via the new
  VulkanPostProcessor::beginFrame() hook in setupFrame.
  This also fixes two latent bugs: (a) fog consumed scratch slots mid-scene
  and bloom's later same-frame cursor reset overwrote them before submission
  whenever fog+bloom were both active; (b) volumetric_fog_data (288 bytes)
  overflowed the 256-byte slots - the memcpy spilled into the neighboring
  slot and the shader's bound range ended before the aspect/fov/noise
  fields. Slots are now 512 bytes (static_assert at the fog call site) and
  the per-frame budget is 32 slots.
- The dedicated tonemap, output-encode, and MSAA-resolve UBOs (each a single
  instance rewritten per frame, i.e. racy; the MSAA one also never flushed)
  are deleted entirely - those passes build their structs on the stack and
  allocate from the scratch ring.
- Static-buffer rewrites orphan instead of racing: getVkBuffer() stamps a
  last-used frame number, and a full-content updateBufferData() on a Static
  buffer used within the last MAX_FRAMES_IN_FLIGHT frames defers the old
  VkBuffer to the deletion queue and writes a fresh one. Offset writes
  (GPU-heap appends to fresh regions) and PersistentMapping buffers (the
  engine holds their mapped pointer and syncs via gr_sync) are exempt.
- gr_sync_wait now honors its contract: VulkanRenderFrame::waitForFinish
  takes a timeout and reports expiry without consuming the fence;
  VulkanRenderer::waitForFrame returns false for fences taken during the
  still-recording frame (previously it waited on a stale or idle slot and
  reported success) and resolves the frame->slot mapping correctly. This
  makes UniformBufferManager's segment-fence throttling actually protective
  on Vulkan for the first time.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Refactor framebuffer and target lifecycle: centralized creation/resizing logic, improved resource management across Vulkan post-processing subsystems.

* Improve Vulkan cross-frame sync with explicit access masks and validation updates

- Unified Vulkan validation flag handling in setup, separating engine debug output from synchronization validation.
- Specified access masks and stage flags to address cross-frame WRITE_AFTER_WRITE vulnerabilities, notably in G-buffer, post-processing, and MSAA resolve paths.
- Adjusted subpass dependencies and image layout transitions to prevent read-after-write and write-after-write hazards, detected using `-gr_sync_validation`.
- Fixed uniform buffer and barrier transitions to ensure proper ordering in render passes across frames.

* Refactor Vulkan render-pass dependencies and sync stages for clarity and future testing

- Clarified dependency logic for cross-pass and cross-frame synchronization, ensuring explicit stage and access flag comments match observed behavior.
- Adjusted subpass dependencies to decouple render-pass compatibility definitions from dependency assumptions.
- Highlighted MoltenVK-specific issues and retained workarounds with notes for re-testing on macOS hardware.
- Updated synchronization barriers and validation notes to address historical inconsistencies across platforms.

* Remove transfer queue support; consolidate all operations into the graphics queue

- Removed transfer queue handling, simplifying buffer and queue logic to use the graphics queue for all transfer operations.
- Updated buffer sharing modes and removed related code paths and parameters for transfer queue families.
- Introduced exclusive buffer access, noting async transfer and queue ownership transfers are future work.
- Limited enabled device features to only those actively used by the renderer, reducing unnecessary feature requests (e.g., robustBufferAccess).

* Refactor frame advancement and descriptor management in VulkanRenderer

- Removed `endFrame()` from `VulkanDescriptorManager`, replacing it with `setCurrentFrame(frameIndex)` for explicit frame synchronization.
- Centralized frame index advancement within `VulkanRenderer` to ensure consistency across components.
- Invalidated cached descriptor set bindings at render-pass boundaries to handle raw recorder staleness correctly.
- Optimized pipeline descriptor rebind logic by removing redundant clears during pipeline changes.

* Refactor `PipelineConfig::hash` and Global descriptor set management for improved hashing and memoization efficiency

- Replaced inefficient shift-based hashing in `PipelineConfig::hash` with a robust `hashCombine` strategy to minimize bucket collisions.
- Introduced `maskBits` helper to streamline bitmask creation for boolean fields.
- Implemented memoization for Global (Set 0) descriptor sets, reducing redundant rebuilds across frames and improving performance.
- Added dirty state tracking (`m_globalSetDirty`) and invalidation logic for Global set updates following relevant input changes (e.g., TLAS or UBO modifications).

* Refactor Vulkan post-processing output logic: extracted shared `encodeToSwapChainPass` method to streamline HDR and SDR output paths.

* Optimize Vulkan descriptor set management with per-frame memoization

- Added memoization for Material (Set 1) and PerDraw (Set 2) descriptor sets to reuse cached sets across frames if inputs remain unchanged.
- Implemented precise input tracking for Material and PerDraw descriptor sets to detect and rebuild only when necessary.
- Reduced redundant Vulkan descriptor allocations and writes, improving performance on repeated or batched draw calls.

* Fix Vulkan access masks for G-buffer and scene pass transitions

- Added `eColorAttachmentRead` to access masks in multiple barriers to ensure proper synchronization for load operations (e.g., `loadOp=eLoad`) across render passes.
- Prevented `READ_AFTER_WRITE` synchronization validation issues in deferred and multi-pass rendering scenarios.
- Updated comments to clarify the reasoning behind the access mask changes and load dependencies.

* Refactor Vulkan resource management and indexed draw logic

- Consolidated destruction queue methods in `VulkanDeletionQueue` using a shared `queueHandle` template to reduce redundancy.
- Introduced `ModelDrawParams` structure and `computeModelDrawParams` function to unify indexed draw parameters for models and shadows.
- Simplified draw logic by eliminating repeated calculation of indexed draw parameters and improving code reuse.

* Add `releaseAnimationSlotRef` for ref-count handling of shared animation texture arrays

- Introduced `releaseAnimationSlotRef` to manage reference counting for shared animation texture-array slots, preventing premature destruction of shared resources.
- Updated `flushTextures` and related methods to integrate `releaseAnimationSlotRef` for shared slot cleanup.
- Refactored staging buffer creation into `createStagingBuffer` for 3D texture uploads and shared path consistency.
- Enhanced texture upload logic with improved tracking and logging for re-upload cases, aiding preloading gap analysis.
- Added instrumentation for on-demand mid-frame texture uploads in `FrameStats`.

* Refactor Vulkan resource and texture management, extend debug logging

- Updated smart pointer usage for consistency across Vulkan resource managers (`std::make_unique`).
- Removed redundant `cubeImageView` member and related cleanup logic; unified sampling view management.
- Added per-instance debug log counters (avoiding static state) for Vulkan texture uploads and bindings, improving visibility during initialization and run-time.
- Optimized `VulkanRenderer` property/feature getters by caching physical device properties and features during device creation.
- Improved memory region flushing logic for out-of-date offsets, aligning with Vulkan Memory Allocator behavior.
- Refactored render pass state tracking to reapply dynamic state reliably after mid-frame render passes.
- Enhanced validation and error handling for mismatched update texture formats.

* fix CI build warning

* Fix on-demand YUV texture locking and set alpha uniform for movie rendering in Vulkan

- Corrected bitmap lock type from `BMP_TEX_OTHER` to `BMP_AABITMAP` for YUV textures during on-demand loading.
- Updated movie rendering to bind alpha from the `MovieData` UBO (set 2, binding 4) for Vulkan, aligning behavior with OpenGL implementation.

* revert inadvertent changes introduced by a squash commit

* address ci issues

* remove wip stuff from code comments that make no sense to people who aren't me

* Remove unused LD_LIBRARY_PATH export and fix shader shadow map sampling.

* Refactor batched rendering uniform setup in Vulkan

- Removed unused fields from `genericData` uniform block, simplifying shader definitions.
- Updated Vulkan material uniform handling to align with OpenGL's batched rendering logic.
- Introduced `batched_data` struct for streamlined UBO updates, improving maintainability and clarity.

* Add `-hdr` command-line option for HDR nits configuration

- Introduced `-hdr` parameter to set paper white and peak luminance in nits, overriding in-game HDR settings.
- Added input validation with error feedback for incorrect format.

* Remove unused `MODEL_SDR_FLAG_SHADOW_MAP` macro from shader flags

* clean up unrelated changes

* Invalidate Vulkan state tracker bindings after external command buffer modifications

- Added `invalidateExternalBindings()` to `VulkanStateTracker` to handle stale pipelines, descriptors, and dynamic states caused by external Vulkan commands (e.g., ImGui renders mid-pass).
- Updated ImGui rendering flow to call `invalidateExternalBindings()` post-draw, ensuring consistent pipeline state for subsequent engine renders.
- Adjusted image usage flags to account for `eTransferSrc` in texture readbacks.
- Fixed barrier `dstAccessMask` and pipeline stage flags to ensure proper synchronization for image transitions.

* Add a few more generically useful lines to the log

* Simplify Qt deployment logic and improve handling for system-installed Qt on Linux

* add cmdline headers to make qtFred compile

* use VK_KHR_synchronization2 for sync and barrier purposes

* Simplify and clean up Qt deployment logic in CMake for qtfred

* update get_bitmap_from_texture to match the new API

* Enhance `-hdr` command-line option to force HDR output and support optional nits configuration with improved input validation and overrides.

* Improve Vulkan robustness, add device feature checks, and refine memory/logging.

- Added depth clamping support gated by device feature checks, with fallback where unsupported.
- Improved deferred lighting UBO overflow handling with user-accessible warnings and buffer size adjustments.
- Enhanced Vulkan logging for validation, allocation, and debugging:
  - Automatic logging for descriptor pool exhaustion/fragmentation.
  - Vertex format and resource destruction checks.
  - Notifications for generic layout transitions and volumetric texture uploads.
- Consolidated shader logic to ensure Vulkan and OpenGL parity.

* Remove HDR10 encode shader and consolidate gamma correction logic for SDR and HDR10. Replace `Verify` assertions with `Assert`

* Add braces to conditional in object lighting logic for clarity

* Refine Vulkan memory allocation checks with `Verification` for improved error handling and messaging.

* Refine VulkanBuffer resize logic to preserve contents with CPU-side copy and add safeguards for non-coherent memory scenarios.

* Add detailed comments explaining descriptor array capacity constants in VulkanDescriptorManager

* Gate shadow support on depth clamp feature availability and clarify its necessity with improved comments.

---------

Co-authored-by: BMagnu <[email protected]>
Co-authored-by: Mara van der Laan <[email protected]>
Co-authored-by: Claude <[email protected]>
2026-07-21 16:58:21 +00:00

836 lines
23 KiB
C++

#include "ddsutils/ddsutils.h"
#include "cfile/cfile.h"
#include "graphics/2d.h"
#include "osapi/osregistry.h"
#ifdef USE_OPENGL_ES
#include "graphics/opengl/es_compatibility.h"
#endif
#define BCDEC_IMPLEMENTATION 1
PUSH_SUPPRESS_WARNINGS
#include "ddsutils/bcdec.h"
POP_SUPPRESS_WARNINGS
/* Currently supported formats:
* DXT1a (compressed)
* DXT1c (compressed)
* DXT3 (compressed)
* DXT5 (compressed)
* uncompressed 1555 (16-bit, 1-bit being alpha)
* uncompressed 8888 (32-bit)
* uncompressed 888 (24-bit, no alpha)
* uncompressed R8 (8-bit, Red channel as luminance)
*/
// power of two check, to verify that a given DDS will always be usable since
// some may not be power-of-2
static inline int is_power_of_two(int w, int h)
{
return ( (w && !(w & (w-1))) && (h && !(h & (h-1))) );
}
static bool valid_dx10_format(const DDS_HEADER_DXT10 &header)
{
switch (header.dxgiFormat) {
case DXGI_FORMAT::DXGI_FORMAT_BC7_TYPELESS:
case DXGI_FORMAT::DXGI_FORMAT_BC7_UNORM:
case DXGI_FORMAT::DXGI_FORMAT_BC7_UNORM_SRGB:
return true;
default:
return false;
}
}
static bool conversion_needed(const DDS_HEADER &dds_header)
{
if ((dds_header.ddspf.dwFlags & DDPF_FOURCC) != DDPF_FOURCC) {
return false;
}
switch (dds_header.ddspf.dwFourCC) {
case FOURCC_DXT1:
case FOURCC_DXT3:
case FOURCC_DXT5:
return !gr_is_capable(gr_capability::CAPABILITY_S3TC);
case FOURCC_DX10:
// anything other than BC7 will end up invalid
return !gr_is_capable(gr_capability::CAPABILITY_BPTC);
default:
break;
}
// anything not specifically listed should end up invalid
return false;
}
// Memory usage for uncompressed textures is quite high. Some MVP assets can
// require well over a GB of VRAM for a single ship after conversion. To help
// alleviate this we need to resize those textures where possible. At 1024x1024
// the memory requirement is reduced to about 36 MB per ship with a full set
// of textures.
//
// NOTE: modifies header!!
//
// returns: number of mipmap levels to skip
static uint conversion_resize(DDS_HEADER &dds_header)
{
const size_t MAX_SIZE = 1024;
uint width, height, depth, offset = 0;
if (dds_header.dwMipMapCount <= 1) {
return 0;
}
width = dds_header.dwWidth;
height = dds_header.dwHeight;
depth = dds_header.dwDepth;
// drop levels until we get to an appropriate size, but make sure we have
// at least 1 mipmap level remaining at the end (in case there's not a full chain)
while (((width > MAX_SIZE) || (height > MAX_SIZE)) && (offset < dds_header.dwMipMapCount-1)) {
// this shouldn't happen, but catch the obscure case (like 8192x4)
if ((width <= 4) || (height <= 4)) {
break;
}
width >>= 1;
height >>= 1;
depth >>= 1;
++offset;
}
// no change needed
if ( !offset ) {
return offset;
}
// update header with new values
dds_header.dwWidth = std::max(1U, width);
dds_header.dwHeight = std::max(1U, height);
dds_header.dwDepth = std::max(1U, depth);
return offset;
}
static int _dds_read_header(CFILE *ddsfile, DDS_HEADER &dds_header, DDS_HEADER_DXT10 *dds_header_dx10 = nullptr)
{
DDS_HEADER_DXT10 dx10_header;
// read the code
int code = cfread_int(ddsfile);
// check it
if (code != DDS_FILECODE) {
return DDS_ERROR_BAD_HEADER;
}
// read header variables
dds_header.dwSize = cfread_uint(ddsfile);
dds_header.dwFlags = cfread_uint(ddsfile);
dds_header.dwHeight = cfread_uint(ddsfile);
dds_header.dwWidth = cfread_uint(ddsfile);
dds_header.dwPitchOrLinearSize = cfread_uint(ddsfile);
dds_header.dwDepth = cfread_uint(ddsfile);
dds_header.dwMipMapCount = cfread_uint(ddsfile);
// skip over the crap we don't care about
cfseek(ddsfile, 11*4, CF_SEEK_CUR);
dds_header.ddspf.dwSize = cfread_uint(ddsfile);
dds_header.ddspf.dwFlags = cfread_uint(ddsfile);
dds_header.ddspf.dwFourCC = cfread_uint(ddsfile);
dds_header.ddspf.dwRGBBitCount = cfread_uint(ddsfile);
dds_header.ddspf.dwRBitMask = cfread_uint(ddsfile);
dds_header.ddspf.dwGBitMask = cfread_uint(ddsfile);
dds_header.ddspf.dwBBitMask = cfread_uint(ddsfile);
dds_header.ddspf.dwABitMask = cfread_uint(ddsfile);
dds_header.dwCaps = cfread_uint(ddsfile);
dds_header.dwCaps2 = cfread_uint(ddsfile);
// Unused but need them checked for later
dds_header.dwCaps3 = cfread_uint(ddsfile);
dds_header.dwCaps4 = cfread_uint(ddsfile);
dds_header.dwReserved2 = cfread_uint(ddsfile);
if (dds_header.ddspf.dwFourCC == FOURCC_DX10) {
dx10_header.dxgiFormat = static_cast<DXGI_FORMAT>(cfread_uint(ddsfile));
dx10_header.resourceDimension = static_cast<D3D10_RESOURCE_DIMENSION>(cfread_uint(ddsfile));
dx10_header.miscFlag = cfread_uint(ddsfile);
dx10_header.arraySize = cfread_uint(ddsfile);
dx10_header.miscFlags2 = cfread_uint(ddsfile);
if (dds_header_dx10) {
memcpy(dds_header_dx10, &dx10_header, sizeof(DDS_HEADER_DXT10));
}
}
// sanitize values
dds_header.dwDepth = std::max(1U, dds_header.dwDepth);
dds_header.dwMipMapCount = std::max(1U, dds_header.dwMipMapCount);
#ifndef NDEBUG
if (dds_header.dwDepth > 1) {
Assertion(dds_header.dwFlags & DDSD_DEPTH, "ERROR: Volume texture missing proper flags!!");
}
#endif
// set pitch manually (per MS docs)
dds_header.dwFlags &= ~DDSD_LINEARSIZE;
dds_header.dwFlags |= DDSD_PITCH;
if (dds_header.ddspf.dwFlags & DDPF_FOURCC) {
int block_size = (dds_header.ddspf.dwFourCC == FOURCC_DXT1) ? 8 : 16;
dds_header.dwPitchOrLinearSize = std::max(1U, (dds_header.dwWidth+3)/4) * block_size;
} else {
// this isn't technically correct, but it's only wrong for formats we don't support
dds_header.dwPitchOrLinearSize = (dds_header.dwWidth * dds_header.ddspf.dwRGBBitCount + 7) / 8;
}
return DDS_ERROR_NONE;
}
static size_t compute_dds_size(const DDS_HEADER &dds_header, bool converting = false)
{
const uint block_sz = 4;
uint d_width = 0, d_height = 0, d_depth = 0;
size_t d_size = 0;
for (uint i = 0; i < dds_header.dwMipMapCount; i++) {
d_width = std::max(1U, dds_header.dwWidth >> i);
d_height = std::max(1U, dds_header.dwHeight >> i);
d_depth = std::max(1U, dds_header.dwDepth >> i);
// When converting we need to pad a bit to compensate for the decompression
// size on smaller mipmap levels. We need to ensure there is always enough
// room to decode an entire 4x4 block in rgba space
if (converting) {
auto sz = std::min(d_width, d_height);
if (sz < block_sz) {
d_size += ((block_sz * block_sz) - (sz * sz)) * d_depth * 4;
}
}
if (dds_header.ddspf.dwFlags & DDPF_FOURCC) {
// size of data block (4x4)
d_size += dds_compressed_mip_size(d_width, d_height, (dds_header.ddspf.dwFourCC == FOURCC_DXT1) ? 8 : 16) * d_depth;
} else {
d_size += d_width * d_height * d_depth * (dds_header.ddspf.dwRGBBitCount / 8);
}
}
// assumes valid cubemap (should be checked elsewhere)
if (dds_header.dwCaps2 & DDSCAPS2_CUBEMAP) {
// the previously computed size is just per face, mult by 6 for full size
d_size *= 6;
}
Assertion(d_size > 0, "ERROR: DDS size computed to 0!!");
return d_size;
}
static int get_bit_count(const DDS_HEADER &dds_header)
{
if (dds_header.ddspf.dwFlags & DDPF_RGB) {
return dds_header.ddspf.dwRGBBitCount;
} else if (dds_header.ddspf.dwFlags & DDPF_FOURCC) {
if (dds_header.ddspf.dwFourCC == FOURCC_DXT1) {
return 24;
} else {
return 32;
}
}
return 0;
}
int dds_read_header(const char *filename, CFILE *img_cfp, int *width, int *height, int *bpp, int *compression_type, int *levels, size_t *size)
{
DDS_HEADER dds_header;
DDS_HEADER_DXT10 dds_header_dx10;
CFILE *ddsfile;
char real_name[MAX_FILENAME_LEN];
int retval = DDS_ERROR_NONE;
int ct = DDS_UNCOMPRESSED;
int is_cubemap = 0;
bool convert = false;
if (img_cfp == NULL) {
// this better not happen.. ever
Assert(filename != NULL);
// make sure there is an extension
strcpy_s(real_name, filename);
char *p = strchr(real_name, '.');
if (p) { *p=0; }
strcat_s(real_name, ".dds");
// try to open the file
ddsfile = cfopen(real_name, "rb");
// file not found
if (ddsfile == NULL)
return DDS_ERROR_INVALID_FILENAME;
} else {
ddsfile = img_cfp;
}
retval = _dds_read_header(ddsfile, dds_header, &dds_header_dx10);
if (retval != DDS_ERROR_NONE) {
goto Done;
}
// check for valid cubemap (we require all 6 faces)
if (dds_header.dwCaps2 & DDSCAPS2_CUBEMAP) {
if ( !(dds_header.dwCaps2 & DDSCAPS2_CUBEMAP_ALLFACES) ) {
retval = DDS_ERROR_CUBEMAP_FACES;
ct = DDS_DXT_INVALID;
goto Done;
}
// it's a valid cubemap, probably
is_cubemap = 1;
}
// determine and verify the type of data for internal use
if (dds_header.ddspf.dwFlags & DDPF_FOURCC) {
switch (dds_header.ddspf.dwFourCC) {
case FOURCC_DXT1:
ct = (is_cubemap) ? DDS_CUBEMAP_DXT1 : DDS_DXT1;
break;
case FOURCC_DXT3:
ct = (is_cubemap) ? DDS_CUBEMAP_DXT3 : DDS_DXT3;
break;
case FOURCC_DXT5:
ct = (is_cubemap) ? DDS_CUBEMAP_DXT5 : DDS_DXT5;
break;
// dxt2 and dxt4 aren't supported
case FOURCC_DXT2:
case FOURCC_DXT4:
retval = DDS_ERROR_UNSUPPORTED;
ct = DDS_DXT_INVALID;
goto Done;
break;
case FOURCC_DX10:
if (valid_dx10_format(dds_header_dx10)) {
// Currently only BC7 is supported
ct = DDS_BC7;
} else {
retval = DDS_ERROR_UNSUPPORTED;
ct = DDS_DXT_INVALID;
goto Done;
}
break;
// none of the above
default:
retval = DDS_ERROR_INVALID_FORMAT;
ct = DDS_DXT_INVALID;
goto Done;
break;
}
} else if (dds_header.ddspf.dwFlags & DDPF_RGB) {
ct = (is_cubemap) ? DDS_CUBEMAP_UNCOMPRESSED : DDS_UNCOMPRESSED;
} else {
// it's not a readable format
retval = DDS_ERROR_INVALID_FORMAT;
goto Done;
}
// maybe do conversion if format not supported
convert = conversion_needed(dds_header);
if (convert) {
// switch to uncompressed format and reset vars
dds_header.ddspf.dwFlags &= ~DDPF_FOURCC;
dds_header.ddspf.dwFlags |= DDPF_RGB;
dds_header.ddspf.dwRGBBitCount = 32;
// NOTE: modifies header and returns offset for mipmap count
dds_header.dwMipMapCount -= conversion_resize(dds_header);
dds_header.dwMipMapCount = std::max(1U, dds_header.dwMipMapCount);
ct = (is_cubemap) ? DDS_CUBEMAP_UNCOMPRESSED : DDS_UNCOMPRESSED;
}
// we don't support compressed, non-power-of-2 images
if ( (ct != DDS_UNCOMPRESSED) && (!is_power_of_two(dds_header.dwWidth, dds_header.dwHeight)) ) {
retval = DDS_ERROR_NON_POWER_OF_2;
goto Done;
}
// stuff important info
if (size)
*size = compute_dds_size(dds_header, convert);
if (bpp)
*bpp = get_bit_count(dds_header);
if (compression_type)
*compression_type = ct;
if (width)
*width = dds_header.dwWidth;
if (height)
*height = dds_header.dwHeight;
if (levels)
*levels = dds_header.dwMipMapCount;
Done:
if (img_cfp == NULL) {
// close file and return
cfclose(ddsfile);
ddsfile = NULL;
}
return retval;
}
static void (*decompress_dds)(const void *in, void *out, int pitch) = nullptr;
static uint32_t BLOCK_SIZE = 0;
//reads pixel info from a dds file
int dds_read_bitmap(const char *filename, ubyte *data, ubyte *bpp, int cf_type)
{
int retval;
size_t size = 0;
CFILE *cfp;
char real_name[MAX_FILENAME_LEN];
DDS_HEADER dds_header;
// this better not happen.. ever
Assert(filename != NULL);
// make sure there is an extension
strcpy_s(real_name, filename);
char *p = strchr(real_name, '.');
if (p) { *p = 0; }
strcat_s(real_name, ".dds");
// open it up and go to the data section
cfp = cfopen(real_name, "rb", cf_type);
// just in case
if (cfp == nullptr)
return DDS_ERROR_INVALID_FILENAME;
// read the header -- if its at this stage, it should be legal.
retval = _dds_read_header(cfp, dds_header);
Assert(retval == DDS_ERROR_NONE);
// this really shouldn't be needed but better safe than sorry
if (retval != DDS_ERROR_NONE) {
cfclose(cfp);
return retval;
}
cfseek(cfp, (dds_header.ddspf.dwFourCC == FOURCC_DX10) ? DX10_OFFSET : DDS_OFFSET, CF_SEEK_SET);
size = compute_dds_size(dds_header); // don't add padding on this one!!
// read in the data
if ( !conversion_needed(dds_header) ) {
cfread(data, 1, (int)size, cfp);
} else {
// Compression format not supported, convert to BGRA
ubyte *comp_data = (ubyte*)vm_malloc(size);
cfread(comp_data, 1, (int)size, cfp);
ubyte *src = comp_data;
ubyte *dst = data;
uint d_width, d_height, d_depth;
size_t data_offset = 0;
// NOTE: this alters the width, height, and depth values in the header,
// so we have to jump through some hoops to get the proper values
const uint mipmap_offset = conversion_resize(dds_header);
const int num_faces = (dds_header.dwCaps2 & DDSCAPS2_CUBEMAP) ? 6 : 1;
const bool has_depth = (dds_header.dwFlags & DDSD_DEPTH) == DDSD_DEPTH;
switch (dds_header.ddspf.dwFourCC) {
case FOURCC_DX10:
decompress_dds = bcdec_bc7;
BLOCK_SIZE = BCDEC_BC7_BLOCK_SIZE;
break;
case FOURCC_DXT5:
decompress_dds = bcdec_bc3;
BLOCK_SIZE = BCDEC_BC3_BLOCK_SIZE;
break;
case FOURCC_DXT1:
decompress_dds = bcdec_bc1;
BLOCK_SIZE = BCDEC_BC1_BLOCK_SIZE;
break;
case FOURCC_DXT3:
decompress_dds = bcdec_bc2;
BLOCK_SIZE = BCDEC_BC2_BLOCK_SIZE;
break;
default:
Error(LOCATION, "Invalid FourCC (%d) for DDS decompression!", dds_header.ddspf.dwFourCC);
break;
}
for (int f = 0; f < num_faces; ++f) {
// if we resized then skip over all of that data (altering values to match pre-resize)
for (uint x = 0; x < mipmap_offset; ++x) {
d_width = std::max(1U, dds_header.dwWidth << (mipmap_offset - x));
d_height = std::max(1U, dds_header.dwHeight << (mipmap_offset - x));
d_depth = has_depth ? std::max(1U, dds_header.dwDepth << (mipmap_offset - x)) : 1U;
src += ((d_width + 3) / 4) * ((d_height + 3) / 4) * d_depth * BLOCK_SIZE;
}
for (uint m = mipmap_offset; m < dds_header.dwMipMapCount; ++m) {
// width/height/depth at post-resize mipmap sizes, so compensate here
d_width = std::max(1U, dds_header.dwWidth >> (m - mipmap_offset));
d_height = std::max(1U, dds_header.dwHeight >> (m - mipmap_offset));
d_depth = std::max(1U, dds_header.dwDepth >> (m - mipmap_offset));
for (uint d = 0; d < d_depth; ++d) {
auto depth_offset = d * d_width * d_height * 4;
for (uint i = 0; i < d_height; i += 4) {
for (uint j = 0; j < d_width; j += 4) {
dst = data + data_offset + depth_offset + ((i * d_width + j) * 4);
decompress_dds(src, dst, d_width * 4);
src += BLOCK_SIZE;
}
}
}
// we need it in bgra, XOR to the rescue
for (size_t x = data_offset; x < (data_offset + (d_width * d_height * d_depth * 4)); x += 4) {
data[x] ^= data[x+2], data[x+2] ^= data[x], data[x] ^= data[x+2];
}
// bump data offset to next layer
data_offset += (d_width * d_height * d_depth * 4);
}
}
vm_free(comp_data);
comp_data = nullptr;
// switch to uncompressed format and reset vars (needed below to get correct bit count)
dds_header.ddspf.dwFlags &= ~DDPF_FOURCC;
dds_header.ddspf.dwFlags |= DDPF_RGB;
dds_header.ddspf.dwRGBBitCount = 32;
}
if (bpp)
*bpp = (ubyte)get_bit_count(dds_header);
// we look done here
cfclose(cfp);
return DDS_ERROR_NONE;
}
int dds_decompress_top_mip_bgra(const char *filename, int cf_type,
int *out_width, int *out_height,
SCP_vector<ubyte> &out_pixels)
{
Assert(filename != nullptr);
// normalize to a .dds extension, same as dds_read_bitmap
char real_name[MAX_FILENAME_LEN];
strcpy_s(real_name, filename);
char *p = strchr(real_name, '.');
if (p) { *p = 0; }
strcat_s(real_name, ".dds");
CFILE *cfp = cfopen(real_name, "rb", cf_type);
if (cfp == nullptr)
return DDS_ERROR_INVALID_FILENAME;
DDS_HEADER dds_header;
DDS_HEADER_DXT10 dx10_header;
int retval = _dds_read_header(cfp, dds_header, &dx10_header);
if (retval != DDS_ERROR_NONE) {
cfclose(cfp);
return retval;
}
// only 2D FOURCC-compressed images are supported here
if (!(dds_header.ddspf.dwFlags & DDPF_FOURCC) ||
(dds_header.dwCaps2 & (DDSCAPS2_CUBEMAP | DDSCAPS2_VOLUME)) ||
(dds_header.dwFlags & DDSD_DEPTH) ||
(dds_header.dwDepth > 1)) {
cfclose(cfp);
return DDS_ERROR_UNSUPPORTED;
}
void (*decode)(const void *, void *, int) = nullptr;
int block_size = 0;
switch (dds_header.ddspf.dwFourCC) {
case FOURCC_DXT1: decode = bcdec_bc1; block_size = BCDEC_BC1_BLOCK_SIZE; break;
case FOURCC_DXT3: decode = bcdec_bc2; block_size = BCDEC_BC2_BLOCK_SIZE; break;
case FOURCC_DXT5: decode = bcdec_bc3; block_size = BCDEC_BC3_BLOCK_SIZE; break;
case FOURCC_DX10:
if (!valid_dx10_format(dx10_header) ||
dx10_header.resourceDimension != D3D10_RESOURCE_DIMENSION::D3D10_RESOURCE_DIMENSION_TEXTURE2D ||
dx10_header.arraySize > 1) {
cfclose(cfp);
return DDS_ERROR_UNSUPPORTED;
}
decode = bcdec_bc7;
block_size = BCDEC_BC7_BLOCK_SIZE;
break;
default:
cfclose(cfp);
return DDS_ERROR_UNSUPPORTED;
}
const int w = static_cast<int>(dds_header.dwWidth);
const int h = static_cast<int>(dds_header.dwHeight);
if (w <= 0 || h <= 0) {
cfclose(cfp);
return DDS_ERROR_INVALID_FORMAT;
}
// BCn data is stored as ceil(w/4) * ceil(h/4) 4x4 blocks; dimensions
// don't have to be multiples of 4.
const int blocks_w = (w + 3) / 4;
const int blocks_h = (h + 3) / 4;
const int padded_w = blocks_w * 4;
const int padded_h = blocks_h * 4;
const size_t compressed_size = static_cast<size_t>(blocks_w) * blocks_h * block_size;
// _dds_read_header leaves the file positioned right after the header
// (including the DX10 sub-header if present), so the next read is the
// top mip's pixel data.
SCP_vector<ubyte> compressed(compressed_size);
const int got = cfread(compressed.data(), 1, static_cast<int>(compressed_size), cfp);
cfclose(cfp);
if (got != static_cast<int>(compressed_size))
return DDS_ERROR_INVALID_FORMAT;
// Decode into a padded buffer so edge blocks have room, then crop to w*h.
SCP_vector<ubyte> decoded(static_cast<size_t>(padded_w) * padded_h * 4);
const int dec_stride = padded_w * 4;
const ubyte *src = compressed.data();
for (int by = 0; by < blocks_h; ++by) {
for (int bx = 0; bx < blocks_w; ++bx) {
ubyte *blk_dst = decoded.data() + (by * 4) * dec_stride + (bx * 4) * 4;
decode(src, blk_dst, dec_stride);
src += block_size;
}
}
out_pixels.assign(static_cast<size_t>(w) * h * 4, 0);
const int dst_stride = w * 4;
for (int y = 0; y < h; ++y) {
std::memcpy(out_pixels.data() + static_cast<size_t>(y) * dst_stride,
decoded.data() + static_cast<size_t>(y) * dec_stride,
static_cast<size_t>(dst_stride));
}
// bcdec outputs RGBA byte-order; swap to BGRA
for (size_t x = 0; x < out_pixels.size(); x += 4) {
std::swap(out_pixels[x], out_pixels[x + 2]);
}
if (out_width) *out_width = w;
if (out_height) *out_height = h;
return DDS_ERROR_NONE;
}
// save some image data as a DDS image
// NOTE: we only support, uncompressed, 24-bit RGB and 32-bit RGBA images here!!
void dds_save_image(int width, int height, int bpp, int num_mipmaps, ubyte *data, int cubemap, const char *filename)
{
DDS_HEADER dds_header;
char real_filename[MAX_FILENAME_LEN];
if (data == NULL) {
Int3();
return;
}
// header size check
if (sizeof(DDS_HEADER) != 124) {
Int3();
return;
}
// not that that's out of the way we can get down to business, constructing a sane filename
memset( &real_filename, 0, sizeof(real_filename) );
int count = os_config_read_uint(NULL, "ImageExportNum", 0);
if (count > 999)
count = 0;
if (filename == NULL) {
if (cubemap) {
sprintf(real_filename, "cubemap%.3d.dds", count++);
} else {
sprintf(real_filename, "image%.3d.dds", count++);
}
os_config_write_uint(NULL, "ImageExportNum", count);
} else {
Assert( strlen(filename) < MAX_FILENAME_LEN-5 );
strcpy_s(real_filename, filename);
char *p = strchr(real_filename, '.');
if (p) { *p = 0; }
strcat_s(real_filename, ".dds");
}
CFILE *image = cfopen( real_filename, "wb", CF_TYPE_CACHE );
if (image == NULL) {
mprintf(("Unable to open DDS image for saving!!\n"));
return;
}
if (num_mipmaps < 1)
num_mipmaps = 1;
// we have a filename and file handle, so now lets create our DDS header...
memset( &dds_header, 0, sizeof(DDS_HEADER) );
uint flags = (DDSD_CAPS | DDSD_PITCH | DDSD_PIXELFORMAT | DDSD_WIDTH | DDSD_HEIGHT);
uint pixel_flags = DDPF_RGB;
uint caps1 = DDSCAPS_TEXTURE;
uint caps2 = 0;
if (bpp == 32) {
pixel_flags |= DDPF_ALPHAPIXELS;
}
if (num_mipmaps > 1) {
flags |= DDSD_MIPMAPCOUNT;
caps1 |= (DDSCAPS_COMPLEX | DDSCAPS_MIPMAP);
}
if (cubemap) {
caps1 |= DDSCAPS_COMPLEX;
caps2 |= (DDSCAPS2_CUBEMAP | DDSCAPS2_CUBEMAP_ALLFACES);
}
dds_header.dwSize = 124;
dds_header.dwFlags = flags;
dds_header.dwHeight = height;
dds_header.dwWidth = width;
dds_header.dwPitchOrLinearSize = (width * (bpp >> 3));
dds_header.dwDepth = 0;
dds_header.dwMipMapCount = num_mipmaps;
dds_header.ddspf.dwSize = 32;
dds_header.ddspf.dwFlags = pixel_flags;
dds_header.ddspf.dwFourCC = 0;
dds_header.ddspf.dwRGBBitCount = bpp;
dds_header.ddspf.dwRBitMask = 0x00ff0000;
dds_header.ddspf.dwGBitMask = 0x0000ff00;
dds_header.ddspf.dwBBitMask = 0x000000ff;
dds_header.ddspf.dwABitMask = (bpp == 32) ? 0xff000000 : 0x00000000;
dds_header.dwCaps = caps1;
dds_header.dwCaps2 = caps2;
// now save it all...
uint dds_id = DDS_FILECODE;
cfwrite(&dds_id, 1, 4, image);
cfwrite(&dds_header, 1, sizeof(DDS_HEADER), image);
int faces = (cubemap) ? 6 : 1;
int f_width = width;
int f_height = height;
int f_size = 0, f_offset = 0;
// cubemaps are written:
// face 1
// all face 1 mipmaps
// face 2
// all face 2 mipmaps
// ... etc.
for (int i = 0; i < faces; i++) {
for (int j = 0; j < num_mipmaps; j++) {
f_size = (f_width * f_height * (bpp >> 3));
cfwrite(data + f_offset, 1, f_size, image);
// increase offset for next mipmap level
f_offset += f_size;
// half width and height for next mipmap level
f_width /= 2;
f_height /= 2;
if (f_width < 1)
f_width = 1;
if (f_height < 1)
f_height = 1;
}
// reset width and height to original values for the next face
f_width = width;
f_height = height;
}
// done! now lets go get something to eat...
cfclose(image);
}
// returns string representation of DDS_** error code
const char *dds_error_string(int code)
{
switch (code)
{
case DDS_ERROR_NONE:
return "No error";
case DDS_ERROR_INVALID_FILENAME:
return "File not found";
case DDS_ERROR_BAD_HEADER:
return "Filecode did not equal \"DDS \"";
case DDS_ERROR_INVALID_FORMAT:
return "DDS was in an unsupported/unknown format";
case DDS_ERROR_UNSUPPORTED:
return "DDS format was known but is not supported (ie. DXT2/DXT4)";
case DDS_ERROR_NO_COMPRESSION:
return "DDS is compressed but compression support is not enabled";
case DDS_ERROR_NON_POWER_OF_2:
return "Cannot load DDS if not power-of-2";
case DDS_ERROR_CUBEMAP_FACES:
return "Cubemaps must have all 6 faces";
default:
return "Abort, retry, fail?";
}
}