Replace conversions between storage types (unorm16, LNS16),
decode types(float16), and codec types (float), with vectorized
versions of the same algorithms.
This change makes the output of the codec invariant across operating systems (removed std::math dependency), and across CPU architectures (NEON vector code now matches the halving reduction patterns used by x86-64).
This patch makes the none (no SIMD), SSE, and AVX2 builds invariant with respect to each other. Prior to this only the SSE2 and 4.1 builds were invariant. This is a fix for #223.
Four classes of change were needed to avoid reassociation rounding differences:
* Horizontal sums in a vector must match the odd-even reduce used by the SIMD cases, even for non-SIMD code, to ensure consistent reduction sums.
* All vector body accumulators that use lane-wise accumulation must use the same accumulator width, irrespective of instruction set, to ensure consistent lane-wise partial sums. For this reason the 1-wide VLA fallback has been removed, as we decided to use 4-wide accumulators.
* AVX2 vector body accumulators must split into two 4-wide operations, and sequentially adding these into the running sum, to ensure consistent accumulator partial sums.
* Vector loop bodies that use a loop tail for accumulation must execute the same number of times, irrespective of instruction set, to ensure that the loop tail runs the same number of times. This is because a the vector body will do odd+even partial sums (reduction across the vector), whereas the loop tail will do linear partial sums (iterating though the remaining tail).
For this patch I've implemented the last bullet simply by rounding down all vector bodies that use accumulators and loop tails to a multiple of 8. This will reduce the number of vectorized loop iterations for 4-wide hardware in some cases, e.g. for an iteration count of 13, the left over is 5, which would have run one more vectorized path. Measured impact is small and only hits some block sizes, although 6x6 is impacted by ~2% on -thorough. This will be optimized in a follow-on patch.
This implementation provides an AVX2 accelerated path for decimation table searches. It primarily helps larger block sizes, as use of weight grid decimation is more likely, and there are more texels to pack out vector operations.
This PR implements the ability to convert floats stored in vfloat4 vectors to fp16 bit patterns stored in vint4 vectors. Scalar versions are also provided as overloaded functions of the same name, for sake of convenience.
The code has been refactored to take advantage of the vectorization opportunities this gives, in particular using NEON (Arm) and F16C (x86-64) ISA support for float<>fp16 conversion. The F16C support is tied to the AVX2 enable config option, it is not a separate enable. In cases where the soft-float code is not needed, code size reduces by ~5KB.
Prior to this PR we had two vec4 implementations for floats, the legacy "float4" and the newer "vfloat4". The newer vfloat4 is a superset of the capabilities of the float4, and using it everywhere would give other opportunities to optimize by e.g. using the conditional select functionality rather than scalarizing chunks of vector code.
This patch is just the basic migration - where possible I've translated the original code without additional refactoring. Further optimizations to make better use of the vfloat4 functionality will come later. Overall this patch has no impact on IQ or performance, but does give a slight reduction in code size.
This replaces the angular_steppings table with a computationally
generated one, where the steppings are evenly spaced 1-40.
This isn't quite as effective, reducing quality by up to 0.05dB,
but improves performance by 3-6%.
This implementation alters the algorithm and vectorizes these two functions.
The algorithm change will have a minor image quality impact; it now uses a faster approximation for selecting which component to assign to a second plane when choosing a partition. There is an insignificant IQ hit on -medium (under < 0.01dB), as we rarely use two planes anyway, and a slightly larger one on -thorough (under < -0.05dB). The significant (>10%) performance gain on -thorough is worth the quality loss (i.e. users are more likely to use -thorough if it's faster, and the overall IQ gain vs medium is still net positive).
This PR vectorizes the implementation of the compute_error_for_weight_set() function using the vector-length agnostic SIMD support. It is bit-exact with the original implementation, so no image diffs are expected.
To make this vectorizable it needs to switch some structures from AOS to SOA, but we can't use these everywhere, so we duplicate some data tables in the decimation table and store both array orderings.
5-7% faster for -thorough
2-4% faster for -medium
1-2% faster for -fast
Neutral for -fastest
This PR adds support for Arm aarch64 builds, including the corresponding NEON accelerated vector library.
As part of this work I also improved testing:
- Native C++ unit tests support using `googletest` integrated into CMake/CTest.
- First unit test suite added, for 4-wide SIMD implementations.
- Command line functional tests can target any build, not just AVX2.
Peter Harris