rd_insert() hands back the chain head it just linked for a position,
and the greedy parser (levels 1-3) already used it to skip the match
search when there is no earlier occurrence. The lazy parser - levels
4-9, which includes the level-6 screenshot encoder - threw it away
and called rd_longest_match() at every position, which reloaded the
head from prev[] and returned 0 when it was empty. On noisy input
that is about half of all positions: a full call, register saves and
the struct loads of the prologue, to learn there was nothing to do.
Both parsers now pass the head in and only call when a chain exists;
rd_longest_match() starts its walk from it instead of re-reading
prev[]. An empty chain returned 0 with no side effects before, so
the parse - and the stream - are unchanged.
Compressed output is byte-identical to before over the level 0-9 x
input chunking x six-input matrix, zlib-wrapped and raw, also under
ASan/UBSan; rzip, rpng, archive and encodings sample tests pass;
builds C89 without new warnings and for aarch64.
screenshot_io_bench, 3840x2160 BGR24 PNG: noise instructions -10.3%
(10.37 G -> 9.30 G), best 393 -> 380 ms, median about -4%; gradient
unchanged.
A screenshot of a core's own frame - no GPU readback, no rescale -
hands the encoder XRGB8888 or RGB565 rows, which it turns into PNG's
R,G,B per row. Both conversions ran a pixel at a time, ahead of
filters that are all vectorised; the BGR24 readback path got the same
treatment in the previous commit.
SSE2 builds each pixel as a 32-bit lane 0x00BBGGRR (R,G,B,0 in
memory) and then closes the gaps without a byte shuffle: in each
64-bit half the odd lane moves down a byte onto the even lane's zero,
and the upper half moves down to follow the lower one. Four pixels
land in the first 12 bytes; the last four are rewritten by the next
store or the scalar tail. XRGB8888 gets its lane from a 16-bit
rotate and two masks, RGB565 from the same shifts and masks as
pixconv_rgb565_to_xbgr8888. NEON reads XRGB8888 as four planes (vld4,
little-endian only) and expands RGB565 with one vsri per channel,
which is exactly the top-bits replication; both store with vst3.
Other targets keep the scalar loops.
PNG output is byte-identical to before: 12 widths from 1 to 1920,
padded and negative pitches, both formats, ASan/UBSan clean. The
conversions match the old loops on SSE2, on NEON (aarch64 under qemu)
and on the scalar path at every width 0-300 and, for RGB565, for all
65536 input values, and never write past the row. rpng sample tests
pass; builds C89 -Wextra without warnings, as C++, and for aarch64
and armv7 NEON.
Per 3840x2160 frame, conversion alone: XRGB8888 3.75 -> 2.45 ms,
RGB565 9.77 -> 4.00 ms.
RetroFW's jz4760 OSS driver moves a node to the codec in 16-byte DMA
units: it drops a write's remainder, and a write under one unit starts
a transfer of none. The threaded pipeline writes any number of frames,
so most writes lost one to three frames - static at its rate of writes -
and a short one could leave the device playing nothing it was given.
On RetroFW a write now goes out in whole units, its remainder held to
lead the next one.
samples/audio/oss_report models that DMA; built with RETROFW=1, which
CI now does, it checks every node is whole units and plays in order.
Fixes#20006
A BGR24 screenshot - the GPU-readback path - has each row turned
into PNG's R,G,B order before filtering. That swap ran a byte at a
time, ahead of filters that are all vectorised, and was two thirds of
rpng_encode_row: about 12 instructions a pixel, 83 million per 4K
capture.
SSE2 has no byte shuffle, but this swap needs none. A 16-byte load
that starts on a pixel boundary holds every byte's partner two places
away in a pattern that repeats every three bytes, so two whole-register
byte shifts and three fixed masks build it; each step completes four
pixels and steps 12 bytes, its last four bytes rewritten by the next
store or the scalar tail. NEON loads the channels as planes (vld3)
and stores them back crossed. Other targets keep the scalar loop.
PNG output is byte-identical to before: 17 widths from 1 to 1920 x 4
heights x smooth and noisy content x both pitch signs, ASan/UBSan
clean. The swap matches the byte loop at every width 0-300 on SSE2,
on NEON (aarch64 under qemu) and on the scalar path, and never writes
past the row. rpng sample tests pass; builds C89 without warnings and
for aarch64 and armv7 NEON.
screenshot_io_bench, 3840x2160 BGR24 PNG: 50 million fewer
instructions per capture (gradient -4.3%, noise -1.4%); gradient
median 69.0 -> 65.9 ms.
Each dynamic block builds its codes by first sorting the used symbols
by frequency. That was an insertion sort, which is cheap for a few
symbols but costs about m*m/4 moves for a full alphabet - and a noisy
block, which every busy screenshot has plenty of, uses all 256
literals, so the sort ran to some 16000 moves per block.
Alphabets of more than 48 symbols now go through a stable byte-wise
radix sort on the frequency: one pass for each byte the largest
frequency uses, which is one or two since a block holds at most
RD_BLOCK_SYMS symbols. The symbols come in in symbol order, so a
stable sort gives the same order the insertion sort did - ties still
fall in symbol order - and the codes and the stream are unchanged.
Small alphabets (distances, code lengths, short blocks) keep the
insertion sort. The ping-pong half uses gl_wt and gl_left, which are
free until the tree build; nothing new is allocated.
Compressed output is byte-identical to before over the level 0-9 x
input chunking x six-input matrix, zlib-wrapped and raw; the sort
itself matches the insertion sort on 200000 random arrays of 49-288
symbols with frequencies up to 32 bits (ASan/UBSan clean); the rzip,
rpng, archive and encodings sample tests pass; builds C89 without new
warnings and for aarch64.
screenshot_io_bench, 3840x2160 PNG: noise instructions -5.8%
(11.17 G -> 10.52 G), best 427 ms -> 392 ms; gradient unchanged.
When a block does not compress - noise, video frames, already-
compressed archive members, the busy parts of a screenshot - it goes
out stored, and every byte of it was moved by a loop that tested for
output room, copied one byte and decremented a counter. On a noisy
4K screenshot that loop was a tenth of the whole encode.
The bytes now go out with memcpy, in runs as long as the remaining
block or the output room allows; a full output still suspends the
copy exactly where the byte loop did, so a caller handing in small
output buffers sees the same stream.
Compressed output is byte-identical to before: the earlier level 0-9
x chunking x six-input matrix (zlib-wrapped and raw, 1 MB output),
and a focused stored-block run - noise, half-noise and 4-bit noise at
levels 1, 6 and 9 into output buffers of 7, 8, 16, 17, 100, 4096,
65537 bytes and 1 MB, all suspending mid-block; ASan/UBSan clean; the
rzip, rpng, archive_zip_methods, archive_stream and encodings sample
tests pass; builds C89 without new warnings and for aarch64.
screenshot_io_bench, 3840x2160 PNG, noise: instructions -9.2%,
best 451.9 ms -> 403.7 ms, median 556.3 -> 492.3 ms; gradient (no
stored blocks) unchanged.
On the threaded pipeline audio_driver_pause_fade() armed the resume
ramp by writing fade_in_pending, a plain bool, on the main thread, and
the producer read and cleared it in audio_driver_submit_width() - on
the thread that submits, which for a core delivering from a thread of
its own is not the main thread. A data race; a resume could also lose
its ramp if the clear landed after a new arm.
fade_in_pending is now a retro_atomic_int_t. The main thread stores it;
whoever applies the ramp takes it with fetch-and against zero
(audio_driver_fade_take(); exchange is not on every retro_atomic.h
backend), in the producer and in audio_driver_arm_resume(). No lock,
and nothing waits.
ff_speedup gains a lane with the producer on a thread of its own while
the main thread ends frames, holds and releases fast-forward and
pauses, the consumer's share taken on the main thread. Under
ThreadSanitizer, which CI already runs ff_speedup with, it reported
this race and nothing else after the fast-forward measurement fix
(e24dafdc), and reports none now. The harness clock can run on real
time for a case with threads.
The CI step runs its lanes at once, in real time, so it takes as long
as the longest lane: rate_control, four settings at each of two device
latencies, 24 s; ff_exit, five transitions, 21 s; setpoint, three
latencies twice over, 18 s.
setpoint and rate_control now take a device latency (setpoint_16,
_32, _64; rate_control_32, _64) and ff_exit a transition (ff_exit_1
to _5), as audio_paced already did, and the step runs all fourteen at
once. The longest is now a single setting: 12 s of wall time for the
step here, three runs, no failures and no retries, on two CPUs. The
whole-case names still run the whole case, and a name that matches
nothing fails instead of passing on no run.
Three copies on the flush path that the next pass overwrote:
- Integer front pair at unity to an s16 device with a master gain:
the batch was copied into the output scratch, then the gain ran over
it in place. The gain now reads the batch and writes the scratch, one
pass (audio_driver_gain_s16(), the same Q16 multiply).
- In-process MIDI synth on the integer path: the game audio was copied
into the writable view, then the synth summed into it. The sum is now
written from the game audio, one pass, and a synth that rendered
nothing leaves the batch where it is, uncopied.
- In-process MIDI synth for a float core: copied in, then summed. The
sum is written in the pass that brings the audio in. The synth renders
before the path is chosen, so one that has nothing this batch no
longer forces an s16 core off its passthrough through two conversions.
Output is unchanged sample for sample. Measured on 800-frame stereo
batches: the float synth sum 167 -> 132 ns per batch; the s16 gain
saves the 23 ns copy, about 0.5% of that pass; a silent synth saves the
copy and, on the float path, an s16 -> float -> s16 round trip.
unity_identity gains lanes: the integer gain at an attenuation and at a
saturating boost against the Q16 oracle; the synth sum on the integer
pair and for a float core against what the copy and the add gave; and
a silent synth leaving the batch on its passthrough, which on the float
path failed before (one s16 -> float and one float -> s16 conversion).
The shared stubs' synth is replaced there by one the cases control.
The fast-forward speed measurement (last_flush_time and its two
averages) is taken by one thread: the submitter on the inline
pipeline, the main thread's frame end on the threaded pipeline and for
a callback core. Other threads wrote it too. Inline, the main thread's
frame end, pause and stop reset it while a core submitting from a
thread of its own was measuring. A callback core's audio thread reset
it beside the main thread's frame-end measurement. On the threaded
pipeline the producer read and reset it, and pipe_ff_frames was a
plain size_t added on the submitting thread and zeroed on the main one.
These are data races, and on 32-bit targets a 64-bit store can tear.
Now only the measuring thread writes the measurement. Any other thread
asks: one flag, stored only when not already up, which the measuring
thread takes before it next measures (audio_driver_ff_mult_sync(),
taken with fetch-and since exchange is not on every retro_atomic.h
backend). An idle frame costs one load. pipe_ff_frames is atomic: added
by the submitter, taken by what is there at the frame end, never stored
to zero. The producer knows the first frame of a hold from a flag of its
own instead of reading the main thread's measurement. A flush outside
fast-forward on the measuring thread still resets outright, which the
inline transport's source stamp overwrites as before.
This replaces the request-handler resets added for pause_fade in
0e35c138 with the same ask every other thread uses.
discrete_multichannel gains a lane with a core submitting from a
thread of its own while the main thread ends frames, holds and releases
fast-forward and pauses. Under ThreadSanitizer it reported nine races
on last_flush_time and pipe_ff_frames on the old code and none now; the
CI step builds it with TSan and runs that lane. Checks that read the
measurement straight after another thread's reset now accept the
pending ask.
check-ff-stall runs in real time, twice, and again under
ThreadSanitizer in CI. Each run spent 0.5 s settling into the hold,
1.5 s stalled and 1 s after it.
The stall is now 1.25 s, the shortest that still takes the old code's
estimate past the stretch's 4x duration bound with margin (from a bit
over 1 s); at 0.5 s the old code passes. A quarter of a second settles
on each side and half a second is judged, as before.
The lane still fails on the code before the gap fix (the stretch off
60 of 60 frames) and passes with it, three runs each variant, and under
ThreadSanitizer.
audio_driver_pause_fade() reset the fast-forward flush measurement on
the main thread. That is right where the measurement is taken there:
the threaded pipeline's producer and a callback core measure at the
frame end. Inline, the thread that submits measures, and a core may
submit from a thread of its own; the main thread's reset then wrote
last_flush_time beside that thread's read-modify-write of it, a 64-bit
store that can tear on 32-bit targets.
Inline, the reset now rides the pause and resume requests pause_fade
already posts, applied by the submitter under the inline claim before
its next flush measures. The main thread resets only for the threaded
pipeline and callback cores, and always without HAVE_THREADS.
The ff_speedup and discrete_multichannel pause lanes pass unchanged.
A stall inside a fast-forward hold was measured as one long flush
interval, and a long enough one pushed the speed estimate past the time
stretch's duration range. The stretch then stayed off for the rest of
the hold: the threaded transport suspends and resumes only from a
drained ring, which fast-forward keeps full; and with Speedup off the
inline flush put back the stamp from before the interval, so every
later flush measured from it.
An interval longer than the average's whole window at 1x
(AUDIO_FF_EXP_AVG_SAMPLES expected intervals) is now a gap, not a speed:
the estimate seeds again at the configured ratio, as after a pause.
Shorter intervals are still measured. One of them can take the estimate
of a hold running at 1x or faster down to about half speed at most,
inside the stretch's range, and any single interval that could push it
out is a gap. The inline, threaded and callback-core paths share the
measurement.
Inline, the stamp is put back only for the Speedup fallback, which may
measure the interval again. Put back otherwise, it stays stale until it
ages into a gap.
The ff_speedup and discrete_multichannel harnesses stall a 2x hold for
three seconds, threaded with and without Speedup, and inline. Inline, a
slow spell of intervals under a gap also takes the estimate out of
range, and the stretch has to be back within a few frames of the hold
returning to 2x. pipeline_wakeups gains a lane on the real wrapper
without Audio Sync, where the consumer primes again after the stall; it
checks that the hold's audio and the stretch come back. The native
transport CI job runs it, also under ThreadSanitizer.
A pause inside a fast-forward hold left the flush timestamp from before
it, so the first flush after the resume measured the whole pause as one
interval. The speed estimate then read the core several times slower
than real time, outside the time stretch's duration range, and the
stretch stayed off until fast-forward was released; inline, only with
Speedup off.
Reset the measurement in audio_driver_pause_fade(), on the main thread
as the frame end resets it. Every pause and resume goes through it, and
so do the jump fades of a state load or core reset, where the reseed is
harmless. The next flush seeds the estimate at the configured ratio, as
the start of a hold does.
The ff_speedup and discrete_multichannel harnesses pause a 2x hold for
125 ms, threaded and inline, and check that the speed estimate and the
stretch hold through it.
In Ozone, opening an image from the file browser with OK and pressing
Up/Down to step to the next image changed the picture but kept the
first image's name as the title.
The title was refreshed by each menu driver on a selection change, and
Ozone only did that when the fullscreen view had been asked for with
START. The viewer opened by OK sets the "shown" flag but not the
"wanted" one, so its title was never refreshed while stepping.
Every step through the viewer - the OK that opens it, Up/Down/Left/
Right, continuous playback - goes through
menu_media_open_selected_file, so the title now belongs there: it takes
the driver's title buffer and rewrites it for the file it opens. Ozone
and XMB pass theirs; no driver can open a file in the viewer and leave
the old name up. A row the viewer cannot show leaves the title alone.
samples/gfx/media_controls: viewer_title drives the real
menu_media_browser.c through OK, Down, a row the viewer cannot show,
Down past it and Up, checking the title each time, plus truncation to
the buffer and no buffer. It fails on the old code.
The frame header refused any JPEG whose component ids were not 1, 2,
3(, 4) or 0, 1, 2(, 3). Adobe's 4-component files - the CMYK and
YCCK the output path has a whole branch for - number their components
'C', 'M', 'Y', 'K', so every one of them failed to load: Photoshop
CMYK JPEGs, and Pillow's or libjpeg's CMYK output. libjpeg takes any
ids; the scans name components by id anyway. Now any ids are taken,
and only two components sharing one is refused.
That also lets through the 3-component files that hold R, G and B
rather than Y, Cb and Cr, which the same check used to refuse (their
ids are 'R', 'G', 'B') and which would otherwise come out with a
YCbCr transform applied to RGB. The colour space is decided as
libjpeg decides it: a JFIF APP0 means YCbCr; else an Adobe APP14's
transform 0 means RGB; else ids 'R', 'G', 'B' do. RGB rows skip the
colour transform and the fused YCbCr kernel and are only put in the
requested byte order. The JFIF segment is now recognised for this.
Decoded pixels are unchanged for every YCbCr and greyscale image in
the 176-file Pillow and 24-file cjpeg corpus (baseline and
progressive, all samplings, restart intervals), through the iterative
decode in both byte orders and the monolithic decode. The 66 Pillow
CMYK cases (baseline and progressive, 1x1 to 1920x1080), which all
failed before, now decode and match libjpeg's CMYK samples (C*K/255
etc. on the Adobe-inverted values) within one level; cjpeg -rgb
output (Adobe transform 0), its progressive form and the same file
with every APP segment stripped (ids alone) match libjpeg within one
level, where before they failed. Of 300 corrupted progressive files,
two that only failed for a mangled component id now decode, as they
do in libjpeg; ASan/UBSan and LeakSanitizer clean over 500 corrupted
files; the rjpeg and image_loader sample tests pass; builds C89, as
C++ and for aarch64.
A progressive JPEG decoded iteratively kept a full-size plane per
component alongside its coefficients, inverse-transformed every
component into its plane one after another, and only then - in
rjpeg_process_image - ran the resample and colour conversion over
the whole image, reading the planes back cold. At 4K 4:4:4 that is
24 MB of planes nobody needed whole and one more trip through DRAM.
The finish phase now goes one MCU row at a time across all
components: dequantise and inverse-transform that row's blocks, then
emit the output rows that are complete - one MCU row behind, exactly
as the baseline fusion does, since the vertical resampler's far row
for the tail of a span is the first plane row of the next. The fused
emitter is armed when the last scan ends; the planes, which the scans
never touch, are allocated ring-sized (three MCU-row spans) on the
iterative path, as baseline's already are, and rjpeg_process_image
finds the output made. The monolithic path keeps whole planes and its
own finish, unchanged.
Decoded pixels are bit-identical to before on 200 images - baseline
and progressive, 4:4:4, 4:2:2, 4:2:0 and cjpeg's 2x1, 1x2, 4x1, 4x2
and 1x4 sampling, restart intervals (every 3 MCU rows, every block),
an optimised progressive script, a jpegtran-progressive transcode,
greyscale, sizes 1x1 to 1920x1080 - through the iterative decode in
both byte orders and the monolithic decode, and on 300 corrupted or
truncated progressive files; ASan/UBSan and LeakSanitizer clean; the
rjpeg and image_loader sample tests pass; builds C89, as C++ and for
aarch64.
Progressive, iterative decode:
3840x2160 4:4:4 peak heap 110.8 MB -> 86.2 MB, LL misses -22%,
best 193.7 ms -> 178.5 ms
3840x2160 4:2:0 peak heap 72.5 MB -> 60.3 MB, LL misses -17%,
best 123.1 ms -> 116.4 ms
1920x1080 4:2:0 peak heap 18.2 MB -> 15.2 MB
GCC 15 warned that sv_ym may be used uninitialized in
rvp8_video_decode: the saved sets were filled under !kf and read back
in the else of a separate kf test, a pairing it does not follow. The
save now fills them for every frame - a key frame with the defaults it
used to reset to at the restore - and the restore copies them back
whenever refresh_entropy_probs is 0. The same entropy results, and
the warning is gone at -O2 and -O3.
Error-resilient (refresh_entropy_probs 0, a key frame every 20) and
normal 176x144 libvpx streams decode byte-exact against FFmpeg before
and after.
Every position a match covers goes into the hash chains so later
matches can reach it, and on the repetitive content screenshots are
made of - flat runs, gradients, upscaled pixel art - that is nearly
every byte. Each one was a call to rd_insert that re-chose the hash
(a run-time CPU-feature branch on a stock x86-64 build), loaded its
three bytes afresh, and was guarded by a per-position bounds test:
42% of a 4K screenshot's encode was spent there.
rd_insert_range() now does a whole interior at once: the bound is
worked out once (rd_insert_end, the same q + 3 <= end), the hash is
chosen once per run, the tables stay in registers, and the shift-xor
hash rolls from one position to the next - ((h << 5) ^ next) & mask is
exactly the hash of the next position, since what the mask drops from
h would be shifted out past it anyway. The chains it builds are the
ones the calls built, position for position, so the output does not
change. rd_insert itself, still used for the position being matched,
is inline.
Compressed output is byte-identical to before for levels 0-9, zlib-
wrapped and raw, fed 1, 7, 4096, 65536 bytes at a time or all at once,
on gradient, noise, flat-run, text, zero and mixed inputs - through the
run-time CRC32 hash, the shift-xor hash (CPU features masked) and the
compile-time SSE4.2 hash; ASan/UBSan clean on both run-time paths; the
deflate, rzip, archive_stream, archive_zip_methods, rpng and other
encodings sample tests pass; builds C89 without new warnings and for
aarch64 with and without +crc.
screenshot_io_bench, 3840x2160 PNG:
gradient instructions -25.6%, best 75.2 ms -> 58.8 ms
noise instructions -4.1%, time unchanged within noise
RSLC_EXTENDED takes the rest of the samplers - sampler1D, sampler3D,
samplerCube and samplerCubeArray of each base, and the 1D, 2D, 2D array,
cube and cube array shadow samplers - with every texture function GLSL
has on them, the shadow ones through the Dref sample and gather
instructions. It takes a stage redeclaring its gl_PerVertex block or
gl_FragCoord, a geometry stage's inputs declared as plain unsized
arrays, gl_PointCoord, gl_PrimitiveIDIn, and GL_EXT_spirv_intrinsics'
spirv_execution_mode with its capabilities and extensions; a float
controls mode declared without them gets the ones it needs. These go to
SPIR-V only; the other destinations refuse a stage that uses them.
A constant division by zero folded to zero; it folds to the infinity or
NaN IEEE gives, as glslang folds it. Cemu's decompiled shaders and the
community graphic packs written for them compile to SPIR-V that draws as
glslang's does, pixel for pixel on lavapipe.
A lookup finds directories as well as files, the root at the empty
path, and searches each directory's sorted entries by bisection instead
of scanning them. A listing's entries carry their place: a file's
offset and size, a directory's first entry and count. zar_index_offset()
tells a caller where the index begins, so it can keep that part of the
file in memory and answer lookups and listings without I/O.
zar_write.c builds an archive front to back through a write callback,
compressing blocks with rzstd where HAVE_RZSTD is defined and finishing
with the footer's SHA-256, so an emulator can make .zar/.wua files
without the reference ZArchive library. Its output reads back with the
reference reader, and the reference writer's output with this one.
Capping an oversized image box-averaged it by a power of two into a
second buffer of the reduced size, then either kept that (when it was
the target) or ran the sinc stage from it, freeing the original in
both cases. A 4K screenshot capped to 1920 held an 8.3 MB copy it did
not need: every averaged pixel lands below every source pixel a later
block still reads (the next block starts at y*f*sw + (x+1)*f, the store
is at y*dw + x, with f >= 2), so the reduction can be made in place.
Now it is. When the box result is the target the buffer is shrunk to
fit with realloc and kept - no allocation at all - except on GEKKO,
whose decoders memalign their output. When the sinc stage follows,
its output buffer and filter are made before the box runs, so a
failure still leaves the image exactly as it was; the box stage itself
cannot fail once its geometry has been checked.
Two things made the in-place loop slower until handled: the compiler
must assume the output aliases the source it is still reading, so a
row's averages are built in a 64-wide local run and copied out; and
the box being a power of two, the four divisions per pixel become
shifts, which give the same result for unsigned sums.
Output is bit-identical to before - pixels, dimensions and return value
- over 352 cases: 11 shapes from 2047x2 to 16385x3, eight caps, 8- and
10-bit, with and without the upscale stage; ASan/UBSan clean, no
leaks. The image_scale_transfer test now asserts the new ownership:
an exact box makes no allocation (it runs with an allocation budget of
0), a box + sinc makes one, and a failed sinc set-up leaves the source
untouched; image_loader passes too. Builds C89, as C++, for aarch64
and with GEKKO.
4K -> 1920x1080: additional image storage 8.3 MB -> 0, 17.9 ms instead
of 20.1 ms (8-bit) and 14.5 instead of 17.2 (10-bit).
An Adam7 image allocated a buffer for each of its seven passes,
converted the pass into it and scattered it into the image, and freed
it - and freed it again and reallocated it every time a pass found
its data not yet inflated, which on a streamed decode is most passes
several times over. Each pass also allocated and freed its own three
scanline buffers.
Now:
- pass 7 (x 0, stride 1) is the image's odd rows, whole, so it is
unfiltered and converted straight into them: the output cursor
steps two image rows a line (out_stride, which the regular path
sets to the width and passes 1-6 to the pass width). Half the
image is no longer written into a buffer and copied out again.
- passes 1-6 share one buffer, allocated once at the largest of them
and kept, through the waits too, until the decode ends.
- the three scanline buffers are one block, reserved once at the
image's full pitch, which no pass exceeds, and kept across passes.
All of it goes back through the buffer pool where one is set (the
pool's three line slots become one).
Decoded pixels are bit-identical to before on a new corpus of 150
interlaced images - grey 1/2/4/8/16-bit, RGB and RGBA 8/16, grey+alpha,
palette 1/2/4/8-bit with tRNS, tRNS on grey and RGB, sizes from 1x1
up through every empty-pass shape, single and many IDATs - and match
Pillow on the 8-bit ones; on the 546-image corpus, its multi-IDAT
re-chunking, the APNG set and 550 corrupted or truncated files;
ASan/UBSan clean, no leaks; the rpng sample tests pass; builds C89,
as C++, as one translation unit, for aarch64 and with GEKKO.
800x600 RGBA interlaced, decoded twice: 196 allocations (88 MB) ->
26 (10.4 MB), instructions -1.9%, D1 and LL data misses -8%.
The VP8L colour-indexing (palette) transform always expanded into a
second full-size buffer, w*h words, and then freed the indexed one.
That buffer is only needed when the indices are packed - 2, 4 or 8 to
a pixel for palettes of 16 colours or fewer - where the expansion runs
top to bottom and would overwrite packed rows not yet read. A palette
of 17 to 256 colours carries one index per pixel, so each index now
becomes its colour where it lies; the fused channel swap already
falls back to the stream's own buffer when there is no expansion
buffer.
Decoded pixels are bit-identical to before, and to libwebp's dwebp,
on palette images of 3, 12, 40, 100, 200 and 250 colours with and
without alpha (packed and unpacked), lossy images with palette-coded
alpha, a palette animation, and 160 corrupted palette files plus the
earlier corrupted sets; ASan/UBSan and LeakSanitizer clean; builds
C89, as C++ and for aarch64.
640x480 with a 40-colour palette: 1.2 MB less allocated,
instructions -7.7%, D1 misses -19%.
Each APNG frame still made a new inflate stream and freed it at the
end of the frame: for the built-in inflater a calloc of 73 KB - its
32 KB back-reference window and Huffman tables zeroed afresh - and for
zlib an inflateInit2 with its window allocation. On small frames that
zeroing cost more than decoding the frame.
trans_stream_backend gains an optional last member, reset: put a
stream back to the state stream_new gave it, keeping its allocations
and whatever define() set. The built-in inflater implements it with
rinflate_reset, which already restores a fresh instance without
re-zeroing the window or tables (a stale window byte is unreachable
once wprior is 0, and tables are rebuilt before use); zlib with
inflateReset, which keeps the window inflateInit2 allocated. Every
other backend lists NULL, and a backend that does not mention the
member gets NULL too.
rpng's buffer pool gains a slot for the stream. A decode takes it -
reset as it is taken - when its backend matches and can reset, and
otherwise makes one as always; a finished, failed or abandoned decode
gives it back, or frees it where there is no pool or no reset. So an
APNG stream inflates every frame with the one inflater, and stills
and every other trans_stream user are untouched.
Composited frames are bit-identical to before on the whole APNG set
and 250 corrupted or truncated files, with the built-in inflater and
with zlib; still PNGs decode identically on the 546-image corpus;
ASan/UBSan clean and no leaks under either backend; the rpng,
archive_file, archive_stream, archive_zip_methods and archive_zst
sample tests pass; builds C89 without new warnings, as one translation
unit and for aarch64.
Two passes of an animation, instructions / data refs / allocated:
128x128, 25 frames -50.6% / -69.5% / 4.1 MB -> 0.54 MB
77x51 Adam7, 10 frames -31.9% / -60.3% / 1.8 MB -> 0.38 MB
640x480, 20 frames -1.3% / -4.1% / 20.3 MB -> 17.5 MB
Every frame of a playing APNG allocated, and freed again afterwards,
the decoder's inflate ring (at least 64 KB), its three scanline
buffers, an Adam7 pass buffer per pass for interlaced frames, and the
frame's whole decoded image - mid-playback, every frame, every loop.
rpng now takes an optional buffer pool (rpng_internal.h): a slot per
role - ring, the three lines, the pass buffer, the output image - each
holding at most one buffer, the larger one when two meet. Every place
rpng allocated one of these takes it from the pool first and falls
back to the allocation it always made, and every place it freed one
gives it back instead; with no pool, giving is free(), so stills and
every other caller behave exactly as before. The output keeps its
32-byte memalign on GEKKO, a pooled image having been allocated there
too. The APNG stream owns a pool, hands each frame's image back once
composited, and releases it on close: once a stream has seen each size
it needs, frames stop allocating these.
Composited frames are bit-identical to before on every APNG in the
set (all colour types and depths, interlaced, palette with tRNS,
hidden default image, multi-fdAT frames, every dispose/blend mix) and
on 250 corrupted or truncated files; still PNGs decode identically on
the 546-image corpus, single and multi-IDAT; ASan/UBSan clean and no
leaks; the rpng sample tests pass; builds C89, as C++, as one
translation unit (griffin), for aarch64, and with GEKKO defined.
Heap allocations for two passes of an animation:
128x128, 25 frames 534 -> 294
77x51 interlaced, 10 frames 704 -> 412
640x480, 20 frames 470 -> 372, 5.6 MB less allocated
What remains per frame is the decoder's own state: the rpng and
process structs, the inflate stream and the IDAT span list.
Every APNG frame was decoded by building a standalone PNG in the
stream's scratch: the file's IHDR (with the frame's size), PLTE and
tRNS, then all of the frame's IDAT/fdAT payloads copied end to end
into one IDAT - the whole compressed frame, every frame, every loop.
The scratch PNG now carries an empty IDAT where the data would go, and
the payloads are handed to rpng as spans of the animation file itself
(rpng_add_idat_spans, internal through rpng_internal.h), read in place
exactly as a still PNG's IDAT chunks are. rpng keeps the base those
spans are relative to - buff_start unless a caller gives its own - and
refuses to mix the two. The APNG part list is the shared span type,
and the duplicate PNG signature goes in favour of rpng_internal.h's.
With the previous change a frame split over several fdAT chunks costs
no more at the seams than one in a single chunk.
Composited frames are bit-identical to before on RGBA, RGB and RGBA at
16 bits, grey and grey 16 interlaced, grey+alpha, palette with tRNS
(interlaced and not), Adam7 RGBA, a hidden default image, frames split
over up to five fdAT chunks, every dispose/blend mix, the earlier ten
animations, and 250 corrupted or truncated files (same frames, same
failures); ASan/UBSan clean; the rpng sample tests pass; builds C89, as
C++, as one translation unit (griffin) and for aarch64.
A 640x480 20-frame animation: instructions -1.4%, data refs -8.2%,
D1 misses -4.7%, 165 KB less allocated. Small frames gain little: the
decoder is still set up afresh per frame, which is the next step.
The compressed stream is fed one IDAT payload at a time. When a
chunk's payload ran out partway through an output slice, the step
returned, and the next one rebound the output to a fresh slice before
handing over the next chunk. To the inflater that rebind turns
everything produced so far into history, so the back-references that
reached across the seam were then served out of its 32 KB window a
byte at a time instead of copied within the buffer, and the window
was committed at every chunk end as well.
libpng and most other writers split the image data into 8 KB IDAT
chunks, so this hit nearly every PNG in the wild. Now, when a chunk
runs out with room left in the slice and another chunk follows, the
next payload is handed over and the inflate carries on into the same
output in the same step. Pacing is unchanged: a step still produces
at most one slice.
Decoded pixels are identical to before on the 546-image corpus (every
colour type and depth, interlaced and not, every filter) and on the
same corpus re-chunked into IDATs of 1, 7, 13, 100, 1000 and 8192
bytes with an empty IDAT mid-stream, which also decode identically to
the single-chunk originals; with the built-in inflater and with zlib;
ASan/UBSan and LeakSanitizer clean; the rpng sample tests pass.
A 3.1 MB RGBA PNG written with 8 KB IDATs now decodes at the cost of
the same image in one IDAT: instructions 664.3M -> 626.1M (-5.7%),
data refs -12.2%. Single-IDAT files are unchanged.
Wii U content did not scan. It now takes the serial lookup by its
product code:
- a WUD, or a Wii U disc by another name (.iso, probed only once the
other detectors find nothing, so other discs read nothing more), by
the "WUP-P-XXXX" its header starts with;
- a WUX, through the new formats/wux, the same octets of the disc it
stands for;
- a Wii U archive (.wua), by the product and company codes of its base
title's meta/meta.xml, read through formats/zar.
formats/wux reads WUX, the Wii U image format that keeps each distinct
sector once, as the WUD it stands for. Like the other readers it does
no file I/O: it asks for each range it needs. Opening reads the
32-octet header alone; the sector index is read a window at a time,
only where a read needs it; a read's data lands in its own destination,
one request per run of sectors the file keeps in order.
formats/zar gains a directory listing, zar_dir_begin() and
zar_dir_step(), each name read whole, the listing's end ZAR_END.
A Wii U release's serial in the database is the product code and the
maker's two characters, or the code alone for a title sold as a
download; a disc does not say its maker. The code is therefore looked
up as a stem, in the Wii U database alone - glob() in a libretrodb
query now matches binary fields, as a serial is kept - and the release
picked is the one of the maker an archive names, else a download for
an archive and a disc release for a disc. The playlist names the
release found.
wux_test is new, zar_test lists directories, and the scan test gains a
Wii U lane: a WUD, a WUX, a disc named .iso, two archives whose update
is listed before the game, and another database holding the codes.
A VP8 image or animation frame with an ALPH chunk built a w*h byte
plane for its alpha - a memcpy of the whole chunk for raw alpha, a
narrowing pass out of the VP8L decode for compressed alpha - then
unfiltered that plane top to bottom in a second pass and merged it
into the decoded pixels in a third, and freed it.
alph_apply() now takes the frame and does it a row at a time: the
row's filtered bytes go into a two-row buffer (the row, and the one
above that the vertical and gradient filters predict from), are
unfiltered there, and merged into the top byte of each pixel while
the row is still in cache. The w*h plane and its two cold passes are
gone; what remains is one 2*w byte buffer. Every check that can fail
(header, short raw data, a VP8L decode that fails) is made before the
first row is written, so a bad chunk leaves the frame opaque exactly
as before.
Output is bit-identical to before on stills with raw and compressed
alpha under every filter (none, horizontal/fast, gradient/best),
quantised alpha, lossy alpha animations from img2webp and ffmpeg, and
200 corrupted files; alpha matches libwebp's dwebp byte for byte;
ASan/UBSan and LeakSanitizer clean; builds C89, as C++ and for
aarch64. A 3655x2053 image with alpha allocates 7.5 MB less per
decode and takes 8-10% fewer data-cache misses; instructions are
unchanged.
An animation frame that covers the whole canvas and is copied rather
than blended - no_blend set, or a source whose alpha bytes are all
0xFF - was decoded into a buffer of its own and then memcpy'd row by
row over the canvas, every pixel of it, before that buffer was freed.
Such a frame replaces every canvas pixel and its buffer is exactly
canvas-sized, so the two now trade places: the decoded frame becomes
the canvas and the old canvas is freed in its place. Nothing the copy
would have overwritten (the key-frame clear, the previous frame's
dispose-to-background) is done first either. The stream's contract
already makes the returned canvas valid only until the next call, and
the channel-order switch converts whichever buffer is the canvas.
Composited frames are bit-identical to before on full-canvas lossy,
lossless and no_blend-with-alpha animations, sub-rectangle ones from
ffmpeg, img2webp (lossless, mixed, lossy) and gif2webp (lossless and
lossy, with disposal), and on the 300 corrupted files; ASan/UBSan and
LeakSanitizer clean; aarch64 builds. A 4 s 640x480 lossy animation
of full frames no longer copies 1.2 MB per frame: data-cache misses
-34%, data refs -0.7%, instructions -0.3% (decoding dominates).
Every AC-3 seek threw the decoder away and made a new one: a calloc of
the whole decoder (some 26 KB) and the 640 cos/sin evaluations of its
transform tables, only to land in the state the old one could be put
back into.
rac3_decoder_reset() now does that: it zeroes the stream state - the
overlap delay, coefficients, grouped-mantissa and E-AC-3 frame state,
scratch - and reseeds the dither, exactly as rac3_decoder_new leaves
them, while the tables and the caller's DRC setting stay. The struct
puts those first so the reset is one memset from the first stream
field to the end. The seek calls it instead of new + free, and can
no longer fail on an allocation.
Output is bit-identical to before for 1/0, 2/0 (AC-3 and E-AC-3),
3/0, 3/1, 4/0 and 5.1 streams, f32 and s16, reads of 1 to 10000
frames with six seeks each (to 0, to a block boundary, mid-block,
past the end); ASan/UBSan and LeakSanitizer clean; the rac3 and
audio_transfer_ac3 sample tests pass; builds C89, as C++ and for
aarch64. 500 seeks with a short read after each on a 5.1 stream:
490 allocations (13.3 MB) -> 7 (0.25 MB), instructions -1.1%.
Redump gives one serial to every release of an Xbox or Xbox 360 title,
and to the extras discs that came with them, so a serial lookup found
the first record - "Halo Movies (Japan)" for a North American Halo.
formats/xdvdfs now reads the regions a title is for: the game region of
an XBE's certificate, and the region of an XEX's security info, each
only where the executable is long enough to hold it. The scanner reads
the region field of the records a serial matches when it has regions to
go by, and takes the first record whose regions overlap them; when none
does, the first record stands in, as before.
The duplicate DB_EXTRACT_* definitions in database_info.c go: the
header carries them, with DB_EXTRACT_REGION added.
Pressing OK on an image in the file browser read and decoded the file
a second time even though its thumbnail was already on screen.
menu_media_open_selected_file only reuses a thumbnail when
gfx_thumbnail_has_file says it holds the file, and that answered yes
only for a live animation session. Every still (PNG, JPEG, BMP, TGA,
single-frame GIF/WEBP) has no session, and neither does an animation
the preview budget refuses - large ones land as a still - nor one that
finished its last loop. All of those were reset and loaded again.
Each thumbnail now records a 64-bit hash of the path it was requested
from (no per-thumbnail path buffer), cleared on reset.
gfx_thumbnail_has_file matches it while the still is shown, or while
its load is in flight and not superseded; a failed or cancelled load
still gets requested again.
samples/gfx/gfx_thumbnail_anim lane 16 drives the real
gfx_thumbnail.c: a still is recognised in flight and once shown, and
not for another path, after a reset, when superseded or when the load
failed. It fails on the old code.
A key frame freed the per-macroblock info array, and the next inter
frame calloc'd it again, so every key frame of a stream playing cost
a free and an allocation of mbw * mbh entries (8160 at 1080p) in the
middle of playback.
The video decoder now owns the array for its life and grows it only
when a key frame brings more macroblocks. A key frame still detaches
it from the decode state, and the first inter frame after one clears
the entries it uses, which is what the calloc gave it, so the decode
reads exactly what it did. A failed begin leaves the array with the
decoder instead of freeing it.
Output is bit-identical to before on every stream in the VP8 set
(altref, error-resilient, partitions, odd sizes, 1080p, noisy), a
keyframe-every-7 stream, one switching between three frame sizes
mid-stream, and 120 corrupted streams; ASan/UBSan and LeakSanitizer
clean; builds C89, as C++ and for aarch64. Heap allocations for the
4 s keyframe-every-7 clip: 23 -> 9.
Turning a NAL into its RBSP moved every byte through a four-way test
for an emulation_prevention_three_byte, one byte at a time, though
such a byte turns up only a few times per megabyte. The scan now
jumps from one 0x03 to the next with memchr, checks the neighbours of
each against the same rule on the same input bytes, and memcpys the
stretches between the ones that qualify.
SPS and PPS NALs were unescaped into a malloc of their own and freed
again. Streams repeat their parameter sets in band, often before
every IDR, so that ran during playback. They now unescape into the
current context's scratch, as slices already do; nothing of a slice is
in it between NALs, a queued slice having taken its buffer along.
The new unescape matches the old byte for byte on 3,000,000 random
buffers of 0 to 63 bytes drawn heavily from 0x00-0x03, under ASan with
exact-size buffers. Decoded pictures are identical to before on
baseline, main with in-band parameter sets every 10 frames, high with
8 slices, B-frames, CAVLC at qp 0, a flat high-bitrate clip and 4:4:4
lossless; the lossless (threaded and not), state and hw sample tests
pass; ASan/UBSan clean; aarch64 builds.
Instructions to decode a whole clip:
640x360 CAVLC qp 0, 6.9 MB 2831.5M -> 2750.8M (-2.9%)
640x360 4:4:4 lossless 2582.6M -> 2560.6M (-0.9%)
640x360 baseline crf 20 808.5M -> 802.8M (-0.7%)
Heap allocations, main with parameter sets every 10 frames: 100 -> 83.
Every Huffman table in a VP8L stream cost four heap operations: a
malloc and free of an 8 KiB symbol scratch and a calloc for the table
itself, then the same again for the code-length table read before it.
An image with a meta-Huffman image has five tables per group and can
have hundreds of groups, so a 3655x2053 lossless WebP made 770 heap
allocations to decode, nearly all of them this, and an animation paid
it again on every frame.
Now:
- the stream decode allocates the symbol scratch once, in the same
block as its code-length scratch, and hands it to every table build;
- the code-length table (three-bit lengths, 7-bit root, so exactly 129
entries and never a second level) lives on the stack, 516 bytes;
- every group's tables are carved in turn out of one arena per stream,
sized at start for the five root tables of every group and doubled
when a second-level table does not fit. A move would invalidate the
pointers, so each table records its offset and the pointers are set
once the last group is read. The arena is zeroed as it is handed
out, exactly as the calloc'd tables were, so an incomplete code reads
the same.
Heap allocations for a decode, old -> new:
3655x2053 lossless 770 -> 23
1024x768 lossless (-z 9) 304 -> 23
640x480 lossless (-z 9) 132 -> 23
10-frame 480x360 animation 896 -> 174
Total bytes allocated falls 8-24%; peak heap rises 0.3-1.6% with the
arena's slack. Instruction counts and data-cache misses are unchanged
within noise: pixel decode dominates the time.
Output is bit-identical to before on lossless (-z 0/6/9, -exact with
alpha), near-lossless, lossy with lossless alpha, still and animated
(lossless and mixed) images, and on 300 corrupted files (same pixels,
same failures); ASan/UBSan and LeakSanitizer clean. Builds C89
-pedantic warning-free, as C++, and for aarch64.
gfx_thumbnail_anim_test stopped linking when the savestate preview
started from the configured state folder: gfx_thumbnail.c now calls
dir_get_ptr(RARCH_DIR_SAVESTATE), which gfx_thumbnail_preview's stubs
already answer and this harness's did not. It answers with an empty
folder, as an unset one reads.
Every Opus packet was decoded into the context's pending buffer and
then copied into the caller's, whether or not the caller had room for
it as it stood.
A read with nothing pending now looks at the next packet's length in
its TOC byte and, when the whole packet fits in what is left of the
caller's buffer, decodes it there and counts it as produced; pending
is not touched. Packets that cannot go out exactly as decoded still
take the pending path: one with pre-skip left to drop off its front,
any packet on a movie timeline (which maps and may overlap or pad
it), and the tail of a read too short for the next packet. The end
bound (end granule, WebM padding) applies by count either way. A
malformed packet whose TOC undercounts it is refused by ropus for
room before anything is decoded, and goes to pending as before, so
errors land where they always did.
Output is bit-identical to before for mono, stereo and 5.1 (mapping
family 1), 2.5, 20, 40, 60 and 120 ms packets, Ogg and WebM, s16 and
f32, reads of 1 to 10000 frames with and without a seek, and for 240
corrupted streams; ASan/UBSan clean, the audio_transfer sample tests
pass.
Decoding a whole stream, 4096-frame reads, instructions / data refs:
stereo 20 ms f32 -0.2% / -0.4% s16 -0.1% / -0.2%
5.1 f32 -1.4% / -8.8% s16 -0.5% / -4.0%
The AAC arm decoded every access unit into a float buffer of its own
(raac interleaving and scaling its planar synthesis output into it),
then copied that buffer into the caller's on an f32 read, or, on an
s16 read, scaled each sample back up by 32768 and rounded it one at a
time. Every sample was written twice, and s16 paid an extra scalar
conversion pass that raac already has in SIMD.
raac now offers the decode in two halves: raac_decode() leaves the
frame inside the decoder, planar, where synthesis put it, and
raac_copy_s16()/raac_copy_f32() interleave any slice of it into a
caller's buffer, converted exactly as raac_decode_s16/raac_decode_f32
would. Those two entry points are now the decode plus a copy of the
whole frame, and the copies keep their SSE2/NEON paths at any offset
and length, with a scalar tail; mono f32 gets a vector path too.
The AAC arm decodes with raac_decode() and each read copies its slice
straight from raac into the caller's buffer, in whichever format it
asked for, so s16 and f32 still mix freely mid-packet. The per-stream
float buffer, its allocation at start and its free are gone.
Output is bit-identical to before for mono, stereo, 3- and 6-channel,
ADTS and M4A, and the doubled-rate (16/22/24 kHz) SBR path, in f32,
s16 and alternating reads of 1 to 10000 frames with a seek, on SSE2
and the scalar build; ASan/UBSan clean, NEON and armv7 build, the
raac, audio_transfer and rmp4 sample tests pass.
Instructions to decode a whole stream, 4096-frame reads:
44.1 kHz stereo M4A s16 168.1M -> 136.4M (-18.8%)
f32 131.4M -> 130.7M (-0.6%)
22 kHz stereo (SBR) s16 419.5M -> 389.8M (-7.1%)
f32 391.6M -> 384.4M (-1.8%)
formats/zar looks files up in a ZArchive and reads them, with no file
I/O of its own. A lookup asks for a directory's entries a run at a
time and for each name only as far as it can match; a read asks for a
block kept as it is straight into its destination, decodes a
compressed block (HAVE_RZSTD) into the destination when the read
wants all of it, and otherwise into one block kept for the reads that
follow.
The scanner takes a .zar by the title of the default.xex it holds.
Each APNG frame is rebuilt as a PNG and handed to rpng, and three CRCs
were computed for it - over IHDR, IEND and the whole of the frame's
compressed data - though that PNG goes nowhere but rpng, which does
not check chunk CRCs. They are left zero.
The region a DISPOSE_PREVIOUS frame saves was reallocated to that
frame's size on every such frame. It is now allocated once, at the
canvas size every frame is checked to fit when the frames are indexed;
and if that allocation fails the region is not saved, rather than the
old buffer being kept at a size that may not match the frame restored
from it.
Four APNGs from FFmpeg and six variants rewritten to dispose to
previous (all frames, and alternating with background and none)
composite identically before and after, nine of them byte-exact
against FFmpeg; the tenth differs from FFmpeg before and after, where
a frame blended over a restored region shows the restored pixels as
the APNG specification has it and FFmpeg does not.
rpng_apng_bounds_test and rpng_decode_corpus_test pass. 0.7% fewer
instructions over a 20-frame animation.
Each game's description, year and manufacturer went through a
PATH_MAX_LENGTH stack copy, a trim, five strstr scans and, for each
HTML code found, string_replace_substring's allocate, copy and copy
back, before a last copy into the field. The replacing was also wrong:
rxml has already decoded the character references, so decoding
&, <, >, ' and " a second time turned a title's
literal "&" (written &amp; in the DAT) into "&".
The text is now trimmed by pointer, with the same whitespace rules as
before, and copied into the field once, bounded, as rxml hands it
over; the exported logiqx_dat_html_code_list table, which nothing else
uses, is gone with the loop.
Fields with entities, surrounding whitespace, over-long text, empty
text and UTF-8 read the same as before; only a doubly-escaped title
differs, and now reads correctly. A 60000-game DAT reads identically
and takes 23% fewer instructions to load and walk.
scan_begin_budget_test passes.
Each of the three UPnP replies (device description, external address,
port mapping) was copied into a NUL-terminated heap buffer, then
copied again by rxml_load_document_string, which keeps its own. The
buffer is now handed over with rxml_load_document_owned, which keeps it
as the document's or frees it on failure, so the second copy and its
allocation are gone and the caller no longer frees it.
natt_desc_parse_test passes.
The normalise stage compacts the input as it turns CR and CRLF into
LF, copying every scalar from its read position to its write position
- onto itself, byte by byte, for the whole file, until the first CR,
which most files never have. While nothing has been compacted the
scalar is already where it belongs, so both positions just advance;
copying starts at the first CR. One scalar a work item, and its UTF-8
validation, as before.
The subtitle_srt tests pass. Loading a 4.4 MB, 40000-cue SRT counts
3.6% fewer instructions in cachegrind; wall time on this box is within
its noise.
rvp8_begin filled the Y, U and V planes with 127 before every frame -
about 3 MB at 1080p - and the decode then overwrote all of it: every
macroblock writes its whole 16x16 (8x8 chroma) area, the 127 / 129
edges intra prediction reads beyond the frame are applied per
macroblock, and motion compensation clamps to the reference. The fill
is gone. WebP lossy, which decodes through rvp8_begin too, loses it as
well.
Seven libvpx streams - odd and tiny sizes (203x117, 17x9), noisy
content, alt-ref, error-resilient, four token partitions and 1080p -
decode byte-identical before and after, six of them byte-exact against
FFmpeg's libvpx decode (the error-resilient one already differed);
clean under ASan and UBSan, and Valgrind memcheck finds no use of an
uninitialised byte on five of them, so nothing read the fill. On the
1080p stream cachegrind counts 4.8% fewer instructions, 25% fewer data
writes and 43% fewer L1 write misses; 6.42 -> 6.31 ms a frame.
A VP8 frame whose refresh_entropy_probs is 0 uses the probability
updates it carries for itself only; the next frame starts from what
this one started from. rvp8_begin recorded the flag, and its comment
left the save and restore to the persistent decoder, which never did
either, so the updates carried on into every later frame.
Error-resilient streams send the flag on every frame, and they decoded
wrong from the first inter frame on.
rvp8_video_decode now keeps the context a frame inherits - coefficient,
MV and intra mode probabilities - before it reads its updates, and
when the frame turns out not to refresh, puts that back afterwards; a
key frame's starting point is the defaults, which are set again (the
mode and MV sets are refilled with theirs when found cleared, as the
next inter frame already does).
Four error-resilient libvpx streams - 352x288, a noisy 320x180, 640x360
with alt-ref and 97x55 with two token partitions - decoded wrong before
and are byte-exact against FFmpeg's libvpx decode now; six other
streams stay byte-exact; clean under ASan and UBSan.
rmp3_stream moved every byte twice on the way in and once more on the
way out:
- Every input byte was copied into the 4 KB hold, and after each frame
the hold's remaining ~3.6 KB was shifted down - about nine times the
compressed stream moved, to reassemble frames that almost never
straddle a window.
- Each decoded frame went into the stream's own PCM buffer and was
copied out: 4.6 KB a frame for stereo s16, 9.2 KB for f32.
Now, with nothing held and the window holding as much as a full hold
would (or the rest of the stream once EOF is stated), the frame is
decoded where it lies, handed exactly the bytes the hold would have
been given, so the decoder sees nothing different; the hold is kept
only for a frame that straddles two windows, and a window that yields
no frame is left in the hold as before. With room in the caller's
buffer for any frame the stream can carry - 1152 frames of up to two
channels, a channel change fixed up in place there - the frame is
decoded straight into it; the synthesis stores sample by sample, so any
alignment will do. The first frame, which sets the stream's channel
count, and short outputs still go through the PCM buffer.
The output is bit-identical before and after on CBR and VBR, mono and
stereo, MPEG-1 and MPEG-2 and ID3-tagged streams, in s16 and f32, with
sliding windows of 700 bytes to the whole file and outputs of 100 to
5000 frames - 224 combinations; rmp3_frame_test passes; clean under
ASan and UBSan. Decoding a 128 kbps stereo file, memcpy and memmove
drop out of the profile and L1 data misses fall by 25%.
The title read takes the executable an Xbox 360 disc boots,
default.xex, ahead of an Xbox one's default.xbe, and reads one on its
own as well: an XEX's execution info, an XBE's certificate or an STFS
package's header, each only as far as the title needs, after a 24-octet
head that tells them apart. The game partition is also looked for where
development images keep it, and a padded root reads as empty.
The scanner takes .xex and .xcp files by their title, and Xbox 360
discs too; a disc of a set gets its place in it ("FO-2046-1"). A .xex
that is no Xbox executable, as an Atari 8-bit one, goes on to the CRC
lookup.
A FLAC's last frame is shorter than its block size, so the decoder
runs out of input partway through the frame it was sized for. rflac
reads that as the end of a span - roll back and wait for more - unless
rflac_set_eof says nothing more comes, and buffer mode, which hands the
decoder the whole file at once, never said so: the last frame was
waited for and lost. A 3.7 s stream at 44.1 kHz read 159744 of its
163170 frames with 4096-frame blocks, short by up to a block on every
file whose length is not a whole number of blocks.
Buffer mode now calls rflac_set_eof with the span it hands over, and
so does the seek table's jump, which hands over the rest of the file.
With the end stated, rflac no longer copies its decoder (4.4 KB) before
every frame for a rewind point that cannot be used: an underrun after
rflac_set_eof is the end of the stream, kept rather than rolled back,
so the point is taken only while more input may come.
Eight FLACs - mono, stereo, 24-bit and 5.1, blocks of 192 to 16384 and
one without a seek table - now read every frame, byte-exact against
FFmpeg's decode; what was read before is an exact prefix of what is
read now; seeks land exactly; audio_transfer_flac_test passes; clean
under ASan. Despite decoding the frames that were lost, cachegrind
counts 2.8% fewer instructions with 192-frame blocks and 0.8% with
1152.
LibretroAdmin