rgif now loads GIFs wherever RetroArch loads images: thumbnails and
backgrounds through task_image, the image viewer core, the file browser
and the image_texture/image_transfer layer underneath them all.
rgif:
- the image_transfer interface the other decoders have:
rgif_set_buf_ptr, rgif_set_avail, rgif_header_ready and
rgif_process_image, giving a 32-bit texture of its own, ARGB32 or
ABGR32 in native byte order
- RGIF_ABGR, for ABGR32 on big-endian targets
- a file not all read yet gives IMAGE_PROCESS_WAIT and is not touched;
the image is decoded in one call once it is all in
Wiring, as HAVE_RTGA is wired:
- IMAGE_TYPE_GIF (image.h) and NBIO_TYPE_GIF (task_file_transfer.h),
both appended so no other value moves
- image_texture_get_type: .gif
- image_transfer: new, free, start, is_valid, set_buffer_ptr, process,
set_avail; the support matrix gains a GIF column
- task_image and task_file_transfer: GIF is an image task
- FILE_TYPE_GIF ("gif", "GIF"): opened as image content
- the file browser's image extensions, the image viewer core's
extensions, the System Information list ("GIF (RGIF)") and
--features ("RGIF"); SUPPORTS_RGIF in config.features.h
- griffin.c, Makefile.common (rgif.o), configure (HAVE_RGIF=yes), and
next to HAVE_RTGA in every platform Makefile, the Android, Apple,
MSVC, UWP and QNX projects and the CI compile matrix
- tools/settings_check.py and tools/stack_budget.py check with it
Also: menu_displaylist's image viewer extension list wrote "tga"
without moving its length on, so "webp" and "dds" landed on top of it
and TGA dropped out of the list; it moves on now.
Tests: rgif_test checks the texture in both byte orders against
rgif_decode, the wait before the whole file is in, and a refused file;
image_decode_fuzz sweeps GIF too (every byte of a small GIF set to
edge values, and every truncation), under ASan and UBSan. The griffin
unity build with every codec, the touched objects in a configured
build, the MinGW/C89 syntax check, the overlap check and the stack
budget pass.
The tree's 3DS layer (the custom crt0 and link spec, system and memory
setup, svchax, the old GPU command layer and the 3dsx/CIA launchers)
lives under os/ctr, beside os/gekko and os/wiiu.
The linear heap allocator is C: the same first-fit pool, so buffers
land where they did, with live allocations found by address in an
open-addressing table instead of a per-allocation tree node, and list
nodes taken from cache-line-aligned slabs and reused. On the same
trace it runs about half the instructions and practically no
malloc/free calls. With the whole heap allocated, a further allocation
now fails instead of starting the heap over and handing out its first
address again, and the free-space queries no longer fault.
samples/frontend/ctr_linear checks placement, alignment, coalescing,
the full-heap case and pages handed back from the heap's end.
The 3DS, Vita and Switch build smb://: the SMB and Kerberos clients
reach the network only through libretro-common's socket helpers, and
the crypto they rest on draws on each console's hardware random number
generator. The Wii U, whose crypto has only a clock to stir, is left
out: SMB derives its signing and sealing keys from that randomness. The
3DS keeps its SMB transfers at 64 KiB, as it does NFS's.
The compile matrix builds the SMB client, the Kerberos client and the
SMB VFS backend for each of the three under their Makefiles' defines.
The 3DS, Vita, Switch and Wii U build nfs:// - the NFS client needs none
of the crypto the RAM-starved consoles leave out, and it reaches the
network only through libretro-common's socket helpers, which these
consoles already build. Where a console's socket layer may not sleep
in a select() with no descriptors, the grace-period wait takes the
platform's own sleep (RARCH_CONSOLE); the 3DS keeps its transfers at
64 KiB, where four connections at the 1 MiB other platforms grow to
would hold 8 MiB of buffers for Wi-Fi that gains nothing from them.
The compile matrix builds the client, its VFS backend and the
prefetcher for each of the four under the defines their Makefiles
set: Wii U against its own wiiu/include, Vita against new stubs of
vitasdk's socket headers.
Every build uses the built-in TLS, SMB and NFS clients, so the
bundled deps/mbedtls, deps/bearssl-0.6 and deps/libsmb2 are removed.
A system library can still stand in for a built-in client, off by
default: --enable-mbedtls and --enable-bearssl replace the TLS client
with a system mbedTLS or BearSSL, and --enable-libsmb the SMB client
with a system libsmb2; a library asked for and not found is a
configure error, and two TLS libraries at once are refused. The
older builtin/system switches go, as do the platform files' bundled
selections, and the libsmb2 backend follows the library's current
share-enumeration API.
Linux-library-net-backends.yml replaces the legacy lane with one lane
per library: each configures with its option, builds RetroArch to
the link, checks the library's backend is linked and the built-in
client is not, and operates it - the cheevos login transport over a
local HTTPS server for mbedTLS and BearSSL, the SMB VFS tests against
a local smbd for libsmb2 (built from upstream; Ubuntu has no package).
The cheevos_login sample links the system BearSSL, the smb_backup
sample carries stand-in libsmb2 headers for its fake share, and the
samples workflow installs libbearssl-dev.
The last builds carrying another TLS library: the 3DS and Vita
makefiles compiled the bundled mbedtls, the Switch linked devkitPro's,
and the OpenDingux family had the bundled mbedtls (dingux) or a
HAVE_OPENSSL that linked a library no backend used (rg350, retrofw).
All six now select HAVE_CRYPTO, the keychain and HAVE_RETROSSL, as the
desktop, mobile and Windows builds do: the TLS 1.2/1.3 client with
certificate verification and the encrypted keychain, from sources the
tree already has, with nothing to link. The crypto's random source
already had a path for each of them (PS_GenerateRandomBytes,
sceKernelGetRandomNumber, randomGet). The 3DS makefile's note that TLS
could not go through griffin no longer holds, the unity lane proving
it, and the mbedtls include path goes with it.
The compile matrix compiles the crypto and keychain in the 3DS and
Vita shapes and the whole TLS client in the Switch shape, with the SDK
stand-ins the random sources need added to the stubs, and the whole
set with a MIPS cross compiler for the OpenDingux family; the matrix
job installs that compiler. The Vita, 3DS, Switch and RetroFW
workflows build the real things.
The configure builds, Android, Apple, UWP and webOS were on the
built-in codec, HAVE_RZSTD; nobody set HAVE_ZSTD, the reference
library, any more; and the console and handheld makefiles - 3DS,
PSP, PS2, Wii U, GameCube, Wii, Vita, Switch, Orbis, Emscripten, DOS,
the Dingux family, the Pandora, the Win32 griffin build - had
neither, so .zst archives, Zstandard CHD hunks, RVZ hashing and
compressed input replays were absent there. HAVE_RZSTD everywhere:
one C file with no dependency, compiled into griffin builds through
griffin.c and into the others through Makefile.common.
USE_CTRULIB_2 now defaults by probing the toolchain's gspgpu.h for
gspPresentBuffer, so plain make builds correctly against both current
libctru and the legacy ctr toolchain with no flag on the make line.
An explicit USE_CTRULIB_2=0/1 still overrides the probe, and a preset
CTRULIB is honored. The salamander makefile gains the same probe, so
the two makefiles agree on any given install.
Legacy libctru remains the out-of-the-box selection, keeping the
ctr-legacy CI lane and existing build setups working unmodified.
Builds on current devkitPro pass USE_CTRULIB_2=1 on the make line,
as the modern CI jobs already do.
Write bin2s output to a temporary .s and assemble it as a separate
step, so a bin2s failure aborts the rule instead of being masked by
the exit status of the assembler at the end of the pipeline. The
shader rules also go through CTR_PICASSO and CTR_BIN2S now, picking
up the .exe suffix on MSYS.
Ignore the .shbin and .shbin.s intermediates and drop them on clean,
so a rule that stops early leaves nothing behind that can reach a
commit.
Run the ctr-shaders recipe as a single shell command that writes
bin2s output to a temporary file and moves it into place only once
picasso and bin2s have both succeeded, so the checked-in
gfx/drivers/ctr_shaders/ctr_sprite_shbin.S survives a partial or
failed regeneration. A trap clears the intermediate .shbin and the
temporary on the way out, leaving no untracked debris in the tree.
Ship the assembled PICA200 shader as
gfx/drivers/ctr_shaders/ctr_sprite_shbin.S and select it whenever
picasso or bin2s are missing from the devkitPro install, so a 3DS
build needs nothing beyond devkitARM itself.
Regenerate the prebuilt shader with 'make -f Makefile.ctr ctr-shaders'
after editing ctr_sprite.vsh or ctr_sprite.gsh.
rthreads.c drives libogc, libctru and the PSP kernel directly on these
platforms rather than dressing them up as pthreads: threads, mutexes
and condition variables map onto LWP, LightLock/CondVar and sceKernel
threads with semaphores. sthread_isself answers on GEKKO,
sthread_raise_current_priority moves the caller up within each
scheduler's band, and an explicit rthreads priority maps across the
whole band on all three. GEKKO timed waits hand LWP the timeout in
the form the linked libogc expects - an absolute deadline on its RTC
clock for the in-tree fork under INTERNAL_LIBOGC, a relative timespec
for upstream - so they expire when asked instead of immediately or
never. 3DS thread creation passes its argument through threadCreate
rather than serializing every launch through a static handoff slot,
joins free the libctru thread object, and timed waits ride
CondVar_WaitTimeout's monotonic nanoseconds. PSP drops the pthreads
port from its link: a detached thread reaps itself through a
done/detached atomic handshake so nothing lingers dormant, and the
pthread_init calls in the PSP/Vita frontend are Vita's alone.
The desktop pthread and Win32 lanes preprocess to byte-identical
code. Makefile.ctr defaults USE_CTRULIB_2=1 to match current libctru,
with 0 still selecting the pre-2.0 CondVar backport; the unread
GX_PTHREAD_LEGACY knob is gone from the GEKKO Makefiles; and
stack_budget measures rthreads.c on its portable lane, since -DPSP on
a host compiler cannot see the PSP SDK headers the native backend
needs. Worst native frame is 48 bytes against the 2 KiB
libretro-common budget. Compiles clean on current devkitPPC (system
libogc and the in-tree fork), devkitARM and pspdev; full 3DS and PSP
builds link end to end.
ZLIB_LIBS is never assigned anywhere, so every makefile target that set
HAVE_ZLIB=1 compiled trans_stream_zlib.c and archive_file_zlib.c against
<zlib.h> and then failed to link -- there is no longer a vendored
implementation behind them. psl1ght hit this first; MSVC hit the
compile-side variant once the compat/zlib headers went away.
Take HAVE_ZLIB out of every platform. RetroArch decodes PNG with rpng,
so the -lpng that pulled zlib in on vita and wiiu was only ever a
convenience for statically linked cores, and android's -lz went with it.
No target enables zlib by default any more: trans_stream_zlib.c is now
reachable only through the explicit HAVE_ZLIB=1 opt-in in
Makefile.common and its matching guard in griffin.
psl1ght additionally drops freetype for stb font: libfreetype pulls
inflate/inflateEnd in through ftgzip, which is the same undefined
reference by another route.
Also repair damage from the earlier -I removal, which deleted the last
entry of several line-continued lists and left the preceding line with a
trailing backslash: vita and ctr swallowed the following 'else' (vita
lost main() entirely), orbis ended up with a literal '\' argument, and
psp1 and griffin were left with an empty continuation and a bare
'CFLAGS +='.
The database sample never built archive_file_7z's strldup dependency;
add compat_strldup.c. test_rpng and the json sample both compiled
trans_stream_zlib.c without a zlib to link against; point them at the
built-in DEFLATE backend instead.
deps/libz is gone, so libretro-common/include/compat/zlib -- the zlib
1.2.8 headers that sat in front of it -- no longer has an implementation
behind it. Every remaining <zlib.h> includer (bsvmovie, archive_file_zlib,
libchdr_zlib, trans_stream_zlib) guards the include on HAVE_ZLIB, so a
build configured against a system zlib picks up the system header and a
build without one never reaches it either way.
Drop the directory and the -I flags that pointed at it across the
platform makefiles, the Xcode projects and the MSVC 6 project.
The rzip sample was still compiling deps/libz directly and hardcoding
-DHAVE_ZLIB while linking trans_stream_zlib.c; it now builds against the
built-in DEFLATE codec alone.
The clean-room DEFLATE/inflate codec in encodings/deflate.c is now the
only decompression path. Every consumer already selected it when
HAVE_ZLIB was undefined -- archive_file_zlib, libchdr's zlib codec and
trans_stream all carry the fallback -- so the vendored zlib copy had no
remaining purpose.
Makefile.common: drop the HAVE_BUILTINZLIB object list and the
HAVE_ZLIB_COMMON aggregate. HAVE_ZLIB=1 now only adds
trans_stream_zlib.o, -DHAVE_ZLIB and $(ZLIB_LIBS). HAVE_CHD is gated on
itself rather than on zlib; HAVE_7ZIP and HAVE_RFLAC were already
outside the zlib gate and are unchanged.
griffin: drop the deps/libz includes and the HAVE_NO_BUILTINZLIB guard,
and unnest the CHD block from HAVE_ZLIB.
Platform makefiles: remove HAVE_BUILTINZLIB/HAVE_ZLIB and -DHAVE_ZLIB,
and drop -lz where zlib was its only consumer. wiiu additionally loses
its portlibs -lpng, which was a convenience for statically linked cores;
RetroArch itself builds HAVE_RPNG. libnx keeps -lpng -lz because its
ffmpeg chain links against both.
qb: drop the HAVE_BUILTINZLIB knob, its check_enabled entry and the
macOS workaround that existed only to avoid the vendored copy.
archive_file_7z.c now reads archives through r7z_archive, and CHD
decodes LZMA through r7z_lzma, so nothing in the tree depends on the
bundled LZMA SDK any more. This deletes it.
The backend rewrite is the substance of the change. r7z_archive parses
from a buffer rather than a stream, so the archive is read in whole
instead of through a CLookToRead2 over a CFileInStream. That suits how
this backend is used: callers already hold the file open and go on to
decode entire members into memory, and 7z cannot decode a solid folder
without holding all of it regardless. It also removes the SDK's
allocator plumbing, its file wrappers and the 16 KiB lookahead buffer
from the context.
One behavioural detail changed for the better. The old iterate step
handed back a pointer into the middle of a decoded folder
(output + offset); the new one hands back the member itself. The caller
in archive_file.c writes exactly `size` bytes from that pointer, so
this is equivalent and no longer depends on the folder buffer
outliving the member.
The per-member compressed size reported during iteration is now the
uncompressed size. It was never meaningful for 7z: members of a solid
folder share one compressed stream and have no individual packed size.
Callers use the value only for progress reporting.
CHD's LZMA path no longer has an SDK implementation to fall back to, so
the HAVE_RLZMA fork in Makefile.common collapses to always building
r7z_lzma. In the griffin unity build the two consumers are separated by
a guard, since libchdr and the 7z backend can each be enabled without
the other but the translation unit may only appear once.
Also drops -Ideps/7zip from every platform makefile, the Xcode header
search paths, the qb configure step and the database sample, and the
now-meaningless _7ZIP_ST define from all of them plus the MSVC project
files. Nothing in the tree references deps/7zip or _7ZIP_ST after this.
Verified: 50 files extracted across 11 archives covering every
supported codec and both header forms, byte-identical to the source
tree; 48,645 captured CHD LZMA streams still decode byte-exact; the
LZMA2, BCJ2 and streaming-LZMA corpora all still pass. Every 7z module
and the rewritten backend build C89 clean with -pedantic-errors -Wall
-Wextra -Werror.
stb_truetype.h and stb_rect_pack.h are no longer referenced by
anything in the tree (the rect packer had been unused since the
plain stb renderer was removed). Drop the vendored headers and the
dead -Ideps/stb include paths from all platform Makefiles.
The DDS loader landed with only the core wiring (Makefile.common,
griffin, qb). Enable it everywhere the WebP loader (HAVE_RWEBP) is
enabled, so DDS decodes on every target rather than only default builds.
- platform Makefiles: mirror every HAVE_RWEBP toggle and inline
-DHAVE_RWEBP with HAVE_RDDS (ctr, dingux, dos, emscripten, griffin,
lfx000, libnx, miyoo, ngc, openpandora, orbis, ps2, psl1ght, psp1,
retrofw, rg350, rg350_odbeta, rs90, switch, vita, webos, wii, wiiu,
msvc).
- MSVC .vcxproj (2010-2022, 360, UWP) and the Android NativeActivity
project: add the HAVE_RDDS preprocessor define next to HAVE_RWEBP.
- Apple .xcodeproj (OSX107, PPC, iOS6-11 incl. Metal/static): add
-DHAVE_RDDS to OTHER_CFLAGS next to -DHAVE_RWEBP.
- pkg/android Android.mk: add -DHAVE_RDDS.
- feature reporting: SUPPORTS_RDDS (config.features.h), the retroarch
--features string, the menu Core Information image-format list, and
the imageviewer core extension list (adds |dds).
All these targets build the image loaders through the griffin unity TU,
which already includes rdds.c under HAVE_RDDS, so defining the flag is
sufficient to compile and enable the loader.
The WebP decoder was only enabled through the qb configure default, so
platforms with static build files never compiled it. Wire HAVE_RWEBP
everywhere HAVE_RPNG is wired, following each file's existing style:
- Variable assignments (routed through Makefile.common's define/object
block or Makefile.griffin's CFLAGS emit): dingux, dos, emscripten,
lfx000, libnx, miyoo, msvc, ngc, retrofw, rg350, rg350_odbeta, rs90,
webos, wii, and every per-platform block in Makefile.griffin, which
also gains the matching CFLAGS emit.
- Platforms with both a griffin branch (inline -D defines) and a
default Makefile.common branch (variable assignments) get both:
ps2, ctr, switch, orbis, wiiu.
- Continuation-style define lists: psl1ght, psp1, vita, openpandora,
and the Android NDK build.
- MSVC project files (griffin-based, PreprocessorDefinitions):
msvc-2010 through msvc-2022, UWP, Xbox 360, and the legacy Android
NativeActivity project.
- Xcode projects (griffin-based, -D flag lists): all eleven, including
the PPC and legacy iOS projects.
The decoder itself is C89-clean scalar C on platforms without SSE2 or
NEON (the SIMD paths are compile-time gated), so no platform needs
anything beyond the define. The salamander Makefiles are deliberately
left alone: the loaders do not decode images beyond their existing
boot path. Console targets are not buildable in this environment; the
edits mechanically mirror the existing HAVE_RPNG wiring in each file,
and the Makefile.common route they rely on is the same one exercised
by the regular Linux build.
* Makefile.ctr: enable gc-sections
This frees ~160k of RAM without any downsides that I know of.
It's not much, but helps pcsx_rearmed on old3ds where it barely fits in RAM.
* remove some dupe logic
Regardless if `save_state_in_background` is enabled or not, it does
exactly the same so the check can be removed.
* optimize RAM state load
content_load_state_from_ram() doesn't need the backup copy,
content_save_state() never touches the ram_buf. It seems the whole
logic was copy-pasted from content_undo_load_state() without thinking.
* avoid useless memory use and copying
Just take the source pointer instead of making a copy and then freeing it.
This is a big deal on slow platforms having to deal with large states,
like psx on old3ds.
* allow the core to disable the savestate undo feature
The undo feature may amount to at least 4 simultaneous buffers:
save undo, load undo, the current save/load buffer, temporary buffer.
When emulating a system with relatively large states on a system with
little RAM (like PSX on old3ds) that amounts to something like ~18MB of
~42MB available to the core. Under memory pressure the libctru variation
in RetroArch tries to salvage some memory from the main stack and linear
memory, but since those are variable, they start to overlap creating
instability.
The solution is to just disable the undo feature.
-ffast-math was added in ff14092d8d with a comment that it "helps SINC
resampler to auto-vectorize". The flag is an alias for "-fno-math-errno
-funsafe-math-optimizations -ffp-contract=fast -fno-honor-infinities
-fno-honor-nans". The last two of those cause compiler complaints
because while the flag was meant for this file, it's shared across the
codebase that includes statements that operate on infinities and NaNs.
GCC 13.3 with -fopt-info-vec reports 18 vectorizations for `-O3
-ffast-math`, 17 for `-O3 -fno-math-errno -funsafe-math-optimizations
-ffp-contract=fast` and also 17 for plain `-O3`. So using the subset of
-ffast-math without the offending flags buys nothing and loses 1
vectorization.
Both GCC and Clang provide the "fast-math" pragma directive, which I add
to this one file that benefits from it, under the condition that it's
supported. -ffast-math is removed from most of the makefiles.
The point of making the default "semi-silent" (each target is printed,
but not the entire invocation verbatim) was to factor out information
about invocation to a one-time target (info) and not repeating it ever
again. With V=1, the info target is silent, because each command
invocation will be printed verbatim, which needs no herald.
The undesired side effect of me introducing the info target and making
it a dependency of the default all target is that info doesn't respect
the --silent make option, so in this use case it was more noisy, not
less.
I believe the most elegant solution that satisfies all use cases is to
stop printing info unconditionally, and instead list it explicitly in
make invocations in CI, together with the all target. Then this info is
kept available for inspection from logs, but it's not repeated on each
invocation. The intention was to make the logs easier to read and scroll
in search for new compiler warnings.
While here:
* remove input_overlay and task_overlay from linking blacklist
* move -Werror to CC invocation since C++ refuses to use it
* copy the quiet build mode from the main Makefile and make it the
default
* temporarily define DEPRECATED for gpu_old.h to allow succesful build
* replace the call to command_event() with one to retroarch_main_quit()
in a failure case
* add missing uncompr.o in Makefile.common for HAVE_BUILTINZLIB
* (3DS) Add bottom screen menu
-> User can save/load state on botom screen with thumbnail.
-> Call a save_state_to_file() when RAM state has data to write a disk.
-> If the bottom screen needs updating, swap the bottom framebuffers.
Add: SAVE/LODE STATE TO RAM
-> This is useful for devices with slow I/O
-> 3DS bottom save state use CMD_EVENT_SAVE_STATE_TO_RAM
-> 3DS bottom load state use CMD_EVENT_LOAD_STATE when RAM state has no data
-> 3DS bottom load state use CMD_EVENT_LOAD_STATE_FROM_RAM when RAM sate has data
* Rewrite path_get_state to retroarch_get_current_savestate_path
* Fix unterminated state_path