Commit Graph
25 Commits
Author SHA1 Message Date
LibretroAdmin 1d1f27a025 config_file: audit and rework — zero-copy pooled parsing (−63% load, −48% write, −99.8% heap ops), injectable file I/O, streaming API, autoconfig profile index (437 opens → 2), 5 pre-existing bugs fixed, fuzzing + regression tests in CI (#19305) 2026-07-30 08:47:22 +02:00
LibretroAdmin 0672da96d6 rhmap/config_file: let a map borrow its string keys
Every RHMAP_*_STR insert stores a private strdup() of the key. For
config_file that copy is pure overhead: the key handed to the map is
always the 'key' field of a struct config_entry_list that the config
file itself owns and that outlives the map at every site --
config_file_deinitialize tears the entry list down after the map,
config_unset removes the map slot before freeing the key, and both
config_file_add_child_list and config_append_file pilfer the donor's
entries before freeing the donor.

Add an opt-in RHMAP_BORROW_KEYS(map) that flips a per-map flag, so
inserts store the caller's pointer and teardown leaves it alone. The
flag lives in the header and is only read on insert, delete and free,
so the lookup path is untouched. Default behaviour is unchanged for
every existing user.

nested_list is the one other string-keyed user in tree and it was
keying off the caller's transient token rather than the copy it had
just made one line above. That is correct today only because the map
duplicates the key; make it key off its own copy so the call site is
right regardless, and bail cleanly if that copy failed to allocate
instead of hashing a NULL key.

Measured on a 1700-entry retroarch.cfg, load plus one
config_get_array() per key, 100 iterations, 5 runs each:

  before   63.2 - 64.6 ms  (mean 63.8)
  after    52.8 - 54.9 ms  (mean 54.0)   -15.4%

cachegrind (--D1=49152,12,64 --LL=1048576,16,64):

  D1 misses  124,714 -> 82,556   -34%
  D refs     5.62M   -> 4.53M    -19%
  Ir         14.67M  -> 11.8M    -20%

Allocation count for the same workload drops from 13,671 to 10,271,
exactly two per key. Identical lookup hit counts before and after;
valgrind reports no leaks and no errors.

Before this change 57% of D1 read misses on a config load were inside
malloc/free rather than in config code at all.
2026-07-27 09:13:38 +02:00
U-DESKTOP-SPFP6AQ\twistedtechre 8c78693e65 libretro-common/include/array/rbuf: fix OOM safety in RBUF_PUSH and RBUF_RESIZE
The RBUF (stretchy buffer) macros had two latent OOM crash
paths and one buffer-overflow path, affecting every RBUF user
in the tree (24 RBUF_PUSH call sites, 20+ RBUF_RESIZE sites
across gfx, menu, input/bsv, network, playlist, tasks,
core_updater_list, m3u_file, nested_list).

=== The three bugs ===

Pre-patch macros:

    #define RBUF_FIT(b, n) \
        ((size_t)(n) <= RBUF_CAP(b) ? 0 \
         : (*(void**)(&(b)) = rbuf__grow((b), (n), sizeof(*(b)))))
    #define RBUF_PUSH(b, val) \
        (RBUF_FIT((b), 1 + RBUF_LEN(b)), \
         (b)[RBUF__HDR(b)->len++] = (val))
    #define RBUF_RESIZE(b, sz) \
        (RBUF_FIT((b), (sz)), \
         ((b) ? RBUF__HDR(b)->len = (sz) : 0))

And rbuf__grow returns:
  * NULL on first-time (buf==NULL) malloc failure
  * buf unchanged on realloc failure for non-NULL buf

BUG 1: first-time PUSH to NULL buf, malloc fails

    int *buf = NULL;
    /* malloc fails */
    RBUF_PUSH(buf, 42);
    /* expands to: rbuf__grow(NULL, 1, sizeof(int)) returns NULL
     * -> b becomes NULL
     * -> (b)[RBUF__HDR(b)->len++] = val
     *    = NULL[((rbuf__hdr*)NULL)-1].len++ = ...
     *    NULL-deref */

BUG 2: subsequent PUSH past cap, realloc fails

    /* buf has len=cap=16, realloc to 32 fails */
    RBUF_PUSH(buf, 42);
    /* rbuf__grow returns original buf unchanged
     * -> b stays same, cap still 16, len still 16
     * -> (b)[16] = val     <-- writes past end of allocation
     * -> len++             <-- len now 17, exceeds cap=16
     *
     * Subsequent PUSHes also write past end, each further
     * bumping len.  Iteration over 0..LEN(b) reads past the
     * allocation.  Memory-corruption territory. */

BUG 3: RESIZE to grow, realloc fails

    /* buf has len=2, cap=16 */
    RBUF_RESIZE(buf, 100);
    /* rbuf__grow returns unchanged buf, cap still 16
     * -> RBUF__HDR(b)->len = 100     <-- len now 100, cap=16
     *
     * Caller iterates 0..100 reading past the 16-elem
     * allocation. */

=== The fix ===

RBUF_FIT and rbuf__grow are left untouched - their contracts
are fine.  The fix is in RBUF_PUSH and RBUF_RESIZE: after
RBUF_FIT, re-check 'RBUF_CAP(b) >= needed'.  If that's false,
the grow failed; skip the write / len bump entirely.

New RBUF_PUSH (do-while idiom, statement-only):

    #define RBUF_PUSH(b, val) \
       do { \
          RBUF_FIT((b), 1 + RBUF_LEN(b)); \
          if (RBUF_CAP(b) >= 1 + RBUF_LEN(b)) \
             (b)[RBUF__HDR(b)->len++] = (val); \
       } while (0)

New RBUF_RESIZE (still an expression):

    #define RBUF_RESIZE(b, sz) \
       (RBUF_FIT((b), (sz)), \
        ((b) && RBUF_CAP(b) >= (size_t)(sz) \
         ? RBUF__HDR(b)->len = (sz) : 0))

Walkthrough for each pre-patch bug:

  * First-time PUSH + malloc fail: RBUF_FIT makes b=NULL.
    RBUF_CAP(NULL)=0.  0 >= 1 is false; no write.  b stays
    NULL, len=0.  Silent no-op, no crash.
  * PUSH at cap + realloc fail: RBUF_FIT leaves b unchanged,
    cap unchanged (say 16, and LEN=16).  16 >= 17 is false;
    no write, no len bump.  Data intact, silent no-op.
  * RESIZE to grow + realloc fail: RBUF_FIT leaves cap
    unchanged.  cap(16) >= sz(100) false; len not assigned.
    buf valid at old len/cap.  Silent no-op.

=== Statement-form do-while(0) compatibility ===

Changing RBUF_PUSH from expression to do-while(0) makes it
statement-only.  Audited all 24 call sites in the tree: every
one uses RBUF_PUSH as a trailing-semicolon statement.  None
uses it as an expression (no 'x = RBUF_PUSH(...)').

The do-while(0) idiom is the canonical C statement-macro form
precisely so it parses correctly as the body of an
unbraced if/else/while/for.  The five call sites in the tree
that do appear in unbraced if/else bodies (gfx_animation.c:850,
852; menu_explore.c:395; menu_explore.c:643;
network/drivers_wifi/connmanctl.c:120) compile correctly
because 'do {} while(0);' is syntactically one statement, and
the trailing semicolon from the call site terminates the
do-while naturally.

RBUF_RESIZE remains an expression (conditional operator form)
to preserve any current expression-context use, though a
tree-wide grep shows all existing call sites are also
statement-context.

=== Verification ===

Unit-tested with a malloc/realloc wrapper that can be toggled
to fail on demand:

  * Test A (first-push OOM): buf stays NULL, len=0 — pre-patch
    segfaulted, post-patch clean.
  * Test B (push at cap, OOM): len=16 cap=16 unchanged, data
    intact — pre-patch wrote past allocation.
  * Test C (RESIZE grow, OOM): len and cap unchanged at old
    values — pre-patch set len=100 with cap=16.

Also compile-tested all direct consumers:
  * gfx/gfx_animation.c
  * input/bsv/uint32s_index.c
  * menu/menu_explore.c
  * network/drivers_wifi/connmanctl.c, nmcli.c
  * core_updater_list.c
  * libretro-common/formats/m3u/m3u_file.c
  * libretro-common/lists/nested_list.c
  * playlist.c
  * tasks/task_content.c

All compile clean with no RBUF-related warnings or errors.

=== Related prior fix ===

Commit 1c7e114 (menu + file_list ...) documents this exact
issue in its commit message under 'Out of scope' and applies
a local workaround in menu_explore.c's ex_arena_grow by
refactoring to avoid the vulnerable RBUF_PUSH-on-failure path.
This patch is the tree-wide fix that makes that workaround
unnecessary, though we leave the menu_explore.c defensive
code in place as it is still correct and has additional
semantics (NULL-propagation through ex_arena_alloc).

=== Thread-safety ===

RBUF operations have always been single-owner (the caller is
responsible for its own serialisation).  This patch doesn't
change that.  The new do-while form is still single-expression
from the concurrency viewpoint.

=== Reachability ===

OOM during RBUF_PUSH / RBUF_RESIZE is reachable from user-
driven paths:
  * playlist growth (every content addition, scan)
  * menu explore indexing (every Explore menu open/rescan)
  * gfx animation queueing (every animated UI transition)
  * core_updater_list population (every online core update fetch)
  * wifi scan results (every scan)
  * input/bsv uint32 indexing (movie record/replay)

All of these are user-reachable and have been silently fragile
against OOM; this patch closes the gap at the foundation.
2026-04-24 03:05:45 +02:00
libretroadmin afc9cb509a CXX_BUILDFIX buildfixes 2024-07-20 19:42:45 +02:00
libretroadmin 60e0dd839a iC comments 2024-07-20 17:54:44 +02:00
libretroadmin 7c6054573e (rhmap) use custom strdup 2024-07-06 00:00:29 +02:00
libretroadmin 57c790646f Buildfix 2024-06-29 03:48:50 +02:00
libretroadmin 47ec7c0782 Try to avoid strdup to avoid portability issues 2024-06-29 03:46:06 +02:00
libretroadmin 40c889b562 Add header include for rhmap.h 2024-06-29 03:25:33 +02:00
Bernhard Schelling 780fdf721e Fix possible hash map corruption when deleting string keys 2022-03-21 15:07:53 +01:00
jdgleaver b1a7d3e995 (config_file) Modify hash map usage for proper compatibility with updated RHMAP implementation 2021-04-28 17:34:24 +01:00
Alcaro 2c5c4647e3 Track the complete string in rhmap 2021-04-28 16:09:45 +02:00
Daniel De Matteis 59952c520b (Mac) Silence more warnings 2021-01-26 20:39:36 +01:00
Bernhard Schelling 9bea4831d7 Add 32-bit key hash map rhmap
Based on ex_hashmap32 in menu_explore.c, modify that to use the new rhmap
2021-01-03 00:52:25 +09:00
Bernhard Schelling 3de2c290fe Mark function possibly unused to suppress warning
Add attribute to avoid the warnings "rbuf__grow' defined but not used [-Wunused-function]" (GCC/Clang) and "unreferenced local function has been removed" (MSVC).
2020-12-10 09:35:08 +09:00
twinaphex 18cb55e207 Remove dynarray.h - rbuf.h replaces it 2020-08-14 16:09:13 +02:00
Bernhard Schelling 48fabb7f88 Simplify gfx_animation by switching from dynarray to RBUF 2020-08-13 05:23:32 +09:00
Bernhard Schelling 4a09907802 Return true when RBUF_TRYFIT is called with 0 2020-08-07 05:19:09 +09:00
Bernhard Schelling 0a67c0416a Change playlist to use dynamic array
Instead of initial 12MB memory allocation, use a dynamically growing array.
This shares the container structure (now called rbuf) with the explore menu.
2020-08-07 03:27:52 +09:00
twinaphex 74cd84399f Cleanups - Replace '== NULL' 2020-01-30 16:47:33 +01:00
orbea 28ff4b391a Clean up white space. 2019-02-03 16:00:50 -08:00
orbea bfc366decc Squeeze extra blank lines with cat(1).
Example:

find . -type f -iname '*.c' | while read -r i; do
  cat -s "$i" > "$i.new"
  mv "$i.new" "$i"
done
2019-01-08 11:04:58 -08:00
orbea f4a5c3bd55 Fix c89 build. 2018-12-15 21:03:28 -08:00
orbea 7c50dfb397 Fix c89 incompatible comments. 2018-12-15 18:55:21 -08:00
Stuart Carnie c00b6e0750 fix: Fix use of freed memory in menu animations
`menu_animation_update` enumerates `menu_animation.list` to process each
`tween`. It was observed that some tweens execute a callback that
pushes more animations via `menu_animation_push`. During the push, if
the tween `list` does not have enough space, a `realloc` occurs,
potentially invalidating the existing list. The remaining pointer access
in menu_animation_update is therefore invalid. Best case is the memory
is unused and thus does not affect the program. Worst case is memory
corruption.
2018-12-15 14:55:10 -07:00