playlist: budgeted resumable parse, and menu first-touch through it

Reading a playlist is tens of milliseconds of JSON parsing and six
allocations per entry, and until now all of it happened in one
blocking call on whichever thread asked - including the menu
thread, on the first touch of a large playlist.  The dirwalk work
capped the directory-enumeration half of that stall; this is the
parsing half.

playlist_read_file becomes a resumable state machine behind
playlist_parse_begin / _step / _end / _abort.  begin() opens the
file, sniffs the format and (for JSON) opens the parser; step()
advances the parse, consulting an optional budget callback between
batches of work and returning 0 when it runs out; end() yields the
finished playlist.  Crucially playlist_init() is now exactly
begin + one unbudgeted step + end, so the blocking and budgeted
paths run the same code and cannot drift apart.

The JSON phase replaces the rjson_parse() callback driver with a
caller-side rjson_next() loop.  This is a transcription, not a
rewrite: the driver's only private state was in_object and the
member-vs-value discrimination, and both are recoverable from the
public context queries (a string at object level with an odd
context count is a member name), so the loop is stateless between
events and can stop anywhere.  The old six-line format budgets at
line-group granularity, and the autofix path-rewriting pass -
previously an unbounded loop in playlist_init - becomes a third
budgeted phase over the same handle.

Wiring: menu_displaylist_set_new_playlist calls the deferred
cached init under the shared per-frame I/O window, exactly as the
dirwalk sites do.  Small and medium playlists complete inside that
one pass and behave as before; a large one yields, the entries are
marked for refresh, and the re-issued request resumes the same
parse rather than restarting it.  RARCH_MENU_CTL_DEINIT abandons a
pending parse before the cache it would install goes away.  The Qt
grid view keeps its blocking cold path - it must return a
populated model to its callers, so budgeting it needs Qt-side
continuation machinery rather than a drop-in swap; the comment
there now says so.

Aborting mid-document exposed the same staged-entry hazard the
oracle found in the finish path: an entry reserved one past the
committed length with members already strdup'd into it.  Both
paths now clear it.

The oracle gains four lanes: budgeted JSON (5000 entries, 23
yields, output identical to the blocking parse entry by entry),
budgeted old format, budgeted autofix, and an abort lane that
abandons a parse mid-document with entries staged.

Verified: playlist.c passes the strict C89 gate; all ten oracle
lanes pass plain and under ASan+UBSan+LSan with zero reports; the
scan oracle (which also links playlist.c) passes plain, ASan+UBSan
+LSan and TSan; the dirwalk oracle still passes; the CI step was
replayed literally; full Linux build links with HAVE_QT=1.

.github/workflows/Linux-samples-tasks.yml

@@ -529,7 +529,12 @@ jobs:
           # malformed-JSON-keeps-partial-data contract, the old
           # six-line format with its trailing metadata block, RZIP
           # compressed input, missing-file-reads-empty, and capacity
-          # clamping.
+          # clamping.  The budgeted lanes then assert the resumable
+          # API (playlist_parse_begin/step/end/abort) produces
+          # playlists identical to the blocking path while yielding
+          # repeatedly, for JSON, the old format and the autofix
+          # pass, and that abandoning a parse mid-document leaks
+          # nothing.
           #
           # The truncated-JSON lane also guards the staged-entry
           # cleanup in playlist_read_file: an entry interrupted

menu/menu_displaylist.c

@@ -199,6 +199,11 @@ void filebrowser_set_type(enum filebrowser_enums type)
  * completes (either way): mark the entries for refresh, exactly as
  * the explore init task does, so the repopulation re-issues the
  * request and consumes the finished listing. */
+static bool menu_displaylist_playlist_within_budget(void *ud)
+{
+   return task_nbio_slice_within_budget(ud, 0, 0);
+}
+
 static void menu_displaylist_dirwalk_refresh(unsigned tag)
 {
    struct menu_state *menu_st = menu_state_get_ptr();
@@ -3486,7 +3491,31 @@ static void menu_displaylist_set_new_playlist(
          playlist_config.capacity = (unsigned)content_favorites_size;
    }
 
-   if (playlist_init_cached(&playlist_config))
+   /* Budgeted first touch: one pass under the shared per-frame I/O
+    * window.  Small and medium playlists finish inside it and this
+    * behaves exactly as the blocking call did; a large one yields,
+    * and the refresh below re-issues the request, which resumes the
+    * same parse where it stopped.  The list is left empty for that
+    * frame rather than stalling the UI for the whole parse. */
+   {
+      nbio_budget_t b;
+      int pl_r;
+
+      task_nbio_slice_open(&b);
+      pl_r = playlist_init_cached_deferred(&playlist_config,
+            menu_displaylist_playlist_within_budget, &b);
+      task_nbio_slice_close(&b);
+
+      if (pl_r == 0)
+      {
+         /* Still parsing: come back next frame. */
+         menu_displaylist_dirwalk_refresh(0);
+         return;
+      }
+      if (pl_r < 0)
+         return;
+   }
+
    {
       playlist_t *playlist                      = playlist_get_cached();
       enum playlist_sort_mode current_sort_mode = playlist_get_sort_mode(playlist);

menu/menu_driver.c

@@ -6876,6 +6876,9 @@ bool menu_driver_ctl(enum rarch_menu_ctl_state state, void *data)
          if (menu_st->flags & MENU_ST_FLAG_DATA_OWN)
             return true;
 
+         /* Abandon a pending deferred playlist parse before the
+          * cache it would install goes away. */
+         playlist_init_cached_defer_abort();
          playlist_free_cached();
 
          /* End any in-flight background directory walk and drop an

playlist.h

@@ -186,6 +186,44 @@ playlist_config_t *playlist_get_config(playlist_t *playlist);
  **/
 playlist_t *playlist_init(const playlist_config_t *config);
 
+/* Resumable playlist parse.
+ *
+ * playlist_parse_begin() opens and sniffs the playlist file and
+ * returns a parse handle (NULL only on allocation failure; a missing
+ * or unreadable file resolves - as it always has - to an empty
+ * playlist at the first step).  playlist_parse_step() advances the
+ * parse, consulting @budget_cb between batches of work when
+ * non-NULL: it returns 1 when the playlist is complete, 0 when the
+ * budget ran out mid-parse (call again), and -1 on a failed parse.
+ * With a NULL @budget_cb the step runs to completion -
+ * playlist_init() is exactly begin + one unbudgeted step + end, so
+ * the blocking and budgeted paths cannot drift apart.
+ * playlist_parse_end() returns the finished playlist (or NULL after
+ * a failure) and frees the handle; playlist_parse_abort() abandons a
+ * parse at any point, releasing everything. */
+typedef struct playlist_parse playlist_parse_t;
+
+playlist_parse_t *playlist_parse_begin(const playlist_config_t *config);
+int playlist_parse_step(playlist_parse_t *p,
+      bool (*budget_cb)(void *), void *budget_ud);
+playlist_t *playlist_parse_end(playlist_parse_t *p);
+void playlist_parse_abort(playlist_parse_t *p);
+
+/* Deferred variant of playlist_init_cached(): same contract, spread
+ * over budgeted steps.  playlist_init_cached_deferred() returns 1
+ * when the requested playlist is cached and ready (cache hit, or the
+ * parse completed within budget), 0 while a parse is pending, -1 on
+ * failure.  While pending, playlist_init_cached_continue() advances
+ * the parse (it never touches the global cache, so a worker task may
+ * drive it; 1 means the parse finished and
+ * playlist_init_cached_finish() - main thread - must install it).
+ * playlist_init_cached_defer_abort() abandons any pending parse. */
+int playlist_init_cached_deferred(const playlist_config_t *config,
+      bool (*budget_cb)(void *), void *budget_ud);
+int playlist_init_cached_continue(bool (*budget_cb)(void *), void *budget_ud);
+int playlist_init_cached_finish(void);
+void playlist_init_cached_defer_abort(void);
+
 /**
  * playlist_free:
  * @playlist        	   : Playlist handle.

samples/playlist/playlist_parity_test.c

@@ -500,6 +500,255 @@ static void lane_capacity(void)
       fprintf(stderr, "[pass] capacity lane\n");
 }
 
+
+/* ------------------------------------------------------------------ */
+/* Budgeted lanes: the resumable API must produce byte-identical      */
+/* playlists to the blocking path, yielding along the way.            */
+/* ------------------------------------------------------------------ */
+
+static bool budget_countdown(void *ud)
+{
+   int *k = (int*)ud;
+   if (*k <= 0)
+      return false;
+   (*k)--;
+   return true;
+}
+
+static char *big_fixture_doc(unsigned entries, const char *base_dir)
+{
+   size_t cap  = (size_t)entries * 160 + 512;
+   char *doc   = (char*)malloc(cap);
+   size_t _len = 0;
+   unsigned i;
+
+   if (!doc)
+      return NULL;
+   _len += (size_t)snprintf(doc + _len, cap - _len,
+         "{\n  \"version\": \"1.5\",\n"
+         "  \"base_content_directory\": \"%s\",\n"
+         "  \"items\": [\n", base_dir);
+   for (i = 0; i < entries; i++)
+      _len += (size_t)snprintf(doc + _len, cap - _len,
+            "    { \"path\": \"%s/dir%03u/game%05u.bin\","
+            " \"label\": \"Game %05u\" }%s\n",
+            base_dir, i % 251, i, i, (i + 1 < entries) ? "," : "");
+   _len += (size_t)snprintf(doc + _len, cap - _len, "  ]\n}\n");
+   return doc;
+}
+
+static void check_playlists_equal(playlist_t *a, playlist_t *b,
+      const char *lane)
+{
+   size_t i, n = playlist_size(a);
+   CHECK(n == playlist_size(b), "%s: sizes differ (%u vs %u)",
+         lane, (unsigned)n, (unsigned)playlist_size(b));
+   if (n != playlist_size(b))
+      return;
+   for (i = 0; i < n; i++)
+   {
+      const struct playlist_entry *ea = NULL;
+      const struct playlist_entry *eb = NULL;
+      playlist_get_index(a, i, &ea);
+      playlist_get_index(b, i, &eb);
+      if (!ea || !eb
+            || !streq(ea->path, eb->path)
+            || !streq(ea->label, eb->label))
+      {
+         CHECK(false, "%s: entry %u differs", lane, (unsigned)i);
+         return;
+      }
+   }
+   CHECK(streq(playlist_get_default_core_path(a),
+               playlist_get_default_core_path(b)),
+         "%s: default_core_path differs", lane);
+}
+
+static void lane_budgeted_json(void)
+{
+   char path[512];
+   char *doc = NULL;
+   playlist_config_t config;
+   playlist_t *blocking = NULL;
+   playlist_t *stepped  = NULL;
+   playlist_parse_t *p  = NULL;
+   unsigned had = failures;
+   unsigned yields = 0;
+   int r;
+
+   if (!(doc = big_fixture_doc(5000, "/games")))
+   {
+      CHECK(false, "budgeted json: doc alloc");
+      return;
+   }
+   snprintf(path, sizeof(path), "%s/big.lpl", fixture_dir);
+   CHECK(write_whole(path, doc), "fixture write");
+   free(doc);
+   config_defaults(&config, path);
+   config.capacity = 8192;
+
+   blocking = playlist_init(&config);
+   CHECK(blocking && playlist_size(blocking) == 5000,
+         "budgeted json: blocking reference");
+
+   p = playlist_parse_begin(&config);
+   CHECK(p != NULL, "budgeted json: begin");
+   for (;;)
+   {
+      int k = 4;   /* four event batches per slice */
+      r = playlist_parse_step(p, budget_countdown, &k);
+      if (r != 0)
+         break;
+      yields++;
+   }
+   CHECK(r == 1, "budgeted json: step result %d", r);
+   stepped = playlist_parse_end(p);
+   CHECK(stepped != NULL, "budgeted json: end");
+
+   /* 5000 entries at ~14 events each vs 4*256-event slices: the
+    * parse cannot possibly fit one slice. */
+   CHECK(yields >= 10, "budgeted json: only %u yields", yields);
+
+   if (blocking && stepped)
+      check_playlists_equal(blocking, stepped, "budgeted json");
+
+   playlist_free(blocking);
+   playlist_free(stepped);
+   if (failures == had)
+      fprintf(stderr, "[pass] budgeted json lane (%u yields)\n", yields);
+}
+
+static void lane_budgeted_old_format(void)
+{
+   char path[512];
+   playlist_config_t config;
+   playlist_t *blocking = NULL;
+   playlist_t *stepped  = NULL;
+   playlist_parse_t *p  = NULL;
+   unsigned had = failures;
+   unsigned yields = 0;
+   int r;
+
+   snprintf(path, sizeof(path), "%s/old.lpl", fixture_dir);
+   CHECK(write_whole(path, old_format), "fixture write");
+   config_defaults(&config, path);
+
+   blocking = playlist_init(&config);
+
+   p = playlist_parse_begin(&config);
+   CHECK(p != NULL, "budgeted old: begin");
+   for (;;)
+   {
+      int k = 1;   /* one line group per slice */
+      r = playlist_parse_step(p, budget_countdown, &k);
+      if (r != 0)
+         break;
+      yields++;
+   }
+   CHECK(r == 1, "budgeted old: step result %d", r);
+   stepped = playlist_parse_end(p);
+   CHECK(stepped && blocking, "budgeted old: results");
+   CHECK(yields >= 1, "budgeted old: no yields at group granularity");
+   if (blocking && stepped)
+      check_playlists_equal(blocking, stepped, "budgeted old");
+   playlist_free(blocking);
+   playlist_free(stepped);
+   if (failures == had)
+      fprintf(stderr, "[pass] budgeted old-format lane (%u yields)\n", yields);
+}
+
+static void lane_budgeted_autofix(void)
+{
+   char path[512];
+   char *doc = NULL;
+   playlist_config_t config;
+   playlist_t *blocking = NULL;
+   playlist_t *stepped  = NULL;
+   playlist_parse_t *p  = NULL;
+   const struct playlist_entry *e = NULL;
+   unsigned had = failures;
+   int r;
+
+   if (!(doc = big_fixture_doc(600, "/games")))
+   {
+      CHECK(false, "autofix: doc alloc");
+      return;
+   }
+   snprintf(path, sizeof(path), "%s/fix.lpl", fixture_dir);
+   CHECK(write_whole(path, doc), "fixture write");
+
+   config_defaults(&config, path);
+   config.capacity      = 8192;
+   config.autofix_paths = true;
+   strlcpy(config.base_content_directory, "/mnt/newgames",
+         sizeof(config.base_content_directory));
+
+   blocking = playlist_init(&config);
+   CHECK(blocking && playlist_size(blocking) == 600,
+         "autofix: blocking reference");
+   playlist_get_index(blocking, 0, &e);
+   CHECK(e && e->path && strncmp(e->path, "/mnt/newgames/", 14) == 0,
+         "autofix: blocking rewrote paths (got \"%s\")",
+         (e && e->path) ? e->path : "(null)");
+
+   /* The fixture write above rewrote the file (autofix saves), so
+    * regenerate it for the stepped run to see identical input. */
+   CHECK(write_whole(path, doc), "fixture rewrite");
+   free(doc);
+
+   p = playlist_parse_begin(&config);
+   CHECK(p != NULL, "autofix: begin");
+   for (;;)
+   {
+      int k = 2;
+      r = playlist_parse_step(p, budget_countdown, &k);
+      if (r != 0)
+         break;
+   }
+   CHECK(r == 1, "autofix: step result %d", r);
+   stepped = playlist_parse_end(p);
+   CHECK(stepped != NULL, "autofix: end");
+   if (blocking && stepped)
+      check_playlists_equal(blocking, stepped, "autofix");
+   playlist_free(blocking);
+   playlist_free(stepped);
+   if (failures == had)
+      fprintf(stderr, "[pass] budgeted autofix lane\n");
+}
+
+static void lane_abort_midway(void)
+{
+   char path[512];
+   playlist_config_t config;
+   playlist_parse_t *p = NULL;
+   unsigned had = failures;
+   int k;
+
+   /* Reuse the big fixture from the budgeted lane. */
+   snprintf(path, sizeof(path), "%s/big.lpl", fixture_dir);
+   config_defaults(&config, path);
+   config.capacity = 8192;
+
+   p = playlist_parse_begin(&config);
+   CHECK(p != NULL, "abort: begin");
+   k = 2;
+   CHECK(playlist_parse_step(p, budget_countdown, &k) == 0,
+         "abort: expected a yield mid-parse");
+   /* Abandon with entries staged and the parser mid-document: LSan
+    * verifies nothing leaks. */
+   playlist_parse_abort(p);
+
+   /* Abort immediately after begin, and end-after-abort misuse
+    * guards. */
+   p = playlist_parse_begin(&config);
+   CHECK(p != NULL, "abort: second begin");
+   playlist_parse_abort(p);
+   CHECK(playlist_parse_end(NULL) == NULL, "end(NULL)");
+   CHECK(playlist_parse_step(NULL, NULL, NULL) == -1, "step(NULL)");
+   if (failures == had)
+      fprintf(stderr, "[pass] abort lane\n");
+}
+
 int main(int argc, char *argv[])
 {
    char cmd[600];
@@ -524,6 +773,10 @@ int main(int argc, char *argv[])
    lane_compressed();
    lane_missing();
    lane_capacity();
+   lane_budgeted_json();
+   lane_budgeted_old_format();
+   lane_budgeted_autofix();
+   lane_abort_midway();
 
    snprintf(cmd, sizeof(cmd), "rm -rf %s", fixture_dir);
    if (system(cmd) != 0) { }

ui/drivers/ui_qt_widgets.cpp

@@ -8754,7 +8754,15 @@ void PlaylistModel::getPlaylistItems(QString path)
        * the whole file parses on the UI thread.  Borrow the menu's
        * cached playlist when it is the same file - the loop below
        * deep-copies every field into QStrings, so nothing outlives
-       * the borrow - and parse only otherwise. */
+       * the borrow - and parse only otherwise.
+       *
+       * The cold path stays blocking deliberately: this function
+       * must return a fully populated model to its callers, so the
+       * budgeted playlist_init_cached_deferred() the menu uses
+       * would need Qt-side continuation (a timer driving the parse
+       * plus a model reset on completion) rather than a drop-in
+       * swap.  The cache borrow above already removes the common
+       * case of that cost. */
       playlist_t *cachedPlaylist = playlist_get_cached();
 
       if (   cachedPlaylist