libretro/RetroArch
1
0 Fork 0
mirror of https://github.com/libretro/RetroArch.git synced 2026-11-07 08:36:02 -10:00
Code Issues Packages Projects Releases 1 Wiki Activity
Files
a4172fdafde1ae1600cb58fb9ab1dd6ee2da4df4
RetroArch/.github/workflows/Linux-samples-tasks.yml
T
LibretroAdmin a4172fdafd CI: the lanes Ubuntu 26.04 broke, and the formats warnings 
Built with ffmpeg 8, libaom 3.13, x265 4.1, glibc 2.43 and clang 21,
five lanes failed on the 26.04 runners:

- AV1 fixtures: a monochrome sequence comes out full range with no
  matrix whatever is asked for; the oracle takes the colour ffprobe
  reads instead of assuming it.
- Preview audio oracle: ffmpeg 8 leaves a timestamp gap at every seam
  of a -stream_loop copy, which the mixer honours and ffmpeg's raw
  decode does not. trailing_large loops its picture and encodes its
  sound as one track.
- rh265 multi-slice oracle: x265 4.1 writes unreadable slice headers
  when two slices are encoded on several threads; those cases encode
  on one worker.
- vkms: the kernel follows the runner's own release, its modules may
  sit under /usr/lib/modules, and modules-extra is fetched only where
  it exists.
- clang -std=c89 -pedantic: glibc's const-generic strchr/strstr under
  _GNU_SOURCE are _Generic; -Wno-c11-extensions goes wherever
  -Wno-c99-extensions already does.

ci_paths filters a file at the top of a samples tree by its own path.
rh264 and rh265 lose the parameters no instantiation reads, the 8-bit
H.264 kernels mark the depth they ignore, and the AV1 WebM fixtures
leave out the MP4 box writer they never use.
2026-11-05 17:36:02 +01:00

2365 lines
121 KiB
YAML
Raw Blame History

name: CI Linux samples/tasks
on:
push:
branches:
- master
# ci_paths: generated by tools/ci_paths.py, do not edit
paths:
- '**'
- '!samples/**'
- '!libretro-common/samples/**'
- '!.github/workflows/*.yml'
- 'libretro-common/samples/encodings/rbzip2/**'
- 'samples/menu/dirwalk/**'
- 'samples/playlist/playlist_parity_test.c'
- 'samples/tasks/autodetect_handoff/**'
- 'samples/tasks/bluetooth_scan/**'
- 'samples/tasks/bps_patch/**'
- 'samples/tasks/bsv_checkpoint/**'
- 'samples/tasks/bsv_replay/**'
- 'samples/tasks/bsv_replay_init/**'
- 'samples/tasks/bsv_statestream/**'
- 'samples/tasks/cloudsync/**'
- 'samples/tasks/content_prefetch/**'
- 'samples/tasks/core_backup/**'
- 'samples/tasks/core_updater/**'
- 'samples/tasks/database/**'
- 'samples/tasks/decompress/**'
- 'samples/tasks/file_transfer/**'
- 'samples/tasks/http/**'
- 'samples/tasks/http_emscripten/**'
- 'samples/tasks/image_decode_slice/**'
- 'samples/tasks/input_remap/**'
- 'samples/tasks/natt_desc/**'
- 'samples/tasks/netplay_nat/**'
- 'samples/tasks/ordering/**'
- 'samples/tasks/overlay/**'
- 'samples/tasks/patch_stream/**'
- 'samples/tasks/pl_thumbnail/**'
- 'samples/tasks/playlist_manager/**'
- 'samples/tasks/save_state/**'
- 'samples/tasks/turbo_bind/**'
- 'samples/tasks/video_shader_parse/**'
- 'samples/tasks/wait_timeout/**'
- 'samples/tasks/watchdog/**'
- 'samples/tasks/wifi_tasks/**'
- 'samples/playlist/**'
- 'samples/ui_companion_log_stub.c'
- '.github/workflows/Linux-samples-tasks.yml'
# ci_paths: end
pull_request:
branches:
- master
# ci_paths: generated by tools/ci_paths.py, do not edit
paths:
- '**'
- '!samples/**'
- '!libretro-common/samples/**'
- '!.github/workflows/*.yml'
- 'libretro-common/samples/encodings/rbzip2/**'
- 'samples/menu/dirwalk/**'
- 'samples/playlist/playlist_parity_test.c'
- 'samples/tasks/autodetect_handoff/**'
- 'samples/tasks/bluetooth_scan/**'
- 'samples/tasks/bps_patch/**'
- 'samples/tasks/bsv_checkpoint/**'
- 'samples/tasks/bsv_replay/**'
- 'samples/tasks/bsv_replay_init/**'
- 'samples/tasks/bsv_statestream/**'
- 'samples/tasks/cloudsync/**'
- 'samples/tasks/content_prefetch/**'
- 'samples/tasks/core_backup/**'
- 'samples/tasks/core_updater/**'
- 'samples/tasks/database/**'
- 'samples/tasks/decompress/**'
- 'samples/tasks/file_transfer/**'
- 'samples/tasks/http/**'
- 'samples/tasks/http_emscripten/**'
- 'samples/tasks/image_decode_slice/**'
- 'samples/tasks/input_remap/**'
- 'samples/tasks/natt_desc/**'
- 'samples/tasks/netplay_nat/**'
- 'samples/tasks/ordering/**'
- 'samples/tasks/overlay/**'
- 'samples/tasks/patch_stream/**'
- 'samples/tasks/pl_thumbnail/**'
- 'samples/tasks/playlist_manager/**'
- 'samples/tasks/save_state/**'
- 'samples/tasks/turbo_bind/**'
- 'samples/tasks/video_shader_parse/**'
- 'samples/tasks/wait_timeout/**'
- 'samples/tasks/watchdog/**'
- 'samples/tasks/wifi_tasks/**'
- 'samples/playlist/**'
- 'samples/ui_companion_log_stub.c'
- '.github/workflows/Linux-samples-tasks.yml'
# ci_paths: end
workflow_dispatch:
permissions:
contents: read
env:
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/master' }}
jobs:
samples-tasks:
name: Build and run samples/tasks
runs-on: ubuntu-latest
env:
APT_PKGS: build-essential zlib1g-dev gcc-mingw-w64-x86-64 wine64 ccache
# One job runs every samples/tasks oracle, most of them three times
# (plain, ASan + UBSan, TSan): 53 steps. Ten minutes was the budget
# when there were fewer; it now runs out partway through and the
# job is cancelled with nothing failing.
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v5
- name: Apt cache key
id: apt
run: sh .github/workflows/scripts/apt-cache-key.sh "$APT_PKGS"
- name: Restore apt cache
uses: actions/cache/restore@v4
with:
path: ~/.apt-cache
key: ${{ steps.apt.outputs.key }}
restore-keys: ${{ steps.apt.outputs.prefix }}
- name: Install dependencies
timeout-minutes: 20
run: sh .github/workflows/scripts/apt-install.sh $APT_PKGS
- name: Save apt cache
if: env.APT_CACHE_SAVE == '1'
uses: actions/cache/save@v4
with:
path: ~/.apt-cache
key: ${{ steps.apt.outputs.key }}
- name: Link with lld
run: sh .github/workflows/scripts/use-lld.sh
- name: Restore ccache
uses: actions/cache/restore@v4
with:
path: ~/.ccache
key: ccache-${{ github.workflow }}-${{ github.job }}-${{ github.sha }}
restore-keys: |
ccache-${{ github.workflow }}-${{ github.job }}-
- name: Enable ccache
run: |
set -eu
export CCACHE_DIR="$HOME/.ccache"
echo "CCACHE_DIR=$CCACHE_DIR" >> "$GITHUB_ENV"
echo "/usr/lib/ccache" >> "$GITHUB_PATH"
ccache -M 250M
ccache -z
- name: Build database_task
shell: bash
working-directory: samples/tasks/database
run: |
set -eu
make clean && make -j$(nproc) all
test -x database_task
- name: Smoke-test database_task argv parsing
shell: bash
working-directory: samples/tasks/database
run: |
set -u
# database_task with no args should print the usage line and
# exit non-zero. This catches build regressions in the argv
# parsing / main() entry path without depending on the task
# queue actually running to completion -- task_push_dbscan
# delegates to task_push_manual_content_scan which does not
# plumb through the caller's completion callback, so the
# loop can only be terminated externally (by timeout or
# signal). The full loop is exercised in local testing; in
# CI we settle for the argv smoke.
#
# We capture the exit status via "|| rc=$?" so that the
# expected non-zero exit does NOT trip the default "set -e"
# that GitHub Actions' bash shell injects (it wraps with
# "bash --noprofile --norc -eo pipefail"). Without this
# guard the step exits at the binary invocation before our
# assertion runs.
rc=0
./database_task > /dev/null 2>&1 || rc=$?
if [[ $rc -eq 0 ]]; then
echo "::error title=Test failed::database_task with no args exited 0; expected non-zero (usage)"
exit 1
fi
echo "[pass] database_task no-args exit=$rc (expected non-zero)"
- name: Build and run archive_name_safety_test, decompress_window_test (plain, ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/decompress
run: |
set -eu
make clean && make -j$(nproc) all
test -x archive_name_safety_test
# Regression test for the Zip Slip / absolute-path defences
# in tasks/task_decompress.c::archive_name_is_safe(). The
# test keeps a verbatim copy of the predicate and runs it
# against 23 safe and unsafe inputs. If task_decompress.c
# ever amends the predicate, the copy here must follow.
timeout 60 ./archive_name_safety_test
echo "[pass] archive_name_safety_test"
# Oracle for the extraction task's pacing, compiled from the
# shipping translation unit against the real unthreaded
# task queue, ZIP backend and shared per-frame I/O window,
# with a virtual clock: a whole archive fits one window, an
# exhausted window still extracts one item per call. A
# DEFLATE member larger than the backend's output window is
# written a window at a time, through a temporary file that
# a CRC failure or a cancel removes, leaving any previous
# file in place.
timeout 60 ./decompress_window_test
make clean >/dev/null
# The same lanes against the built-in inflater, which builds
# without HAVE_ZLIB use and which rebinds its output window
# differently from zlib.
make -j$(nproc) all BUILTIN_DEFLATE=1
timeout 60 ./decompress_window_test
make clean BUILTIN_DEFLATE=1 >/dev/null
make all SANITIZER=address,undefined -j$(nproc)
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
timeout 120 ./decompress_window_test
make clean >/dev/null
make all SANITIZER=thread -j$(nproc)
TSAN_OPTIONS=halt_on_error=1 timeout 120 ./decompress_window_test
make clean >/dev/null
- name: Build and run cloudsync_path_safety_test
shell: bash
working-directory: samples/tasks/cloudsync
run: |
set -eu
# Regression test for the manifest-key check in
# tasks/task_cloudsync.c::task_cloud_sync_fetch_server_file.
# A hostile sync server's manifest key could escape the
# cloud-sync directory via ".." or an absolute component, or
# crash the fetch with a key that has no '/'. The test
# #includes the real predicate (tasks/task_cloudsync_path.c).
make clean && make -j$(nproc) all
test -x cloudsync_path_safety_test
timeout 60 ./cloudsync_path_safety_test
echo "[pass] cloudsync_path_safety_test"
- name: Build and run cloudsync_manifest_adds_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/cloudsync
run: |
set -eu
# tasks/task_cloudsync.c's updated manifests, added to from
# four threads and the task's own at once with no lock: each
# entry is pushed on a list the task thread folds in. Every
# entry must be there once, with its own hash; TSan is what
# sees the threads share anything without ordering, ASan an
# entry freed twice or not at all.
make clean && make -j$(nproc) cloudsync_manifest_adds_test
timeout 60 ./cloudsync_manifest_adds_test
make clean SANITIZER=address,undefined && make -j$(nproc) cloudsync_manifest_adds_test SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
timeout 120 ./cloudsync_manifest_adds_test
make clean SANITIZER=thread && make -j$(nproc) cloudsync_manifest_adds_test SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 120 ./cloudsync_manifest_adds_test
make clean >/dev/null
echo "[pass] cloudsync_manifest_adds_test"
- name: Build and run cloudsync_pacing_test (ASan + UBSan + LSan, TSan)
shell: bash
working-directory: samples/tasks/cloudsync
run: |
set -eu
# tasks/task_cloudsync.c's diff runs as many steps as the shared
# per-frame I/O window grants and hashes the current file 256 KiB
# at a time, carrying a large file over to the next run instead
# of reading it whole in one (the frame loop, with Threaded Tasks
# off). Against a stub window: a 9 MiB file spread over runs with
# its MD5 unchanged, small files several to a run, the floor
# alone finishing the diff, and a 9 MiB download reported from
# another thread hashed by the task over many runs instead of
# by the callback. The directory walk listing the local files
# runs within the grant too and lists what dir_list_new() did,
# an unreadable root still stopping the sync. Pushed on the real
# task queue and run to its
# end, the sync must release everything (LeakSanitizer).
make clean && make -j$(nproc) cloudsync_pacing_test
timeout 120 ./cloudsync_pacing_test
make clean SANITIZER=address,undefined && make -j$(nproc) cloudsync_pacing_test SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
timeout 300 ./cloudsync_pacing_test
make clean SANITIZER=thread && make -j$(nproc) cloudsync_pacing_test SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 300 ./cloudsync_pacing_test
make clean >/dev/null
echo "[pass] cloudsync_pacing_test"
- name: Build and run the cloudsync hand-over tests, locked path (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/cloudsync
run: |
set -eu
# Completion callbacks hand manifest entries and downloads to
# the task thread on lock-free lists where the atomics backend
# has pointer operations, and under tcs_manifest_lock where it
# does not. Every backend this runner has, has them, so
# FORCE_LOCK=1 builds tasks/task_cloudsync.c with
# TASK_CLOUDSYNC_FORCE_LOCK: the locked path on the host's own
# backend, where TSan sees an append or hand-over the lock
# does not cover. Same tests, same checks, other path.
for t in cloudsync_manifest_adds_test cloudsync_pacing_test cloudsync_commit_test; do
make clean $t FORCE_LOCK=1
timeout 120 ./$t
make clean $t FORCE_LOCK=1 SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
timeout 300 ./$t
make clean $t FORCE_LOCK=1 SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 300 ./$t
done
make clean >/dev/null
echo "[pass] cloudsync hand-over, locked path"
- name: Check cloud-sync commits on the content thread
shell: bash
working-directory: samples/tasks/cloudsync
run: |
set -eu
. "$GITHUB_WORKSPACE/.github/workflows/scripts/wine-server.sh"
make clean SANITIZER=address,undefined && make -j$(nproc) cloudsync_commit_test SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=halt_on_error=1 ./cloudsync_commit_test
make clean SANITIZER=thread && make -j$(nproc) cloudsync_commit_test SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 ./cloudsync_commit_test
make clean CC=x86_64-w64-mingw32-gcc LDFLAGS=-static && make -j$(nproc) cloudsync_commit_test CC=x86_64-w64-mingw32-gcc LDFLAGS=-static
WINEDEBUG=-all wine ./cloudsync_commit_test.exe
make clean
- name: Build and run cloudsync_async_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/cloudsync
run: |
set -eu
# network/cloud_sync_driver.c runs a blocking driver's calls
# (SMB, NFS) on a worker and hands results back through
# cloud_sync_poll(), so a slow server never stalls the thread
# driving the sync - the frame loop with Threaded Tasks off.
# Against a driver that blocks until released: every call
# returns at once, handlers run on the polling thread in call
# order, one driver call at a time, names copied, refused calls
# failing, and a new worker for the sync after one that ended
# or failed to begin. On the old direct calls the test hangs.
make clean && make -j$(nproc) cloudsync_async_test
timeout 60 ./cloudsync_async_test
make clean SANITIZER=address,undefined && make -j$(nproc) cloudsync_async_test SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
timeout 120 ./cloudsync_async_test
make clean SANITIZER=thread && make -j$(nproc) cloudsync_async_test SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 120 ./cloudsync_async_test
make clean >/dev/null
echo "[pass] cloudsync_async_test"
- name: Build and run natt_desc_parse_test
shell: bash
working-directory: samples/tasks/natt_desc
run: |
set -eu
# Regression test for the UPnP IGD description parser
# (network/natt_desc.c::natt_parse_desc_node). A router's
# description XML is untrusted; the old parser crashed on a
# leaf or empty root node and never reached a nested
# <service>. The test #includes the real parser and feeds it
# hand-built hostile trees.
make clean && make -j$(nproc) all
test -x natt_desc_parse_test
timeout 60 ./natt_desc_parse_test
echo "[pass] natt_desc_parse_test"
- name: Build and run http_method_match_test (ASan)
shell: bash
working-directory: samples/tasks/http
run: |
set -eu
# Regression test for the heap-buffer-overflow fix in
# tasks/task_http.c::task_push_http_transfer_generic().
# Pre-fix, an unsafe memcmp(method, "GET", 3) read past
# the strdup'd buffer when method was NULL or shorter
# than 3 bytes. Build under AddressSanitizer so a
# regression to the unsafe expression is caught at the
# bounds level, not just the truth-table level. If
# task_http.c amends the GET-dispatch predicate, the
# copy in http_method_match_test.c must follow.
make clean SANITIZER=address && make -j$(nproc) all SANITIZER=address
test -x http_method_match_test
timeout 60 ./http_method_match_test
echo "[pass] http_method_match_test"
- name: Build and run video_shader_wildcard_test (ASan)
shell: bash
working-directory: samples/tasks/video_shader_parse
run: |
set -eu
# Regression test for the buffer-overflow defences in
# gfx/video_shader_parse.c::video_shader_replace_wildcards_impl().
# Pre-fix, _len = strlcpy(replace_text, source, 256)
# returned strlen(source) regardless of the 256-byte
# cap. When source was a $CONTENT-DIR$ / $PRESET-DIR$
# / $CORE$ value longer than 256 bytes (DIR_MAX_LENGTH
# is 4096, NAME_MAX_LENGTH 256), the subsequent
# memcpy(dst + prefix, replace_text, _len) read off
# the end of replace_text and the strlcpy's size
# argument PATH_MAX_LENGTH - prefix - _len underflowed
# size_t. Build under AddressSanitizer so the OOB
# read on replace_text and the underflow-driven OOB
# write on dst are both caught at the bounds level.
# If video_shader_parse.c amends the post-replacement
# arithmetic, the verbatim copy in
# video_shader_wildcard_test.c must follow.
make clean SANITIZER=address && make -j$(nproc) all SANITIZER=address
test -x video_shader_wildcard_test
timeout 60 ./video_shader_wildcard_test
echo "[pass] video_shader_wildcard_test"
- name: Build and run cloudsync_savefile_owner_test (ASan)
shell: bash
working-directory: samples/tasks/cloudsync
run: |
set -eu
# Regression test for the save-data rollback cloud sync could
# cause at startup. A core holds its save RAM in memory and
# writes it back when content closes, so a sync that replaces
# the file in between has its work overwritten and the stale
# result uploaded over the server's copy.
# save.c::content_savefile_is_live() is the ownership check
# tasks/task_cloudsync.c consults before it touches a local
# file; the test pins that predicate and the manifest key
# lookup the carry-forward uses to recover the hash the last
# sync recorded. Build under AddressSanitizer so a regression
# in the list walk is caught at the bounds level. If save.c
# amends the predicate, the verbatim copy in
# cloudsync_savefile_owner_test.c must follow.
make clean SANITIZER=address && make -j$(nproc) all SANITIZER=address
test -x cloudsync_savefile_owner_test
timeout 60 ./cloudsync_savefile_owner_test
echo "[pass] cloudsync_savefile_owner_test"
- name: Build and run input_remap_bounds_test (ASan)
shell: bash
working-directory: samples/tasks/input_remap
run: |
set -eu
# Regression test for the input-remap analog-axis OOB-
# write fixes in configuration.c::input_remapping_load_file()
# and input/input_driver.c. Pre-fix a malformed .rmp
# could supply a remap target outside [0, RARCH_ANALOG_
# BIND_LIST_END), which the use sites in input_driver.c
# then indexed into a fixed-size analog_value[][8] array
# without bounds (button -> analog branch) or with a
# broken bound that compared elements against bytes
# (analog -> analog branch using sizeof instead of
# ARRAY_SIZE). Build under AddressSanitizer so any
# reintroduction of either primitive is caught at the
# bounds level. If configuration.c or input_driver.c
# amends the relevant predicates, the verbatim copies
# in input_remap_bounds_test.c must follow.
make clean SANITIZER=address && make -j$(nproc) all SANITIZER=address
test -x input_remap_bounds_test
timeout 60 ./input_remap_bounds_test
echo "[pass] input_remap_bounds_test"
- name: Build and run turbo_bind_bounds_test (ASan + UBSan)
shell: bash
working-directory: samples/tasks/turbo_bind
run: |
set -eu
# Turbo Bind and Turbo Button hold a RetroPad ID that indexes
# input_config_bind_order[24] in the menu's left/right
# handlers and input_remap_ids[port][] in the turbo paths of
# input_driver.c. The test walks both settings in both
# directions with wrap-around on and off, feeds them values
# from outside their range, checks the load-time clamp in
# configuration.c and the empty-bind guard on the turbo
# clear, with the tables at their exact sizes so either
# sanitizer reports an index past an end. If
# menu_setting.c, configuration.c or input_driver.c change
# those functions, the copies in turbo_bind_bounds_test.c
# must follow.
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
test -x turbo_bind_bounds_test
UBSAN_OPTIONS=halt_on_error=1 timeout 60 ./turbo_bind_bounds_test
echo "[pass] turbo_bind_bounds_test"
- name: Build and run bsv_replay_bounds_test (ASan)
shell: bash
working-directory: samples/tasks/bsv_replay
run: |
set -eu
# Regression test for the .bsv replay-file event-count
# bounds in input/bsv/bsvmovie.c. Pre-fix a malformed
# replay file with key_event_count > 128 or
# input_event_count > 512 caused a heap-buffer-overflow
# write into bsv_movie_t (a calloc'd struct) with
# attacker-chosen 8/12-byte payloads at attacker-chosen
# offsets up to ~500KB. Build under AddressSanitizer
# so any reintroduction of the unbounded count is caught
# at the bounds level. If input/bsv/bsvmovie.c amends
# the predicate, the verbatim copy in
# bsv_replay_bounds_test.c must follow.
make clean SANITIZER=address && make -j$(nproc) all SANITIZER=address
test -x bsv_replay_bounds_test
timeout 60 ./bsv_replay_bounds_test
echo "[pass] bsv_replay_bounds_test"
- name: Build and run bsv_checkpoint_bounds_test (ASan + UBSan)
shell: bash
working-directory: samples/tasks/bsv_checkpoint
run: |
set -eu
# A checkpoint in a .bsv replay carries its state, encoded
# and compressed sizes, all from the file. The test feeds
# bsv_movie_load_checkpoint() checkpoints whose sizes
# disagree, truncated ones and skipped ones, with buffers at
# their exact sizes, so an overflow, a double free or a
# leak is reported. If input/bsv/bsvmovie.c changes that
# function, the verbatim copy in
# bsv_checkpoint_bounds_test.c must follow.
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
test -x bsv_checkpoint_bounds_test
UBSAN_OPTIONS=halt_on_error=1 timeout 60 ./bsv_checkpoint_bounds_test
echo "[pass] bsv_checkpoint_bounds_test"
- name: Build and run bsv_replay_init_test (plain, ASan + UBSan)
shell: bash
working-directory: samples/tasks/bsv_replay_init
run: |
set -eu
# Regression oracle for replay start-up and shutdown in
# tasks/task_movie.c and input/bsv/bsvmovie.c (#19622).
# Links the shipping units against a stub core and the real
# task queue / intfstream / VFS, in the iOS configuration
# (RZSTD on, STATESTREAM off). Pre-fix a replay truncated
# inside its first checkpoint, or halted before its first
# frame, left BSV_FLAG_MOVIE_END set with no handle: the
# runloop then paused after every frame, for every content,
# until the process was restarted. Also covers halting a
# recording before its first frame leaving the session stuck
# "recording". If runloop.c changes its per-frame replay
# protocol, frame() in the test must follow.
make clean && make -j$(nproc) all
test -x bsv_replay_init_test
timeout 120 ./bsv_replay_init_test
echo "[pass] bsv_replay_init_test"
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
# allocator_may_return_null: the test feeds a checkpoint too big
# to allocate and expects malloc to return NULL.
ASAN_OPTIONS=detect_leaks=1:allocator_may_return_null=1 UBSAN_OPTIONS=print_stacktrace=1 \
timeout 120 ./bsv_replay_init_test
echo "[pass] bsv_replay_init_test (ASan + UBSan)"
# STATESTREAM=1: the deduplicating checkpoint encoding desktop
# builds use, and the lanes that exercise it.
make clean STATESTREAM=1 SANITIZER=address,undefined && make -j$(nproc) all STATESTREAM=1 SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:allocator_may_return_null=1 UBSAN_OPTIONS=print_stacktrace=1 \
timeout 120 ./bsv_replay_init_test
echo "[pass] bsv_replay_init_test (STATESTREAM, ASan + UBSan)"
- name: Build and run netplay BSV recording tests
shell: bash
working-directory: samples/tasks/bsv_replay_init
run: |
set -eu
. "$GITHUB_WORKSPACE/.github/workflows/scripts/wine-server.sh"
make clean && make -j$(nproc) NETPLAY=1 SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:allocator_may_return_null=1 \
timeout 120 ./bsv_replay_init_test
make clean && make -j$(nproc) NETPLAY=1 STATESTREAM=1 SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:allocator_may_return_null=1 \
timeout 120 ./bsv_replay_init_test
make clean && make -j$(nproc) NETPLAY=1 CC=x86_64-w64-mingw32-gcc LDFLAGS=-static
WINEDEBUG=-all timeout 120 wine ./bsv_replay_init_test.exe
- name: Build and run bsv_statestream_test (plain, ASan + UBSan)
shell: bash
working-directory: samples/tasks/bsv_statestream
run: |
set -eu
# Statestream checkpoints decoded by the shipping
# input/bsv/bsvmovie.c with HAVE_STATESTREAM on, which no
# other lane builds. Block and superblock indices, the
# superblock sequence and its length all come from the
# replay file: the test round-trips recorded checkpoints,
# then feeds sequences naming undefined superblocks and
# blocks, a sequence longer than the superblock list, a
# recording after a short sequence, and a header with no
# block layout. The oom and oomload lanes fail each
# allocation of a checkpoint write and of a load in turn
# (malloc, calloc and realloc are wrapped at link time) and
# need every one to fail cleanly, with no leak. Each lane
# takes its name as an argument to run alone.
make clean && make -j$(nproc) all
test -x bsv_statestream_test
timeout 120 ./bsv_statestream_test
echo "[pass] bsv_statestream_test"
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=halt_on_error=1:print_stacktrace=1 \
timeout 120 ./bsv_statestream_test
echo "[pass] bsv_statestream_test (ASan + UBSan)"
# Again with every object hashing alike (constant_hash.h),
# so inserts share buckets and the oom lanes also fail the
# buckets' own growth.
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined \
EXTRA_CFLAGS='-include $(CURDIR)/constant_hash.h'
ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=halt_on_error=1:print_stacktrace=1 \
timeout 120 ./bsv_statestream_test
echo "[pass] bsv_statestream_test (colliding hash)"
- name: Build and run bps_patch_bounds_test (ASan)
shell: bash
working-directory: samples/tasks/bps_patch
run: |
set -eu
# Regression test for the .bps patch parser bounds in
# tasks/task_patch.c::bps_apply_patch. Pre-fix a
# malicious .bps could write attacker-chosen bytes
# past the malloc'd target_data buffer (heap-buffer-
# overflow WRITE), read past source_data (info leak),
# and read past modify_data via TARGET_READ. The
# size-prelude was also unbounded, allowing 32-bit
# truncation of modify_target_size to drive a
# smaller-than-expected allocation. Build under
# AddressSanitizer so any reintroduction is caught
# at the bounds level. If task_patch.c amends the
# action-loop predicates, the verbatim copy in
# bps_patch_bounds_test.c must follow.
make clean SANITIZER=address && make -j$(nproc) all SANITIZER=address
test -x bps_patch_bounds_test
timeout 60 ./bps_patch_bounds_test
echo "[pass] bps_patch_bounds_test"
- name: Build and run patch_stream_test (ASan/UBSan)
shell: bash
working-directory: samples/tasks/patch_stream
run: |
set -eu
# Differential harness for the streaming patch appliers in
# tasks/patch_stream.c. Each streamed applier must produce
# output byte-identical to the whole-buffer applier it
# mirrors in tasks/task_patch.c, at every chunk size - the
# oracle for IPS, UPS and BPS is a copy of that whole-buffer
# logic held in the test, so if task_patch.c changes its
# appliers the copy has to follow. The .xdelta case checks
# the delegation to the VCDIFF decoder instead, since that
# decoder is verified separately against the reference
# implementation.
#
# The cases that matter most are the ones a whole-buffer
# applier cannot have: an empty source, a source that ends
# short of its declared length, and chunk boundaries landing
# inside a patch command. A truncated feed must be refused
# rather than answered with a target built from a partial
# source.
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
test -x patch_stream_test
timeout 900 ./patch_stream_test
echo "[pass] patch_stream_test"
- name: Build and run task_file_transfer_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/file_transfer
run: |
set -eu
# Oracle for tasks/task_file_transfer.c, compiled from the
# tree so it exercises the shipping translation unit. The
# handler is a switch whose states advance at most one step
# per tick, so what matters is how many ticks a load takes,
# whether every input reaches a terminal at all, and what a
# caller reading task_get_progress() sees on the way - a
# test that only checks the bytes passes on every defect
# below. Lanes run to a tick cap rather than looping, so a
# task that never finishes reports instead of hanging CI.
#
# Pinned here: a NULL path and a parse callback that
# returns success without setting is_finished, both of
# which once left the task running forever with no flag set
# (a per-frame call under the regular queue, a hot spin on
# the worker thread under the threaded one); progress
# reaching 100 on every completion path, not just the
# single-tick one; the tick that opens the file also
# reading, which one byte either side of the threshold used
# to decide; a file truncated underneath an in-flight fill
# cancelling rather than parsing a buffer whose tail was
# never written; and every audio type reaching the mixer
# arm, since M4A and OPUS were once absent from that switch
# while both push paths set them, so those sounds loaded
# and silently did not play.
#
# Two lanes guard the 256 MiB prefix cap that cancelled
# every large non-faststart video. The real-demuxer lane
# pins the premise the cap got wrong: a genuine h264 MP4
# (real ftyp/mdat/moov, real coded frames) is written with
# a ~300 MiB free box pushing moov to EOF - a valid file,
# the layout most cameras produce - and the real rmp4, via
# image_transfer's avail-aware open, must answer need_more
# on a 1 MiB prefix and decode real frames once fully
# resident. Anyone restoring a prefix ceiling on the
# "a still only needs a few per cent" argument is refuted
# by that lane directly. The task lane pins the
# mechanism: a still ready only at EOF, over a 300 MiB
# file, must finish uncancelled with every byte resident.
# On the capped tree it stops at exactly 268435456 and
# cancels - no thumbnail, no animation - and the oracle
# exits non-zero.
#
# The conc lane runs disjoint handles on many threads at
# once. data_transfer synchronises nothing by design and
# the task layer gives one handle to one task, so it is not
# looking for contention on a shared handle - it checks
# that the spine keeps no hidden static state, which a
# sanitizer build can answer and inspection cannot.
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
test -x task_file_transfer_test
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 600 \
./task_file_transfer_test
ASAN_OPTIONS=detect_leaks=1 timeout 600 \
./task_file_transfer_test conc
echo "[pass] task_file_transfer_test (ASan)"
make clean SANITIZER=thread && make -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 600 \
./task_file_transfer_test
TSAN_OPTIONS=halt_on_error=1 timeout 600 \
./task_file_transfer_test conc
echo "[pass] task_file_transfer_test (TSan)"
- name: Build and run pl_thumbnail_wait_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/pl_thumbnail
run: |
set -eu
# The thumbnail tasks park while a transfer runs, and a
# cancelled one keeps its handle until the transfer's
# callback has written into it.
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=print_stacktrace=1 \
timeout 120 ./pl_thumbnail_wait_test
echo "[pass] pl_thumbnail_wait_test (ASan + UBSan)"
make clean SANITIZER=thread && make -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 120 ./pl_thumbnail_wait_test
echo "[pass] pl_thumbnail_wait_test (TSan)"
- name: Build and run core_crc_slice_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/core_updater
run: |
set -eu
# Oracle for the resumable CRC in
# libretro-common/streams/interface_stream.c and the sliced
# hashing loop tasks/task_core_updater.c drives it with.
#
# intfstream_get_crc() consumes a whole stream in one call,
# so the cost of the tick that invoked it was a function of
# core size and nothing else - 43ms for a 40MB core measured
# cold on NVMe, an order of magnitude worse on the SD-card
# and spinning-disk targets, and the updater hashes the
# installed core on every update check. intfstream_crc_step()
# hashes a bounded slice and resumes, so the work spreads
# across ticks.
#
# The property that needs pinning is not "the CRC is
# correct" - a whole-stream call gets that right too - but
# that no single tick hashes the whole file while the answer
# still matches the unsliced value at every chunk size,
# including sizes that do not divide the file evenly and a
# final slice of one byte. Both are asserted against
# intfstream_get_crc() as the oracle.
make clean && make -j$(nproc) all
test -x core_crc_slice_test
timeout 600 ./core_crc_slice_test
echo "[pass] core_crc_slice_test"
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 600 \
./core_crc_slice_test
echo "[pass] core_crc_slice_test (ASan)"
make clean SANITIZER=thread && make -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 600 ./core_crc_slice_test
make clean
echo "[pass] core_crc_slice_test (TSan)"
- name: Build and run get_list_refresh_flags_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/core_updater
run: |
set -eu
# Regression lane for the menu refresh flags around the
# core updater get-list task, against the shipping
# tasks/task_core_updater.c, tasks/task_http.c,
# task_queue.c and net_http.c.
#
# The Core Downloader entry point sets
# MENU_ST_FLAG_ENTRIES_NONBLOCKING_REFRESH before pushing
# the get-list task, and only the task's retirement
# callback clears it again. The callback reads
# refresh_menu through task->state, so the handle has to
# stay attached to the task until retrieval; a handler
# that frees it early turns the clear into a silent no-op,
# MENU_ENTRIES_NEEDS_REFRESH() then stays false for the
# rest of the session, and one visit to Core Downloader
# wedges every deeper menu until restart. The test drives
# the real push end to end against a loopback
# .index-extended server, both values of refresh_menu, and
# asserts the flags clear and the refresh gate comes back.
make -f Makefile.refresh_flags clean && make -f Makefile.refresh_flags -j$(nproc) all
test -x get_list_refresh_flags_test
timeout 600 ./get_list_refresh_flags_test
echo "[pass] get_list_refresh_flags_test"
make -f Makefile.refresh_flags clean SANITIZER=address,undefined && make -f Makefile.refresh_flags -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 600 \
./get_list_refresh_flags_test
echo "[pass] get_list_refresh_flags_test (ASan)"
make -f Makefile.refresh_flags clean SANITIZER=thread && make -f Makefile.refresh_flags -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 600 \
./get_list_refresh_flags_test
make -f Makefile.refresh_flags clean
echo "[pass] get_list_refresh_flags_test (TSan)"
- name: Build and run update_installed_scan_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/core_updater
run: |
set -eu
# Regression lane for the Update Installed Cores scan in
# tasks/task_core_updater.c, against the shipping task, list
# parser, HTTP task and task queue. Every list entry and
# every CRC chunk is a work item of the shared per-frame I/O
# window (task_nbio_slice_*), so with the unthreaded queue a
# 512-entry list finishes in a few dozen frames rather than
# one frame per entry; a CRC resumed on a later tick does not
# repeat the per-core stats. Both are asserted: the gather
# count, and path_is_valid() calls counted at the link
# boundary. The get-list lane forces the window to its
# one-item floor and asserts the listing is parsed a line
# per gather into a private list, while the caller's list -
# read by the Core Downloader's menu callbacks every frame -
# only ever holds its old contents or the complete new ones.
# The scan lanes also count core info reads at the link
# boundary: Update Installed Cores' fetch reads info for its
# installed cores alone. The status lanes pin that a fetch
# finding no installed cores still ends in 'all cores
# updated' and a failed fetch in 'core list failed'.
# The refused-download lane refuses a changed core's transfer
# and asserts the run still completes: a download that
# cannot start fails instead of waiting on an extraction.
# The CRC cache lane counts bytes hashed at the link
# boundary: a repeat run hashes no unchanged core, a touched
# core is hashed again, a changed one is hashed and
# downloaded, and a corrupt cache falls back to hashing all.
make -f Makefile.update_installed clean && make -f Makefile.update_installed -j$(nproc) all
test -x update_installed_scan_test
timeout 600 ./update_installed_scan_test
echo "[pass] update_installed_scan_test"
make -f Makefile.update_installed clean SANITIZER=address,undefined && make -f Makefile.update_installed -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 600 \
./update_installed_scan_test
echo "[pass] update_installed_scan_test (ASan)"
make -f Makefile.update_installed clean SANITIZER=thread && make -f Makefile.update_installed -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 600 \
./update_installed_scan_test
make -f Makefile.update_installed clean
echo "[pass] update_installed_scan_test (TSan)"
- name: Build and run staged_install_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/core_updater
run: |
set -eu
# End-to-end lane for the staged core install against the
# shipping updater, HTTP, decompress, archive, rzip and core
# backup code, serving real deflated zips: with 'Compress
# Automatic Core Backups' off the replaced core is moved into
# the backups as it is; the history is pruned; a move refused
# at the link boundary falls back to a compressed copy; with
# the setting on the backup is compressed as before; a new
# core that cannot be moved into place leaves the installed
# one as it was.
make -f Makefile.staged_install clean && make -f Makefile.staged_install -j$(nproc) all
timeout 600 ./staged_install_test
echo "[pass] staged_install_test"
make -f Makefile.staged_install clean SANITIZER=address,undefined && make -f Makefile.staged_install -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 600 \
./staged_install_test
echo "[pass] staged_install_test (ASan)"
make -f Makefile.staged_install clean SANITIZER=thread && make -f Makefile.staged_install -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 600 \
./staged_install_test
make -f Makefile.staged_install clean
echo "[pass] staged_install_test (TSan)"
- name: Build and run download_backup_wait_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/core_updater
run: |
set -eu
# Regression lane for the core download task's wait on its
# automatic backup in tasks/task_core_updater.c. The threaded
# queue frees a retired backup task on the main thread while
# the download handler runs on the worker, so completion has
# to come from the backup's callback rather than from reading
# the task. One lane retires and frees the backup between two
# download ticks (a handler that reads it is a
# heap-use-after-free under ASan); the other cancels the
# download while the backup runs, so the backup's callback
# fires after the download's handle is gone.
make -f Makefile.download_backup clean && make -f Makefile.download_backup -j$(nproc) all
test -x download_backup_wait_test
timeout 600 ./download_backup_wait_test
echo "[pass] download_backup_wait_test"
make -f Makefile.download_backup clean SANITIZER=address,undefined && make -f Makefile.download_backup -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 600 \
./download_backup_wait_test
echo "[pass] download_backup_wait_test (ASan)"
make -f Makefile.download_backup clean SANITIZER=thread && make -f Makefile.download_backup -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 600 \
./download_backup_wait_test
make -f Makefile.download_backup clean
echo "[pass] download_backup_wait_test (TSan)"
- name: Build and run pfd_switch_cores_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/core_updater
run: |
set -eu
# Regression lane for the Play Feature Delivery 'switch
# installed cores' task in tasks/task_core_updater.c, built
# with ANDROID defined against stand-ins for the five
# play_feature_delivery_* calls. The switch task learns of
# each install's completion and error from the install's
# callback rather than by reading the task, which the queue
# frees as it retires it; with the threaded queue, a wait
# that reads the task moves on before the install is retired
# and the next push is refused, skipping cores. The other
# lanes end the switch on a failed install, and cancel the
# switch while an install runs. The recovery lanes leave
# the <core>.bak an interrupted install moves the core to,
# with the core missing: the next install must put it back
# if it fails and drop it if it succeeds or play feature
# delivery already installed the core.
make -f Makefile.pfd_switch clean && make -f Makefile.pfd_switch -j$(nproc) all
test -x pfd_switch_cores_test
timeout 600 ./pfd_switch_cores_test
echo "[pass] pfd_switch_cores_test"
make -f Makefile.pfd_switch clean SANITIZER=address,undefined && make -f Makefile.pfd_switch -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 600 \
./pfd_switch_cores_test
echo "[pass] pfd_switch_cores_test (ASan)"
make -f Makefile.pfd_switch clean SANITIZER=thread && make -f Makefile.pfd_switch -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 600 \
./pfd_switch_cores_test
make -f Makefile.pfd_switch clean
echo "[pass] pfd_switch_cores_test (TSan)"
- name: Build and run core_backup_io_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/core_backup
run: |
set -eu
# Oracle for the file I/O in tasks/task_core_backup.c,
# compiled from the tree so it exercises the shipping
# translation unit. As in save_state and
# image_decode_slice, the clock is the test's instrument:
# cpu_features_get_time_usec is wrapped so the tick budget is
# assertable exactly rather than measured against a wall
# clock.
#
# The throughput lanes pin the budget that replaced a fixed
# one-quantum-per-tick loop: a 4KB quantum capped a backup at
# ~245KB/s whatever the device could do, and an automatic
# backup runs on every core update over cores that are
# routinely tens of megabytes. The exhausted-budget lane
# asserts the floor from the other side - with a clock that
# spends the budget on its first observation the handler must
# still move exactly one quantum per tick and still produce a
# correct backup, which is the old behaviour reached through
# the new code and what the slowest supported storage gets.
#
# The read-amplification lane counts bytes at the VFS
# boundary, which is the only place they can be counted:
# interposing libc read()/write() silently reports zero here,
# because glibc's stdio reaches them through internal aliases
# that never go via the PLT. It measures 2x when the CRC is
# unknown - the handler hashes the core to name the backup,
# then reads it again to copy it - and 1x when the caller
# supplies a CRC. The second figure is the one that matters:
# tasks/task_core_updater.c hands its already-computed
# local_crc to the automatic backup precisely so that pass is
# skipped, and nothing in the type system enforces it. Pass
# 0 from the updater and every automatic backup silently
# doubles its read I/O.
#
# The restore lane round-trips a backup byte for byte. It is
# worth noting why it is asserted rather than skipped: it
# used to skip, silently, on every run. The suite stubbed
# frontend_driver_get_core_extension() with the wrong
# signature - const char *f(void) against the real
# bool f(char *, size_t) - which compiles, because nothing
# includes the real declaration, but left core_backup.c
# comparing an extension against uninitialised stack. Every
# archive read as CORE_BACKUP_TYPE_INVALID,
# task_push_core_restore() refused, and the lane printed SKIP
# while the suite reported PASSED. Both refusal paths are
# now CHECK failures: a fixture this test builds itself
# cannot legitimately be unavailable to it.
#
# The interrupted-restore lanes stop a restore one quantum
# in, by cancelling it (as the exit drain does) and by
# failing its writes (a full disk), and require the
# installed core to be byte-identical afterwards with no
# temporary left: the restore writes beside the core and
# renames over it only once complete. The symlink lane
# requires a core reached through a link to keep the link
# and have its target replaced.
make clean && make -j$(nproc) all
test -x core_backup_io_test
timeout 600 ./core_backup_io_test
echo "[pass] core_backup_io_test"
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 600 \
./core_backup_io_test
echo "[pass] core_backup_io_test (ASan)"
make clean SANITIZER=thread && make -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 600 ./core_backup_io_test
make clean
echo "[pass] core_backup_io_test (TSan)"
- name: Build and run image_decode_slice_test (ASan + UBSan)
shell: bash
working-directory: samples/tasks/image_decode_slice
run: |
set -eu
# Oracle for the shared decode window in tasks/task_image.c,
# compiled from the tree so it exercises the shipping
# translation unit. The clock is the test's instrument: the
# sample provides cpu_features_get_time_usec() itself,
# advancing a fixed quantum per observation, so the decode
# loops are budget-bound deterministically and the cost of a
# gather is readable as the fake-clock delta across
# task_queue_check() - no wall-clock measurement, no
# flakiness.
#
# The regression it pins has two faces, both verified to
# fail it: the pre-window code, where eight concurrent
# decodes each took a full frame_duration per gather (~8
# frames of decode in one gather), and the subtler
# period == allowance variant of the window itself, where a
# task that spends its whole allowance has aged the window a
# full period and the next task in the same gather resets
# it - the multiplication surviving a fix that reads
# correct. That variant passed inspection and failed
# simulation, which is why this oracle asserts on measured
# gather cost rather than on the code's shape.
#
# It also checks the decode itself end to end - dimensions
# and a pixel spot-check on every delivered image - so the
# budget rework cannot silently break what it budgets.
make clean && make -j$(nproc) all
test -x image_decode_slice_test
timeout 600 ./image_decode_slice_test
echo "[pass] image_decode_slice_test"
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 600 \
./image_decode_slice_test
echo "[pass] image_decode_slice_test (ASan)"
- name: Build and run scan_begin_budget_test (plain, ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/database
run: |
set -eu
# Pacing + throughput oracle for the manual content scanner's
# BEGIN/END split in tasks/task_database.c, compiled from the
# tree so it exercises the shipping translation units.
#
# The regression it pins: MANUAL_SCAN_BEGIN used to perform
# the entire recursive directory walk, the whole DAT load and
# the playlist setup in one handler invocation, and
# MANUAL_SCAN_END flushed every accumulated result (each add
# a linear playlist_entry_exists probe) plus a sort and a
# write in another - 166 ms measured for the 5000-file
# fixture, a hard frame freeze whenever Threaded Tasks is
# off. Both are now sequences of sub-states under the shared
# per-frame I/O window (tasks/task_nbio_slice.c, the same
# window the file-transfer spine uses), with completed
# sub-states collapsing into one gather so small libraries
# keep single-tick latency.
#
# Three wall-clock assertions guard the plain build, because
# any one alone can be satisfied by a scanner that is worse:
# pacing (no gather over the window plus bounded slack),
# first feedback within two frames of the push, and total
# scan time within a factor of the blocking walk + DAT parse
# it replaced - a scanner that no longer freezes but takes
# twice as long fails the third lane. Playlist parity (every
# fixture file present exactly once) is asserted in every
# mode.
#
# The sanitizer runs use the 'sanitize' mode, which runs
# every lane but skips the wall-clock assertions a sanitized
# build cannot honestly meet. Those lanes are where the
# in-flight resources the split introduced - an open
# directory iterator, an open DAT stream over a half-filled
# buffer, an open flush playlist - are made to die at every
# phase: a 14-point cancellation sweep lands a cancel in
# setup, mid-walk, mid-DAT-read, content iteration and
# mid-flush, and LeakSanitizer decides whether the teardown
# released everything (it already found one 16-byte-per-scan
# leak that predates the split). The threaded lane runs the
# same scan under the threaded queue and is the TSan target:
# the shared window deliberately opts threaded queues out of
# its static state, and this is where a mistake in that
# opt-out would surface.
#
# Two dedup lanes guard the playlist path-hash index that
# replaced the flush's quadratic existence probes: a
# differential lane asserts the index answers exactly like
# the linear playlist_entry_exists scan over plain, bare-
# archive, inside-archive and case-variant paths with fuzzy
# archive matching both off and on, and a no-overwrite
# rescan lane re-scans the fixture into the populated
# playlist and asserts zero duplicates, driving the index's
# budgeted seeding at scale. The throughput factor is
# tightened to reject the pre-index quadratic flush.
#
# The big-DAT lane scans 50 files against a 300k-game DAT
# and asserts the same pacing budget: no gather may grow
# with DAT size now that the rxml parse and the search-
# index build step under the shared window. Its label
# check samples the end of the list, so an index that
# silently covered only a prefix fails correctness, and
# three extra cancel points land inside the budgeted read
# and parse of the large document.
make clean && make -j$(nproc) all
test -x scan_begin_budget_test
timeout 600 ./scan_begin_budget_test
echo "[pass] scan_begin_budget_test"
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 900 \
./scan_begin_budget_test sanitize
echo "[pass] scan_begin_budget_test (ASan)"
make clean SANITIZER=thread && make -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 900 \
./scan_begin_budget_test sanitize
make clean
echo "[pass] scan_begin_budget_test (TSan)"
- name: Build and run database_scan_test (ASan + UBSan)
shell: bash
working-directory: samples/tasks/database
run: |
set -eu
# End-to-end oracle for tasks/task_database.c, built from the
# shipping translation units. Alongside the database pairing
# and playlist assertions it already carries, three memory
# safety invariants of the manual scan state machine are
# pinned here:
#
# - a content list whose entries exactly fill
# string_list_new()'s initial capacity must leave
# DATABASE_SCAN_ITERATE_NEXT at the M3U or the end state.
# An exhausted index must never reach
# MANUAL_SCAN_ITERATE_CONTENT, which reads its entry
# before any bounds check;
#
# - a content directory with nothing scannable in it must be
# rejected as invalid content by the database branch of
# manual_scan_begin_dir_list(), the same rule the
# non-database branch applies, instead of iterating an
# empty list;
#
# - task_database_cue_prune() and gdi_prune() must leave the
# entry being scanned alone when a cue or gdi sheet names
# itself, because the serial lookup reads that path next.
#
# The first and third are bounds and lifetime properties that
# a plain build walks straight past, so this lane is sanitized
# only and LeakSanitizer decides the teardown as well.
make clean SANITIZER=address,undefined && make -j$(nproc) database_scan_test SANITIZER=address,undefined
test -x database_scan_test
scan_root="$(mktemp -d)"
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 900 \
./database_scan_test "$scan_root"
rm -rf "$scan_root"
make clean
echo "[pass] database_scan_test (ASan)"
- name: Build and run database_scan_test (TSan, no threads, no readers)
shell: bash
working-directory: samples/tasks/database
run: |
set -eu
# The scan's reader tasks under TSan, with a stub SMB backend
# so they take smb:// paths; then the same checks
# in a build without threads, with the budget test too, and in
# one whose readers are compiled in but never start.
make clean SANITIZER=thread && make -j$(nproc) database_scan_test SANITIZER=thread \
SCAN_DEFINES=-DHAVE_SMBCLIENT
scan_root="$(mktemp -d)"
TSAN_OPTIONS=halt_on_error=1 timeout 900 ./database_scan_test "$scan_root"
rm -rf "${scan_root:?}"
make clean && make -j$(nproc) database_scan_test scan_begin_budget_test HAVE_THREADS=0
scan_root="$(mktemp -d)"
timeout 900 ./database_scan_test "$scan_root"
rm -rf "${scan_root:?}"
budget_root="$(mktemp -d)"
TMPDIR="$budget_root" timeout 600 ./scan_begin_budget_test
rm -rf "${budget_root:?}"
make clean && make -j$(nproc) database_scan_test SANITIZER=address,undefined \
SCAN_DEFINES=-DSCAN_READERS_MAX=0
scan_root="$(mktemp -d)"
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 900 \
./database_scan_test "$scan_root"
rm -rf "${scan_root:?}"
make clean
echo "[pass] database_scan_test (TSan, HAVE_THREADS=0, SCAN_READERS_MAX=0)"
- name: Build and run pl_manager_budget_test (plain, ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/playlist_manager
run: |
set -eu
# Oracle for the playlist manager tasks' load phase,
# compiled from the shipping translation unit against the
# real task queue, the real playlist reader and the real
# shared per-frame I/O window, with a virtual clock so the
# pacing assertions are exact.
#
# Both handlers were already per-entry state machines with
# one exception: the load read the whole playlist in a
# single blocking call, which for a large collection is
# tens of milliseconds of JSON on the thread driving the
# frame loop whenever Threaded Tasks is off.
#
# The pacing lane isolates that phase: the per-entry
# iteration costs the same number of invocations either
# way, so the difference between a paced and an unpaced run
# is the load. The cancel lane, under LSan, covers
# abandoning a task while its playlist is still being read.
make clean >/dev/null
make all -j$(nproc)
./pl_manager_budget_test
make clean >/dev/null
make all SANITIZER=address,undefined -j$(nproc)
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
./pl_manager_budget_test
make clean >/dev/null
make all SANITIZER=thread -j$(nproc)
TSAN_OPTIONS=halt_on_error=1 ./pl_manager_budget_test
make clean >/dev/null
- name: Build and run overlay_budget_test (plain, ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/overlay
run: |
set -eu
# Oracle for the overlay loader's pacing, compiled from the
# shipping translation unit against the real task queue and
# the real shared per-frame I/O window, with image loading
# stubbed and a virtual clock so the assertions are exact.
#
# The loader used to chunk by a count derived from the
# workload - half an overlay's descriptors, a quarter of
# the overlays - which is not pacing: a bigger overlay just
# meant bigger chunks, so the job always landed in the same
# two or four handler invocations however long each took.
# The scaling lane is the one that pins the fix: work per
# invocation must not grow with the size of the overlay.
#
# LSan also guards the image list, which used to be
# released only when a load was cancelled and therefore
# leaked once per successful overlay load.
make clean >/dev/null
make all -j$(nproc)
./overlay_budget_test
make clean >/dev/null
make all SANITIZER=address,undefined -j$(nproc)
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
./overlay_budget_test
make clean >/dev/null
make all SANITIZER=thread -j$(nproc)
TSAN_OPTIONS=halt_on_error=1 ./overlay_budget_test
make clean >/dev/null
- name: Build and run overlay_visibility_bounds_test (ASan + UBSan)
shell: bash
working-directory: samples/tasks/overlay
run: |
set -eu
# Builds the shipping input/input_overlay_alpha.c and
# led/drivers/led_overlay.c against a stub driver. Which images
# the overlay LED driver hides: a pack's _led images, else the
# image at a ledN_map slot whatever its desc does, and never an
# index off the page - led_map comes from the config with no
# range of its own. And the per-poll alpha pass: an image is set only when
# its alpha changes, a page load sets them all again, and an
# LED going out and back on between polls is shown again.
make -f Makefile.visibility clean >/dev/null
make -f Makefile.visibility all SANITIZER=address,undefined -j$(nproc)
ASAN_OPTIONS=detect_leaks=1 \
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
./overlay_visibility_bounds_test
make -f Makefile.visibility clean >/dev/null
- name: Build and run content_prefetch_test (plain, ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/content_prefetch
run: |
set -eu
# Oracle for the content prefetch task - the path a menu
# content load defers to, streaming the bytes in a budgeted
# slice per tick while the frontend keeps running, so the
# load takes a finished buffer instead of blocking on a
# read.
#
# Delivery is by ownership transfer from the task callback,
# so this is as much a memory-correctness test as a
# behavioural one, and it runs under all three flavors. One
# lane drives the THREADED queue deliberately: the other
# lanes run the handler on the calling thread, where a TSan
# run would prove nothing about the worker-to-callback
# hand-off that is the only place a race could live.
make clean >/dev/null
make all -j$(nproc)
./content_prefetch_test
make clean >/dev/null
make all SANITIZER=address,undefined -j$(nproc)
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
./content_prefetch_test
make clean >/dev/null
make all SANITIZER=thread -j$(nproc)
TSAN_OPTIONS=halt_on_error=1 ./content_prefetch_test
make clean >/dev/null
- name: Build and run bluetooth_scan_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/bluetooth_scan
run: |
set -eu
# The Bluetooth scan task on the real threaded task queue,
# with the discovery window shortened. The scan used to wait
# out the window inside one handler call, holding the task
# thread; it now begins, is parked, looks every poll whether
# the driver is done, and ends. Pins that begin and end run
# once each a window apart, that another task runs during the
# window, that the callback fires, and that a driver done
# early ends the scan then.
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
timeout 60 ./bluetooth_scan_test
make clean SANITIZER=thread && make -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 60 ./bluetooth_scan_test
echo "[pass] bluetooth_scan_test"
- name: Build and run wifi_tasks_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/wifi_tasks
run: |
set -eu
# The Wi-Fi tasks on the real threaded task queue, with a
# driver whose scan takes a while. Connect, Disconnect and the
# Wi-Fi Enabled toggle pushed during the networks list's
# periodic scan were turned away; pins that they are held and
# run after it in order, one on the driver at a time, and that
# only a scan behind a scan is refused.
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
timeout 60 ./wifi_tasks_test
make clean SANITIZER=thread && make -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 60 ./wifi_tasks_test
echo "[pass] wifi_tasks_test"
- name: Build and run wait_timeout_test (plain, ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/wait_timeout
run: |
set -eu
# task_queue_wait() waits indefinitely, which is wrong for
# anything depending on a machine we do not control - the
# netplay MITM query waits on an HTTP round trip to the
# lobby server, and a server that never answers used to hang
# the frontend outright.
#
# These lanes pin both directions: a wait that CAN be
# satisfied returns as soon as it is, rather than sitting
# out its timeout, and one that cannot gives up at roughly
# the deadline instead of never. Both queue implementations
# are covered, since the bound is a wrapped condition rather
# than per-implementation code.
make clean >/dev/null
make all -j$(nproc)
./wait_timeout_test
make clean >/dev/null
make all SANITIZER=address,undefined -j$(nproc)
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
./wait_timeout_test
make clean >/dev/null
make all SANITIZER=thread -j$(nproc)
TSAN_OPTIONS=halt_on_error=1 ./wait_timeout_test
make clean >/dev/null
- name: Build and run task_ordering_test (plain, ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/ordering
run: |
set -eu
# Pins the task queue's visibility contract, which is what
# made a movie-record ordering guard fail.
#
# retro_task_regular_gather() lifts every running task off
# the queue and puts each one BACK as it processes it, so
# whether a finder called from inside a handler sees a
# sibling depends on where that sibling sits in the pass.
# At least one invocation reports "nothing in progress"
# while the task is genuinely pending - which an ordering
# guard only has to get wrong once. A main-thread flag is
# visible from inside a handler and is what the guards use
# now; the ordering lane fails if a task guarded that way
# runs before the one it waits on.
make clean >/dev/null
make all -j$(nproc)
./task_ordering_test
make clean >/dev/null
make all SANITIZER=address,undefined -j$(nproc)
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
./task_ordering_test
make clean >/dev/null
make all SANITIZER=thread -j$(nproc)
TSAN_OPTIONS=halt_on_error=1 ./task_ordering_test
make clean >/dev/null
- name: Build and run autodetect_handoff_test (plain, ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/autodetect_handoff
run: |
set -eu
# Autoconfig reports from a driver's own thread are made on
# the main thread, in order, where the queue's answer is exact.
make clean >/dev/null
make all -j$(nproc)
./autodetect_handoff_test
make clean >/dev/null
make all SANITIZER=address,undefined -j$(nproc)
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
./autodetect_handoff_test
make clean >/dev/null
make all SANITIZER=thread -j$(nproc)
TSAN_OPTIONS=halt_on_error=1 ./autodetect_handoff_test
make clean >/dev/null
- name: Build and run task_watchdog_test (plain, ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/watchdog
run: |
set -eu
# Oracle for the task queue's slow-handler watchdog,
# compiled against the shipping queue. With Threaded Tasks
# off, handlers run on the thread that drives the frame
# loop, so a handler that does not return promptly is a
# visible stall - and the queue is the only place that can
# attribute one to a specific task instead of to "something
# in the frame".
#
# The lanes pin: a handler inside its budget is silent; one
# that overruns is reported once, attributed to that task,
# with a duration above the budget; the threaded queue is
# deliberately not measured, because a long handler on a
# worker is what the worker is for; and an unregistered
# watchdog never fires, which is the default and the state
# release builds ship in.
make clean >/dev/null
make all -j$(nproc)
./task_watchdog_test
make clean >/dev/null
make all SANITIZER=address,undefined -j$(nproc)
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
./task_watchdog_test
make clean >/dev/null
make all SANITIZER=thread -j$(nproc)
TSAN_OPTIONS=halt_on_error=1 ./task_watchdog_test
make clean >/dev/null
- name: Build and run nat_task_serialisation_test (plain, ASan + UBSan)
shell: bash
working-directory: samples/tasks/netplay_nat
run: |
set -eu
# Oracle for the NAT traversal task's request serialisation,
# compiled from the shipping translation unit against the
# real task queue with the natt_* device layer faked.
#
# Only one NAT task may run at a time - both requests mutate
# the single shared request object - and that exclusion used
# to be enforced by blocking the calling thread, which is
# the one driving the menu, for the duration of UPnP
# discovery. The lanes pin the replacement: a push issued
# while a task is in flight returns without running it to
# completion, a close followed by an open runs as both in
# that order rather than being coalesced (coalescing would
# leak the port mapping), queued requests actually start
# from the task callbacks, and an overflowing request is
# refused rather than dropped.
make clean >/dev/null
make all -j$(nproc)
./nat_task_serialisation_test
make clean >/dev/null
make all SANITIZER=address,undefined -j$(nproc)
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
./nat_task_serialisation_test
make clean >/dev/null
- name: Guard against unbounded task_queue_wait in network/ and tasks/
shell: bash
run: |
set -eu
# task_queue_wait(NULL, ...) does not mean "wait for my
# task" - a NULL condition makes both queue implementations
# loop until the queue is EMPTY. A caller blocking on one
# HTTP round trip therefore also waits out every unrelated
# task in flight: a content scan, a core download, a
# thumbnail pack. Where a wait is still necessary it must
# name a condition that describes the caller's own work, so
# the block is bounded by that work and nothing else.
#
# This does not sanction waiting - the menu paths are held
# to the stricter no-wait guard below. It stops the
# unbounded form from coming back where waits remain.
hits=$(grep -rn --include='*.c' --include='*.cpp' \
'task_queue_wait[[:space:]]*([[:space:]]*NULL' \
network/ tasks/ || true)
if [[ -n "$hits" ]]; then
printf '%s\n' "$hits"
printf '::error title=Unbounded task_queue_wait::Pass a condition describing your own work; NULL waits for the entire queue to drain.\n'
exit 1
fi
printf 'No unbounded task_queue_wait in network/ or tasks/.\n'
- name: Guard against task_queue_wait on interactive menu paths
shell: bash
run: |
set -eu
# task_queue_wait() spins the calling thread until a task
# finishes. Reached from the menu it is a hard UI freeze
# for the duration of the work - a database scan on slow
# storage is seconds - and on Android it is an ANR. The
# menu teardown waits that used to live here were replaced
# by cancellation plus a stale-completion generation guard
# (menu_explore_cancel_init_task / menu_dbinfo_cancel_task).
#
# Scope is mechanical: menu/, the task translation units the
# menu drives directly, and the content load - a chain of
# main-thread tasks that closes content from the frame loop.
# The remaining wait sites (NAT traversal, the CLI database
# scan, the exit path) are tracked separately; widening this
# guard is what should happen as each of those is converted.
hits=$(grep -rn 'task_queue_wait' menu/ network/ command.c \
tasks/task_menu_explore.c \
tasks/task_database_info.c \
tasks/task_netplay_nat_traversal.c \
tasks/task_content.c || true)
if [[ -n "$hits" ]]; then
printf '%s\n' "$hits"
printf '::error title=Blocking wait on a menu path::task_queue_wait() must not be reachable from the menu. Use cancellation with a stale-completion guard, or a deferred continuation.\n'
exit 1
fi
# The close's blocking helpers wait inside task_save.c, where
# the grep above cannot see them: the menu and the tasks must
# not call them. The load closes through
# retroarch_main_deinit_begin/pending/finish instead.
hits=$(grep -rn 'content_wait_for_\(save\|load\)_state_task' \
menu/ tasks/ --include='*.c' \
| grep -v '^tasks/task_save.c:' || true)
if [[ -n "$hits" ]]; then
printf '%s\n' "$hits"
printf '::error title=Blocking close on a menu path::content_wait_for_*_state_task() blocks the frame loop. Close through retroarch_main_deinit_begin/pending/finish from a main-thread task.\n'
exit 1
fi
grep -q 'retroarch_main_deinit_pending' tasks/task_content.c || {
printf '::error title=Blocking close on a menu path::the content load no longer waits for state tasks from the frame loop.\n'
exit 1
}
printf 'No task_queue_wait or blocking close on the guarded menu paths.\n'
- name: Build and run playlist_parity_test (plain, ASan + UBSan)
shell: bash
working-directory: samples/playlist
run: |
set -eu
# Parity oracle for playlist.c's read path (playlist_init /
# playlist_read_file), compiled from the shipping
# translation unit. Pins the reader's observable behaviour
# ahead of its restructuring into a budgeted resumable
# parse: the full JSON schema, the
# malformed-JSON-keeps-partial-data contract, the old
# six-line format with its trailing metadata block, RZIP
# compressed input, missing-file-reads-empty, and capacity
# clamping. The budgeted lanes then assert the resumable
# API (playlist_parse_begin/step/end/abort) produces
# playlists identical to the blocking path while yielding
# repeatedly, for JSON, the old format and the autofix
# pass, and that abandoning a parse mid-document leaks
# nothing. The cache-reuse lane guards the size stamp the
# cache keys on: it must be the on-disk size for every
# format and compression combination - including after the
# autofix pass saves the file it fixed - or the cached
# playlist is never reused and the menu re-parses on every
# navigation.
#
# The switch and supersede lanes cover a field report -
# MaterialUI on Android showing one playlist's titles under
# another's heading, where selecting a row launches content
# from the wrong system. The deferred cached init yields
# mid-parse, and the menu reads playlist_get_cached() while
# building its list, so leaving the previous playlist
# installed until the new one is ready is user-visible and
# wrong. The lanes assert on EVERY yield that the cache is
# either empty or already the requested playlist, including
# when a request is superseded before it finishes, and that
# the parse really yielded - a lane that stops exercising
# its own window is worse than no lane.
#
# The pump and rebuild-reuse lanes cover issue #19427
# (large playlists flashing the empty-list placeholder):
# a yielded read must complete on frames alone, and the
# rebuild that follows completion must be served from the
# cache just installed - a restart there loops, one flash
# per completion. Pinned: the recorded
# base_content_directory under an autofix-off config, and
# the size stamp left stale by the autofix pass's save.
#
# The truncated-JSON lane also guards the staged-entry
# cleanup in playlist_read_file: an entry interrupted
# mid-object is reserved but never committed, and its
# already-parsed members must be freed on the error path -
# the leak the lane originally flushed out. LSan gates it
# here permanently. No threads on this path, so no TSan
# flavor.
make clean >/dev/null
make all -j$(nproc)
./playlist_parity_test
make clean >/dev/null
make all SANITIZER=address,undefined -j$(nproc)
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \
./playlist_parity_test
make clean >/dev/null
- name: Build and run dirwalk_budget_test (plain, ASan + UBSan, TSan)
shell: bash
working-directory: samples/menu/dirwalk
run: |
set -eu
# Oracle for menu/menu_dirwalk.c, the budgeted, cancellable
# directory enumeration behind interactive menu list
# building, compiled from the tree so it exercises the
# shipping translation unit.
#
# The regression it pins: filebrowser_parse and the
# playlist-directory displaylists walk and sort their
# directory synchronously on the thread that drives the UI,
# a stall proportional to directory size. menu_dirwalk
# caps it: one budgeted synchronous pass under the shared
# per-frame I/O window (small and medium directories
# complete on the spot, identical result, single tick),
# with oversized walks - and oversized sorts, which for a
# 40k-entry directory cost more than the walk - continuing
# on a task as a resumable enumeration plus a resumable
# bottom-up mergesort, ordered by the exact comparator
# dir_list_sort() uses.
#
# The clock is the test's instrument (the save_state
# pattern): the sample provides cpu_features_get_time_usec()
# itself. Step 0 reads the real clock for the pacing and
# bench lanes; a fixed virtual step per observation makes
# window exhaustion deterministic for the deferral, cancel
# and threaded lanes on any machine at any cache
# temperature. The build also shrinks the sort-machinery
# constants (run length, merge batch, synchronous-sort
# ceiling) so run sorting, the element-granular resumable
# merge, and sort-only deferral are all driven across many
# invocations by a 200-entry fixture.
#
# Parity is the contract: a consumed listing equals
# dir_list_new() plus the same sort, entry for entry with
# attributes, across the flag and sort-mode combinations
# the displaylist callers use. The pacing lane asserts no
# single main-thread stall - request, task gather, or
# consume - exceeds the window plus bounded slack on a
# 40k-file flat fixture (the walk under test is
# non-recursive: the menu browses one level, so the files
# sit flat, which is also what a large ROM directory looks
# like). The bench lane holds the hard precondition:
# wall-clock from first request to consumed result may not
# regress past the blocking walk + sort it replaces. The
# cancel lane supersedes and cancels mid-walk and drains
# stale completions, with LeakSanitizer judging the
# teardown; the threaded lane hands the walk to a worker
# and is the TSan target for the handler-to-main-callback
# hand-off.
make clean && make -j$(nproc) all
test -x dirwalk_budget_test
timeout 600 ./dirwalk_budget_test
echo "[pass] dirwalk_budget_test"
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 900 \
./dirwalk_budget_test sanitize
echo "[pass] dirwalk_budget_test (ASan)"
make clean SANITIZER=thread && make -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 900 \
./dirwalk_budget_test sanitize
make clean
echo "[pass] dirwalk_budget_test (TSan)"
- name: Build and run save_state_io_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/save_state
run: |
set -eu
# Oracle for the file I/O in tasks/task_save.c, compiled
# from the tree so it exercises the shipping translation
# unit. As in image_decode_slice, the clock is the test's
# instrument: the sample provides
# cpu_features_get_time_usec() itself, advancing a fixed
# step per observation, so the tick budget is assertable
# exactly rather than measured against a wall clock. Step 0
# is a budget that never expires; step >= the budget is one
# that expires on first observation.
#
# The throughput regression it pins: both handlers used to
# transfer exactly one SAVE_STATE_CHUNK per queue tick,
# which caps a save at chunk * tick_rate (~6 MB/s) whatever
# the device can do - a 16 MB state took 164 ticks, 2.7s at
# 60Hz, while one 100 KB write costs ~62us of a 16667us
# frame. A tick is now bounded by time and transfers as
# many quanta as fit. Because a time budget is only an
# improvement if it cannot be worse than the fixed count it
# replaced, the exhausted-budget lane asserts the fallback
# directly: with a clock that spends the budget on its first
# observation the handler must still transfer exactly one
# quantum per tick and still produce a correct file. That
# is the old behaviour reached through the new code, and it
# is what the slowest supported storage gets.
#
# The rest are the defects the audit turned up, each
# verified to fail this oracle on the pre-fix tree:
#
# - The rastate block walker looped on 'input < stop', so
# it could enter the body with one byte left and read the
# block length out of input[4..7] - six bytes past the
# end. content_deserialize_state had the same shape one
# level up, reading seven bytes of magic and input[7] for
# the version with no length check at all. Savestates
# are shared, synced and recovered off failing media, and
# the load handler's own short-read path produces exactly
# this buffer, so this is a format parser on hostile
# input. ASan reports both for any state truncated to
# 9..15 bytes.
#
# - A block's declared 32-bit length was never checked
# against the bytes actually remaining and was then
# passed to core_unserialize as the size of the buffer at
# that offset. The stub core here reads every byte it is
# told the buffer holds before judging the size, exactly
# as a real core does, so ASan - not the stub's own
# opinion - is what catches an over-long block. A stub
# that checked size first would pass against the unfixed
# walker.
#
# - That length was also assembled as input[7] << 24 on an
# int, undefined for every top byte >= 0x80, i.e. every
# length at or above 2 GB - the range a corrupt length
# lands in, and the value the bound check is asked to
# trust. UBSan is the assertion.
#
# - A failed core_serialize left data NULL and size 0, at
# which point every test in the save handler read as
# success and the task reported COMPLETED. The file had
# already been opened and truncated, so the user was told
# the save succeeded and the slot held zero bytes.
#
# - A failed open was a bare return: not errored, not
# finished, so the queue re-entered the task every tick
# and it retried forever. Save tasks are
# TASK_TYPE_BLOCKING, so that one task held the single
# blocking slot for the rest of the session - no state
# could be saved or loaded again without a restart. The
# exclusion rule that makes it severe is itself pinned,
# together with the cleanup the refusal path owes (the
# task, its title, the state struct and the caller's
# serialized buffer), which is why leak detection is on.
#
# The undo-snapshot lane is the one that needs the counting
# allocator (-Wl,--wrap=malloc in the sample's Makefile).
# content_load_state_cb retakes the undo snapshot on every
# state load, and that capture used to free its destination
# and malloc a new one each time; it now keeps the outgoing
# allocation as a spare and swaps the two. Pointer identity
# cannot see the difference - a state-sized block is mmap'd,
# munmap hands the region back, and the next request of the
# same size is usually given the same address - so a run of
# captures looks identical either way while only one of them
# pays the first-touch faults. Counting state-sized malloc
# calls is the discriminator: eight captures allocate two
# buffers after the change and eight before it.
#
# The conc lane runs the same round-trip under a threaded
# queue, where the transfer loop executes on the worker
# while content_deserialize_state and the undo bookkeeping
# stay on the gathering thread. It is one task at a time by
# design, not by simplification: TASK_TYPE_BLOCKING means
# the frontend cannot produce concurrent save tasks, so a
# lane that pushed them would be asserting against a
# configuration that cannot occur. What it asks TSan is
# whether a loop that newly carries a deadline across
# iterations kept any of it shared.
make clean && make -j$(nproc) all
test -x save_state_io_test
timeout 300 ./save_state_io_test
timeout 300 ./save_state_io_test conc
echo "[pass] save_state_io_test"
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 300 \
./save_state_io_test
ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=print_stacktrace=1 \
timeout 300 ./save_state_io_test conc
echo "[pass] save_state_io_test (ASan)"
make clean SANITIZER=thread && make -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 300 \
./save_state_io_test
TSAN_OPTIONS=halt_on_error=1 timeout 300 \
./save_state_io_test conc
echo "[pass] save_state_io_test (TSan)"
# The same oracle with HAVE_COMPRESSION, as the frontend
# ships it, states going through rzip. The lane that needs
# it runs only here: a save over a slot keeps the file that
# was there for undo, and an undo puts that file back byte
# for byte - compressed or not, whatever the setting has
# become - rather than inflating it on every save and
# deflating it afresh on undo.
make clean RZIP=1 && make -j$(nproc) all RZIP=1 SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 300 \
./save_state_io_test_rzip
ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=print_stacktrace=1 \
timeout 300 ./save_state_io_test_rzip conc
echo "[pass] save_state_io_test_rzip (ASan)"
- name: ccache statistics
if: ${{ !cancelled() }}
continue-on-error: true
run: ccache -sv
- name: Save ccache
# Only master's pushes keep a cache: pull requests restore
# master's, so one push does not evict the next one's.
if: success() && github.event_name == 'push' && github.ref == 'refs/heads/master'
uses: actions/cache/save@v4
with:
path: ~/.ccache
key: ccache-${{ github.workflow }}-${{ github.job }}-${{ github.sha }}
samples-tasks-http:
name: Build and run samples/tasks/http
runs-on: ubuntu-latest
env:
APT_PKGS: build-essential zlib1g-dev ccache
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@v5
- name: Apt cache key
id: apt
run: sh .github/workflows/scripts/apt-cache-key.sh "$APT_PKGS"
- name: Restore apt cache
uses: actions/cache/restore@v4
with:
path: ~/.apt-cache
key: ${{ steps.apt.outputs.key }}
restore-keys: ${{ steps.apt.outputs.prefix }}
- name: Install dependencies
timeout-minutes: 20
run: sh .github/workflows/scripts/apt-install.sh $APT_PKGS
- name: Save apt cache
if: env.APT_CACHE_SAVE == '1'
uses: actions/cache/save@v4
with:
path: ~/.apt-cache
key: ${{ steps.apt.outputs.key }}
- name: Link with lld
run: sh .github/workflows/scripts/use-lld.sh
- name: Restore ccache
uses: actions/cache/restore@v4
with:
path: ~/.ccache
key: ccache-${{ github.workflow }}-${{ github.job }}-${{ github.sha }}
restore-keys: |
ccache-${{ github.workflow }}-${{ github.job }}-
- name: Enable ccache
run: |
set -eu
export CCACHE_DIR="$HOME/.ccache"
echo "CCACHE_DIR=$CCACHE_DIR" >> "$GITHUB_ENV"
echo "/usr/lib/ccache" >> "$GITHUB_PATH"
ccache -M 250M
ccache -z
- name: C89 syntax check for the HTTP task translation units
shell: bash
run: |
set -eu
# Fast guard for the C89 dialect rules these files are built
# under. RetroArch's full C89 build lives in retroarch.yml
# (make C89_BUILD=1), which does catch violations but only
# after a configure and a complete build; a syntax-only pass
# over the handful of translation units this area touches
# fails in seconds instead.
#
# It exists because a mixed declaration slipped into
# task_core_updater.c and reached master: the C99-style
# "declare where first used" habit is silent under the
# default build and only surfaces here.
# -Werror=declaration-after-statement is the specific rule
# that catches it; the rest of the flag set mirrors the
# C89_BUILD block in the top-level Makefile. _GNU_SOURCE is
# added because -ansi otherwise hides the POSIX prototypes
# (strdup and friends) that these files legitimately use,
# which would produce a wall of int-conversion noise
# unrelated to the dialect rules being checked.
C89="-std=c89 -ansi -pedantic -Werror=pedantic -Wno-long-long"
C89="$C89 -Werror=declaration-after-statement -Wno-variadic-macros"
C89="$C89 -D_GNU_SOURCE"
INC="-Ilibretro-common/include -I. -Ideps"
DEF="-DRARCH_INTERNAL -DHAVE_NETWORKING -DHAVE_THREADS -DHAVE_COMPRESSION"
rc=0
for f in libretro-common/net/net_http.c \
libretro-common/queues/task_queue.c \
tasks/task_http.c \
tasks/task_core_updater.c \
tasks/task_pl_thumbnail_download.c \
tasks/task_core_backup.c \
core_backup.c \
libretro-common/streams/interface_stream.c \
tasks/task_file_transfer.c \
tasks/task_nbio_slice.c \
libretro-common/lists/dir_list.c \
libretro-common/formats/logiqx_dat/logiqx_dat.c \
libretro-common/formats/xml/rxml.c \
database_info.c; do
if gcc -fsyntax-only $C89 $INC $DEF "$f"; then
echo "[pass] C89 $f"
else
echo "::error title=C89 violation::$f"
rc=1
fi
done
exit $rc
- name: Build and run http_url_dedup_test (ASan)
shell: bash
working-directory: samples/tasks/http
run: |
set -eu
# Regression test for the duplicate-download suppression key
# in tasks/task_http.c and tasks/task_http_emscripten.c.
# connection_url was a fixed char[NAME_MAX_LENGTH] filled by
# strlcpy, but task_http_finder() compares that stored copy
# against the raw candidate URL - so past NAME_MAX_LENGTH the
# two could never compare equal and the "concurrent download
# of the same file is not allowed" guard returned false for
# every candidate. Playlist thumbnail URLs exceed it
# routinely, and both tasks then reached their completion
# callback and raced filestream_write_file() over one path.
#
# The failure is one-directional: truncation cannot conflate
# two distinct URLs, since the candidate is never truncated
# to match. So the test asserts both that the owned key
# self-matches at any length and that distinct long URLs
# stay distinct - the latter guarding against a future "fix"
# that hashes the key to a fixed width and introduces the
# false positive this bug did not have. It keeps the old
# fixed-buffer shape as a second oracle so a revert is
# caught. Both NAME_MAX_LENGTH values are covered: 256 on
# most targets, 128 on the small-path platforms.
make -f Makefile.dedup clean SANITIZER=address && make -f Makefile.dedup -j$(nproc) all SANITIZER=address
timeout 60 ./http_url_dedup_test
make -f Makefile.dedup clean SANITIZER=address && make -f Makefile.dedup -j$(nproc) all SANITIZER=address \
CFLAGS_EXTRA=-DNAME_MAX_LENGTH=128
timeout 60 ./http_url_dedup_test
make -f Makefile.dedup clean
echo "[pass] http_url_dedup_test"
- name: Build and run http_emscripten_park_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/http_emscripten
run: |
set -eu
# tasks/task_http_emscripten.c built natively against stand-in
# Emscripten headers: the task parks while its fetch is in
# flight and the fetch's settle wakes it.
make clean SANITIZER=address,undefined && make -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=print_stacktrace=1 \
timeout 60 ./http_emscripten_park_test
echo "[pass] http_emscripten_park_test (ASan + UBSan)"
make clean SANITIZER=thread && make -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 60 ./http_emscripten_park_test
echo "[pass] http_emscripten_park_test (TSan)"
- name: Build and run http_emscripten_headers_test (ASan + UBSan)
shell: bash
working-directory: samples/tasks/http
run: |
set -eu
# Header marshalling for tasks/task_http_emscripten.c, which
# sits between two representations: RetroArch's raw
# CRLF-delimited blob and the NULL-terminated key/value array
# emscripten_fetch wants, plus the response blob that has to
# become the same "Name: Value" header block net_http.c
# produces - through net_http_headers_compact(), which the test
# links rather than copies. Neither conversion existed while the backend was
# on emscripten_async_wget2_data, which has no header support
# at all - the `headers` argument was accepted and dropped on
# every entry point, so cloud sync could not authenticate.
#
# The parsing is where the sharp edges are: hand-built blobs
# in task_push_webdav_move() and
# task_push_http_transfer_with_content() mean bare LF,
# missing trailing CRLF, empty values, embedded colons and
# malformed lines all reach it. A header dropped here is an
# auth failure rather than a crash - silent, and awkward to
# trace back - so the degenerate cases are worth pinning.
# Pure logic, so it runs natively without emcc.
make -f Makefile.emsc_headers clean SANITIZER=address && make -f Makefile.emsc_headers -j$(nproc) all SANITIZER=address
timeout 60 ./http_emscripten_headers_test
make -f Makefile.emsc_headers clean SANITIZER=undefined && make -f Makefile.emsc_headers -j$(nproc) all SANITIZER=undefined
UBSAN_OPTIONS=print_stacktrace=1 timeout 60 \
./http_emscripten_headers_test
make -f Makefile.emsc_headers clean
echo "[pass] http_emscripten_headers_test"
- name: Build and run http_task_lifetime_test (ASan, TSan)
shell: bash
working-directory: samples/tasks/http
run: |
set -eu
# Lifetime contract for task->state between tasks/task_http.c
# and libretro-common/queues/task_queue.c. Links the real
# task_http.c and the real task_queue.c; only
# msg_hash_to_str() and task_window_progress_cb() are
# stubbed, since neither participates in lifetime.
#
# This is the lane the other HTTP samples could not cover:
# they drive net_http.c directly, which exercises the
# protocol engine well and the task layer not at all - and
# the task layer is where the crash was.
# task_http_transfer_handler() freed its http_handle_t as
# soon as it set RETRO_TASK_FLG_FINISHED, but a task stays
# findable long after its handler is done:
# retro_task_threaded_find() scans the running, finished AND
# retiring lists, and task_http_finder() dereferences
# task->state on every candidate. So task->state dangled for
# the whole window between the last handler tick and full
# retirement, and any concurrent push racing a finishing
# download read freed memory.
#
# It survived for years because connection_url was an inline
# char[NAME_MAX_LENGTH]: reading a freed-but-still-mapped
# array yields stale bytes and strcmp() plods through them.
# Making connection_url a pointer turned the same read into
# "load the freed chunk's tombstone and dereference it" - a
# hard SIGSEGV inside task_http_finder on Windows. The
# pointer change did not introduce the defect; it removed the
# padding hiding it.
#
# The shape is the one from that backtrace: threads push GETs
# for a small set of URLs while the queue finishes and
# retires earlier ones, so every push runs task_queue_find()
# over all three lists and collides with retirement
# continuously. ASan is what makes it assertable - without
# it a use-after-free on a recently freed chunk reads
# plausible bytes and passes. TSan matters as much, since
# the race is what makes the window reachable.
#
# To confirm this test still bites, move the teardown in
# task_http.c from task_http_transfer_cleanup() back into
# task_http_transfer_handler() and rerun under ASan.
make -f Makefile.task_lifetime clean SANITIZER=address && make -f Makefile.task_lifetime -j$(nproc) all SANITIZER=address
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
timeout 300 ./http_task_lifetime_test
echo "[pass] http_task_lifetime_test (ASan)"
make -f Makefile.task_lifetime clean SANITIZER=thread && make -f Makefile.task_lifetime -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 300 \
./http_task_lifetime_test
make -f Makefile.task_lifetime clean
echo "[pass] http_task_lifetime_test (TSan)"
- name: Build and run http_transfer_stress_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/http
run: |
set -eu
# Transport-level oracle for libretro-common/net/net_http.c
# as driven by tasks/task_http.c, against a real loopback
# server. Framing correctness comes first - Content-Length,
# chunked across chunk sizes 1..65536, and
# connection-close-delimited, then the same again with the
# server dribbling a few bytes per write so every header
# line, chunk header and chunk boundary is torn across
# receive calls.
#
# Two things make the tick-cost lane mean anything. It
# interposes socket() to pin SO_RCVBUF, because per-update
# byte yield otherwise tracks host /proc/sys/net tuning and
# autoscaling; and it sleeps the task-queue tick period
# between updates, because without that the client polls
# faster than the peer can fill the socket, every recv()
# takes a sliver, and the number reports the server's write
# rate rather than the structural per-tick cap - it swung
# between 263 and 93207 B/tick across runs. Sleeping the
# period lets the receive buffer saturate, which is the
# regime RetroArch actually runs in and the only one where
# ticks-per-body is stable.
#
# The peak-window ceiling for the streaming sink is paired
# with a contrast case asserting the buffered path *exceeds*
# it, so the pair cannot pass by both paths happening to stay
# small on a given host. The teardown lane deinits,
# re-inits and transfers again: calling net_http_deinit() is
# what makes a leak in the DNS cache and connection pool
# visible to LeakSan at all, since both lists hang off
# file-scope globals and otherwise count as still reachable.
make -f Makefile.stress clean SANITIZER=address,undefined && make -f Makefile.stress -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 600 \
./http_transfer_stress_test
echo "[pass] http_transfer_stress_test (ASan)"
make -f Makefile.stress clean SANITIZER=thread && make -f Makefile.stress -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 600 \
./http_transfer_stress_test
make -f Makefile.stress clean
echo "[pass] http_transfer_stress_test (TSan)"
- name: Build and run http_transport_failure_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/http
run: |
set -eu
# What a failed transfer is allowed to hand back, across
# libretro-common/net/net_http.c and tasks/task_http.c.
#
# net_http_new() allocates response.data up front as the
# receive buffer - 64KiB of plain malloc(). On a transport
# failure nothing is written into it and response.len stays
# 0, but the pointer is non-NULL, and net_http_data()'s
# accept_err returned it, so every failed download published
# uninitialised, unterminated heap as
# http_transfer_data_t::data. On a torn body it was worse:
# the caller got the Content-Length remainder as the length,
# over bytes that never arrived. Meanwhile task_http.c gated
# task_set_error() on RETRO_TASK_FLG_MUTE, which only ever
# suppressed the on-screen notification - so muted transfers
# (achievements, netplay lobby, core list refresh) reported
# failure to their callbacks as success.
#
# Four failure lanes entering net_http_update() at four
# different points, plus two contrast lanes: a 200 must still
# deliver body, headers and status, and a 404 must still
# deliver its body under accept_err, or the fix could be
# satisfied by an accessor that returns NULL rather more
# often than that and cloud sync's WWW-Authenticate path
# would break silently. Both contrast lanes pass on the
# pre-fix tree, which is what makes them a guard. Two task
# lanes run the same failure through the real task_http.c and
# task_queue.c, muted and unmuted.
#
# getaddrinfo is wrapped at link time so the DNS lane cannot
# quietly start testing a successful lookup on a runner whose
# resolver answers for everything. Leak detection is on
# because the fix changes ownership: net_http_delete() is now
# the only thing that frees the receive buffer on failure.
#
# To confirm this still bites, drop the `status < 0` guard
# from net_http_data() and rerun.
make -f Makefile.transport_fail clean SANITIZER=address,undefined && make -f Makefile.transport_fail -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1 timeout 300 \
./http_transport_failure_test
echo "[pass] http_transport_failure_test (ASan)"
make -f Makefile.transport_fail clean SANITIZER=thread && make -f Makefile.transport_fail -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 300 \
./http_transport_failure_test
make -f Makefile.transport_fail clean
echo "[pass] http_transport_failure_test (TSan)"
- name: Build and run http_resume_test (ASan + UBSan, TSan)
shell: bash
working-directory: samples/tasks/http
run: |
set -eu
# An interrupted download to a file (task_push_http_download_file,
# as the core updater and thumbnails use) resumes with
# "Range: bytes=N-" and If-Range on the first response's strong
# ETag. A 206 from the right offset continues the file; a 200
# (resource changed) starts it over; a 206 from the wrong offset,
# a first response without Accept-Ranges or a validator, and a
# connection that keeps dropping (more than three resumes) fail
# and remove the partial file. Every interrupted download used
# to fail outright.
make -f Makefile.resume clean SANITIZER=address,undefined && make -f Makefile.resume -j$(nproc) all SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 \
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 timeout 300 \
./http_resume_test
echo "[pass] http_resume_test (ASan)"
make -f Makefile.resume clean SANITIZER=thread && make -f Makefile.resume -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 300 ./http_resume_test
make -f Makefile.resume clean
echo "[pass] http_resume_test (TSan)"
- name: Build and run HTTP initialization failure test
shell: bash
working-directory: samples/tasks/http
run: |
set -eu
for threads in 0 1; do
make -f Makefile.init_failure clean
make -f Makefile.init_failure THREADS=$threads
timeout 60 ./http_init_failure_test
done
make -f Makefile.init_failure clean
- name: Build and run http_limits_test (ASan + UBSan, threaded and not)
shell: bash
working-directory: samples/tasks/http
run: |
set -eu
# net_http.c included whole, with the clock and the socket
# calls stubbed. The DNS cache is capped (it grew by one entry
# per distinct host:port for the whole session), evicts least
# recently used first, and never evicts a lookup still in
# flight. The pool keeps at most 8 idle connections, closing
# the longest-idle first, and closes rather than reuses one
# idle for more than 30 s. On a system without IPv6, an IPv6
# address listed first no longer fails the transfer, and one
# mid-list no longer hides the addresses after it. Bracketed
# IPv6 literals ([::1]:8080) parse in URLs and redirects and
# reach the resolver without their brackets.
for threads in 1 0; do
make -f Makefile.limits clean
make -f Makefile.limits THREADS=$threads SANITIZER=address,undefined
ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=halt_on_error=1 \
timeout 60 ./http_limits_test
done
make -f Makefile.limits clean
- name: Build and run HTTP DNS connect lifetime test
shell: bash
working-directory: samples/tasks/http
run: |
set -eu
for ssl in 0 1; do
make -f Makefile.dns_lifetime clean
make -f Makefile.dns_lifetime SSL=$ssl
timeout 60 ./http_dns_lifetime_test
done
make -f Makefile.dns_lifetime clean
- name: Build and run http_dns_evict_deadlock_test (ASan, TSan)
shell: bash
working-directory: samples/tasks/http
run: |
set -eu
# Deadlock regression for
# net_http_dns_cache_remove_expired(), which runs with the
# DNS cache lock held and used to sthread_join() the resolver
# thread of any entry it evicted. net_http_resolve() takes
# that same lock twice, so a join against a thread blocked
# acquiring the lock the joiner holds never returns.
#
# The window is not theoretical: entry->addr stays NULL for
# the whole resolution, so the fail-timeout arm fires after
# 30s - and getaddrinfo() against a blackholed resolver
# routinely blocks longer. The entry looks expired precisely
# while its own resolver is still running, hanging the task
# thread (and in unthreaded builds the frontend) over nothing
# worse than a slow DNS server.
#
# Reproducing that honestly would mean a 30s wait against an
# unreachable nameserver, so the test wraps two symbols at
# link time instead: getaddrinfo, to park the resolver
# mid-flight on one nominated hostname, and
# cpu_features_get_time_usec, to age the entry past the
# timeout instantly. --wrap rather than a strong definition
# because both live in libretro-common objects linked into
# the test.
#
# Because the failure mode is a hang rather than a wrong
# answer, the sweep runs behind a watchdog: on timeout the
# test reports and exits non-zero instead of blocking CI.
make -f Makefile.dns_evict clean SANITIZER=address && make -f Makefile.dns_evict -j$(nproc) all SANITIZER=address
ASAN_OPTIONS=detect_leaks=1 timeout 300 \
./http_dns_evict_deadlock_test
echo "[pass] http_dns_evict_deadlock_test (ASan)"
make -f Makefile.dns_evict clean SANITIZER=thread && make -f Makefile.dns_evict -j$(nproc) all SANITIZER=thread
TSAN_OPTIONS=halt_on_error=1 timeout 300 \
./http_dns_evict_deadlock_test
make -f Makefile.dns_evict clean
echo "[pass] http_dns_evict_deadlock_test (TSan)"
- name: ccache statistics
if: ${{ !cancelled() }}
continue-on-error: true
run: ccache -sv
- name: Save ccache
# Only master's pushes keep a cache: pull requests restore
# master's, so one push does not evict the next one's.
if: success() && github.event_name == 'push' && github.ref == 'refs/heads/master'
uses: actions/cache/save@v4
with:
path: ~/.ccache
key: ccache-${{ github.workflow }}-${{ github.job }}-${{ github.sha }}
Reference in new issue View Git Blame Copy Permalink