A sprite reading the buffer it draws into, linear, from beside where
it draws, is a blur in place: its texture is held for the render pass
and read as the buffer was before it, rather than the pass ending at
every column whose page the last one wrote. The rule matched a UV box
exactly two texels larger than the draw on both axes at a bias of one;
a blur sampling an axis on texel centres, or at the texture's edge,
has fewer extra texels there, and one reaching a few texels aside has
a larger bias. The rule now takes zero to two extra texels an axis and
a bias up to four.
A game blooming through such a blur, a column of 32 at a time: 80
render passes a frame become 33, with 2130 of every 2400 passes
before holding one to three primitives.
tests/swdraw blur_in_place: a buffer blurred in place a column at a
time every frame, brightened until white and read back on every
renderer, in at most two render passes a frame on paraLLEl-GS.
The GS loop waits for ring work with the timed wait, for its wedge
bound; that wait parked at once on an empty ring, where the untimed
one spins its budget first. A park is the asymmetric eventcount's
process-wide barrier and a kernel sleep/wake pair, paid for work a
few packets away. The timed wait now spins the same budget before
the park; the bound covers the park alone, as before.
A game in play: 38 barriers a frame become 24 (strace -c on
membarrier over 100 frames).
A target drawn at another buffer width has its pages moved to that
width's places first, a copy a page. When the draw keeps nothing of
the target (no blend reading it, no mask, no test) and covers every
page at the new width, what is moved is overwritten at once;
RelayoutTarget now takes that and leaves the new texture unwritten
instead. A game redrawing a buffer at two widths in turn every frame:
32 image copies a frame on the Vulkan renderer become none, the
pictures the same.
tests/swdraw width_overwritten: a buffer drawn four pages wide and
then five every frame, each draw covering all forty pages, then read
as a texture: the same picture on the software renderer, paraLLEl-GS
and the Vulkan renderer, and no image copy a frame on the latter.
A colour draw at another buffer width over a depth buffer's pages had
the whole depth converted to colour at the depth's width, and the
converted target then laid out again at the draw's width with a copy
a run of pages. RelayoutTargetWidth now takes the texture the pages
come from and the conversion shader, and converts each run straight
into its place; the full-size conversion and the copies after it are
gone. Taken when the target has no dirty rects (they are in the old
layout) and both are at one scale; otherwise as before.
A game filling its depth buffer's RGB through a narrow colour frame
every frame: 62 image copies a frame on the Vulkan renderer become
32, the pictures the same.
tests/swdraw depth_narrowed: a depth buffer ten pages wide drawn every
frame, a colour draw two pages wide with its alpha masked over its
pages, and the pages read back as a texture at that width: the same
readback and picture on the software renderer, paraLLEl-GS and the
Vulkan renderer, and no image copy a frame on the latter.
The three GIF paths' pending-read counts sat 92 bytes apart inside
Gif_Path, next to the packet state the EE writes while the GS thread
decrements them; every decrement of one path dragged the line the EE
was working on. They live in their own 64-byte slots now; Gif_Path
keeps its readAmount field at zero so the savestate layout is as
before.
The AVX2 setup code made the Gouraud colour steps of a primitive in
scratch slots of the scanline temporaries, among them temp.uv_minmax.
That slot is not a temporary: with a constant LOD (TEX1.LCM, or UV
coordinates, or every level past MXL) BeginDraw puts the mip level's
clamp there once for the whole draw, and the scanline code masks every
texel coordinate with it. A primitive after the first found a colour
step there instead - a small negative number for a ramp - and a repeated
texture's coordinates, masked by that, reached megabytes past the level:
a racing game's race start read 8 MB past an 8x8 level and the
rasterizer thread died in the fetch.
The steps go in temp.lod instead, which the scanline code writes before
it reads in the per-pixel LOD path and never reads in the constant one.
tests/swdraw: mip_shaded, a mipmapped PSMT8 texture repeated across
Gouraud triangles at a constant LOD of 1. The swdraw lane holds its
picture on four rasterizers to the one on one, and the vknegotiate lane
holds the software renderer's, on one and on four, to Vulkan's and
paraLLEl-GS's; before, the three differed.
Every change of texture allocated and wrote a TFX texture descriptor set,
and every change of target a target set: 462 sets a frame for 580 draws
in a racing game, as its draws cycle through their textures, each a
vkAllocateDescriptorSets and a vkUpdateDescriptorSets (and on a driver
allocating sets from the heap, a malloc).
The sets made on a command buffer are kept in a table keyed by what they
bind (the two views, the sampler, the layouts), cleared with the pool
when the command buffer comes round again; a draw whose binding an
earlier draw on the buffer had takes that set. The same game now makes
157 sets a frame, every frame hash-identical.
tests/swdraw: texture_alternate, sixteen sprites a frame reading two
textures in turn. tests/vknegotiate counts the Vulkan renderer's
descriptor sets a steady frame: at most eight (five: the two textures,
the target, the readback and the present), where a set a draw made
nineteen.
Three things were made anew every frame a game streams its textures:
The texture slab kept only power-of-two images up to 1024 a side, and
the extents textures are cut to by their clamps rarely are: 34x113,
640x448 and the like, a hundred VkImages a frame in a racing game, each
made and destroyed with its views. The slab now keys on the exact
dimensions, levels and layers, any size up to 4096 a side, in buckets
walked newest first; the byte budget and its flush stay as they were.
Granite's staging pool kept 32 blocks of 64 KB, two megabytes, where a
frame's uploads ran to three; and an upload larger than a block took a
block of its own that the pool never kept, so every texture past 64 KB
made a VkBuffer for itself. The pool keeps 256 blocks' worth of bytes
and keeps oversized blocks within that budget, handing one back to the
next request it fits; a plain request still takes a plain block.
The scratch buffer a frame's render passes fill (32 MB, mostly coarse
tile lists) was dropped when full and a new one made. A full one is set
aside with the timeline value whose signal says the GPU is done with it,
and the next scratch of its domain takes it back once that value is
reached; up to four are kept.
In the racing game a steady frame on paraLLEl-GS went from 100 images
and 9 buffers created to none, every frame hash-identical.
tests/swdraw: stream_clamped, a 1024x120 texture streamed six times a
frame (each upload larger than a staging block) and read through a
region clamp 34x113. tests/vknegotiate counts a steady frame's images
and buffers on paraLLEl-GS: none.
InvalidateVideoMem took every colour draw over a source's pages as
rewriting them, whatever the draw's frame mask kept. A game that keeps a
PSMT8H texture in the top byte of its depth buffer's words and draws RGB
over those pages through a 32-bit frame with the alpha masked lost the
texture every frame: the source was dropped and made again, its memory
hashed (1 MB a frame here) for the hash cache to find the texture it
already had; past the cache's size, uploaded again.
The frame mask of the draw doing the write now reaches the invalidation,
and a 32-bit write whose mask keeps every bit a source reads (the top
byte for 8H, its nibbles for 4HL and 4HH, the low three bytes for 24-bit)
leaves that source alone, in the per-page walk and in the same-base
removals.
tests/swdraw: alpha_under_depth_big, the alpha uploaded as PSMT8H and
read as a texture too large for the hash cache, the masked colour draw
one page wide at the depth buffer's base over its first page.
tests/vknegotiate counts the Vulkan renderer's uploads over a steady
frame: one (the buffer's RGB the depth draw dirtied), where the dropped
texture's page made two.
The device negotiation hands the core the frontend's vkGetInstanceProcAddr,
and the libretro Vulkan interface asks that every entry point be fetched
through it; the GSdx Vulkan renderer fetched them from libvulkan's own
instead, so a frontend wrapping the loader's - tests/vknegotiate counting
the calls a frame makes - saw nothing of this renderer's calls. The
instance and device functions now load through the one the frontend gave,
which for a frontend passing the loader's through is the same thing.
Each render pass bound its textures through a bindless descriptor set
of its own, allocated from the pool with the pass's count; a driver
that backs every variable-count set with its layout's whole capacity
(lavapipe: four megabytes a set) then paid that for every pass, and a
loading screen's hundreds of passes a second grew past memory within a
minute. A pool now holds one set of every descriptor it has, allocated
once, and each pass takes a range of it, written at its place while
the set stays bound (update after bind); the shader indexes its
textures from the pass's base (GlobalConstants.bindless_base). A
descriptor-buffer pool already worked this way. Granite's
BindlessAllocator, unused, goes.
tests/vknegotiate counts descriptor set allocations (descsets) and
checks a steady frame makes none.
A sky's clouds, read through a palette the game rebuilds in a target
every frame, came out as a hatched blotch at 4x and were right at
native. Three things at scale, each pinned by a case:
A target preloaded from local memory was upscaled with a linear
filter (the bilinear upload hack, on by default and through the GameDB
for a hundred titles), so a palette entry's block of fragments held a
gradient of its neighbours. A pixel of local memory is now a block of
the scale's fragments, as the GS has it; the hack, its config and its
GameDB fix go.
A palette copied from a target is one value an entry: its native
pixel's, read at the place among the pixel's fragments where the draw
at scale put the native sample (NativeSample: the first fragment's
centre on the kept grid, the pixel's middle under the Normal and
Native offsets, between two fragments at an even scale, with a linear
fetch). The copy that kept every fragment as a sample of its own, and
let each fragment of a draw read its own, gave every fragment a
different entry; the readback at scale took the block's middle for
every draw, half a pixel off on the kept grid, and takes the same
place now.
A point is drawn at the pixel its vertex is in, whatever the vertex's
fraction: its centre for a sized point, which spans half the size
either way, its origin for one expanded into a quad; a sized point at
scale was centred on the pixel's corner and covered a quarter of it.
tests/swdraw: palette_bilinear, palette_bilinear_drawn and
palette_shaded, native and at 4x on both renderers in
tests/vknegotiate, which runs palette_engulfed at 4x too;
tests/gsclut/clut_from_target checks the entry's sample.
A sky blended by the alpha of a gradient drawn into a small buffer,
then taken by the wide buffer around it, came out grey from a fresh
start: the wide buffer's draws had masked alpha, so its alpha was not
valid, and the first read needing it laid local memory's flat alpha
over the whole buffer, the gradient's page included. Four things in
CopyTargetPagesInto, and one each side of it:
A channel the older target holds that the new one has not drawn is
the new one's from the copy on: its other pages take local memory's
first, as a preloaded target has them, and the channel is marked
valid, so a read of it finds the copied page rather than refilling
it. The alpha range grows over the older target's, or a blend by the
copied page's alpha was skipped as flat. Both keep their alpha as the
GS does before the copy; scaled, an alpha past 0x80 (a palette's)
cannot be. The pages taken count as drawn, and the target is valid
over them, so a readback carries them to local memory - the older
target, parked, leaves them to it.
CreateTarget scaled a new target's alpha by setting the flag over
whatever it held; it now scales the texture, and only where the alpha
allows.
A state load dropped every target and marked only the pages whose
bytes the state changed as restored, so a target made again over an
unchanged page started blank past its first draw. TargetsRestored
marks the dropped targets' pages, whether the state changed them or
not.
tests/swdraw: engulfed_alpha, on both renderers in tests/vknegotiate
and the software renderer in tests/swdraw, with a savestate round
trip.
A sky drawn once into the alpha of a wide buffer, its palette drawn
into a small one, and the gradient under it drawn into half a page,
came out white or black from a fresh start, and black again from a
state the hardware renderer had saved. Four things in the texture
cache, all of the same kind - a page of local memory holds several
targets' pixels, a channel each, and the cache treated it as one
target's:
A target leaving the cache skipped any page a transfer had written,
whatever the transfer wrote: a 24-bit Z laid over the sky's pages by a
depth buffer of another width marked their RGB dirty, and the alpha
holding the sky was never written to local memory. A dirty rect now
takes away only the channels it was written with, in DropKeepsPage and
in the read's own dirty check, and the page stamps a read and a drop
leave - m_page_read_draw and DropBatch::page_draw - are per channel, so
a read of a target's RGB leaves the page's alpha to whoever holds it. A
dropped target claims only the channels it has drawn.
A target made over an older one of another width left the older one
in the lists and started blank where its pages were; the palette
lookup, going by the drawn rect's bounds, then read the blank page.
CopyTargetPagesInto carries the older target's pages into the new one
page for page, as RelayoutTargetWidth lays them out, and parks the
older one; a display target, which spans the buffers drawn inside it,
is left out.
A texture based at a block inside a page of a target was read from
local memory, which had never seen it. The GS adds each texel's block
to the base, so a read kept to one page whose blocks share no bit with
the base's block in it is the target's texels from that block's place
on: LookupSource serves it from the target, at that offset.
A state the hardware renderer saved wrote every target back whole, a
depth buffer's conversion over the sky's alpha; ReadbackAll now goes by
the drop rule, and a target made over pages a state restored, which
the cache cannot know the GPU had, starts from local memory
(mem_restored, TakeRestoredPages).
tests/swdraw: alpha_dirtied_parked, palette_engulfed and
block_offset_read pin the four, on both hardware renderers and across a
state saved by one and loaded.
A colour target whose first pages a new target takes over (PreloadTarget
copies them into the new one and let the old target go) is now parked
as a target dropped by age is: its pages go to local memory when they
are read. A page a later target holds is written for the channels that
target does not hold: a 24-bit target, or one drawn with its alpha
masked, leaves the page's alpha to the older target. Without this, a
buffer whose alpha is laid down once and read every frame as PSMT8H
through a palette, under a 24-bit depth buffer and effects drawn with
their alpha masked, read zeros once an effect was drawn at the depth
buffer's base: the sky of a game that keeps its clouds in that alpha.
A state taken for another instance to load writes the targets back to
local memory first, so it holds what the GPU drew; a state for this
instance (runahead) keeps them on the GPU as before.
tests/swdraw's alpha_under_depth, _wide and _strip pin the three
layouts on the software renderer, Vulkan and paraLLEl-GS, through a
state save and load; the harness takes 16 kicks.
A colour draw at the base of a depth buffer laid out at another buffer
width gets the depth buffer's contents laid out at its own width, page
for page as local memory holds them, before it draws. A striped double
clear also covers a 32-bit colour with its alpha masked under a 24-bit
Z: both write the same three bytes, so the clear fills the whole page
column in colour and keeps the alpha.
Together these give a bloom pass drawn a page column at a time
the bright pass the GS draws, where it came out as bright bands
a column wide, at every scale.
tests/swdraw/column_passes draws such a pass twice over a depth buffer
ten pages wide and reads back the second row of pages, on the software
renderer, Vulkan and paraLLEl-GS.
A local to host transfer is read out of local memory when the FIFO is
first read, as the other renderers do. A read that takes all of it
gets the pixels written straight into its buffer; one that takes a
part, or a new transfer that comes first, reads it into the FIFO's own
buffer, from which reads are served as before.
tests/swdraw/readback_parts reads a transfer in two parts with a packet
between them, and upload_clut_between has a palette load read the
first half of an upload before the second arrives; both run on Vulkan
and paraLLEl-GS in tests/vknegotiate, and on the software renderer.
Every variable in paraLLEl-GS's own sources (gs/) is declared with its
type, named in full where it is nested. The anonymous structs a
variable referred to have names, the lambdas held in variables are
static functions or written out where they were called, and
RangeMerger hands its merged ranges back to the caller instead of
calling one. gs_util, page_tracker and gs_interface build to the same
code size; in gs_renderer, copy_blocks is now small enough to be
inlined into its callers.
On a device with a real texture barrier (GL_ARB_texture_barrier or
NV_texture_barrier), a batch drawn as runs samples its render target
directly, as one draw in groups with a barrier before each: a primitive
starts a group when it reads where the group wrote, writes where it
read, or, blending, writes where it wrote. Nothing is copied. A batch
with a primitive reading what it writes, a second alpha pass, colour
clip or primitive-ID DATE reads a copy of the whole target instead,
refreshed run by run; other devices copy as before.
tests/gsautoflush/runs_groups pins the grouping against drawing the
primitives one by one. ASan/UBSan and TSan clean on the gsautoflush
suite; the feedback cases draw the GS's picture on GL and Vulkan.
Where auto flush would end a batch -- a primitive reading what an
earlier one of the batch drew -- the batch now keeps going, recording a
segment boundary instead (up to 64 a batch). The hardware renderer
draws the whole batch with one texture cache lookup and one state
setup, as a draw per run of primitives between boundaries: before each
run after the first, the copy of the target the draw samples is
refreshed over just the texels that run reads, from what the runs
before it drew. Every backend's RenderHW and CopyRect take it as is;
nothing is backend specific.
The renderer says per batch whether it can (a texture at the frame,
same format and width, colour, unmasked, no hack paths); when it
cannot, or on the software renderer, the batch is handed to Draw one
segment at a time, each with its own vertices and trace, as flushing
at each boundary did. Auto flush's detection looks back over the
current segment only, against what the segments before it drew. A
draw whose setup merged or split the batch's primitives draws each of
its primitives as a run of its own, which still sees every write
before it.
On a batch of 7200 sprites chained across 1500 boundaries, the GS
thread's time on llvmpipe goes from 126 to 86 ms; a batch with no
feedback costs what it did.
tests/swdraw/feedback_runs: four chains, six sprites long, interleaved
into one batch that splits five times. The software renderer's memory
and the OpenGL and Vulkan renderers' picture must hold the seed halved
six times in every chain's last block; drawn without refreshing the
copy between runs, it is black.
Two swdraw cases: three sprites in one batch each reading, at half, the
block the one before wrote a page to its left, the third holding the
seed halved three times; and three sprites over one area each reading
exactly its own pixels, the area halved three times. The software
renderer's GS memory must hold those words (swdraw), and the OpenGL and
Vulkan renderers must draw the same picture, the one those words make
(glnegotiate). Without auto flush, the first case's second and third
blocks come out black on both.
On drivers with no copy_image, CopyRect copies with glBlitFramebuffer,
whose destination rectangle took the source rect's offset on top of the
one it was given, so a copy from a rect not at the origin landed that
far right and down of where it was asked to. The blit now writes at the
destination offset, as the copy_image paths and the other devices do.
tests/gscopyrect/copyrect_dest pins where each path's copy lands.
A primitive that samples the frame nearest, each vertex's texel within a
texel of its pixel and inside any region clamp -- a sprite, or a
triangle writing alpha alone -- reads only the pixels it writes. The
hardware renderer draws a batch of those reading the target in place,
with a barrier between overlapping primitives, which sees every earlier
write; auto flush no longer ends such a batch at an overlap. Every
primitive of a batch is classified, its first included, and one that
reads elsewhere, after an overlap was let through, flushes before it.
A batch an overlap was let through for is drawn in place. The software
renderer, and a device without texture barriers or framebuffer fetch,
flush as before.
tests/gsautoflush/pixel_feedback pins which overlaps stay in a batch.
The vertex kick handlers come in two sets, plain and auto-flush, and
which one a draw kicks through is chosen when a register that decides it
is written -- TEX0 and TEX2, TEST, FRAME and ZBUF of the current context,
PRIM and PRMODE -- and changes only when the answer does: the draw state
reads what it draws. Every other draw kicks through the plain handlers,
which do not ask, where before every kick asked whenever auto flush was
on (by default in the software renderer, and in the hardware renderer
for every game the database turned it on for).
Auto flush is now always on, for every primitive, in both renderers.
The hardware level setting, the software switch, their core options and
the GameDB fix and its entries go.
tests/gsautoflush/autoflush_select pins the choice against the state
after every register write.
It drew small point lists into 32-bit buffers on the CPU so a CLUT load
from them would find the palette in local memory. A CLUT load reads a
palette the GPU drew from its target, or reads the target back where
the GPU palette path does not apply. Its GameDB entries go with it.
A CLUT load skips bringing local memory up to date only where the draw
will read the palette from the target holding it on the GPU: CSM1 with
its CSA in the first sixteen blocks. A CSM2 load, or a 16-bit palette
past block 15, falls back on the palette built from local memory, so
the target holding its blocks is now read back before the load reads
memory.
tests/gsclut/clut_load_sync pins the choice against GSClut::Read32's.
It drew palettes on the CPU into local memory -- lists of points into a
32-bit buffer, and textured sprites from memory -- so a CLUT load from
there would find them. A CLUT load already takes a palette a target
holds from the target, on the GPU, assuming its full alpha range, and
one a parked target drew is written to memory before the load reads it.
Its GameDB entries go with it.
A target dropped by age leaves the lists but keeps its texture. What it
drew goes to local memory, by the same rule as a drop, only when that
memory is read: an EE readback, a CLUT load, a local copy, a source or a
target's dirty area loading from memory, or a primitive drawn on the
CPU. A write over its pages retires them instead, and one left with
nothing local memory lacks goes. Parked targets are the first to go
over the memory budget, and are all written back when the cache is read
back whole.
A target nothing reads again now costs no readback and no wait.
tests/gstc/park_flush pins the rule against the memory each read should
see.
The colour target holding the pages a constant Z writes now gets the
value by a fill on the GPU, from a one-texel target cleared to it, in
place of having those pages uploaded again from local memory; local
memory still gets the Z, and every other target over the pages reloads
them from there. The pages go where the target's own layout puts them,
and pages past its width within its buffer width widen it.
A draw on an existing target's pages past that target's width, within
its buffer width, widens it the same way rather than going into a
target of its own. A widened target keeps the width for the next target
made at its address, so it is sized once.
It drew two kinds of sprite on the CPU so their pixels would reach local
memory: the clouds drawn at load time into a 32-bit buffer's alpha, read
back every frame afterwards as an 8-bit indexed texture, and a decal
drawn into a buffer's alpha with RGB masked. Both are targets dropped
while their contents were still on the GPU only, which the cache now
writes back as it drops them; its GameDB entries go with it.
On the GS a page keeps what was last written to it until something
writes it again, and a game can read it any time later. A target the
cache drops by age or over its memory budget now writes the pages of
what it drew since its last read back to local memory first, except
pages a transfer has written since, pages a target drawn after it still
holds, and pages holding newer pixels: those a later target dropped in
the same pass takes, and those a read has put there, which the cache
stamps per page with the draw they came from. The drops of one pass are
written out together and deleted after.
tests/gstc/drop_writeback runs random histories of draws, transfers and
drops against the memory they should leave.
A sprite with ZTST ALWAYS and one Z writes that Z into memory in the Z
layout. When the whole pages it writes belong to a colour target drawn
after any depth buffer there, and the draw's own colour is on other
pages, the Z is written into local memory and those pages of the colour
target are refreshed from it, with no depth buffer made for the draw.
A buffer one page column wide drawn in turn at two neighbouring pages,
each draw laying the next column's starting value down as Z, now works
as drawn.
GSC_BurnoutGames, which collapsed such a pass into one full-screen draw,
is removed; its GameDB entries keep the sky and cloud handling through
GSC_BlackAndBurnoutSky. tests/gstc/zwrite_colour pins the rule against
the GS's PSMCT32 and PSMZ32 addressing.
FileSystem::FindFiles filled a std::vector of entries holding
std::string names, walking the tree with std::string paths; HostFS.h
pulled <string> and <vector> into the sixteen files that include it,
most of them for the path separator alone.
HostFS.cpp is now HostFS.c, C89, with the same walk and filters:
FileSystem_FindFiles fills a FILESYSTEM_FIND_RESULTS array whose names
it mallocs, FileSystem_FreeFindResults frees them, and KEEP_ARRAY keeps
what is there as before. The header is C with extern "C" for its C++
includers. The IOP host directory listing owns its results and walks
them by index; the BIOS search and the BIOS core option free theirs
when done. An unused path-append template went with the .cpp.
A tree built before this has HostFS.o made from the .cpp and a depfile
naming it, which stops the build ("No rule to make target") or, with
the depfile gone, links the C++ object. The Makefile now drops, before
reading the depfiles, the depfile and object of any C source whose
depfile still names a .cpp, so a rename from C++ to C rebuilds.
tests/hostfs builds HostFS.c as C89 and walks a real folder through
libretro-common: files and folders as the flags select, hidden entries
only when asked, a wildcard, an exact name, full and relative paths,
the recursive walk not entering a hidden folder unless asked, KEEP_ARRAY,
a broken link left out, a missing folder, and the results freed and
reused. The same checks pass against the C++ version it replaces.
The ubsan row built with -fsanitize=undefined alone, which prints a
report and carries on: a suite that reached undefined behaviour still
passed, and the row was green whatever UBSan found. With the round-mode
fix reverted, dbload's apply lane reports the read past the name table
in this configuration and still exits 0.
-fno-sanitize-recover=undefined makes the first report end the run.
Every suite passes with it.
With the automatic game fixes off, an entry that sets eeDivRoundMode
and not eeRoundMode was reported by eeRoundMode's name: the mode left
at MaxCount indexed the four-entry name table one past its end, and
the pointer read there was printed as a string. The message for
applying it printed eeRoundMode's number the same way. Both now name
the division mode.
tests/patch's dbload lane applies every entry's fixes with the
automatic fixes on and off, as a VM start does, so a log line that
indexes a table with a mode the entry left unset shows up in the
sanitizer runs; it reported this one under UBSan.
The database was a std::unordered_map<std::string, GameEntry>, each
entry holding std::strings, std::vectors of fixes, hacks, memory card
filters and dynamic patches, and its own std::unordered_map of patch
blocks, with member functions over them and std::optional through the
parser.
Now the file is read into a few pools shared by every entry (text,
game fixes, speed hacks, GS fixes, patch blocks, dynamic patch
entries); while it is read an entry names its runs by offset, and
once the pools are final the entries are made with pointers into them.
Serials are found through an open-addressed index, built as the file
is read so a serial repeated in another case is still refused before
it is parsed. Memory card filters are stored already joined with '/'.
GameDatabase_FindGame/FindPatch/ApplyGameFixes/ApplyGSHardwareFixes
replace the namespace and members; DynamicPatch is a pair of pointer
and count runs; ParseSpeedHackName takes a pointer and a length. The
GS fixes skipped under manual hacks were also collected into a string
nothing read, which is gone.
Checked by dumping every one of the 12807 entries through both the old
and the new code, every field plus what both apply functions do with
the automatic fixes on and off: byte for byte the same, and clean under
ASan, UBSan and LeakSan across an unload and reload. tests/patch's
dbload lane now also loads a small database whose every field is known,
including the eight kinds of line that must be refused, and checks
what each field reads back as, lookups in any case, the default patch
block as fallback only when there is one, and unloading.
Open and Recreate each held the index and blob file names as two
PATH_MAX_LENGTH arrays on the stack, which put their frames past the
4 KiB that tests/stackframe allows, and the suite failed on them. The
two names now share one malloc'd buffer, freed before returning; an
allocation failure is a cache that did not open, as any other.
GSFunctionMap and GSLocalMemory moved onto the C flat map
(GSFlatMap.c) and GSLocalMemory onto GSPage2Tile.c, but these three
suites build those units by hand and were not given the new sources,
so all three stopped linking. They compile them now; gssearch builds
both as C89 -pedantic with the suite's sanitizer flags, as it does
its own main.c.
The patch store was a std::vector<IniPatch> exported to Counters and
VMManager, each walking it for the places it wanted; a 'bytes'
payload was a std::vector<u8> inside each patch; the dynamic patches
were copies of the game database's vectors; the parser split lines
into std::vector<std::string_view>; the loaders took std::string and
went through std::string buffers, std::optional and FileSystem's
std::vector of results.
Now:
- The patches are a plain array in Patch.cpp, read through
LoadedPatchCount/LoadedPatch, and applied by ApplyLoadedPatches with
a mask of places, in load order as before. A payload is a malloc'd
run with its length, freed by ForgetLoadedPatches, and only
allocated once the line has passed every check.
- The dynamic patches' entries are copied into one array of their
own, a pattern run then a replacement run per patch, so a reload of
the game database cannot leave them dangling.
- The parser splits into five (pointer, length) fields on the stack,
with the stripping and empty-field rules SplitString had, and the
command table takes the same.
- LoadPatchesFromZip and LoadPatchesFromDir take C strings. The zip
member is inflated into a malloc'd buffer. The folder is walked with
retro_opendir directly, with FindFiles's filter (files, hidden ones
included, "*.pnach", sizable) and its order, and each file is read
with filestream_read_file, which terminates it. The "not found"
message no longer has a newline in the middle of the path.
tests/patch gains a lane that links Patch.cpp against the real
libretro-common directory, file and inflate code: pnach files for the
CRC in either case and with suffixes beside another CRC's, a .txt, a
hidden file, a directory named like a pnach and an empty folder; and
archives written by Python's zipfile with the member deflated, stored
under an upper-case name, only under a subfolder, and behind a
comment, plus every truncation and every one-byte corruption of one.
It fails if the CRC match turns case-sensitive.
pcsx2/StringView.h/.cpp held prefix/suffix tests, a lower-casing copy
and the pnach "key = value" split, all on std::string_view. Each had
one or two callers:
- MemoryCardFile's ".bin" checks and pcap_io's device-name prefix are
libretro-common's string_ends_with/string_starts_with, on strings
that are NUL-terminated anyway.
- VMManager::IsElfFileName compares the last four characters to
".elf" without case.
- The game database lower-cases its serial keys with an ASCII loop.
- The pnach reader splits at '=' itself, with the stripping it already
had for the comma-separated fields; that stripping now passes
isspace an unsigned char, as it must for bytes past 0x7f.
The files are gone from the Makefile, CMake and the test builds that
linked them.
The objects' depfiles (-MMD) name every header an object includes as
a prerequisite. When a header is deleted, make reads the depfile, finds
a prerequisite with no rule and stops with "No rule to make target
'pcsx2/StringView.h'" before building anything; the tree has to be
cleaned by hand. -MP adds an empty target for each header, so a
deleted one only makes its objects out of date, and they rebuild.
Depfiles written before this have no such targets, so the build tag
gains "deps-mp": a tree built before this cleans itself once, by the
mechanism a toolchain change already uses.
Each byte of a 'bytes' payload was parsed as two characters in base
16, and the parse stops at the first non-hex character, so "0g" was
taken as a byte of 0 and the line loaded, where the error branch for
a payload that is not hex was meant to refuse it. Both characters
are now checked.
tests/patch only fed the parser the game database's own lines, which
are all well formed. It now also loads eighteen pnach lines that are
not: spacing around '=' and the commas, a trailing comment, no '=',
a place out of range, signed or not a number, a payload not hex or
not whole bytes, an unknown CPU or operand size, too few fields, and
an address or value past its field. Each must load or be refused as
before, and what loads must carry the place, address, value and
payload written.
pcsx2/ParseNumber.h was a set of templates over std::from_chars,
std::optional and std::string_view. It is now C89: parse_number_u64,
_u32, _u8, _s32, _float and _bool take a pointer and a length, write
*out only on success and return nonzero when they did. The integer
rules are std::from_chars's: no whitespace, '+' or base prefix, a '-'
only for the signed form, letters as digits past 9 in either case,
stop at the first non-digit, and an out-of-range value is no value.
The bool and float forms keep the old word matching and rstrtof_len.
Callers converted: the SettingInfo default/min/max/step helpers,
the pnach line parser (place, address, value, 'bytes' payload) and
the game database's speedHacks, gsHWFixes and patch-CRC fields.
tests/parsenumber builds the header as C89 -pedantic and holds both
the C and C++ builds to std::from_chars on fixed edge cases (range
limits, signs, partial runs, bases 2-36) and 200000 generated runs,
plus the bool and float forms: 1.2M checks. It fails when the
overflow check or the sign handling is broken.
The game-database lane had stopped linking: its stubs for the three
GS lookup functions still had the std::string_view/std::optional
signatures from before GSHwHack.cpp took a pointer and length and
returned s16, so the suite failed on every run.
It also set SANFLAGS to empty before reading SANITIZER, discarding
the flags the CI matrix passes, so the ASan and UBSan jobs ran it
unsanitized. SANFLAGS is now kept and SANITIZER added to it.
With the flags honoured, the TSan build no longer linked:
libretro-common's .c files went through the C++ driver, which gives
retro_procbarrier's __tsan_acquire/__tsan_release declarations C++
linkage. They are built by the C compiler now, as in the core.
swdraw --bios reports the frames its case needs, and glnegotiate runs
both renderers that long (at least 30) rather than 30 for every case,
so what it compares is each case's finished picture: move_width_change,
whose transfer lands late, showed a black frame about one run in twelve
at the 30th. It joins the cases compared, with the CPU sprite lane
taking its case's count too.
A download texture is a plain pointer freed with gs_download_texture_free
where its owner ends - the texture cache's three kept textures and read
batch slots in its destructor, ReadbackAll's batch at each flush and on
the way out - and CreateDownloadTexture returns one on every device, the
D3D ones included. The SW rasterizer takes the processor list as a
pointer and count. The code generator's label wrapper takes const char*,
as xbyak's jumps already do; the ring heap's make_shared constructs
without forwarding and its unused make/destroy go; the page loopers take
the callback by const reference. std::array members and tables become C
arrays, numeric_limits become constants, is_same a trait of our own,
is_trivially_copyable the compiler builtin, the unused std::hash
specialisation goes, the std math and string calls are the C ones of the
same precision, and GS includes the C headers rather than their <c...>
forms.
The hash cache's purge list is sorted with qsort, swaps are written out,
the register handler and GL state tables are filled by loop and memset,
and the <algorithm>, <iterator> and <utility> includes nothing used any
more leave eleven files. The Apple Vulkan library lookup builds its two
candidate paths in a malloc'd buffer rather than a std::string.
The paraLLEl-GS renderer, the HW texture cache, the SW renderer's
rasterizer and texture cache, the HW renderer's CPU-sprite textures and
rasterizer, the GL context and stream buffers, the Vulkan null texture
and the factories behind them (GLContext, GLStreamBuffer, GSTextureVK,
the download textures, GSRasterizerList) are plain pointers made with
new and deleted where their owner ends. A draw's temporary depth
stencil and texture copy go back to the pool at the draw's two exits
in place of a recycling deleter; GSreopen's freeze data is malloc'd
and freed on every path; the ring heap's constructors run without an
exception guard, there being no exceptions. GSDownloadTexturePtr stays,
the D3D devices returning it too.
A texture's alpha range is a GSAlphaMinMax of two bytes rather than a
std::pair, the texture cache's hardware-clear verdict a three-way enum
rather than an optional<bool> (unknown stays distinct from no, as the
optional's empty state was), and the render window comes back by value.
The software rasterizer's job queue takes its worker's start and draw
callbacks as function pointers with a context, the rasterizer list
keeping a small Worker record per thread, in place of three
std::functions and their captures; the Vulkan device's preinitialised
buffer takes its fill as a function pointer. The game database's hack
lookups take the name as pointer and length, and GS.h stops including
<optional>, <string> and <string_view>.
LibretroAdmin