* docs(cheats): community cheat store design and plan (#1071) Refs #1071 Co-Authored-By: Claude Opus 5.5 <[email protected]> * docs(cheats): map printf_notify to app toasts Refs #1071 Co-Authored-By: Claude Opus 5.5 <[email protected]> * feat(cheat-import): BigPEmu API map; int_full settings omitted; printf_notify ignored Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(cheat-import): refuse side effects in message args; scope Unsupported.call to API gaps Co-Authored-By: Claude Opus 5.5 <[email protected]> * feat(cheat-import): committed cartridge identity table (hashes only) scan_cartridges.py walks a ROM directory (read-only) and writes a sorted FNV1a64 -> CRC32 table with size, title and name; no ROM bytes or paths. load_cartridges rejects duplicate fnv/crc, bad hex and format != 1. The generated cartridges.json (152 rows) lets CI map community script hashes to the CRC32 the app binds packs to without any ROMs. Co-Authored-By: Claude Opus 5.5 <[email protected]> * feat(cheat-import): community cheat pack generator (#1071) build_packs.py converts every pinned community C cheat source with convert.py and writes the site/cheats tree: per-cartridge .cht packs (one per CRC32, sources merged, identical cheats deduped, same-name cheats suffixed), manifest.json, the unsupported.json worklist with per-call API usage, index.html and README. Deterministic, bounded fetch of one codeload tarball per source, --source-dir for local checkouts, --summary and --bump for the bump workflow. sources.json pins jguffer/BigPEmu_Scripts at 2421150. Co-Authored-By: Claude Opus 5.5 <[email protected]> * data(cheats): generate community packs from jguffer/BigPEmu_Scripts@2421150 23 cartridge packs (72 cheats, 9 partial) plus the unsupported.json worklist (55 entries), manifest and index, generated with build_packs.py from the pinned codeload tarball; byte-identical to a local checkout run. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(cheat-import): worklist unmatched literals beside a hit; bound fetch errors and member count - identity row for table-missing literals in a file that also has a hit - http.client.HTTPException during fetch is a FetchError (exit 2) - tar expansion cap counts a header block per member and caps member count - Collector uses the api_map given to build() Co-Authored-By: Claude Opus 5.5 <[email protected]> * data(cheats): regenerate for generator fix (unmatched-literal identity rows) Co-Authored-By: Claude Opus 5.5 <[email protected]> * feat(site): publish community cheat packs under /cheats/ build_site.py copies the committed site/cheats tree verbatim; check_site.py gains check_cheats (manifest format/sha256/bytes/CRC guard, unlisted .cht, unsupported.json, index.html links, 256 KiB manifest and 1 MiB tree limits) with unit tests. pages.yml watches site/cheats and tools/cheat-import. The converter README now documents the author's permission, the credit, the sources -> cartridges -> packs workflow and how to add a source; the community_test harness and its fixtures are removed (the committed packs and the regeneration diff are the conversion contract). Co-Authored-By: Claude Opus 5.5 <[email protected]> * ci(cheats): regen-diff job, concurrency, weekly pin-bump PR cheat-import.yml: add site/cheats/** to the path filters, a per-ref concurrency group, drop the community_test.py fetch step (the files are gone), and add a packs job that regenerates twice, diffs the runs and the committed tree, flags unmapped BigPEmu calls, then runs the site-check tests and build_site + check_site. cheat-packs-bump.yml: weekly (Mon 06:23 UTC) poll of each pinned github source; on movement, bump the pin, regenerate, validate, force-push the fixed branch cheats/pin-bump and open or refresh a develop PR (validation failures head the body instead of being dropped), then dispatch cheat-import.yml on the branch because GITHUB_TOKEN PRs do not fire pull_request workflows. test_committed_sources no longer pins the commit/date value: every bump PR would otherwise fail the unit tests; only the shape is asserted. Refs #1071 Co-Authored-By: Claude Opus 5.5 <[email protected]> * ci(cheats): gate bump workflow to the upstream repo, drop bot trailer Only libretro/virtualjaguar-libretro opens bump PRs (the Provenance-Emu fork runs crons too), with a 30 minute job timeout. The bot-authored bump commit no longer carries a Claude co-author trailer. Refs #1071 Co-Authored-By: Claude Opus 5.5 <[email protected]> * ci(cheats): pin pycparser, force LF, add job timeouts Pin pycparser==3.11 (newest release that installs on the Python 3.10 CI leg; 2.22, 2.23, 3.0, 3.11 and 3.12 all regenerate byte-identical packs). Add a root .gitattributes forcing LF for the generator and generated tree so pack bytes cannot drift by checkout. Give both jobs a 20 minute timeout. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(cheats): dedupe pack header title, refuse control-character paths The pack's first line printed the title twice when the cartridge name began with the title. Print the name alone in that case. Fetch and local-dir reads now fail closed on any path containing a control character, since names can reach ::warning:: annotations and PR bodies. Co-Authored-By: Claude Opus 5.5 <[email protected]> * data(cheats): regenerate packs with deduplicated header title Line 1 of each pack no longer repeats the title when the cartridge name already starts with it. Regenerated from the pinned source commit; two runs and a --source-dir run are byte-identical. Co-Authored-By: Claude Opus 5.5 <[email protected]> * ci(cheats): even indentation on a bump-workflow continuation line editorconfig-checker wants multiples of 2 in workflow files. Refs #1071 Co-Authored-By: Claude Opus 5.5 <[email protected]> --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
36 KiB
Community cheat store: design (#1071)
Follow-up to #1050 / #1052. CI converts pinned community C cheat sources with the existing
tools/cheat-import/convert.py, commits the packs under site/cheats/, the project site serves
them, and the desktop app installs them per cartridge from a "Community cheats" section.
Permission: jguffer (author of jguffer/BigPEmu_Scripts and #1050) gave the maintainer permission
to convert and redistribute his scripts. Every pack and the app credit him.
scripts/cheat_aliens_vs_predator.c is excluded: it credits values to a third-party script
bundled with BigPEmu.
Prototype (61 sources at 24211503, local corpus): 17 cartridge packs convert today. One
converter fix (accept bigpemu_register_setting_int_full as a non-boolean, therefore omitted,
setting; writes that depend on it are still refused by the existing dependency tracking) lifts it
to 22. The rest go to the worklist: pointer/array declarations (6), CD titles, RAM reads,
printf_notify, >10 settings, unknown cartridge hashes (no corpus cart).
BigPEmu API map (user request): tools/cheat-import/bigpemu-api.json lists every BigPEmu call the
sources use, what it does (from the community sources, not an SDK), our equivalent, and its status
(supported, setting-omitted, ignored, unsupported; anything else is unmapped). The
generator records per-source API use in unsupported.json (calls, top-level api_usage), packs
carry a note for ignored calls, and CI warns on unmapped calls, so the worklist grows from data.
Mapping printf_notify as ignored (message only) adds Iron Soldier (3/3): 23 packs.
Two PRs into develop: (1) data + CI + site (Refs #1071), (2) app (Closes #1071).
Data and CI
Scope: everything except the app UI. Converter = tools/cheat-import/convert.py, unchanged in
logic (one factoring: split cht_text into cht_header + cht_entries, see §5; plus the
coordinator's register_setting_int_full fix, which only widens the unsupported-setting path).
Committed outputs live under site/cheats/; the Provenance-fork "Publish site" cron copies them
verbatim within the hour, exactly like site/thumbnails/.
1. tools/cheat-import/sources.json (input, hand-edited / bumped by CI)
{
"format": 1,
"sources": [
{
"id": "jguffer",
"kind": "github",
"repo": "jguffer/BigPEmu_Scripts",
"branch": "main",
"commit": "24211503339a969f8494b8e189b8f2aa0cb9eba6",
"commit_date": "2026-04-15",
"include": ["scripts/cheat_*.c"],
"exclude": [
{"path": "scripts/cheat_aliens_vs_predator.c",
"reason": "credits values to a third-party script bundled with BigPEmu (\"Taken from Whitehouse avp_common.c script\"); permission covers only the author's own work"}
],
"notes": {
"scripts/cheat_baldies.c": "disc title; v1 packs cover cartridges only (converter refuses discs, app has no disc CRC)"
},
"author": "jguffer",
"credit": "Cheats by jguffer (https://github.com/jguffer/BigPEmu_Scripts), converted by the Virtual Jaguar project with the author's permission.",
"permission": "Author granted conversion and redistribution to the maintainer (see libretro/virtualjaguar-libretro#1071, #1050). Repository has no license file.",
"license": null
}
]
}
kind:"github"now;"local"reserved for an in-repo directory of manual ports ({"id":"vj","kind":"local","path":"tools/cheat-import/ports","include":["*.c","*.cht"]}; a.chtthere is copied through verbatim after guard validation). Schema only; not created now.branchis what the bump workflow polls;commitis the pin;commit_date(upstream committer date, ISO) is written by the bump and feeds the deterministicupdatedfields (§6).include/excludeare fnmatch globs relative to the repo root. Onlyscripts/cheat_*.cis ever fetched; the generator refuses any include pattern not ending in.c.notes[path]is a free-text line appended to that file's worklist entry whatever its outcome.- Pinning: commit only. The generator fetches ONE tarball per source
(
https://codeload.github.com/<repo>/tar.gz/<commit>, limit 32 MiB, extracted in memory, only members matchingincludekept, each capped at 1 MiB). A commit is content-addressed; per-file sha256 pins are redundant as input and go to the OUTPUT instead (manifest +.chtheader) as provenance.community-cases.jsonstyle per-file hashes are therefore dropped (§7).
2. tools/cheat-import/cartridges.json (committed identity table, hashes only)
{"format": 1, "cartridges": [
{"fnv1a64": "75F2216BD4F54AC2", "crc32": "5E2CDBC0", "size": 4194304,
"title": "Doom", "name": "Doom - Evil Unleashed (1994)"}
]}
Sorted by fnv1a64 (upper-case hex, 16 chars); crc32 upper-case 8 chars; size = native payload
bytes; title = display title the manifest shows; name = No-Intro-style full name (region/rev)
for disambiguation when one title has several revisions. Uniqueness: fnv1a64 unique; crc32
unique (a collision fails the generator; it would break the app's binding anyway).
Built by tools/cheat-import/scan_cartridges.py --rom-dir DIR [--merge cartridges.json] --output NEW.json:
walks *.j64/*.jag/*.rom/*.bin, reuses convert.rom_bytes (same copier-header refusal) and
convert.fnv1a64 + zlib.crc32, pre-fills name from the file stem and title from the stem up to
the first (. --merge keeps existing curated title/name for known hashes and only appends.
Output is a new file (never overwrites). The maintainer edits titles and opens a PR; review = the
diff. Never ROM bytes, never file paths, in the committed file.
3. Output layout under site/cheats/ (committed)
site/cheats/manifest.json packs index (schema §4)
site/cheats/unsupported.json worklist
site/cheats/packs/<CRC32>.cht one pack per cartridge CRC32 (upper-case hex)
site/cheats/index.html static human page, same pattern as site/thumbnails/index.html
site/cheats/README.md provenance + credit + how it is generated (copied verbatim)
build_site.py adds, next to the thumbnails block: copytree(SITE_SRC/"cheats", out/"cheats").
index.html is NOT in ALL_PAGES (it is noindex, hand-written, like the thumbnails one) and is
regenerated by build_packs.py so its game list cannot go stale: title, cart name, CRC, cheat
count, partial flag, link to the pack, credit line, and a link to unsupported.json.
One pack per CRC32, merging every source that names that cartridge. Rules:
- Order: sources in
sources.jsonorder, files in sorted path order, settings in registration order. Stable across runs. - Dedupe: identical
(casefold(name), code)pairs collapse to one entry (kasumi_ninja vs kasumi_ninja_infinite_health). The manifest lists both files as contributors. - Same name, different code (mc3d_classic vs mc3d_virtual "Infinite X"): the later entry gets the
suffix
(<module.title>)if the two modules' category titles differ, else(<file stem minus cheat_>). Mandatory becauseparseChtmerges CONSECUTIVE identically named entries into one toggle; the suffix(Part N)is reserved and already refused by the converter. - Cross-source interaction: impossible by construction (each BigPEmu module reads only its own
settings, so no
(A && B)guard can span files); the converter's combination check stays per-source. What CAN happen across sources is address overlap with different values. The generator computes each cheat's byte image; overlapping pairs are shipped (the app applies in a fixed order, same as any libretro-database set) and recorded in the manifest asoverlaps: [[i, j], ...]plus a# note:comment in the.cht. Not refused. - A source whose literals hit two table rows (two revisions) is converted twice, once per digest, and lands in two packs.
- Partial sources (
--allow-partialsemantics): only complete toggles are exported; every omission is anotes[]row on the pack AND a row inunsupported.json. - A source with no table hit, a module-level
Unsupported/ParseError, or zero converted toggles produces no pack entry and one worklist row.
unsupported.json:
{"format": 1, "entries": [
{"source": "jguffer", "path": "scripts/cheat_baldies.c", "sha256": "...",
"stage": "module", // "identity" | "module" | "setting"
"crc32": null, "setting": null,
"reason": "unsupported initialisation call bigpemu_register_setting_int_full",
"note": "disc title; v1 packs cover cartridges only ..."}
]}
stage=identity reasons: "no known cartridge for literals [..]" / "no 64-bit literal found".
stage=setting rows carry crc32 and setting. Sorted by (source, path, setting).
4. site/cheats/manifest.json (exact schema; flat for app/src/neuro/neuro_json.h, which is the app's only JSON reader: doubles, strings, arrays, objects, sorted keys)
{
"format": 1,
"generated_from": {
"converter_sha256": "<sha256 of tools/cheat-import/convert.py>",
"generator_sha256": "<sha256 of tools/cheat-import/build_packs.py>",
"cartridges_sha256": "<sha256 of cartridges.json>",
"sources": [{"id": "jguffer", "repo": "jguffer/BigPEmu_Scripts",
"commit": "2421150...", "commit_date": "2026-04-15",
"url": "https://github.com/jguffer/BigPEmu_Scripts/tree/2421150...",
"author": "jguffer", "credit": "...", "permission": "..."}]
},
"packs": [
{
"crc32": "5E2CDBC0", "fnv1a64": "75F2216BD4F54AC2",
"title": "Doom", "name": "Doom - Evil Unleashed (1994)",
"file": "packs/5E2CDBC0.cht", "bytes": 1234,
"sha256": "<sha256 of the .cht bytes>",
"updated": "2026-04-15",
"credit": "Cheats by jguffer (...), converted ... with the author's permission.",
"cheats": 8, "writes": 21, "partial": false, "exceeds_core_limit": false,
"sources": [{"source": "jguffer", "path": "scripts/cheat_doom.c",
"sha256": "197cc6...", "converted": 8, "omitted": 0}],
"notes": [{"source": "jguffer", "path": "scripts/cheat_atari_karts.c",
"setting": "Unlock All Challenges, Cups, Racers",
"reason": "unsupported call bigpemu_jag_read8 (...)"}],
"overlaps": []
}
],
"unsupported": "unsupported.json",
"counts": {"packs": 20, "cheats": 60, "partial_packs": 4, "unsupported_entries": 41}
}
- Pack version =
sha256. No separate version field: it changes iff the bytes change, and the app can recompute it from a stored file offline (update check = compare manifestsha256with the local file's hash; integrity check after download = same value). Hex strings everywhere; counts/bytes as integers (exact in double). updated= maxcommit_dateover the pack's contributing sources: deterministic, human-meaningful, no build clock.creditis pre-rendered per pack (joined with;when several sources contribute) so the app shows one string without templating.packssorted bycrc32;sources/notesinside a pack in the generation order of §3.- URL: the app resolves
fileagainsthttps://jaguar.provenance-emu.com/cheats/; the manifest carries no absolute URL so a mirror or local build serves the same bytes.
5. .cht content
Factor convert.cht_text(title, crc, entries, source_name, source_sha) into
cht_header(crc, comment_lines) + cht_entries(entries); cht_text keeps its signature and
output byte-for-byte (existing tests stay green). build_packs.py writes:
# Community cheats for Doom (Doom - Evil Unleashed (1994)); nothing is enabled automatically.
# Credit: Cheats by jguffer (https://github.com/jguffer/BigPEmu_Scripts), converted by the Virtual Jaguar project with the author's permission.
# Source: jguffer/BigPEmu_Scripts@24211503339a969f8494b8e189b8f2aa0cb9eba6 scripts/cheat_doom.c
# Source SHA256: 197cc6ff...
# Omitted: "Unlock All Challenges, Cups, Racers" (scripts/cheat_atari_karts.c): unsupported call bigpemu_jag_read8 (...)
# Overlap: cheat 2 and cheat 5 write the same bytes; later wins (app order).
# Cartridge revision guard is enforced by the Virtual Jaguar app.
vj_rom_crc32 = "5E2CDBC0"
cheats = 8
cheat0_desc = "Infinite Ammo"
cheat0_code = "..."
cheat0_enable = false
Parser facts (app/src/cheats.cpp): parseCht skips blank and # lines and any key not matching
cheat<N>_; loadChtFile scans only vj_rom_crc32 (duplicate/malformed = whole set blocked).
So comments and unknown top-level keys are safe. No new key is needed: the app derives the
version by hashing the file; credits/notes for the UI come from the manifest, not from parsing
comments. quoted() already refuses quotes/backslashes/control chars in names; the generator
additionally strips CR/LF from any text it puts in a comment line.
6. Determinism and the regen check
json.dumps(obj, indent=2, sort_keys=True, ensure_ascii=True) + "\n"; LF only; files opened withnewline="\n"; no timestamps, hostnames, paths or Python version anywhere in outputs.- All orderings fixed (§3). Tarball extraction order ignored (members sorted by path).
index.htmlis generated from the manifest only.- Regen check (CI, §8):
build_packs.py --out $RUNNER_TEMP/cheatsthendiff -r $RUNNER_TEMP/cheats site/cheats; any diff fails with the diff in the log. Second invariant in the same job: run the generator twice to separate temp dirs anddiff -rthem (catches nondeterminism independently of what is committed). - Converter invariant run on every real source during generation:
module.convert(digest ^ 1)must yield nocode(wrong-hash activation = hard failure), replacing the same check incommunity_test.py.
7. Generator CLI and tests
tools/cheat-import/build_packs.py (Python 3.10+, pycparser; stdlib tarfile/urllib):
build_packs.py --sources sources.json --cartridges cartridges.json --out DIR
[--source-dir ID=PATH ...] # local checkout instead of tarball fetch (tests, offline)
[--allow-network/--no-network] # default: network only when no --source-dir covers a source
[--summary OLD_MANIFEST NEW_MANIFEST] # prints Markdown added/changed/removed per game (bump PR body)
[--max-pack-bytes 65536] [--max-total-bytes 1048576]
Exit 0 = wrote manifest.json, unsupported.json, packs/, index.html, README.md into DIR
(refuses a non-empty DIR without its own .vj-cheat-packs marker, like build_site.py); exit 1 =
validation failure (collision, budget, wrong-hash activation, bad schema); exit 2 = fetch failure.
Zero converted packs overall is exit 1 (an empty store is never published silently).
Identity scan per file: re.findall(r"0x[0-9A-Fa-f]{9,16}", text) on the uncommented source,
intersect with cartridges.json keys (case-insensitive). Literals that are not in the table are
listed in the worklist reason so a corpus gap is visible.
Tests test_build_packs.py (synthetic only, same style as test_convert.py; reuse its source()
builder, ROM, DIGEST): two-sources-one-CRC merge with exact-duplicate dedupe; same-name
different-code suffixing; two-revision source yields two packs; no-table-hit, module-error and
setting-error worklist rows with the right stage; notes appended; excluded file listed with
reason and never parsed; include not ending in .c refused; overlap recorded; exceeds_core_limit
set at 257 writes; wrong-hash activation fails; determinism (two runs byte-equal); pack/total byte
budgets; --summary output for an added, a changed and a removed cheat; scan_cartridges.py
on synthetic ROM bytes (+ copier-header refusal, --merge keeps curated titles).
community_test.py, test_community.py, community-cases.json: deleted. The committed packs
plus the regen-diff job ARE the conversion contract (a changed value shows as a diff in
site/cheats/packs/*.cht in the PR that changed the converter), the fetch is now the tarball
step, and the wrong-hash check moved into the generator. README.md loses its "does not
redistribute" paragraph and the community_test section; it gains: permission statement (verbatim
from #1071), the credit requirement, the sources/cartridges/pack workflow, and "how to add a
source" (PR with author permission note).
8. Workflows (CI budget: nothing runs on unrelated PR pushes)
cheat-import.yml (extend). Triggers: push/pull_request with paths
tools/cheat-import/**, site/cheats/**, the workflow file; workflow_dispatch. Add
concurrency: {group: cheat-import-${{ github.ref }}, cancel-in-progress: true}. Jobs:
converter(existing matrix 3.10/3.13):unittest discover(now includes generator tests); drop thecommunity_test.py --fetchstep.packs(3.13 only,needs: converter):build_packs.py --out $RUNNER_TEMP/a, again to/b,diff -r a b,diff -r a site/cheats, thenpython3 scripts/build_site.py --releases scripts/testdata/releases.json --out $RUNNER_TEMP/site && python3 scripts/check_site.py $RUNNER_TEMP/siteso a manifest/.chtthe site check rejects never merges.
cheat-packs-bump.yml (new). schedule: cron '23 6 * * 1' (Monday) + workflow_dispatch;
permissions: {contents: write, pull-requests: write}; concurrency: {group: cheat-packs-bump, cancel-in-progress: false}; GH_TOKEN: ${{ github.token }} (no PAT exists; GITHUB_TOKEN is the
choice). Steps:
- For each
kind=githubsource:gh api repos/<repo>/commits/<branch> --jq .sha,.commit.committer.date. No pin moved ->exit 0(log only). - Write new
commit/commit_dateintosources.json(via a--bump ID=SHA:DATEmode ofbuild_packs.pyso the JSON formatting stays canonical), regenerate tosite/cheats/, run the unit tests and the same regen/site checks as jobpacks. Any failure: open/refresh the PR anyway, body headed "VALIDATION FAILED", with the log tail; never push a red pin silently. build_packs.py --summary <old manifest from git show HEAD:site/cheats/manifest.json> <new>-> Markdown: per game added/changed/removed cheats, new/removed packs, worklist delta, byte totals.- Branch
cheats/pin-bump(fixed name; force-push), commitdata(cheats): bump jguffer/BigPEmu_Scripts to <sha7>(+ attribution lines),gh pr create --base developif no open PR for that head, elsegh pr editbody. Body = step-3 summary, step-2 results, upstream compare link,Refs #1071, and the note below. gh workflow run cheat-import.yml --ref cheats/pin-bump.workflow_dispatchandrepository_dispatchare the documented exceptions to "GITHUB_TOKEN events create no runs", and the run attaches to the branch head SHA so it shows in the PR's Checks tab. The PR body states: "Opened by Actions with GITHUB_TOKEN:pull_requestworkflows did not fire. The converter workflow was dispatched on this branch; to run the full PR suite, close and reopen the PR or push an empty commit." Repo settings required: "Allow GitHub Actions to create and approve pull requests" (confirmed on). Branch protection ondevelopis unchanged; a maintainer merges.
pages.yml: add site/cheats/** and tools/cheat-import/** to paths. (Dead publisher here,
but the filter documents intent; the Provenance-fork cron publishes regardless.)
check_site.py: new check_cheats(out): cheats/manifest.json parses, format == 1, every
packs[].file exists, its sha256/bytes match, its vj_rom_crc32 equals crc32, no .cht in
packs/ is unlisted, unsupported.json parses, index.html exists and every local link in it
resolves, sizes within §9. Skipped only when cheats/ is absent (older checkouts).
site/pages/privacy.html: one row "Community cheats (jaguar.provenance-emu.com/cheats/)":
manifest + pack download, no identifiers sent, cached locally.
9. Size and limits
- Pack: hard max 64 KiB (
--max-pack-bytes); realistic worst case is 256 writes * 14 chars ~ 4 KiB per cheat, 10 settings -> ~40 KiB. Manifest hard max 256 KiB.site/cheats/total hard max 1 MiB (--max-total-bytes); today's expectation: ~20 packs, < 40 KiB total, manifest ~15 KiB. CheatManager::kCoreLimit = 256applies to the sum of ENABLED segments. A pack is written as-is;writes(sum of all segments) andexceeds_core_limit(writes > 256) are in the manifest so the app can warn "enabling every cheat in this pack exceeds the core limit" without parsing. The converter already refuses a single cheat over 256 writes.- Fetch: tarball 32 MiB cap, 60 s timeout, each kept member 1 MiB cap (converter's own limit).
10. Risks
- Upstream has no license file; redistribution rests on the author's message.
permissiontext and per-pack credit are the mitigation; a withdrawal means deletingsite/cheats/packs/*in one PR. - Values are BigPEmu-authored; nothing here proves gameplay correctness. Packs ship off by default and partial; the manifest says which settings were omitted. Worklist entries are converter work, not app work.
- The 64-bit-literal scan is heuristic: a non-hash literal that collides with a table row would bind a script to the wrong cartridge. Collision with a real FNV1a64 is negligible, but the generator lists all literals in the worklist so review sees them.
cartridges.jsononly grows by maintainer PR from the private corpus; six scripts currently have no table hit and stay on the worklist until someone with those carts scans them.- Cross-source address overlap is shipped, not refused (§3); the app's fixed order makes it deterministic but a user enabling both gets last-writer semantics.
- The app matches
.chtsets by name stem, not CRC; the app side must store downloaded packs where its "Community" section binds them byvj_rom_crc32, or stem matching will not find<CRC>.cht. - Bump PRs via GITHUB_TOKEN never get the full
pull_requestsuite until a human close/reopen; the dispatched converter run and the in-body validation are the gate until then. community_test.pyremoval drops the Python-3.10 contract run on real sources; thepacksjob runs on 3.13 only. The converter's unit tests still run on both.
App
Scope: the "Community cheats" section of the Cheats window and everything behind it. The data side
(design-data.md) is fixed: manifest.json (§4) at https://jaguar.provenance-emu.com/cheats/, packs at
cheats/packs/<CRC32>.cht, pack version == sha256 of the bytes, vj_rom_crc32 guard inside.
Facts the design leans on (verified in the tree):
CheatManager::rescan()lists<dir>/*.chtand<dir>/Atari - Jaguar/*.chtonly, non-recursive, so a sub-folder of<data>/cheats/is invisible to the stem-matched library path.loadChtFilealready enforces thevj_rom_crc32guard;parseChtignores#lines and non-cheatN_keys.- The core (
src/core/cheat.c) appends segments in the orderretro_cheat_setis called and stops at 256; the index is only a tag. So the batch order inapply()IS the write order and the 256 cap is "first 256 segments in that order". - The app has ONE downloads switch:
library.art(LibraryService::downloadArt); RetroAchievements images already treat it as the global offline switch (Achievements::updateImageNetwork). - Background HTTP pattern:
Worker(library_service.h) +http::geton the worker + results handed over under a mutex +pump()once per UI frame on the main thread (LibraryService::updateDatabase,ra::ImageCache).ra::ImageCacheinjectsfetchas astd::functionfor tests. The app has no SHA-256 anywhere. neuro::Json(parse never throws, depth 32, doubles/strings/arrays/objects) reads the manifest as-is.
1. Storage (<data>/cheats/community/)
<data>/cheats/community/manifest.json last manifest fetched, verbatim bytes (cap 256 KiB)
<data>/cheats/community/packs/<CRC32>.cht installed pack, bytes exactly as served (cap 64 KiB)
<data>/cheats/community/packs/<CRC32>.json manifest `packs[i]` object for that pack, Json::dump() at install
<CRC32> is upper-case 8 hex (%08X of GameIdentity.crc), same as the manifest/URL. A pack is "installed"
iff <CRC32>.cht exists; the .json sidecar is what the UI shows offline (title, credit, cheats, writes,
partial, exceeds_core_limit, notes[], updated, sha256). If the sidecar is missing (hand-copied file) the section
says "credits unavailable: use Update" and still works. Last manifest check time: cheats.community.checked
(unix seconds) in config.ini. No separate index file: the directory is the index.
Install is atomic and verified in memory before anything is written: http::get(url, resp, 64 KiB) ->
resp.status == 200 -> resp.body.size() == pack.bytes -> sha256(body) == pack.sha256 ->
loadChtText(body, set) parses with >= 1 cheat and set.revisionBound && set.restriction.empty() && set.romCrc == crc -> writeFileAtomic(<CRC>.cht) then writeFileAtomic(<CRC>.json) (temp + rename, per-process
temp name, already in paths.cpp). Any failure leaves the previous install untouched. loadChtText is
loadChtFile minus the readFile (tiny refactor; loadChtFile calls it).
2. CheatManager changes
enum class Origin { Database, Community }and a second set:ChtSet community_; bool communityLoaded_;withsetCommunityDir(dir)(<data>/cheats/community/packs/).gameLoaded()callsloadCommunity()afterpickSet(): path<dir>/<%08X of crc>.cht, only whencrc != 0;loadChtFile+ the same guard check asloadSelectedSet(a mismatched or unbound file is loaded with every cheat marked unsupported andrestrictionset, never silently applied).reloadCommunity()is public for the store to call after an install/update/remove of the current game's pack. Accessorconst ChtSet* communitySet() const. Independent ofselected_,matches_,library_: the database set stays exactly as today.- Toggles:
isEnabled(c, Origin),setEnabled(c, on, Origin),setEnabledByName(name, on)searches database, then community, then user (CLI--cheat-ongains packs for free). Ini keys: databaseon.<id>, communitycp.<id>;enabled_holds the prefixed key, so ids can never collide.idis unchanged (FNV-1a of desc + "\n" + code): across a pack update an unchanged cheat keeps its toggle, a cheat whose text OR code changed gets a new id and comes back OFF (stated in the UI once per update: "N cheats changed and were turned off").loadCommunity()prunescp.*keys that match no cheat in the loaded pack (stale after an update), so the ini never grows. apply()order: database set (index i), community pack (indexkCommunityIndexBase = 5000 + i), user codes (10000 + i). Rationale: the core writes in batch order andapply()is the only writer, so "later wins" is deterministic; the CRC-bound pack is more specific than a stem-matched database file, so it overrides it on an overlapping address; user codes are explicit intent and win over everything.appliedSegments_sums all three and the existing >kCoreLimitwarning stays the single gate; the manifest'swrites/exceeds_core_limitlet the UI warn BEFORE the user ticks everything.enabledCount(),disableAll()cover the pack too.- Uninstall (
Remove): deletes<CRC>.cht+.json, erases everycp.*key of the game's ini, re-applies. Reinstalling starts with everything off.gameClosed()clearscommunity_likeset_.
3. Update and merge rules
- User codes (
UserCheat,user.N.*) are never read or written by install/update/remove. A pack cheat the user copied into Manual codes is an ordinary user code: unaffected, applied last, duplicates the write (harmless). - Update = install over the existing file (same verification); toggles follow §2 by id. There is no merge of cheat lists: the pack file is always exactly the served bytes.
- Manifest no longer lists an installed CRC: the file stays, the section shows "No longer offered by the store
(installed copy kept)", toggles keep working,
Removeis offered,Updateis not. - Manifest sha256 differs from the installed
.json's sha256 (not a rehash of the file: the sidecar is the record of what was verified at install): stateUpdateAvailable. If the.chton disk hashes differently from its sidecar (user edited it), the section says "edited locally" andUpdatereplaces it.
4. Fetch flow
CheatStore (app/src/cheat_store.{h,cpp}), no SDL/ImGui/GL, two injected hooks so the whole thing runs
synchronously in tests: fetch(url, maxBytes, body, err) -> bool (default http::get, status != 200 is an
error) and run(job) (default app.cheatNet.post on a Worker owned by App, created in cheatsInit;
tests run the job inline). Results cross back under a mutex and pump() (called from drawCheats, main
thread) applies them: writes the cache files, updates PackState, asks CheatManager::reloadCommunity().
stop() sets the cancel flag and joins (app shutdown).
- Manifest fetch: when the Cheats window becomes visible with a cartridge loaded AND downloads are on AND
http::available()ANDnow - cheats.community.checked >= 6 h; and on the "Check for updates" button (no interval). Never at app start and never from the library/scan code. Install/Update fetch the pack using the cached manifest's sha256; a server-side change between the two shows as "pack changed on the server: check for updates again", never as a silent install. - Limits: manifest 256 KiB, pack 64 KiB (
maxBytes, data §9); http.cpp's own 10 s connect / 20 s stall / 120 s total;format != 1or a pack entry missingcrc32|file|sha256|bytesis "store manifest not understood (format N)", entries with a bad CRC/sha hex are skipped individually.filemust be a relative path with no..; resolved againstkCheatStoreBase(https://jaguar.provenance-emu.com/cheats/, config overridecheats.community.urlfor the loopback test server, likelibrary.art_mirror). - Errors are a status line inside the section (
CheatStore::status(), same idea asLibraryService::status()), never a toast: "Could not reach the cheat store: . Installed packs still work." A failed manifest fetch does not touch the cached manifest or the check time (so the next window open retries). - Downloads off (
library.artfalse): no fetch; the section shows the installed pack if any, otherwise "Downloads are off (Library: Download art and cheat packs). Turn it on to see community packs." The Library checkbox label becomes "Download art and cheat packs"; the key stayslibrary.art. - Hardcore: install/update/remove are data-only and stay enabled; toggles sit inside the existing
BeginDisabled(hardcore)andblocked()keepsapply()empty. The section shows one line: "Installed; cheats cannot be enabled in hardcore mode."
5. UI (cheats_ui.cpp, drawCommunity)
Position: after "Database cheats", before "Manual codes", header "Community cheats". drawSetCheats becomes
drawCheatRows(cm, set, Origin) and both sections use it (checkbox + wrapped label + tooltip = code/note).
| State | Shown |
|---|---|
| no game | section absent (existing "Load a cartridge or disc" text) |
| disc (crc == 0) | "Community packs cover cartridges only." |
| downloads off, nothing installed | the note from §4 |
| no manifest yet / fetching | "Checking the community store..." (spinner text), button disabled |
| not offered, nothing installed | "No community pack for this cartridge (CRC 5E2CDBC0)." + Check for updates |
| offered, not installed | title, "N cheats (M writes) by , updated ", partial note, Install |
| installed, current | credit line, cheat rows, partial notes, Remove, Check for updates |
| update available | same + warning "Update available (pack changed )", Update first, Remove |
| no longer offered | rows + "No longer offered by the store; installed copy kept", Remove |
| offline / error | rows (if installed) + the status line |
Always under the header when a pack is known: credit as TextDisabled wrapped (the manifest's pre-rendered
string); notes[] as a collapsed TreeNode("N settings could not be converted") listing setting: reason;
exceeds_core_limit as a warning "Enabling every cheat here exceeds the core's 256-write limit; the first 256
writes win" (the existing red line still reports the actual count). Hardcore line per §4. Buttons go through
ui::Flow. Library-wide affordance: v1 is the loaded game only. The manifest is small, so nothing stops a
later "games in your library with a pack" list on the Library tab, but it needs tile plumbing and a second
CheatStore consumer; the issue's wording "(or the loaded game)" allows deferring it, and the site's
cheats/index.html already lists every pack.
6. Tests (app/tests, no network)
vj_cheats_test gains src/cheat_store.cpp + src/sha256.cpp + src/neuro/neuro_json.cpp; fixtures
fixtures/community/manifest.json and fixtures/community/5E2CDBC0.cht (guard = the Test Game CRC used in
testRevisionGuard). Fetcher is a std::map<url, body> lambda; run executes inline.
testSha256: FIPS vectors ("", "abc", the 448-bit block) and a > 64 KiB buffer.testManifestParse: good;format 2; a pack with a bad sha hex is skipped and the rest kept; truncated JSON; oversize body (fetcher reports "response too large") -> error status, cache untouched.testInstallVerify: sha mismatch, byte-count mismatch,vj_rom_crc32for another CRC, missing guard, a body with no cheats, HTTP 404: each leaves no.cht/.jsonand sets the matching status; success writes both andcommunitySet()has the cheats, all off.testCommunityToggles: enable two, "restart" (new manager) -> still on; install an updated fixture where one cheat's code changed -> that one off, the other on, stalecp.key gone;Remove-> ini has nocp.keys.testApplyOrder: db set + pack + user code all on -> batch isreset, db entries,set 5000+i,set 10000+i;appliedSegments()is the sum; the warn fires above 256.testStates: offered/not installed, installed/current, update available (manifest sha changed), dropped from manifest, disc identity (crc 0) never loads or fetches,library.artoff never calls the fetcher, a hardcoreblockedreturning true still installs butapply()sends onlyreset.testNoManifestRetryGate: fetch within 6 h is skipped; "Check for updates" forces it.
7. Files
| File | Change | ~Lines |
|---|---|---|
| app/src/sha256.{h,cpp} | new, FIPS 180-4, one function | 120 |
| app/src/cheat_store.{h,cpp} | new: manifest/PackMeta parse, verify, paths, states, store service with fetch/run hooks, pump, status |
450 |
| app/src/cheats.{h,cpp} | Origin, community set + guard, cp. prefix, prune, apply order, loadChtText, reloadCommunity |
+130 |
| app/src/cheats_core.cpp | App::cheatNet Worker, store wiring (dirs, base URL override, downloads flag, hooks), stop on shutdown |
+40 |
| app/src/cheats_ui.cpp | drawCommunity, drawCheatRows refactor |
+160 |
| app/src/app.h / app.cpp | CheatStore cheatStore; Worker cheatNet; + stop |
+10 |
| app/src/library_ui.cpp | checkbox label | 1 |
| app/cmake/VJCheatTests.cmake, app/CMakeLists.txt | sources, fixtures | +10 |
| app/tests/cheats_test.cpp + fixtures/community/* | §6 | +300 |
8. Docs
site/pages/app.html: the Cheats bullet and gallery caption mention community packs (install, credits, off by default, update, cartridges only); a short paragraph under Cheats.site/pages/privacy.html: the row the data design specifies (service: this project's website; when: opening the Cheats window with a cartridge loaded and downloads on, at most every 6 h, or on Check for updates / Install; sent: a request for the pack index and the pack named by the cartridge's CRC32, nothing else); the local-files list gainscheats/community/.tools/cheat-import/README.md: one cross-link paragraph: packs built by CI are served by the store and installed from the Cheats window; manual.chtuse stays as documented.docs/WHATSNEWline;docs/app-debug-tools.mdunchanged.
9. Risks
- SHA-256 is new code in the trust path: pinned by test vectors, and a wrong hash fails closed (no install).
- A pack beyond 256 enabled writes starves user codes, which come last; the manifest warning plus the existing count line are the mitigation, not a fix.
- The one downloads switch now also gates cheats; a user who turned art off for bandwidth loses the store silently unless the section's note is read.
- Stale cached manifest after the CI bump: at most 6 h plus the window-open gate; acceptable for cheats.
- Hand-copied or edited files under
packs/bypass the store's verification but not the CRC guard.