11 Commits
Author SHA1 Message Date
Mike Grimes b2b201bc16 dat: generate native readers for typed HSD archive loading (#3658) 2026-10-11 19:01:44 +00:00
79b13a3246 dat: type stage and fighter external attributes (#3652)
* dat: type rumble patterns, extend color animation tables

lbRumbleData entries are rumble scripts (HSD_PadRumbleAdd), not color
animations: lbRumbleEntry with a null-terminated HSD_Rumble list.
LbBf's and items' color animation tables run to the next object.
DatCompile creates the object's directory, which a fresh build dir lacks.
Trailing relocations go from 283 to 221.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: type stage parameters, color animation scripts and scene leftovers

- UnkStageDat_x8_t: camera_anim, anim_loops (x28), x2C counted by x30;
  HSD_AObjDesc.obj_id as the HSD_Joint the loader falls back to.
- Hoist the yakumono params of Great Bay, Heal, Yoshi's Story, Final
  Destination, Onett, Yoshi's Island, Pokemon Stadium, Mushroom Kingdom II
  route and Trophy collection into their headers and type them by archive.
- Stage color animation scripts (grMaterial_801C9604) as DAT_SCRIPT
  ColorOverlay_x8_t pointers; zako spawn descs as extent arrays.
- dat_symbols.txt: GrOp camera wrappers, GrTDk/GrTKp light copies; drop
  the PlGn trees at 0x5D7C8/0x5D978, which aren't material animations.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: type the remaining untyped publics

- yakumono_param in the 28 stages whose code reads none of it: one zero
  word, typed u32.
- MemSnapIconData and MemCardIconData are null-terminated lists of banner
  and icon pointers; icon_data is u8** in lbsnap.c and lbcardgame.c.
- The name entry lists' counts ran to the end of the data; each ends in a
  pointer to an empty string. mnNameDefaultName* are only that pointer.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: grPStadium_YakumonoParam's diffuse color has an alpha byte

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: type orphan model desc lists in GrGb, GrTe, ItCo, fighters

Each is a one-element list of a DynamicModelDesc right after it, like the
existing model desc leftovers; unreferenced relocations 903 -> 293.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: type ftData.ext_attr as a union of each fighter's attributes

ext_attr points to union ftData_ExtAttr (ftdata.h), a member per fighter
kind. Plain-data unions with DAT_IF members now choose a member, like
unions with pointers, instead of checking the whole (largest) union.
Fix attribute fields read as pointers that hold scalars (Link, Purin,
Samus), and create obj/ dirs before compiling samples.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: type ftData's wait animation lists

ftData.x24 and x28 are WaitStruct lists ending in -1; the walk reads them
through the union's integer view.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: choose a layout for every item's special attributes

Unk4 (coins) is it_2E5A_Attrs, Charizard's flames their own two floats,
and items whose code reads no attributes an extent of unread words.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: note the unreferenced words after fighters' item articles

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: bound tagged union inference and readback by the selected member

* dat: keep one shared sample-directory fix

* dat: clean up includes after moving attribute types

* dat: type empty stage parameters as null pointers

* dat: resolve void pointer type names from DWARF

* dat: keep shared attribute layouts in module headers

* dat: remove obsolete attribute header includes

---------

Co-authored-by: Robin Avery <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: itsgrimetime <[email protected]>
2026-10-08 14:35:52 -07:00
e865ef5df4 dat: infer byte-command CPU scripts (#3651)
* dat: walk byte command scripts, size CPU scripts

DAT_BYTE_SCRIPT(length) sizes scripts of byte commands by an expression
in the command's first byte; cpuCommandLength gives a CPU command's
length as ftCo_800B4880 reads it. DAT_SCRIPT and DAT_BYTE_SCRIPT also
apply on pointer typedefs, for arrays of scripts.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: note PlSb's unreferenced subroutines

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: reference the CPU interpreter for command lengths

* dat: tidy CPU script declarations and traversal

* docs: clarify CPU script annotations and walker comments

---------

Co-authored-by: Robin Avery <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: itsgrimetime <[email protected]>
2026-10-08 09:20:49 -07:00
355e23c02f dat: extend fighter and stage item inference (#3650)
* dat: add Leadead's last attribute byte, refresh TODO

Leadead's attributes have a byte at 0x1F that the sample dropped. TODO.md
drops findings that no longer occur and adds the walk's current ones.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: count debug and sound-test tables

* dat: select fighter, Kirby copy and stage item attributes

Bind Article::kind for fighters' own items and Kirby's copies, and select
the remaining character, stage and head-only item layouts. Ambiguous
item-attribute unions go from 106 to 33 and walk gaps from 57 to 19.

Tooling:
- Expressions take C's ?:, in melee-dat and the native evaluator, so a
  macro can hold a table: ftData_ItemKind and ftKbCopy_ItemKind map
  fighter and slot to the kind each *_Init_OnLoad registers.
- A chosen union member's DAT_BIND applies to it (walk and native).
- Untyped roots keep their loader bindings, for the types dat_symbols.txt
  gives them (Kirby's copies get fighter_kind).

Types:
- itSpecialAttrsHead, Redead's x0 completed, is the record monsters' and
  stage items' attributes start with: Tingle, Whispy's apples, the ZG
  shell, the Arwing laser and Like-Like now use it.
- Crazy Hand's laser and bullet share Master Hand's layouts.
- gr_itkind is the whole kind.
- Five unused AnimJoint aliases that misread their trees are dropped.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: type stages' item script tables and quake model lists

ALDYakuAll is a table of GroundItemScript, a scripted pointer, as far as
the data runs, so its scripts are reached and their aliases go. Each
stage's unreferenced list after quake_model_set is typed by address.
Trailing relocations go from 613 to 463.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: type ItCo's copied attributes and joint lists, count CPU scripts

ItCo's copy of the common items' data types each item's attributes as
its original's kind; Food's and Unown's copied joint lists after their
model descs are typed by address. Unown's swarm shares Unown's
attributes. Fighter_804D64FC.cmdscripts holds 0x3E CPU scripts.
Trailing relocations go from 463 to 283.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: preserve handoff fixes and test binding and count semantics

* dat: remove a redundant header and format union bindings

* dat: create the samples' object directory before compiling

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: express item lookups as tuple matches

* dat: wrap tuple bindings in annotation macros

* dat: name match wrappers after their annotation

* dat: use C-like syntax for match cases

* dat: simplify match cases to variadic expressions

---------

Co-authored-by: Robin Avery <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: itsgrimetime <[email protected]>
2026-10-07 20:40:19 -07:00
Mike Grimesanditsgrimetime e35493e5ba dat: share native storage for array-element references (#3645)
Co-authored-by: itsgrimetime <[email protected]>
2026-10-06 16:07:05 +00:00
Mike Grimes 779690777b Preserve repeated opaque DAT references and verify missing pointers (#3640) 2026-10-06 07:27:41 +00:00
Mike Grimesanditsgrimetime b578ad37ff dat: select more common-item attribute layouts (#3637)
Co-authored-by: itsgrimetime <[email protected]>
2026-10-06 00:06:03 +00:00
Mike Grimesanditsgrimetime 29b5bd518c dat: count nested fighter visibility lists (#3636)
Co-authored-by: itsgrimetime <[email protected]>
2026-10-05 23:48:03 +00:00
Mike Grimesanditsgrimetime ea2995d5cb dat: honor DAT_COUNT on inline arrays (#3635)
Co-authored-by: itsgrimetime <[email protected]>
2026-10-05 12:19:27 -07:00
Robin Avery 71929abafe dat: type orphan scenes and scripts, count more tables (#3634)
Sample coverage: 99.90% to 99.98% of archive bytes (about 196 KB more
explained); every sample matches.

Annotations and dat_symbols.txt:
- DAT_TERMINATED(value, n): terminators n elements long (LbAd's sound
  load lists end in two)
- DAT_COUNT on pointer typedefs, for lists of counted lists, and nested
  member names in expressions (x0.dynamicsNum)
- DAT_SCRIPT length expressions can end a script with a 0-word command;
  colAnimCommandLength for color animation scripts
- dat_symbols.txt: terminated:V for null-terminated list roots, and
  script:S for scripts nothing points to
- the native archive interface supports all of the above

Types:
- counted: spline control points and segments, hurtbox inits, stage
  parameter rows, stage joint position lists, fighter sound id lists,
  absorb descs, ftLoadCommonData's tables, dynamic bone sets
- typed: color animation scripts, target test touch lines, Mewtwo's and
  Jigglypuff's target stage params, damage shift lists
  (ftDamageShifts), fog animations (SceneFogAnim, two words)
- toy background animations loaded with their own types

Unreferenced data, typed by address:
- stage camera, fog and light wrappers, and whole orphan scenes
- orphan model descs and their lists
- more unreferenced animation and model trees; Link's and Young Link's
  rooted at their joints rather than inside them
- 131 fighter and item scripts nothing points to
- ItCo's byte-for-byte copy of the common item data
2026-10-05 13:51:35 -04:00
Robin AveryandClaude Opus 5.5 d52f449432 dat: implement C deserialization codegen, improve coverage (#3632)
A C library, tools/dat-cli/native, that reads the archives into the game's own types on any platform, from tables `melee-dat native codegen` writes from the DWARF. It's a port of the walk that converts as it goes, so `melee-dat native expect` and the library's trace must agree: they do for all 7048 archives on 64-bit and 32-bit little-endian and 32-bit big-endian (under qemu), and every converted object reads back against its data. The dat preset builds it with the host's compiler and runs its tests in ctest.

- the walk lays queued objects out as they were reached, so typedef tags
  apply (the CPU attack lists' DAT_TERMINATED was ignored)
- CPU attack tables typed (ftCo_AttackList); stage param types moved into
  their headers; stage param fields typed as their data
- 47 dat_symbols.txt aliases that read stage data as material animations
  removed
- generated C casts only between different types; untyped data typed from
  the fields that point to it
- `samples report --incomplete`, why units aren't complete
- a `native` dev shell with the DWARF/dat tools, qemu and a ppc32 cross
  compiler; the default shell is what CI needs
- clang-tidy's checks stay on src/: tools/ and the generated tables opt out

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* dat: native schema as descriptors per type, a file per header, enum type indices

`melee-dat native codegen` writes a directory instead of one tables.c:
each type is a named `dat_type_*` descriptor with its members, in
types/<header>.c for the header that declares it, along with what refers
to it (pointers, arrays); roots are in roots/<module>.c; DAT_TYPE_* and
DAT_NAME_* are enums. Descriptors use designated initializers, refer to
types by name and keep expressions next to their members, with the
annotation as written in a comment where it isn't plain.

- dat/schema.h: the schema's types and the macros descriptors are written
  in; DAT_NONE is 0, so every field defaults to zero
- members, expressions, scripts, binds and roots are pointers, not ranges
  of shared tables; macros expand to a depth when evaluated
- the schema is generated when configuring, again when its inputs change
- codegen split into generator.rs (the rows) and emit.rs (the C)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 13:13:55 -04:00