* dat: replace symbol wildcards with explicit archive entries
* dat: match symbol list formatting and ordering
---------
Co-authored-by: itsgrimetime <[email protected]>
* dat: type rumble patterns, extend color animation tables
lbRumbleData entries are rumble scripts (HSD_PadRumbleAdd), not color
animations: lbRumbleEntry with a null-terminated HSD_Rumble list.
LbBf's and items' color animation tables run to the next object.
DatCompile creates the object's directory, which a fresh build dir lacks.
Trailing relocations go from 283 to 221.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: type stage parameters, color animation scripts and scene leftovers
- UnkStageDat_x8_t: camera_anim, anim_loops (x28), x2C counted by x30;
HSD_AObjDesc.obj_id as the HSD_Joint the loader falls back to.
- Hoist the yakumono params of Great Bay, Heal, Yoshi's Story, Final
Destination, Onett, Yoshi's Island, Pokemon Stadium, Mushroom Kingdom II
route and Trophy collection into their headers and type them by archive.
- Stage color animation scripts (grMaterial_801C9604) as DAT_SCRIPT
ColorOverlay_x8_t pointers; zako spawn descs as extent arrays.
- dat_symbols.txt: GrOp camera wrappers, GrTDk/GrTKp light copies; drop
the PlGn trees at 0x5D7C8/0x5D978, which aren't material animations.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: type the remaining untyped publics
- yakumono_param in the 28 stages whose code reads none of it: one zero
word, typed u32.
- MemSnapIconData and MemCardIconData are null-terminated lists of banner
and icon pointers; icon_data is u8** in lbsnap.c and lbcardgame.c.
- The name entry lists' counts ran to the end of the data; each ends in a
pointer to an empty string. mnNameDefaultName* are only that pointer.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: grPStadium_YakumonoParam's diffuse color has an alpha byte
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: type orphan model desc lists in GrGb, GrTe, ItCo, fighters
Each is a one-element list of a DynamicModelDesc right after it, like the
existing model desc leftovers; unreferenced relocations 903 -> 293.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: type ftData.ext_attr as a union of each fighter's attributes
ext_attr points to union ftData_ExtAttr (ftdata.h), a member per fighter
kind. Plain-data unions with DAT_IF members now choose a member, like
unions with pointers, instead of checking the whole (largest) union.
Fix attribute fields read as pointers that hold scalars (Link, Purin,
Samus), and create obj/ dirs before compiling samples.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: type ftData's wait animation lists
ftData.x24 and x28 are WaitStruct lists ending in -1; the walk reads them
through the union's integer view.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: choose a layout for every item's special attributes
Unk4 (coins) is it_2E5A_Attrs, Charizard's flames their own two floats,
and items whose code reads no attributes an extent of unread words.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: note the unreferenced words after fighters' item articles
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: bound tagged union inference and readback by the selected member
* dat: keep one shared sample-directory fix
* dat: clean up includes after moving attribute types
* dat: type empty stage parameters as null pointers
* dat: resolve void pointer type names from DWARF
* dat: keep shared attribute layouts in module headers
* dat: remove obsolete attribute header includes
---------
Co-authored-by: Robin Avery <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: itsgrimetime <[email protected]>
* dat: walk byte command scripts, size CPU scripts
DAT_BYTE_SCRIPT(length) sizes scripts of byte commands by an expression
in the command's first byte; cpuCommandLength gives a CPU command's
length as ftCo_800B4880 reads it. DAT_SCRIPT and DAT_BYTE_SCRIPT also
apply on pointer typedefs, for arrays of scripts.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: note PlSb's unreferenced subroutines
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: reference the CPU interpreter for command lengths
* dat: tidy CPU script declarations and traversal
* docs: clarify CPU script annotations and walker comments
---------
Co-authored-by: Robin Avery <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: itsgrimetime <[email protected]>
* dat: add Leadead's last attribute byte, refresh TODO
Leadead's attributes have a byte at 0x1F that the sample dropped. TODO.md
drops findings that no longer occur and adds the walk's current ones.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: count debug and sound-test tables
* dat: select fighter, Kirby copy and stage item attributes
Bind Article::kind for fighters' own items and Kirby's copies, and select
the remaining character, stage and head-only item layouts. Ambiguous
item-attribute unions go from 106 to 33 and walk gaps from 57 to 19.
Tooling:
- Expressions take C's ?:, in melee-dat and the native evaluator, so a
macro can hold a table: ftData_ItemKind and ftKbCopy_ItemKind map
fighter and slot to the kind each *_Init_OnLoad registers.
- A chosen union member's DAT_BIND applies to it (walk and native).
- Untyped roots keep their loader bindings, for the types dat_symbols.txt
gives them (Kirby's copies get fighter_kind).
Types:
- itSpecialAttrsHead, Redead's x0 completed, is the record monsters' and
stage items' attributes start with: Tingle, Whispy's apples, the ZG
shell, the Arwing laser and Like-Like now use it.
- Crazy Hand's laser and bullet share Master Hand's layouts.
- gr_itkind is the whole kind.
- Five unused AnimJoint aliases that misread their trees are dropped.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: type stages' item script tables and quake model lists
ALDYakuAll is a table of GroundItemScript, a scripted pointer, as far as
the data runs, so its scripts are reached and their aliases go. Each
stage's unreferenced list after quake_model_set is typed by address.
Trailing relocations go from 613 to 463.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: type ItCo's copied attributes and joint lists, count CPU scripts
ItCo's copy of the common items' data types each item's attributes as
its original's kind; Food's and Unown's copied joint lists after their
model descs are typed by address. Unown's swarm shares Unown's
attributes. Fighter_804D64FC.cmdscripts holds 0x3E CPU scripts.
Trailing relocations go from 463 to 283.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: preserve handoff fixes and test binding and count semantics
* dat: remove a redundant header and format union bindings
* dat: create the samples' object directory before compiling
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: express item lookups as tuple matches
* dat: wrap tuple bindings in annotation macros
* dat: name match wrappers after their annotation
* dat: use C-like syntax for match cases
* dat: simplify match cases to variadic expressions
---------
Co-authored-by: Robin Avery <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: itsgrimetime <[email protected]>
* dat: validate packed motions from the original DOL
* dat: align symbol fixtures with EditorConfig
---------
Co-authored-by: itsgrimetime <[email protected]>
Sample coverage: 99.90% to 99.98% of archive bytes (about 196 KB more
explained); every sample matches.
Annotations and dat_symbols.txt:
- DAT_TERMINATED(value, n): terminators n elements long (LbAd's sound
load lists end in two)
- DAT_COUNT on pointer typedefs, for lists of counted lists, and nested
member names in expressions (x0.dynamicsNum)
- DAT_SCRIPT length expressions can end a script with a 0-word command;
colAnimCommandLength for color animation scripts
- dat_symbols.txt: terminated:V for null-terminated list roots, and
script:S for scripts nothing points to
- the native archive interface supports all of the above
Types:
- counted: spline control points and segments, hurtbox inits, stage
parameter rows, stage joint position lists, fighter sound id lists,
absorb descs, ftLoadCommonData's tables, dynamic bone sets
- typed: color animation scripts, target test touch lines, Mewtwo's and
Jigglypuff's target stage params, damage shift lists
(ftDamageShifts), fog animations (SceneFogAnim, two words)
- toy background animations loaded with their own types
Unreferenced data, typed by address:
- stage camera, fog and light wrappers, and whole orphan scenes
- orphan model descs and their lists
- more unreferenced animation and model trees; Link's and Young Link's
rooted at their joints rather than inside them
- 131 fighter and item scripts nothing points to
- ItCo's byte-for-byte copy of the common item data
A C library, tools/dat-cli/native, that reads the archives into the game's own types on any platform, from tables `melee-dat native codegen` writes from the DWARF. It's a port of the walk that converts as it goes, so `melee-dat native expect` and the library's trace must agree: they do for all 7048 archives on 64-bit and 32-bit little-endian and 32-bit big-endian (under qemu), and every converted object reads back against its data. The dat preset builds it with the host's compiler and runs its tests in ctest.
- the walk lays queued objects out as they were reached, so typedef tags
apply (the CPU attack lists' DAT_TERMINATED was ignored)
- CPU attack tables typed (ftCo_AttackList); stage param types moved into
their headers; stage param fields typed as their data
- 47 dat_symbols.txt aliases that read stage data as material animations
removed
- generated C casts only between different types; untyped data typed from
the fields that point to it
- `samples report --incomplete`, why units aren't complete
- a `native` dev shell with the DWARF/dat tools, qemu and a ppc32 cross
compiler; the default shell is what CI needs
- clang-tidy's checks stay on src/: tools/ and the generated tables opt out
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dat: native schema as descriptors per type, a file per header, enum type indices
`melee-dat native codegen` writes a directory instead of one tables.c:
each type is a named `dat_type_*` descriptor with its members, in
types/<header>.c for the header that declares it, along with what refers
to it (pointers, arrays); roots are in roots/<module>.c; DAT_TYPE_* and
DAT_NAME_* are enums. Descriptors use designated initializers, refer to
types by name and keep expressions next to their members, with the
annotation as written in a comment where it isn't plain.
- dat/schema.h: the schema's types and the macros descriptors are written
in; DAT_NONE is 0, so every field defaults to zero
- members, expressions, scripts, binds and roots are pointers, not ranges
of shared tables; macros expand to a depth when evaluated
- the schema is generated when configuring, again when its inputs change
- codegen split into generator.rs (the rows) and emit.rs (the C)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* variable types fixes and finding targets for cleanup
* Small tweaks to dat file annotation groundwork
* Fixing types
* Rework item enum sections
* Move item section sizes into enum (instead of using defines)
Archive byte coverage (matching samples and inferred data) goes from 86.98%
to 99.80%, and complete units from 106 to 345 of 838. Sample matches are
unchanged apart from the two known mismatches.
Tooling:
- `samples report` prints the archive bytes covered, and `--missing` lists
the data the base doesn't infer by the field that reaches it.
- dat_symbols.txt lines are `name = dat:address;` like symbols.txt, with the
dat in the section's place. `*` for the address matches a public symbol
by name, as before; an address gives a C alias for data no public symbol
names. Type and count are separate attributes mirroring the annotations:
`type:T`, `count:N` and `extent` (instead of `count:*` and `T[N]`).
- `DAT_SCRIPT` takes a table (fighters) or a length expression in
`_command` (`itCommandLength`, items); the generic commands 0-9
(`Command_Execute`) are shared by every script. Scripts' extents count
as typed data.
- hsd.rs keeps archive tables in file order, as the game reads them, and
refuses malformed archives.
- Clippy comes from the flake.
Types:
- `HSD_VtxDescList::vertex` is an `HSD_VertexArray` blob; palettes are
stored rounded up to 16 entries; collision vertices, lines and joints are
counted; the particle banks run as far as the data does; item state
scripts are scripts; `FtPartsDesc.vis_table` rows are
`FtPartsVisLookup*`; `HSD_AObjDesc.obj_id` is any descriptor; every
effect archive's table is an `EffectDataTable`.
Unused data: only names, and the pointers from them, reach archive data,
so data nothing points to is export leftovers. About 3,800 aliases type
the joint, animation and material animation trees among it whose every
part is well-formed (PlFe's and TyMcCmDs's unused models among them),
with no new walk gaps or sample mismatches.
Co-authored-by: Claude Opus 5.5 <[email protected]>
Samples are now chosen across every archive by a `samples pick` step: for
each type and union variant, the fewest instances that together show every
case its fields take anywhere in the game (pointers null, relocated,
unrelocated or -1; float and integer classes; bitfields; nonzero padding),
preferring instances the walk found clean. Each archive's choice is
rewritten only when it changes. Sample-mode C goes from 12,130 samples
(245k lines) to 347 (13k lines), and every unit is in objdiff, samples or
not. MELEE_DAT_SAMPLES_ALL still samples everything.
Elided data is declared as the type the walk reached it as, spelled as the
pointers to it spell it (`Mtx`, `HSD_DisplayList[n]`, `u16[n]`), instead
of `UNK_T`. `MtxPtr`, `Mtx44Ptr` and `ROMtxPtr` are redeclared with
`DAT_TYPE` in dat_macros.h for the DWARF build, which uses aurora's SDK
headers.
Every piece of each archive outside the samples is now its own symbol in
the target, and the base infers each piece the walk explains on its own,
instead of only public symbols and data the samples point to, each with
everything it reaches. So what's inferred no longer depends on what's
sampled: units without samples show all their data, and 106 units are
complete in both modes. Bytes a pointer says are bytes (texels, strings,
keyframes) and command scripts up to their end command count as explained.
Each unit's C is now `src/<unit>.h`, declaring the archive's public
symbols, its samples and what they point to, and `src/<unit>.c` only for a
unit with samples, instead of a header and source per root in a folder.
A unit without samples links its base object from its inferred data alone.
Clippy is in the dev shell, and its warnings in dat-cli are fixed.
A union member condition that can't be evaluated now stops the search
instead of falling through to a later member, so `ftData_Item` uses
`DAT_IF(true)` as its catch-all.
Co-authored-by: Claude Opus 5.5 <[email protected]>
* dat: reject missing archive inputs during configuration
* dat: require every target symbol to be inferred for completeness
* dat: check inferred symbol identity and size before archive walks
---------
Co-authored-by: itsgrimetime <[email protected]>
Textures, palettes and display lists are typed with the sizes the game
computes:
- HSD_ImageDesc::image_ptr is HSD_ImageData, sized by
GXGetTexBufferSize(width, height, format, mipmap, maxLOD + 1)
(exact for all 168 mipmapped textures)
- HSD_TlutDesc::lut is u16, counted by n_entries
- HSD_PObjDesc::display is HSD_DisplayList, n_display << 5 bytes, as
GXCallDisplayList takes it
Annotation expressions can call functions the tool ports (only
GXGetTexBufferSize so far), and float fields convert to integers as C
does. DAT_BLOB on a u8 typedef names a format of opaque bytes; raw u8
data is never inferred. Keyframe streams are HSD_FObjData, counted by
their length, and DAT_TERMINATED compares elements smaller than a word
whole (FigaTree::nodes, up to -1).
.1.inferred holds only the archive's public symbols and the data the
samples point to; one is inferred when everything it reaches before
another of them or a sample is explained. The walk records how far each
object's typed data reaches, so counted and DAT_EXTENT arrays count
whole, and counted plain data is one object. Zeros up to the next word,
or 32 bytes for GX data, are padding. Generated C declares untyped data
as UNK_T instead of DatBlob arrays.
EffectDataTable types the effect descriptors after its particle banks
(Ef: 0.3% to 99.3% of relocations explained).
Fixes:
- types.bin dropped the tables DAT_SCRIPT reads, so every script opcode
past the first ten was unknown (6265 false findings)
- the types hash stopped at forward declarations and DAT_TYPE names, so
changes to many HSD records never rebuilt their archives
objdiff now infers 74% of the archives' global data (from about 1%);
14125/14127 samples match.
Co-authored-by: Claude Opus 5.5 <[email protected]>
Units are grouped by module, the first two letters of the archive's name:
every build directory holds <module>/<archive> (Pl/PlMr), and objdiff
names them dat/Pl/PlMr, like the code's main/.
The target objects held only the samples, so units showed every symbol
matching while not complete. Now they hold the whole archive in two
sections, which objdiff lists by name:
- .0.sampled: the sampled objects, with pointers as relocations
- .1.inferred: the archive's other global symbols (its publics, and the
data the samples point to), uninitialized, so objdiff never diffs
their bytes
The base defines the globals the walk explains in its own .1.inferred
(target/<unit>.rest.o, linked with the C), so objdiff pairs them by name
and matches them by size; the rest show as missing. A global is
explained when it and everything it reaches before another global or a
sample is typed data with no unexplained relocation. Each target
symbol's offset in the archive is its virtual address in a .note.split,
as decomp-toolkit writes for split code, which objdiff shows. objdiff
takes section kinds from their ELF type, so the names are free; the
numbers set the order.
Raw u8 data says nothing of its format, so it isn't explained. DAT_BLOB
on a u8 typedef names one: HSD_FObjData, the keyframe streams of
HSD_FObjDesc and FigaTrack, sized by DAT_COUNT(length). DAT_TERMINATED
compares elements smaller than a word whole (FigaTree.nodes, up to -1).
The walk records how far each object's typed data reaches, so counted
and DAT_EXTENT arrays count whole. Generated C declares untyped data as
UNK_T instead of a DatBlob array.
---------
Co-authored-by: Claude Opus 5.5 <[email protected]>
Symbol names and scope in the samples:
- Data the archive doesn't name is named by the field that first reaches
it and its offset (`child_x1A0`, `x1C_4_x2818`), instead of `x<OFFSET>`,
and is local to its unit in both objects: `LOCAL` (static, used) in the
generated C. Only the archive's public symbols are global.
- Data the samples point to but don't define starts with its root
(`ftDataMario_x0_common_attr_x3AC8`), and is declared as its type where
the walk typed it as one record, instead of as bytes. Externs start with
the unit (`GrFz_<extern>`).
- `LOCAL` and `DatBlob` live in a `macros.h` that `samples macros` writes
into the build directory, like dtk's `macros.inc`, instead of
`dat_macros.h`, which is for annotating the DWARF.
Annotations:
- `DAT_TERMINATED(value)` replaces `DAT_NULLTERM`: a list ends at an
element whose first word is `value` (`GX_VA_NULL` for vertex descriptor
lists, now walked).
Build:
- `types/` and `stamp/` merge into `metadata/`; clang-tidy is off for the
generated sources.
- The dat-cli README is brought up to date.
Co-authored-by: Claude Opus 5.5 <[email protected]>
Of the 30 roots the code loads into a `T**`, 28 are tables: null-terminated
lists (map_plit in the stage archives, itemdata, the scene light and model
lists) or fixed-count tables (the staff roll models, the name tables). The
walk followed only their first entry: when the loader gives a root's type,
dat_symbols.txt couldn't add a count.
- dat_symbols.txt takes a `count:N` or `count:*` attribute, which also
applies to roots whose loader gives their type: the walk lays out that many of the root's
type, or as many as fit before the next symbol or pointer target (a
null-terminated list's null entry included).
- 17 table roots get counts.
- The per-archive types hash includes the count.
Explained relocations 95.0% -> 95.9% (Gm 98.4%, Gr 95.3%, If 99.5%,
Mn 99.3%); samples unchanged.
Co-authored-by: Claude Opus 5.5 <[email protected]>
- Per-archive type dependencies: `samples types` hashes the types each
archive's roots reach and rewrites the hash only when it changes, so a
header change rebuilds only the archives that use it (a stage type: the
71 Gr units, in 13 s).
- objdiff `complete` for units whose types explain the whole file: every
relocation, every public symbol, no walk issues (49 of 765).
- Dat types declared in .c files are hoisted into their TU headers, so
their data can be sampled; `types unhoisted` lists what's left (none).
ftLoadCommonData, plLoadCommonData and the menu data tables are struct
tags, since each shares its name with its archive symbol.
- Coverage: scene and stage model-group animation/light lists are
DAT_NULLTERM, the stage model group's fields are named, and the debug
and sound test name tables are sized (91.5% -> 95.0% of relocations
explained).
- `types duplicates` finds records defined twice; ftData_x38 is merged
into AbsorbDesc.
- Archive externs: named slots Melee's loader sets to NULL. The walk
counts them as pointers, and the samples relocate them to their names.
- Generated headers group declarations as Samples, Externs and Elided;
data that isn't written as C is a sized `DatBlob` array.
13564 of 13566 samples match, as before; the DOL still matches.
Co-authored-by: Claude Opus 5.5 <[email protected]>
- No header code in base objects: jobj.h redeclared the static inline
HSD_JObjRefThis without `inline`, which made clang emit it (and
iref_INC, with its assert's file name) wherever the header is included.
`samples report` lists anything a base object has outside .data.
- The base object's .data is laid out in the target's order, so objdiff's
section match agrees with the per-symbol one. clang lays variables out
where an initializer first points to them, not where they are defined,
so each unit is compiled with -fdata-sections and linked (ld.lld -r)
with target/<unit>.ld, which `samples slice` writes. Every unit's .data
bytes now equal the target's.
- Formatting in place: codegen writes src/ and the format step formats it
there, with a stamp in stamp/, so there is one set of generated sources.
Also:
- The build runs without the dev shell's environment, as objdiff runs it:
NEWLIB_INCLUDE and AURORA_SRC are cached (from the environment when set;
configuring without them fails), and the LLVM tools and cargo are found
by absolute path. Configuring through a symlinked path fails: the build
then names files by two paths, and ninja reran CMake and rebuilt
everything on every build.
- User-side switches in the build's cache: MELEE_DAT_SAMPLES_ALL (archive
globs) samples every typed object of those archives, and
MELEE_DAT_SAMPLES_EXCLUDE (type globs) keeps types out of the samples.
- objdiff is pinned to v3.8.2, which no longer needs the Cargo.lock patch.
11237 of 11239 samples match, as before; the DOL still matches.
Co-authored-by: Claude Opus 5.5 <[email protected]>
The dat samples no longer go through dtk, the DOL or MWCC: the archives
weren't built with MWCC, so matching them only needs a pipeline that
compiles to the same bytes. They now build in their own CMake preset,
`dat`, in `build/GALE01/dat`, which is also their objdiff project.
- `cmake/Dat.cmake`: per archive, four ninja steps: slice the archive into
a target object (`samples slice`), generate C from it (`samples
codegen`), format it with the repository's `.clang-format`, and compile
it with the DWARF build's flags. `samples project` writes `objdiff.json`
last. `compile_commands.json` gives clangd the same flags.
- The DWARF build is per game version (`MELEE_VERSION`).
- `types export` writes a compact, deduplicated types file (postcard,
1.8 MB), so each step loads the types in milliseconds instead of reading
the 160 MB ELF.
- Samples are per archive and named as the archive names them (its public
symbol, else `x<OFFSET>`). The generated C is a directory per archive
with a header and source per root; a unit file includes the sources.
Pointers between roots go through the headers' extern declarations.
- Union objects are declared as the member their tag chose; generated
comments are Doxygen.
- `configure.py`, the root objdiff project and `melee-dtk` know nothing of
dats; `config/GALE01/dat.yml` loses its samples section.
- nix: `melee-dat-samples` is the CMake DWARF build with the samples on.
- `HSD_RObjDesc_u` is back in its original member order: designated
initializers don't need a pointer member first.
11237 of 11239 samples match; the two that don't are in TODO.md.
Co-authored-by: Claude Opus 5.5 <[email protected]>
See tools/dat-cli/README.md.
The walk now reads every archive, including the packed Pl??AJ.dat
animation files, and types 99.5% of their public symbols. Symbols the
loaders don't record, because their names are built at runtime, are
typed by name patterns in config/GALE01/dat_symbols.txt (fighter
animations, the HSD exporter's *_joint/*_matanim_joint/... names,
textures and palettes as byte arrays, SISLib text, stage parameters,
scenes and Kirby's copy abilities). New annotations cover what C types
can't: DAT_NULLTERM lists (also on typedefs, for nested ones like the
envelope lists), DAT_SCRIPT command streams (fighter action scripts,
with lengths from the game's own table), DAT_EXTENT on pointers, and
DAT_TYPE on typedefs.
`melee-dat symbols coverage` reports how many relocations the walk
explains (91.5%) and classifies the rest: gaps where a type is missing
or wrong, data trailing a typed object where nothing points, and data
nothing references at all. It can group them by archive, by the object
they follow or by untyped field, and writes JSON shaped like objdiff's
report.json.
`melee-dat samples` checks the types themselves. It picks one instance
of each type, and of each variant its tagged unions choose, then writes
the instance's bytes as a target object and C initializers generated
from the current types, compiled with the game's MWCC command. objdiff
compares them, so a pointer typed as an integer, data in padding or a
layout MWCC disagrees with shows up as a mismatch. Everything is
generated under build/GALE01/dat, which is an objdiff project. All HSD
and SDK samples match; one item type doesn't yet.
The DWARF build now compiles aurora's headers in their console layout
rather than TARGET_PC, which had made GXBool an int and grown GXTexObj
and GXTlutObj. dat_macros.h fills in the GX vertex functions aurora's
console branch lacks, until they're upstreamed.
Fixed along the way: stage touch-line callbacks return hitbox data
(lbColl_80008D30_arg1), not DynamicsDesc; Kirby's copy table is one
entry per FighterKind, so every hats[] read was a fighter off, and each
copy now has a struct; ftData_x20.x0 is the shield's root joint;
FigaTrack has a byte at +7; HSD_RObjDesc_u lists a pointer member first
so C initializers can hold every variant.
Co-authored-by: Claude Opus 5.5 <[email protected]>
See tools/dat-cli/README.md.
The macros in dat_macros.h write annotations into the DWARF built by the
ppc-dwarf CMake preset (melee-dwarf in nix). The tool is AI-assisted and
incomplete; tools/dat-cli/TODO.md lists what's missing. It will
eventually hook into objdiff and generate its own coverage/progress
metrics.
The annotations are parsed by tools/dat-cli/src/interop/dwarf/
annotation.rs and expr.rs. The expressions are a bit complex because
they need to describe conditionals that reference struct fields, but
wouldn't be that hard to implement for downstream tooling in other
languages. This makes the DWARF valuable to ports and other use cases
for reading dat files.
In the DWARF build, the archive loaders record which type each symbol
is loaded as: the lbArchive loaders through DAT_ROOTS, and
HSD_ArchiveGetPublicAs, a typed HSD_ArchiveGetPublicAddress that
replaces most call sites. Checking those types against the archives
turned up wrong ones, which are fixed here: DynamicsTemplate is split
from DynamicsDesc, UnkStageDat's shadow materials, HSD_AObjDesc.obj_id,
the particle banks, the item throw table, and a number of void*
destinations. placeholder.h and m2c_macros.h move to
libs/doldecomp/include alongside dat_macros.h.
---------
Co-authored-by: Claude Opus 5.5 <[email protected]>
