Followed [the
guide](https://github.com/Xeeynamo/sotn-decomp/wiki/Decompilation#add-new-overlay)
which seemed to work relatively painlessly for this overlay.
I hit an issue in `make-config.py` where no duplicates were found and
have added a separate commit which fixed that. Otherwise no issues with
guide.
```
josh@JoshsPC:/mnt/c/dev/sotn-decomp$ tools/make-config.py tt_001
✔ generating psx splat config
✔ splitting config/splat.us.tt_001.yaml
✔ adjusting files at src/servant/tt_001
✔ disassembling matched functions
✖ finding duplicates across overlays
Traceback (most recent call last):
File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1132, in <module>
raise e
File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1126, in <module>
make_config(args.input, args.version)
File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1106, in make_config
if found > 0:
TypeError: '>' not supported between instances of 'NoneType' and 'int'
```
I was also able to remove the weird `goto block` on `RicMain`.
The HD version seems to be an older version. I am not entirely sure of
the US changes, but it seem to fix a series of bugs when Richter gets
grabbed by Gaibon? There's a new debug string which ints
`g_Player.unk7A` being named as `run_disable_f` but I did not yet rename
it.
Allow to compile weapon overlays as a single C files. The benefit is to
disallow symbols to be exported by statically compile them all but the
header. We would no longer need to use `OVL_EXPORT` as long as the
symbols are marked as `static`. The frames data is also generated as an
array of data that can be just included into their respective C files,
generated by JSON files on the fly.
I changed `animset` to `frameset` as the data does not represent a set
of animations but rather a set of frames, where every frame has multiple
sprites. The information on how the frames are played (e.g. animation)
is found elsewhere.
Following the full list of changes to achieve this project:
* The `animset` from splat is now dummied out
* The `frameset` JSON is now generated by the asset manager
* The `frameset` JSON is now compiled into `src/weapon/w_0xx_y.h`
* The `header.c` has been moved into `shared.h`
* For simplicity I added a `#define g_Animset w_000_1`. I did not want
to hack the asset manager too much to force the name to be `g_Animset`
and I did not want to change all the symbols yet.
* Simplified `weapon_pc.c` as we now only need the exported header
* Always try to build the most up-to-date asset manager: this should
avoid weird errors whenever people pull new changes
As the asset manager now can accept those `config` files, now the CLI
supports the following commands:
* `stage extract`
* `stage build`
* `config extract` (NEW)
* `config build` (NEW)
The config structure is heavily inspired to Splat to maintain
consistency and continuity.
I moved *some* of the PSX-specific stuff out of the common `Makefile`
and into the `Makefile.psx.mk` -- there is still more that could be done
here, but this should be enough to have the correct rules for PSX and
PSP with the desired assembler.
Also threw in a bonus feature where I've removed the `move` macro from
`macro.inc` because `maspsx` does this automagically for you now.
Mostly identical to MAD and PSP. There are four very different functions
from US that needs to be decompiled to be 100% complete:
* [x] `e_misc.c` `EntityMessageBox`
* [x] `e_misc.c` `EntityRelicOrb`
* [x] ~`e_stage_name.c` `StageNamePopupHelper`~ it does not exist
* [x] `e_stage_name.c` `EntityStageNamePopup`
These four very different functions are the only ones that use the JP
text instead of the US one. This overlay also lacks of `BlitChar`.
`EntityStageNamePopup` is very similar to the PSP counterpart. I used
@joshlory scratch to match the HD part.
`EntityRelicOrb` had some #ifdef between US and BETA (aka the MAD
overlay). I found some code that crossed between BETA and HD, so I just
put an `#else` after `VERSION_US`. We are starting to reconstruct how
the source code originally evolved across different game builds.
This aims to deprecate all the Splat tools in `tools/splat_ext` in
favour of a more centralised asset manager. This brings the following
advantages:
* Much faster extraction
* Faster build
* Automatically define `static` symbols or unique names whenever
`static` is not possible
* Allow to embed assets into the output binary
* Drastically simplify `Makefile` by removing all the asset build rules
* Avoid situations where it is not possible to extract and build assets
that is not 4-byte aligned
This is achieved by having the splat YAML targeting a normal C file as
data and have an external tool to take care of the following:
1. Extract asset files straight from the overlay binary file into human
readable file in `assets/st/STAGE_NAME`
2. Build assets as header files that go into `src/st/STAGE_NAME` to just
include them from any C file
This requires each stage header to have the following new format: please
see `src/st/nz0/header.c`
Built assets in `src/st` are ignored by Git.
As for now, for simplicity sake, the steps `make extract_assets` and
`make build_assets` are just executed within `make extract` exclusively
for the US version.
I plan to auto-generate files such as `src/st/nz0/tile_data.c`.
For a first iteration I am aiming to handle the following:
* [X] Extract rooms: `assets/st/*/rooms.json`
* [X] Extract room layers: `assets/st/*/entity_layouts.json`
* [X] Extract tilemap data: `assets/st/*/tilemap_*.bin`
* [X] Extract tilemap definitions: `assets/st/*/tiledef_*.json`
* [X] Extract sprites: `assets/st/*/sprites.json`
* [x] Extract entity layouts
* [X] Build rooms: `src/st/*/rooms.h`
* [X] Build room layers: `src/st/*/layers.h`
* [X] Build tilemap data: `src/st/*/tilemap_*.h`
* [X] Build tilemap definitions: `src/st/*/tiledef_*.h`
* [x] Build sprites (aka `g_SpriteBanks`)
* [x] Build entity layouts
* [x] Allow the tool to suggest how to adjust the Splat config for each
overlay
I want the tool to cover the following stages:
* [x] CEN
* [x] DRE
* ~MAD~ I do not think this can be done, it is way too different from
the other overlays
* [x] NO3
* [x] NP3
* [X] NZ0
* [x] ST0
* [X] WRP
* [x] RWRP
* ~WRP (PSP)~ Maybe in a follow-up PR
For a later iteration I plan on extracting and build:
* Entity GFX thingie
* The CLUT thingie in the header
* Uncompressed GFX data
* Cutscene data
* Blueprints
* The `src/config_us.h` thingie
---------
Co-authored-by: Josh Lory <josh.lory@outlook.com>
This updates the Makefile so that targets which depend on phony targets
(like `*_dirs`) have direct dependencies on those targets rather than
relying on transient dependencies from dependent targets. Other targets
explicitly create parent directories for their files.
This resolves#739, making it possible to using make-4.4 to build in
parallel.
Extract ST/WRP out of #1119 . All the function symbols should have been
cross-referenced. There as some PSX functions missing from PSP and some
new functions from PSP that are not present on PSX (e.g.
`st_init_wrp.c`).
The files `st_debug.c` and `e_breakable.c` are shared between WRP PSX
and WRP PSP. Everything else from PSP is isolated into its own folder. I
had to do some tricks on the YAML config to allow shared code.
`ST_WRP_MERGE = st_debug e_breakable` in the `Makefile` is a bit
annoying as MWCC complains about every single minute detail from the C
source that has been already decompiled for the PSX US build.
`EntityWarpSmallRocks` is matching on PSP but I couldn't extract the
rodata without having a bunch of linker errors. This might be a Splat
issue. I need to investigate further.
`func_psp_09244760` is soooo interesting. The values from `0x11` to
`0x17` matches the Entity IDs that are unique to the WRP overlay. This
aligns to what we have in `typedef enum EntityIDs`.
Overall I am very excited to the recent discoveries from the PSP build!
100% decompiled.
`US` adds some padding to have the file exactly 40KB long. The HD
version does not add any padding, hence why I ended up producing
`tt_000_raw.bin`. I would have used just `tt_000.bin` as a name as found
in `disks/pspeu/PSP_GAME/USRDIR/res/ps/hdbin/tt_000.bin`, but on
Windows-based file system it would collide with `TT_000.BIN` due to the
OS having case insensitive names.
I modified `make clean` as I found annoying that `VERSION=hd make clean`
would wipe out `us` build stuff.
The only different function in HD is `ProcessEvent`, which has a weaker
check. Another hint suggesting HD being older than US.
`s32 _unused[26];` added enough padding in the bss section to get an
🆗 . I am pretty sure it is unused data because the final binary is
not aligned by any power of 2.
Mostly a proof of concept to extract resources from `BIN/F_GAME.BIN`.
There are tons of hacks to make this work:
* Tweaked png2s to decode binaries into PNGs
* Add a `config/gfx.game.json` that describes how to extract different
sprites out of a binary file
* Coded the palette location by reading the `clut` parameter in the
source code. A clut value of `0x196` corresponds to `"palette": "0x96"`
or `"palette": 150`.
* Coded the bits per pixel for a couple of 8-bit images
* Coded the palette location of those graphics that expects the palette
from DRA.BIN instead of F_GAME.BIN
* For all the undiscovered palette, I called the files `unk` and
defaulted their palette to grey
The expectation is that sprites will slowly be documented and added into
`config/gfx.game.json` by modders and enthusiasts. This does not yet
pack back these PNGs into the binary file. The way these assets are
extracted is probably not final or perfect, but we have to start from
somewhere.
Extract the memory card icons out of DRA.BIN and SEL.BIN.
I also took the opportunity to extract `g_MemcardPortMask` into its own
source file `save_mgr_pre.c`. Unfortunately this is required to keep the
original data order. This suggests that most likely the icon palette and
bitmap were baked into the original `save_mgr.c` as byte arrays. I
decided to take a different approach and extracted them as PNGs for
better moddability.
I had to spin-off Splat into a new fork due to some breaking changes on
0.18.0.
Previously if either the YAML or the symbol list were changed, we were
forced to either manually delete the `asm` folder of the specific
overlay or to invoke `make clean`, where the latter forced us to
re-extract all the overlays via `make -j extract`.
That changes with this PR. How `make` works is `target: dep1 dep2` where
if one of the dependencies has the last modified date greater than the
target, the rule is triggered again. Previously we were extracting
overlays doing `make extract_stcen`. But since every overlay extraction
generates a linker script I now changed the rules to do `make
build/us/stcen.ld` to extract the same overlay. As I explained above, if
either the YAML or one of the related symbols changes, the linker
modified date will be older and the extraction for that overlay to be
trigger again.
The above allowed me to stop polluting the repo root with the linker
scripts as they are now moved into `build/us` or `build/hd` depending of
what you're trying to build. If you do `make extract` twice in a row you
will be welcomed with a `make: Nothing to be done for 'extract'.`.
There are still some instances where you **might** need `make clean`
beforehand, especially when modifying the `segment` section in the YAML
or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I
want to see if the current solution will be enough.
### Please try removing `make clean` from your workflow, once this is
merged, to quickly detect possible problems with this new approach.
I am also planning to make more substantial changes on our build-chain
like this PR or #660 in the future. I am aiming to small incremental
changes over time in case I break someone's flow or detect design flaws.
I am also considering the breaking changes introduced in make 4.4, which
will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a
deadline to finish all the new build-chain work.
---------
Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
