This will decompile all the functions but the new `EntityRWarpRoom` from
the RWRP overlay.
Note that as `e_misc.c` has a function with a different castle flag, I
had to use a shared header. Everything else was identical.
---------
Co-authored-by: bismurphy <tjmurphy@mit.edu>
Mostly identical to MAD and PSP. There are four very different functions
from US that needs to be decompiled to be 100% complete:
* [x] `e_misc.c` `EntityMessageBox`
* [x] `e_misc.c` `EntityRelicOrb`
* [x] ~`e_stage_name.c` `StageNamePopupHelper`~ it does not exist
* [x] `e_stage_name.c` `EntityStageNamePopup`
These four very different functions are the only ones that use the JP
text instead of the US one. This overlay also lacks of `BlitChar`.
`EntityStageNamePopup` is very similar to the PSP counterpart. I used
@joshlory scratch to match the HD part.
`EntityRelicOrb` had some #ifdef between US and BETA (aka the MAD
overlay). I found some code that crossed between BETA and HD, so I just
put an `#else` after `VERSION_US`. We are starting to reconstruct how
the source code originally evolved across different game builds.
Now with the maspsx changes, this can also match by using
SPRITESHEET_PTR.
Tried to document all the variables, use flags when possible, etc, so
hopefully this is mostly complete. Should be exciting to have
RenderEntities and RenderPrimitives both working now!
This aims to deprecate all the Splat tools in `tools/splat_ext` in
favour of a more centralised asset manager. This brings the following
advantages:
* Much faster extraction
* Faster build
* Automatically define `static` symbols or unique names whenever
`static` is not possible
* Allow to embed assets into the output binary
* Drastically simplify `Makefile` by removing all the asset build rules
* Avoid situations where it is not possible to extract and build assets
that is not 4-byte aligned
This is achieved by having the splat YAML targeting a normal C file as
data and have an external tool to take care of the following:
1. Extract asset files straight from the overlay binary file into human
readable file in `assets/st/STAGE_NAME`
2. Build assets as header files that go into `src/st/STAGE_NAME` to just
include them from any C file
This requires each stage header to have the following new format: please
see `src/st/nz0/header.c`
Built assets in `src/st` are ignored by Git.
As for now, for simplicity sake, the steps `make extract_assets` and
`make build_assets` are just executed within `make extract` exclusively
for the US version.
I plan to auto-generate files such as `src/st/nz0/tile_data.c`.
For a first iteration I am aiming to handle the following:
* [X] Extract rooms: `assets/st/*/rooms.json`
* [X] Extract room layers: `assets/st/*/entity_layouts.json`
* [X] Extract tilemap data: `assets/st/*/tilemap_*.bin`
* [X] Extract tilemap definitions: `assets/st/*/tiledef_*.json`
* [X] Extract sprites: `assets/st/*/sprites.json`
* [x] Extract entity layouts
* [X] Build rooms: `src/st/*/rooms.h`
* [X] Build room layers: `src/st/*/layers.h`
* [X] Build tilemap data: `src/st/*/tilemap_*.h`
* [X] Build tilemap definitions: `src/st/*/tiledef_*.h`
* [x] Build sprites (aka `g_SpriteBanks`)
* [x] Build entity layouts
* [x] Allow the tool to suggest how to adjust the Splat config for each
overlay
I want the tool to cover the following stages:
* [x] CEN
* [x] DRE
* ~MAD~ I do not think this can be done, it is way too different from
the other overlays
* [x] NO3
* [x] NP3
* [X] NZ0
* [x] ST0
* [X] WRP
* [x] RWRP
* ~WRP (PSP)~ Maybe in a follow-up PR
For a later iteration I plan on extracting and build:
* Entity GFX thingie
* The CLUT thingie in the header
* Uncompressed GFX data
* Cutscene data
* Blueprints
* The `src/config_us.h` thingie
---------
Co-authored-by: Josh Lory <josh.lory@outlook.com>
`sotn_str` can now handle escaped quotes, right parenthesis, and more in
in `_S` strings. This allows all of the strings in `dra/menu.c` to be
encoded as static C-strings.
This also converts the spell button sequences to UTF-8.
---------
Co-authored-by: Jonathan Hohle <jon@ttkb.co>
We have now fully decompiled every function which had a DECOMP_ME_WIP
comment, and we do not expect to ever have a need to add one in the
future.
Therefore, the function finder can be simplified by removing the process
of searching for these.
With no DECOMP_ME_WIP strings, `get_c_files` will never append anything
to `files`, so it always returns an empty list. We can therefore remove
the function.
With `c_files` always being an empty list, iterating over it will do
nothing, so we can remove `c_files` and the entire block of `for c_file
in c_files`.
Those are all the changes being made here, pretty simple.
This took me a lot of manual work. But I think I confirmed a pattern
that will help me to automate all of this for all the next stage
overlays that will be imported in the repo.
I noticed some stages with only one room having more than two layers or
more than two tile definitions, it might be either debugging or unused
content? I did not have time to explore any of that.
I modified the `tiledef` splat extension to greatly minimise the set-up
and noise.
Still work in progress. I removed splat as a submodule and started using
it as a pip package instead. Everything is matching but the memory card
icons part in both DRA and SEL. I still have no idea what the issue is.
Once this PR is good to be merged, we can get rid of the splat fork too.
I had to cherry-pick some existing changes from another branch, but that
part is tested and working.
For the blueprint stuff, I changed the fields from the strings `"TRUE"`
and `"FALSE"` to native JSON boolean values `true` and `false`. That
made things much easier to parse.
The parsing mechanic is straightforward. I'd like to have @bismurphy
review on the blueprint parser and asset changes specifically.
Secondary function for the Skull Shield. Gets spawned by the ShieldSpell
that just got merged.
I also noticed too late that my previous PR reverted the update to
mwccgap since I hadn't updated that; this PR includes a fix to that
accidental revert.
This one was a beast, but nice to get it working.
This seemed to match the same entity extension as weapon 29. Weapons 28
and 29 are both shields, so I decided to rename the Weapon29 struct to
be Shield instead. We will see if it continues to match for the future
shields.
This had a ton of FIX(), enums, #defines, etc, so hopefully I got them
all :)
Made a small tweak to mwccgap so that we don't compile a .c file twice
if we don't need to... compiling psp used to take 2.7s and now takes
2.0s on my laptop - so not super noticeable at this point, but makes
sense not to compile things twice!
Also added sha1sum for the wrp.bin to avoid people copying in the wrong
file and having a bad time (dont know why I didnt catch that pspeu means
psp eu.. not psp us!)
When using the `dec` script, it is possible to select a function where
there are multiple functions sharing a name. The test case I had for
this was `EntityEquipItemDrop`. If you run `dec EntityEquipItemDrop`, it
will tell you that there are 4 occurrences of functions with that name,
and list the four, telling you to re-run with -n or -f to specify which.
If you choose to specify option #0, with `dec EntityEquipItemDrop -n 0`,
then the number_occurence variable will be set to zero. But the
if-statement `if number_occurrence` fails, because the value is zero,
which is parsed as False. Decompilation therefore fails, as if you had
not passed `-n`.
This change explicitly tests for None, so that zero is a valid
selection.
Thanks to @sozud for the hint and @bismurphy for the refactoring idea.
The function `PreventEntityFromRespawning` on PSP hints that the struct
starts 8 bytes earlier. Also there's a missing `nop` at
`PreventEntityFromRespawning`, suggesting the function had to be moved
to the previous function.
I noticed that this struct had some overlap with other values in memory,
so I have pulled all those values into this struct.
The boundaries of this struct are uncertain and are a matter of ongoing
research.
OK. Here goes. Version 0.0.1 alpha of the MWCC Global Assembly Processor
(mwccgap). It's currently very simple/limited, but it appears to work
for `src/servant/tt_000/10E8.c`.
There is lot more that can be done to improve mwccgap - i.e. supporting
.rodata migration would be a good addition, but let's see how far we can
get with it in it's current state.
Note that the Makefile could do with some improvements - we don't nede
to use mwccgap for any C file that *dont* have INCLUDE_ASM macros (it's
a waste of time) so these could be ignored, i.e. for SSSV I do something
like this to find the files that need fixing up:
```
GLOBAL_ASM_C_FILES := $(shell $(GREP) GLOBAL_ASM $(SRC_DIR) </dev/null 2>/dev/null)
```
.. although this is perhaps too simple given that SOTN has a mix of PSP
and PSX functions (and therefore there may be INCLUDE_ASM for a PSX
function but none for PSP functions...
In CUE files `FILE` is a stateful, global declaration that applies to
all following `TRACK`s. `sotn-disk` was treating `FILE` declarations as
`TRACK` delimiters which would result in incorrect parsing of CUE files
with a single `FILE`, but multiple tracks.
Now when reading a `FILE` declaration, the path is stored and processing
continues. A `TRACK` declaration will use the previously defined path,
and if a previous track had been started, append that previous track to
the list.
This also builds `sotn-disk` using the local repository instead of
pulling the latest commit from GitHub. The target depends on `sotn-disk`
sources and will rebuild as necessary (or with `make
~/go/bin/sotn-disk`).
As an aside,
[pull/232](https://github.com/Xeeynamo/sotn-decomp/pull/232) ran into
this same error, but fixes in a slightly different way that leaves side
effects that may make supporting things like `INDEX` (for extracting the
placeholder audio, for example) more error prone in the future.
Co-authored-by: Jonathan Hohle <jon@ttkb.co>
Having `platform: psx` will not correctly disassemble PSP functions with
opcodes that are not part of the R3000 (the PSX CPU) architecture. These
functions would be disassembled by Splat as a bunch of `.word
0xBLAHBLAH`.
Changing the platform to `psp` introduces all sort of new challenges.
Function prototypes needs to be declared earlier. But also the MWCC
compiler will not accept the `%lo` and `%hi` dialect from GNU AS. There
were some patches on `mwcpp.py` to use `la SYMBOL_NAME` that would
expand into a `lui / addiu` combo. But even though symbols needs to be
declared like function prototypes at the top of the file. This is simply
not feasible on bigger overlays.
As a solution to the problem above, I replaced the existing patches by
converting instructions into `.word`. The overlay cannot longer be
relocated with this approach, but it is not an issue as the final goal
is to decompile these functions any way.
The labels in the jump table has the same problem, which forced me to
change the segment type from `rodata` to `data.
This is just another single step to create the conditions to start
including bigger re-compilable PSP overlays. I am sure the `mwcpp.py`
solution will be thrown into the bin at some point, but this PR improves
our current situation.
TT_000 is the first overlay from PlayStation 1 that we are now able to
compile from the source and produce a 1:1 binary. This lead me to start
exploring the same overlay from the game Castlevania: Dracula X
Chronicles, which contains a PSP re-build of Symphony of the Night.
This PR adds all the infrastructure to add the same flow for a PSP
matching decomp. Use `export VERSION=pspeu` and then the usual `sotn`
command to splat the overlay, build it and check if it matches. Running
`make extract_disk` should not be necessary as the same ISO used from
`VERSION=hd` is also used for `pspeu`, so you will probably have it
extracted already.
Some important notes about the PSP build:
* The whole PSP build seems to be compiled with `-O0`, which makes it
much easier to decompile
* Having ŧhe PSX, PSP and Saturn builds will allow to easily
cross-reference the code and reduce fake matches
* `disks/pspeu/PSP_GAME/USRDIR/res/ps/hdbin/tt_000.bin` is the HD PSX
build
* `disks/pspeu/PSP_GAME/USRDIR/res/ps/PSPBIN/tt_000.bin` has the same
code from the HD build, but for PSP
* `disks/pspeu/PSP_GAME/USRDIR/res/ps/PACK_E/TP00.BIN` is the same as
above, but it packs both overlay and graphics. This is the file the PSP
game seems to actually use
* The PSP build uses the Metrowerk CodeWarrior's C compiler, which is
very different from the GCC one used on PSX.
* Thanks to @mkst lead, we found a way to still use the GNU assembler
and linker
* MWCC uses [wibo](https://github.com/decompals/WiBo/), a think
compatibility layer to run Windows CLI tools on Linux. It is much more
lightweight than Wine.
* MWCC does not support the `INCLUDE_ASM` dialect, so the custom
pre-processor `tools/mwcpp` had to be created
* The exact MWCC compiler version is unknown, but I suspect it is `build
147`
* I am not yet sure of any implications for using GNU AS and GNU LD
instead of the MW correspondent tools
* Not all the functions can be correctly disassembled, spimdisasm will
just produce a bunch of `.word 0x________` due to the in-progress effort
of adding the Allegrex-specific opcodes support
---
TO-DO list before marking the PR as ready:
- [X] Add PSP build to the CI
- [x] Add progress reporting to the PSP build
- [x] Integrate source file from `src/servant/tt_000_psp` to
`src/servant/tt_000` to promote the psp build as first-citizen
---
TO-DO in a follow-up PR:
* Figure out what `header` is: can we extract it as assembly code? or
maybe as custom re-compilable asset via splat? Is it a MW stuff or a
Castlevania-specific file?
* Get rid of the last line in `Makefile.psp.mk`
I don't have the frogress secrets etc. to fully test this but it seemed
to work on a dry run:
```
{'timestamp': 1707176381, 'git_hash': '5f3c92dc729020b7e8a6d7a98ac7549fc697089c', 'categories': {'code': {'main': 19812, 'main/total': 108268}, 'functions': {'main': 201, 'main/total': 534}}}
```
Another disappointing non-match, even closer than #1008 . There is only
one registry swap in one single line. I've been permuting for a day
without success. Putting this here if anyone wants to give it a try:
https://decomp.me/scratch/AuSvv
A side-project I've been working on that helped me understanding some
structures that are now a bit better documented in #1034 . Richter
sprites are extracted as usual. Doing `python3
tools/splat_ext/spritesheet.py decode disks/us/BIN/ARC_F.BIN assets/arc
alucard disks/us/BIN/F_GAME.BIN 1` can extract Alucard sprites too.
There is no repacking yet as I want to avoid using splat for this one.
This is a proof-of-concept for Japanese sotn_strs. The macro `_SJ` is
used to designate them. I only converted one string in config_jp to
serve as a test for the pipeline. I moved sotn_str to a folder and added
some tests. I renamed it since python doesn't like modules with - in the
name. The code could be cleaner and the conversion table could use some
more work but I think this is a reasonable start.
Mostly a proof of concept to extract resources from `BIN/F_GAME.BIN`.
There are tons of hacks to make this work:
* Tweaked png2s to decode binaries into PNGs
* Add a `config/gfx.game.json` that describes how to extract different
sprites out of a binary file
* Coded the palette location by reading the `clut` parameter in the
source code. A clut value of `0x196` corresponds to `"palette": "0x96"`
or `"palette": 150`.
* Coded the bits per pixel for a couple of 8-bit images
* Coded the palette location of those graphics that expects the palette
from DRA.BIN instead of F_GAME.BIN
* For all the undiscovered palette, I called the files `unk` and
defaulted their palette to grey
The expectation is that sprites will slowly be documented and added into
`config/gfx.game.json` by modders and enthusiasts. This does not yet
pack back these PNGs into the binary file. The way these assets are
extracted is probably not final or perfect, but we have to start from
somewhere.
Once decomp.me is updated I will update the SOTN preset to include
`--expand-li`. Let me know if you run into any issues but this builds an
🆗 binary for me.
