Still work in progress. I removed splat as a submodule and started using
it as a pip package instead. Everything is matching but the memory card
icons part in both DRA and SEL. I still have no idea what the issue is.
Once this PR is good to be merged, we can get rid of the splat fork too.
Having `platform: psx` will not correctly disassemble PSP functions with
opcodes that are not part of the R3000 (the PSX CPU) architecture. These
functions would be disassembled by Splat as a bunch of `.word
0xBLAHBLAH`.
Changing the platform to `psp` introduces all sort of new challenges.
Function prototypes needs to be declared earlier. But also the MWCC
compiler will not accept the `%lo` and `%hi` dialect from GNU AS. There
were some patches on `mwcpp.py` to use `la SYMBOL_NAME` that would
expand into a `lui / addiu` combo. But even though symbols needs to be
declared like function prototypes at the top of the file. This is simply
not feasible on bigger overlays.
As a solution to the problem above, I replaced the existing patches by
converting instructions into `.word`. The overlay cannot longer be
relocated with this approach, but it is not an issue as the final goal
is to decompile these functions any way.
The labels in the jump table has the same problem, which forced me to
change the segment type from `rodata` to `data.
This is just another single step to create the conditions to start
including bigger re-compilable PSP overlays. I am sure the `mwcpp.py`
solution will be thrown into the bin at some point, but this PR improves
our current situation.
TT_000 is the first overlay from PlayStation 1 that we are now able to
compile from the source and produce a 1:1 binary. This lead me to start
exploring the same overlay from the game Castlevania: Dracula X
Chronicles, which contains a PSP re-build of Symphony of the Night.
This PR adds all the infrastructure to add the same flow for a PSP
matching decomp. Use `export VERSION=pspeu` and then the usual `sotn`
command to splat the overlay, build it and check if it matches. Running
`make extract_disk` should not be necessary as the same ISO used from
`VERSION=hd` is also used for `pspeu`, so you will probably have it
extracted already.
Some important notes about the PSP build:
* The whole PSP build seems to be compiled with `-O0`, which makes it
much easier to decompile
* Having ŧhe PSX, PSP and Saturn builds will allow to easily
cross-reference the code and reduce fake matches
* `disks/pspeu/PSP_GAME/USRDIR/res/ps/hdbin/tt_000.bin` is the HD PSX
build
* `disks/pspeu/PSP_GAME/USRDIR/res/ps/PSPBIN/tt_000.bin` has the same
code from the HD build, but for PSP
* `disks/pspeu/PSP_GAME/USRDIR/res/ps/PACK_E/TP00.BIN` is the same as
above, but it packs both overlay and graphics. This is the file the PSP
game seems to actually use
* The PSP build uses the Metrowerk CodeWarrior's C compiler, which is
very different from the GCC one used on PSX.
* Thanks to @mkst lead, we found a way to still use the GNU assembler
and linker
* MWCC uses [wibo](https://github.com/decompals/WiBo/), a think
compatibility layer to run Windows CLI tools on Linux. It is much more
lightweight than Wine.
* MWCC does not support the `INCLUDE_ASM` dialect, so the custom
pre-processor `tools/mwcpp` had to be created
* The exact MWCC compiler version is unknown, but I suspect it is `build
147`
* I am not yet sure of any implications for using GNU AS and GNU LD
instead of the MW correspondent tools
* Not all the functions can be correctly disassembled, spimdisasm will
just produce a bunch of `.word 0x________` due to the in-progress effort
of adding the Allegrex-specific opcodes support
---
TO-DO list before marking the PR as ready:
- [X] Add PSP build to the CI
- [x] Add progress reporting to the PSP build
- [x] Integrate source file from `src/servant/tt_000_psp` to
`src/servant/tt_000` to promote the psp build as first-citizen
---
TO-DO in a follow-up PR:
* Figure out what `header` is: can we extract it as assembly code? or
maybe as custom re-compilable asset via splat? Is it a MW stuff or a
Castlevania-specific file?
* Get rid of the last line in `Makefile.psp.mk`
Occasional maintenance:
* asm-differ: latest commit
* m2c: latest commit
* maspsx: latest commit
* spimdisasm: 0.18.0 latest
* splat: 0.17.3
The latest version of splat is 0.19.1 but a breaking change in
[0.18.0](https://github.com/ethteck/splat/pull/294) is preventing me to
upgrade further ([discussion
here](https://discord.com/channels/710646040331681844/813939516385525790/1172978921000669274))
Some YAML were malformed and since splat 0.17.0 there are additional
checks to ensure they are compliant. There are also new checks that
prevents a malformed symbol list including duplicates, which I fixed
too.
Occasional maintenance:
* asm-differ: latest commit
* m2c: latest commit
* maspsx: latest commit
* spimdisasm: 0.18.0 latest
* splat: 0.17.3
The latest version of splat is 0.19.1 but a breaking change in
[0.18.0](https://github.com/ethteck/splat/pull/294) is preventing me to
upgrade further ([discussion
here](https://discord.com/channels/710646040331681844/813939516385525790/1172978921000669274))
Some YAML were malformed and since splat 0.17.0 there are additional
checks to ensure they are compliant. There are also new checks that
prevents a malformed symbol list including duplicates, which I fixed
too.
[As
mentioned](https://discord.com/channels/1079389589950705684/1135205782703570994/1154948845638254672),
since #595 some tools are not able to pick up some function names as the
symbol names are gone. This PR adds a quick tool to re-generate that
list from a map file and integrates it in the CI flow.
EDIT: CI is failing because the running Linter is from `master`, where
`mapfile-parser` is not installed.
There is a parsing bug in the mapfile-parser that prevents to parse the
customised weapon map to report the progress.
I noticed progress for weapon was not reported after merging #465 .
Apparently mapfile-parser is very sensible and it does not like map
files produced by custom linker scripts.
Splits `BIN/WEAPON0.BIN` in its own individual C files and assets. There
are 59 overlays 59 PNGs extracted as part of it. The `.data`, `.text`
and `.sbss` section are correctly split. I did not yet split the
`.rodata` but I assume it will be so small we can do it on-the-go.
This is fully integrated in the build process, it gives an 🆗 once
built back and it is fully integrated in the CI. We did not yet know how
weapons work in-game and we already have a few instances in the code
where we call `D_8017A000();` and similar. I think it is time to start
documenting them.
Thanks to @bismurphy <bismurphy@users.noreply.github.com> for addressing `analyze_calls.py` for this specific
PR.
This adds the ability to scrape decomp.me for functions. We just look
for ps1 and the same function name so collisions are possible. The
in-source entries are considered definitive and scraped entries are
labeled in the output so that it's known they may be less reliable.
There's many scratches that we don't have labeled in the source so this
should help avoid duplicated effort.
Here's an example of the output:
https://gist.github.com/sozud/2fa067f9434a326020f09f8fe6a2e75d
The idea behind this script is making it easier to keep track of what is
remaining to work on in the project. Remaining functions are sorted by
the length and number of branches, and whether they have jump tables or
not. This gives a sense of the difficulty of the function and should
make it easier for new contributors to find a function to work on.
Decomp.me WIP links are also included. The script looks for comments
with the following format and includes them in the output:
```
DECOMP_ME_WIP func_800F9E18 https://decomp.me/scratch/VmuNt
```
The output of the script looks like this:
https://gist.github.com/sozud/da6744b3fe58309e2d989ca4198e9515
It might be nice to integrate this with the CI as well. A further
improvement would be listing whether the function is a duplicate or not.
