Mostly an inconsequential function, but we can learn a whole lot from
this one.
1. Lots more symbols added to PSP. This function touches a lot of
previously-unseen things in PSP.
2. This is the first PSP function to print a string. Importantly, PSP
puts strings in `data`, not `rodata`! Luckily, the process of pulling
them in is simple and easy, as demonstrated here.
3. I modified the Makefile for the -O4 stuff. At this point I think DRA
is `O4` and all the overlays are `O0`. We can always make the logic more
complicated later if we need to.
4. We discovered a discrepancy in the size of DR_ENV on PSP. I suspect
we will have more SDK differences, but we will find them function by
function. I decided to set the size in a variable so that we can put all
the version differences at the top of the file, instead of them being
more scattered.
I think that's it!
Had to tweak some file splits, but ultimately I managed to decompile all
of the functions in a PSP file, and then line up the two DRA codebases
so that PSP could pull from the existing one. Of course, some of the
functions also needed tweaking to end up matching on PSP.
Glad to be able to prove this is possible in DRA!
Picked a random `.c` file from PSP to decompile.
This caused me to discover that this function requires `-O4,p`, which is
new. All the other -O4 files also match with this, which is good.
Also back-ported my changes to PS1.
Some tiny changes to the texture displaying tools that I made due to
xeeynamo finding a tiny bug; not worth a whole PR so I just tossed them
into this one.
Decided to tackle this whole file. It mostly matches with the existing
DRA, so I've been back-porting the required changes to get the matches.
Everything here matches in both.
The only issue is the deadstripping, which we still need to figure out.
I'm not sure why there's a file split at the end of this PSP file, where
there isn't one in the PS1 game. I'll try removing the split once the
functions are done on each side of the split.
Also adjusted the makefile filter logic so that we only need to give the
name of the file, rather than the full path. This makes the file list
more concise. If we run into collisions (two files of the same name, in
different overlays, which need different optimizations) we can revisit
this.
I picked a random function from PSP and figured out what it was in PS1.
Turns out it was one of these two.
`SetPrimRect` and `SetTexturedPrimRect` happen to be pulled into a .h
file already. The source code there required a line swap to get a PSP
match.
Interestingly, these are compiled with -O4. I'm guessing for now that
functions with -O4 are grouped together into the same .c files (that is,
some .c files were compiled with -O4, rather than some functions being
-O4). Therefore, I adjusted the PSP makefile so that we can specify what
object files use -O4. This is the same approach I used in Aironauts when
I found some functions being GCC 2.7 and some being 2.8. Not sure if
this will be the strongest approach long term, but I think that will
become more clear as we begin crunching through PSP.
I moved *some* of the PSX-specific stuff out of the common `Makefile`
and into the `Makefile.psx.mk` -- there is still more that could be done
here, but this should be enough to have the correct rules for PSX and
PSP with the desired assembler.
Also threw in a bonus feature where I've removed the `move` macro from
`macro.inc` because `maspsx` does this automagically for you now.
Merging `wrp_psp/warp.c` into `wrp/warp.c`.
I imported all the BSS data into `wrp_psp/bss.c`. I do not feel it is
yet the right time to split the bss section into the individual files as
I feel it might have a ripple effect to every other overlay.
I confirm empty arrays on PSX are in data, but on PSP they are in bss.
This is currently forcing me to do some `#ifdef` here and there. It
should get solved once the bss is split into their respective original
files.
I feel very positive about the file split in WRP. So far it seems to be
accurate. I really hope to spread this new pattern to other overlays
too. It should make importing new stage overlays in the future much
easier.
Quite a big one. I merged `EntitySoulStealOrb` across all overlays by
normalising the symbol names. The symbol types was all over the place,
so I had to fix that as well.
I solved the s16/u16 ifdef between PSX and PSP. The only part I couldn't
match without a ifdef was `angle = (angle - 1) * 8;` and `angle = (angle
* 8) - 8;`.
I also decompiled the exclusive PSP functions `func_psp_0923AD68` and
`func_psp_0923B2F0` and merged the PSX and PSP `e_particles.c` code. The
exclusive functions required a file split.
Straightforward PR. Interestingly importing the data doesn't seem to
result to a match ad there's some `0x10` aligned padding added. Either
we know too little about importing data with MWCC or the file split was
wrong and the data belonged together with other functions in another C
file.
For now I had to use a `#ifdef`. It will probably get forgotten until we
decide to import the data from WRP PSP.
Extract ST/WRP out of #1119 . All the function symbols should have been
cross-referenced. There as some PSX functions missing from PSP and some
new functions from PSP that are not present on PSX (e.g.
`st_init_wrp.c`).
The files `st_debug.c` and `e_breakable.c` are shared between WRP PSX
and WRP PSP. Everything else from PSP is isolated into its own folder. I
had to do some tricks on the YAML config to allow shared code.
`ST_WRP_MERGE = st_debug e_breakable` in the `Makefile` is a bit
annoying as MWCC complains about every single minute detail from the C
source that has been already decompiled for the PSX US build.
`EntityWarpSmallRocks` is matching on PSP but I couldn't extract the
rodata without having a bunch of linker errors. This might be a Splat
issue. I need to investigate further.
`func_psp_09244760` is soooo interesting. The values from `0x11` to
`0x17` matches the Entity IDs that are unique to the WRP overlay. This
aligns to what we have in `typedef enum EntityIDs`.
Overall I am very excited to the recent discoveries from the PSP build!
Highlights:
* `abs` is a real function the compiler can optimise and inline (thanks
@Mc-muffin for the discovery)
* `-Op` is required to avoid using `div` when dividing integer numbers
* A file split in TT_000 is required to match on PSP
* `UpdateAnim` had the wrong signature
* Splatting the PSP build has been an excellent idea
Matches `func_801713C8`.
This is an important milestone as it is the first function using the
`SEH` instruction. I got this new compiler (and many more) from someone
in ObscureGamers that kindly shared at
https://archive.org/download/compilers-no-license .
This still gives us no clue on what `f32` and the various `.i.hi` stuff
was actually meant to be.
OK. Here goes. Version 0.0.1 alpha of the MWCC Global Assembly Processor
(mwccgap). It's currently very simple/limited, but it appears to work
for `src/servant/tt_000/10E8.c`.
There is lot more that can be done to improve mwccgap - i.e. supporting
.rodata migration would be a good addition, but let's see how far we can
get with it in it's current state.
Note that the Makefile could do with some improvements - we don't nede
to use mwccgap for any C file that *dont* have INCLUDE_ASM macros (it's
a waste of time) so these could be ignored, i.e. for SSSV I do something
like this to find the files that need fixing up:
```
GLOBAL_ASM_C_FILES := $(shell $(GREP) GLOBAL_ASM $(SRC_DIR) </dev/null 2>/dev/null)
```
.. although this is perhaps too simple given that SOTN has a mix of PSP
and PSX functions (and therefore there may be INCLUDE_ASM for a PSX
function but none for PSP functions...
I improved the PSP Makefile to always remove `*.post.c` regardless if
the compiler fails or succeeds. This also helps when invoking `differ`
on a broken build.
The permuter uses the last Makefile line to create a `compiler.sh`
script used to make a bunch of iterations. Previously it was using `rm
$<.post.c || true` as compiler. Now that I compacted everything into a
single line it can at least import a C file. The produced `compiler.sh`
is still wrong as it cannot properly take in consideration the
`mwcpp.py` tool and the parameters to pass to it.
---
Before this fix:
```bash
$ ./import.py sotn-decomp/src/servant/tt_000/10E8.c sotn-decomp/asm/pspeu/servant/tt_000/nonmatchings/10E8/func_80171ED4.s
Compiler type: gcc
Function name: func_80171ED4
Compiler: python3 tools/mwcpp.py {input} -o {output}
Assembler: mips-linux-gnu-as -march=vr4300 -mabi=32 {input} -o {output}
src/servant/tt_000/10E8.c:1:10: fatal error: servant.h: No such file or directory
1 | #include <servant.h>
| ^~~~~~~~~~~
compilation terminated.
Failed to preprocess input file, when running command:
cpp -P -undef src/servant/tt_000/10E8.c -D__sgi -D_LANGUAGE_C -DNON_MATCHING -DNONMATCHING -DPERMUTER '-D_Static_assert(x, y)=' '-D__attribute__(x)=' '-DGLOBAL_ASM(...)=' '-D__asm__(...)='
```
With this fix:
```bash
$ ./import.py sotn-decomp/src/servant/tt_000/10E8.c sotn-decomp/asm/pspeu/servant/tt_000/nonmatchings/10E8/func_80171ED4.s
Compiler type: gcc
Function name: func_80171ED4
Compiler: python3 tools/mwcpp.py '&&' '((MWCIncludes=bin/' bin/wibo bin/mwccpsp.exe -gccinc -Iinclude -D_internal_version_pspeu -O0 -c -lang c -sdatathreshold 0 src/servant/tt_000/10E8.c.post.c '&&' rm 'src/servant/tt_000/10E8.c.post.c)' '||' '(rm' src/servant/tt_000/10E8.c.post.c '&&' exit '1))' {input} -o {output}
Assembler: mips-linux-gnu-as -march=vr4300 -mabi=32 {input} -o {output}
In file included from include/game.h:78,
from include/servant.h:4,
from src/servant/tt_000/10E8.c:1:
include/primitive.h:146:8: warning: extra tokens at end of #endif directive [-Wendif-labels]
146 | #endif PRIMITIVE_H
| ^~~~~~~~~~~
Preserving no macros. Use --preserve-macros='<regex>' to override.
usage: mwcpp.py [-h] [--version VERSION] [-o OUTPUT] input
mwcpp.py: error: unrecognized arguments: ((MWCIncludes=bin/ bin/wibo bin/mwccpsp.exe -gccinc -Iinclude -D_internal_version_pspeu -O0 -c -lang c -sdatathreshold 0 src/servant/tt_000/10E8.c.post.c && rm src/servant/tt_000/10E8.c.post.c) || (rm src/servant/tt_000/10E8.c.post.c && exit 1)) /tmp/permutereepb8_ih.c
Warning: failed to compile .c file.
```
TT_000 is the first overlay from PlayStation 1 that we are now able to
compile from the source and produce a 1:1 binary. This lead me to start
exploring the same overlay from the game Castlevania: Dracula X
Chronicles, which contains a PSP re-build of Symphony of the Night.
This PR adds all the infrastructure to add the same flow for a PSP
matching decomp. Use `export VERSION=pspeu` and then the usual `sotn`
command to splat the overlay, build it and check if it matches. Running
`make extract_disk` should not be necessary as the same ISO used from
`VERSION=hd` is also used for `pspeu`, so you will probably have it
extracted already.
Some important notes about the PSP build:
* The whole PSP build seems to be compiled with `-O0`, which makes it
much easier to decompile
* Having ŧhe PSX, PSP and Saturn builds will allow to easily
cross-reference the code and reduce fake matches
* `disks/pspeu/PSP_GAME/USRDIR/res/ps/hdbin/tt_000.bin` is the HD PSX
build
* `disks/pspeu/PSP_GAME/USRDIR/res/ps/PSPBIN/tt_000.bin` has the same
code from the HD build, but for PSP
* `disks/pspeu/PSP_GAME/USRDIR/res/ps/PACK_E/TP00.BIN` is the same as
above, but it packs both overlay and graphics. This is the file the PSP
game seems to actually use
* The PSP build uses the Metrowerk CodeWarrior's C compiler, which is
very different from the GCC one used on PSX.
* Thanks to @mkst lead, we found a way to still use the GNU assembler
and linker
* MWCC uses [wibo](https://github.com/decompals/WiBo/), a think
compatibility layer to run Windows CLI tools on Linux. It is much more
lightweight than Wine.
* MWCC does not support the `INCLUDE_ASM` dialect, so the custom
pre-processor `tools/mwcpp` had to be created
* The exact MWCC compiler version is unknown, but I suspect it is `build
147`
* I am not yet sure of any implications for using GNU AS and GNU LD
instead of the MW correspondent tools
* Not all the functions can be correctly disassembled, spimdisasm will
just produce a bunch of `.word 0x________` due to the in-progress effort
of adding the Allegrex-specific opcodes support
---
TO-DO list before marking the PR as ready:
- [X] Add PSP build to the CI
- [x] Add progress reporting to the PSP build
- [x] Integrate source file from `src/servant/tt_000_psp` to
`src/servant/tt_000` to promote the psp build as first-citizen
---
TO-DO in a follow-up PR:
* Figure out what `header` is: can we extract it as assembly code? or
maybe as custom re-compilable asset via splat? Is it a MW stuff or a
Castlevania-specific file?
* Get rid of the last line in `Makefile.psp.mk`
