This took me a while. I used a [throw-away
script](https://gist.github.com/Xeeynamo/58da1ff8f3831d0ba5d23da27cbca025)
to help me with the quest, but I still had to manually check every
single YAML subsegment.
I transported over what we were able to decompile from main. For a few
functions I added the signatures and documented existing DRA code,
especially on the sound department (got inspired by the recent @sozud
PRs).
All the `.text` part from `main.exe` is now completely extracted as
C/ASM where appropiate. Almost all the functions have their original
PSY-Q names but 7 of them: `func_80012DBC`, `func_80012F84`,
`func_80017008`, `func_80017078`, `func_8001929C`, `func_80021F0C` and
`func_800286E0`. I did not feel confident enough to rename them, so I
left them be. The rest of the functions I am 99% sure they are all
accurate.
I am now excluding the
[asm/](https://github.com/Xeeynamo/sotn-decomp/tree/master/asm/us/main/psxsdk)
folder from the repo. It was useless.
This research confirms me the game uses the PSY-Q 3.5 libraries, with
the exception of `LIBGPU.LIB`, which is from PSY-Q 3.0 for some unknown
reason.
EDIT: `make format` was not taking care of duplicated symbols in our
symbol list. To speed-up my work, all the duplicate symbols (duplicates
= name AND offset) are now removed.
Previously if either the YAML or the symbol list were changed, we were
forced to either manually delete the `asm` folder of the specific
overlay or to invoke `make clean`, where the latter forced us to
re-extract all the overlays via `make -j extract`.
That changes with this PR. How `make` works is `target: dep1 dep2` where
if one of the dependencies has the last modified date greater than the
target, the rule is triggered again. Previously we were extracting
overlays doing `make extract_stcen`. But since every overlay extraction
generates a linker script I now changed the rules to do `make
build/us/stcen.ld` to extract the same overlay. As I explained above, if
either the YAML or one of the related symbols changes, the linker
modified date will be older and the extraction for that overlay to be
trigger again.
The above allowed me to stop polluting the repo root with the linker
scripts as they are now moved into `build/us` or `build/hd` depending of
what you're trying to build. If you do `make extract` twice in a row you
will be welcomed with a `make: Nothing to be done for 'extract'.`.
There are still some instances where you **might** need `make clean`
beforehand, especially when modifying the `segment` section in the YAML
or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I
want to see if the current solution will be enough.
### Please try removing `make clean` from your workflow, once this is
merged, to quickly detect possible problems with this new approach.
I am also planning to make more substantial changes on our build-chain
like this PR or #660 in the future. I am aiming to small incremental
changes over time in case I break someone's flow or detect design flaws.
I am also considering the breaking changes introduced in make 4.4, which
will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a
deadline to finish all the new build-chain work.
---------
Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
## What is this
Required for #465 . I re-wrote how the build process works for the
`BIN/WEAPON0.BIN` ~~overlay~~ file.
## A bit of a background
On the PSP build I found `WEAPON0.BIN` is just a blob with `f0_000.bin`,
`w0_000.bin`, `f0_001.bin` and so on concatenated as a single file. `f0`
contains the graphics (or CHR) and `w0` contains the actual overlay (or
PRG). The individual overlays share a great amount of functions I
documented in `config/symbols.us.weapon.txt`.
## The problem
Splat creates its own `weapon.ld` linker script, insisting to link all
the overlays as a single binary file. While this is not technically
correct, it creates a bigger problem: symbols cannot be shared. In 58
overlays there are 58 `DestroyEntity`. Other decomps are
circumnavigating the issue by adding a prefix. Following the same
pattern we could have `W000_DestroyEntity`, `W001_DestroyEntity` and so
on. But this is not maintainable and it does not reflect our long-term
vision of having shared code between overlays.
## The solution
I created the dedicated linker script `weapon0.ld` to output the
individual overlay `w0_000.bin` and similar. `f0_000.bin` is also
created from the correspondent PNG. Once all the overlays and graphics
are created I am simply concatenating them back to `WEAPON0.BIN`. This
is much more efficient in terms of separation of concerns,
maintainability, build speed and it also allows to use the same symbol
names across overlays. Also very important, the generated individual
overlays matches the structure of what is found in the PSP build.
## Side note
I thought it was wise to prepare the Makefile and the linker script to
also allow rebasing and compiling for WEAPON1.BIN. The generated file
does not currently match and it will probably need a bit more research.
I left the build infrastructure there. To compile the non-matching
WEAPON1 just modify `$(BUILD_DIR)/WEAPON0.BIN: $(WEAPON0_FILES)` to
`$(BUILD_DIR)/WEAPON0.BIN: $(WEAPON0_FILES) $(WEAPON1_FILES)`.
# What
Adds a custom debug menu. It is written in C and it is meant to replace
the Bat familar `SERVANT/TT_000.BIN`. Once loaded you can long-press
SELECT+START to soft-reset the game and keep using the debug menu
everywhere, including when playing with Richter or during the credits.
One key requirement to run this is to have an emulator that emulates the
8MB of RAM. This is a key requirement to have the debug module surviving
soft-resets or accessing to the in-game menu. I personally used [PCSX
Redux](https://github.com/grumpycoders/pcsx-redux) to build this module.
I am not sure about the compatibility with other emulators. This does
**NOT** work on real hardware and it is a choice by design. The debug
module is intended to test different areas of the game and help
decompiling. It is not intended to be used in normal gameplay.
# Build
Simply invoke `make disk_debug` to create a disk image of the game in
`build/` with the debug module replacing the Bat familiar.
# Usage
On PCSX Redux go to Configuration, Emulation and tick the box `8MB`.
## Loading the module
You need to enable the Bat Card from the menu. This will load the debug
module from the disk. If you want to re-load the module you need to
select another Familar Card, un-pause, pause again and select the Bat
Card once more.
## The main screen
You will know you have loaded the debug module when you see the blue
rectangle on the top right. You can press R2 to cycle between the menus.
Some menus will temporarily freeze the game, some not. To quickly return
in-game you can either press TRIANGLE or START. To bring back the paused
debug menu just press R2 once again.
## Debug Mode
### Stage
Teleports the player to a different stage. It is not stable and it can
crash often.
### Player
Switches between Alucard and Richter. Currently switching from Richter
to Alucard consistently crashes the game.
### No Clip
Allows to freely move the player within the room and without the
collisions on. Once the flag is enabled from the debug menu, pressing L2
while in-game will temporarily freeze the player movement and make it
immune to collision checking. You can then press the directional buttons
to slowly move the player or you can hold CROSS to move it faster. You
can also use SQUARE or CIRCLE to cycle between the player frames. Press
L2 again to deactivate the NoClip mode.
### Frame by frame
Freezes the game outside the debug module. Press L1 to advance by 1
frame. Hold L2 to put the game in slow-motion.
### Show hitboxes
As shown in the image
### Show debug messages
When the debug menu is un-paused, prints on the top left all the debug
messages from the game itself.
### Show collision layer
This prints the internal collision value for every 16x16 tile on the
screen. Look the CheckCollision function for more information on how
each printed value is used.
### Show draw calls
Shows the maximum GPU resource usage since the game started. Currently
only the `max` option works. The `current` option will not show
anything.
### Show HBlank
Prints the current horizontal blank interrupt count.
## Entity Spawn
Allows to immediately spawn new entities in the current map
### Mode
There are three list of entities, each one with their own ID: DRA, Stage
and RIC. The option RIC is hidden if Richter is not the current playing
character. As the list of entities per stage is maintained manually,
stage entities might not be available for all stages. The `Alloc` shows
how many entities are reserved or actually used. Pressing the SQUARE
button here will destroy all the entities within that range.
### ID
Press Left or Right to cycle between the different IDs available. Some
of them might crash the game immediately once spawned. Press CROSS to
immediately spawn the entity.
### Params
Each entity might have its own parameters. Sometimes the flag 0x8000 is
used, which can be toggled with the SQUARE button. Press CROSS to
immediately spawn the entity.
### Entity preview
Shows the entity before spawning it. This is turned off by default as it
can immediately crash by cycling through the available entity IDs.
### Place entity
Pressing CROSS will allow to move the entity across the screen before
placing it. Press CROSS again to place the entity and return to the
previous screen. Press SQUARE to quickly place multiple entities of the
same type.
## Sound player
There are three macro categories the sound player is split into. For
what is currently known only the sounds within the Kind 3 changes based
on the loaded stage.
### Stop all sounds
This will also disable the SPU IRQ, effectively unlocking the frame
rate.
### Load Stage
Loads a different sound font than the current loaded stage. This can
help to quickly preview and test SFXs from other stages without
necessarily moving the player there.
### Load Servant
Loads the sound font of a specific servant without necessarily equipping
the Familiar Card.
## Castle flags
Preview all the flags used to modify the behaviour of different parts of
the two castles.
### Edit mode
You can move the cursor with the directional buttons and flip the flag
with CROSS. Press L1 or R1 to cycle between the pages. The cursor warps
when reaching the border of the flag grid, allowing a faster navigation.
### View mode
Allows to move between the flags more flexibly.
### Listen mode
Listens for the modified flags while playing. Every time a flag is
modified the offset and its value is registered on the top left up to 4
rows. When all the rows are occupied, new values will just remove the
oldest one. The last modified flag will always be displayed at the
bottom.
---------
Co-authored-by: Alejandro Asenjo Nitti <sonicdcer@users.noreply.github.com>
Co-authored-by: sozud <sozud@users.noreply.github.com>
Fully splits `libsnd` into individual files following the original PSY-Q
SDK file names. The same is done with `libcd` and `libspu` just to
define the boundaries of `libsnd`. I also correctly names **all** the
function symbols from the sound library with their right offset. We
previously had some wrong symbol offsets, leading to the inability to
create their C counterparts with `INCLUDE_ASM`. This PR will start to
open the doors to decompiling `main` or understand better how `DRA`
communicates with `main`.
This took an incredible amount of time as I had to do it manually. I
would have written a script, but there are some variants I would prefer
to handle manually. To achieve this I used the [PSY-Q
signatures](e9e46e7e13/350/LIBSND.LIB.json).
I also discovered the game uses PSY-Q 3.5, which brings some piece of
mind to what they used to develop the game.
After this PR my plan is to:
* Finish to split the other PSY-Q libs
* Move `asm/main/*.s` into their own C files
* Include `main` progress in our reporting
Following the incredible work of @bismurphy in #396, this will integrate
the tool in the CI. The markdown file `gh-duplicates/functions.md` will
have clickable links to navigate right to the SVG call tree of the
correspondent functions.
---------
Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
Very happy to have this working!
Now, when you run `python3 analyze_calls.py ALL`, it will generate the
call graphs for every function in the game, and save as an SVG file in
the directory `tools/generated_graphs`. It will then finish by creating
an `index.html` that links to every one of those graphs. Each graph's
bubbles are clickable to take you to the next graph.
I have deployed an example of the result of this, linked in the sotn
decomp Discord, but because it is hosted on my personal computer, I will
not be linking it here (accessible to the wider internet and crawlers,
etc). It would be great if this script could be worked into the CI to
create a publicly visible list of functions linking to graphs, which
would automatically update whenever the repo gets updated.
As always, be sure to run `make force_extract` before invoking this.
It will be very exciting once this is up and running!
This is the script demonstrated in Discord, with both graphical and
command line interfaces.
I won't say it's particularly good Python, but it gets the job done for
now :)
This traverses all functions in /asm (after doing `make force_extract`),
and logs all functions called by every function. This takes several
seconds, so the result of this traversing is saved to a text file called
sotn_calltree.txt and used for future runs. I've added this to the
gitignore. As mentioned in the comments at the top of the Python file,
this file can be deleted if the user would like to re-generate the call
tree (mainly if symbols have been renamed, which would change the
generated assembly files).
This is very much meant to be a first draft; it may have bugs and it
definitely has ugly, low-quality code (I had to do a whole bunch of
special cases for assembly that calls functions indirectly, for
example), but at least for now this could be a useful starting point.
Hopefully myself or others will make improvements into the future!
This adds some new Makefile commands:
build_saturn_toolchain: Downloads GCCSH, and builds two docker
containers. One container is binutils for SuperH, and the other has
Dosemu.
build_saturn: Copies GCCSH, the source code, and asm into the build
directory and runs the GCCSH via Dosemu in the docker container.
Everything is kept in the same directory since it's hard to use Dosemu
otherwise.
check_saturn: Dumps the object files to binary using the binutils Docker
container, then checks the hashes.
diff_saturn: Compares the binaries and outputs a diff to
build/saturn/$FILENAME-diff.txt. Used like FILENAME=GAME.PRG make
diff_saturn.
I've added the source for game.c and t_bat.c. I've replaced one of the
asm functions in t_bat.c to show that it's possible to match simple
functions. I'm not able to generate linker scripts yet so functions that
rely on symbols from other overlays generally can't be matched yet.
This workflow is not all that clean so I'm open to ideas on improving
it. I tried using various versions of
https://github.com/decompals/old-gcc to try to get a native compiler but
couldn't get a matching build. I'm not 100% sure that this version of
GCCSH is correct but initial results seem promising.
As of right now, it's not possible to run this in CI since the dosemu
container changes the USER and Github won't let you write to disk if you
do that.
https://docs.github.com/en/actions/creating-actions/dockerfile-support-for-github-actions#user
