Commit Graph

163 Commits

Author SHA1 Message Date
Luciano Ciccariello
151346d62c
Update clang-format as self-contained binary (#1257)
Version 18.1.6 pulled from
https://github.com/xeeynamo/sotn-decomp/releases and compiled with the
following:

```Dockerfile
FROM alpine:latest
RUN apk add --no-cache \
    build-base \
    cmake \
    git \
    ninja \
    libstdc++ \
    zlib-dev \
    python3
ENV LLVM_VERSION=llvmorg-18.1.6
RUN mkdir -p /opt/llvm \
    && cd /opt/llvm \
    && git clone --depth=1 --branch $LLVM_VERSION https://github.com/llvm/llvm-project.git \
    && mkdir -p llvm-project/build \
    && cd llvm-project/build \
    && cmake -G Ninja \
        -DLLVM_ENABLE_PROJECTS=clang \
        -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_CXX_FLAGS="-static-libstdc++ -static-libgcc" \
        -DCMAKE_EXE_LINKER_FLAGS="-static" \
        -DLLVM_ENABLE_RTTI=ON \
        ../llvm \
    && ninja clang-format \
    && strip bin/clang-format
FROM scratch
COPY --from=0 /opt/llvm/llvm-project/build/bin/clang-format /clang-format
ENTRYPOINT ["/usr/local/bin/clang-format"]
```
`docker build --output . .`

Original source code:
https://github.com/llvm/llvm-project/tree/llvmorg-18.1.6
commit: `1118c2e05e67a36ed8ca250524525cdb66a55256`
2024-06-04 21:07:50 +01:00
Luciano Ciccariello
5362c8b8da
Add ST/WRP from PSP (#1172)
Extract ST/WRP out of #1119 . All the function symbols should have been
cross-referenced. There as some PSX functions missing from PSP and some
new functions from PSP that are not present on PSX (e.g.
`st_init_wrp.c`).

The files `st_debug.c` and `e_breakable.c` are shared between WRP PSX
and WRP PSP. Everything else from PSP is isolated into its own folder. I
had to do some tricks on the YAML config to allow shared code.

`ST_WRP_MERGE = st_debug e_breakable` in the `Makefile` is a bit
annoying as MWCC complains about every single minute detail from the C
source that has been already decompiled for the PSX US build.

`EntityWarpSmallRocks` is matching on PSP but I couldn't extract the
rodata without having a bunch of linker errors. This might be a Splat
issue. I need to investigate further.

`func_psp_09244760` is soooo interesting. The values from `0x11` to
`0x17` matches the Entity IDs that are unique to the WRP overlay. This
aligns to what we have in `typedef enum EntityIDs`.

Overall I am very excited to the recent discoveries from the PSP build!
2024-05-25 11:56:06 -07:00
Mark Street
b6a0ddbeb6
Add mwccgap (#1131)
OK. Here goes. Version 0.0.1 alpha of the MWCC Global Assembly Processor
(mwccgap). It's currently very simple/limited, but it appears to work
for `src/servant/tt_000/10E8.c`.

There is lot more that can be done to improve mwccgap - i.e. supporting
.rodata migration would be a good addition, but let's see how far we can
get with it in it's current state.

Note that the Makefile could do with some improvements - we don't nede
to use mwccgap for any C file that *dont* have INCLUDE_ASM macros (it's
a waste of time) so these could be ignored, i.e. for SSSV I do something
like this to find the files that need fixing up:
```
GLOBAL_ASM_C_FILES := $(shell $(GREP) GLOBAL_ASM $(SRC_DIR) </dev/null 2>/dev/null)
```
.. although this is perhaps too simple given that SOTN has a mix of PSP
and PSX functions (and therefore there may be INCLUDE_ASM for a PSX
function but none for PSP functions...
2024-05-18 11:41:00 +01:00
Jonathan Hohle
76d1584c89
Replace BIOS Trampolines (#1133)
This replaces several dozen BIOS trampolines from the extracted ASM to
"decompiled" source. These are modeled on the `INCLUDE_ASM` macro, but
generate the instructions necessary for each trampoline directly instead
of importing an extracted source file.

Because these trampolines never return, and GCC 2.6 doesnt appear to
have builtins for leaving off the return jump postamble, these will
likely need to remain assembly.

This also changes the `main.elf` target to depend on `main.ld`, and
undefined symbols files, allowing `make build` to regenerate those files
if necessary.

Co-authored-by: Jonathan Hohle <jon@ttkb.co>
2024-05-17 08:45:26 +01:00
Jonathan Hohle
20a1fb6998
Support Single BIN CUE Files (#1127)
In CUE files `FILE` is a stateful, global declaration that applies to
all following `TRACK`s. `sotn-disk` was treating `FILE` declarations as
`TRACK` delimiters which would result in incorrect parsing of CUE files
with a single `FILE`, but multiple tracks.

Now when reading a `FILE` declaration, the path is stored and processing
continues. A `TRACK` declaration will use the previously defined path,
and if a previous track had been started, append that previous track to
the list.

This also builds `sotn-disk` using the local repository instead of
pulling the latest commit from GitHub. The target depends on `sotn-disk`
sources and will rebuild as necessary (or with `make
~/go/bin/sotn-disk`).

As an aside,
[pull/232](https://github.com/Xeeynamo/sotn-decomp/pull/232) ran into
this same error, but fixes in a slightly different way that leaves side
effects that may make supporting things like `INDEX` (for extracting the
placeholder audio, for example) more error prone in the future.

Co-authored-by: Jonathan Hohle <jon@ttkb.co>
2024-05-14 19:59:56 +01:00
Luciano Ciccariello
15865c38ba
Completes w_051 (#1122)
All the stuff marked as `static` does not have the `g_` prepended as by
definition they are not global
2024-05-12 18:31:16 +01:00
Luciano Ciccariello
aa6c5692fb
More TT_000 PSP functions matching (#1115) 2024-04-23 20:06:05 +01:00
Luciano Ciccariello
9e7e87eaac
Add TT_000 from the HD version (#1111)
100% decompiled.

`US` adds some padding to have the file exactly 40KB long. The HD
version does not add any padding, hence why I ended up producing
`tt_000_raw.bin`. I would have used just `tt_000.bin` as a name as found
in `disks/pspeu/PSP_GAME/USRDIR/res/ps/hdbin/tt_000.bin`, but on
Windows-based file system it would collide with `TT_000.BIN` due to the
OS having case insensitive names.

I modified `make clean` as I found annoying that `VERSION=hd make clean`
would wipe out `us` build stuff.

The only different function in HD is `ProcessEvent`, which has a weaker
check. Another hint suggesting HD being older than US.

`s32 _unused[26];` added enough padding in the bss section to get an
🆗 . I am pretty sure it is unused data because the final binary is
not aligned by any power of 2.
2024-04-04 22:28:57 +01:00
Mark Street
a5ffd53cff
Update maspsx (#1100)
Also tweaked the Dockerfile since this is what I use to build the
project.
2024-03-15 14:04:57 -07:00
Mark Street
dfc74d4acd
Migrate .rodata to functions for main.exe (#1078)
I feel like getting all the rodata migrated will make it easier to
update splat (also makes it easier to match functions that use rodata).
2024-02-24 11:35:01 +00:00
Mark Street
41ad71e388
Pass undefined_syms files to linker (#1065)
Creating a PR for this, will look at updating splat in a separate PR
2024-02-18 14:57:03 +00:00
Luciano Ciccariello
78953c01aa
Allow spritesheet to be extracted from the CLI (#1035)
A side-project I've been working on that helped me understanding some
structures that are now a bit better documented in #1034 . Richter
sprites are extracted as usual. Doing `python3
tools/splat_ext/spritesheet.py decode disks/us/BIN/ARC_F.BIN assets/arc
alucard disks/us/BIN/F_GAME.BIN 1` can extract Alucard sprites too.
There is no repacking yet as I want to avoid using splat for this one.
2024-02-05 21:57:48 +00:00
Luciano Ciccariello
8eb7795b4f
Decompile some PSX SDK functions (#1002) 2024-01-21 17:58:32 +00:00
sozud
abca5d44d2
POC for JP sotn_strs (#986)
This is a proof-of-concept for Japanese sotn_strs. The macro `_SJ` is
used to designate them. I only converted one string in config_jp to
serve as a test for the pipeline. I moved sotn_str to a folder and added
some tests. I renamed it since python doesn't like modules with - in the
name. The code could be cleaner and the conversion table could use some
more work but I think this is a reasonable start.
2024-01-17 08:27:17 +00:00
sozud
23a143b295
Mark SEL handwritten functions as asm (#977) 2024-01-14 11:27:34 -08:00
Luciano Ciccariello
70d2a82688
Fix reporting for the RIC overlay (#880)
Re-generate symbols for `build/us/ric.map`. Also I moved the
`--no-default` as it was previously ignored.
2023-12-24 12:29:21 -05:00
Mark Street
fd49b82d1f
Update maspsx (--expand-li) (#877)
Once decomp.me is updated I will update the SOTN preset to include
`--expand-li`. Let me know if you run into any issues but this builds an
🆗 binary for me.
2023-12-20 21:13:37 +00:00
ser-pounce
ce30caa958
Makefile refactoring (#866)
Needed a break from all the asm. I realize that Makefile changes aren't
always appreciated as they can break things in unexpected ways,
especially if they come from newbies to the project (ahem), so feel free
to close this if it's not needed.

One comment about removing `/**/*.s` and `/**/*.c`: as far as I could
see there were no subdirectories, so didn't seem needed.
2023-12-18 21:22:42 +00:00
Luciano Ciccariello
21f158a7c7
Load assets json (#851)
Uses a small JSON parsing library called cJSON (MIT license) to parse
the assets `equipment.json` and `accessory.json` and load them in memory
rather than hard coding it or pre-building it.
2023-12-17 14:10:11 +00:00
Luciano Ciccariello
b851d5e4d2
Import 4A538, demo and 5087C data (#835)
As per title. I also added a new tool called `dirt_patch`. As I
mentioned in our Discord server there is some left-over data from
previous dev builds in DRA.BIN and potentially elsewhere too. The tool
uses the new file `config/dirt.us.json` which stores a list of patches
to avoid crazy hacks and `#ifdef` for the sake of getting a match. I
hope this tool will not be abused.
2023-12-12 10:36:34 -08:00
Luciano Ciccariello
e9bed91931
Extract memory card icons (#166)
Extract the memory card icons out of DRA.BIN and SEL.BIN.


![image](https://user-images.githubusercontent.com/6128729/227794798-4dd071f9-512b-4c72-8f5d-fcbf7a615519.png)

I also took the opportunity to extract `g_MemcardPortMask` into its own
source file `save_mgr_pre.c`. Unfortunately this is required to keep the
original data order. This suggests that most likely the icon palette and
bitmap were baked into the original `save_mgr.c` as byte arrays. I
decided to take a different approach and extracted them as PNGs for
better moddability.

I had to spin-off Splat into a new fork due to some breaking changes on
0.18.0.
2023-12-11 23:12:13 -08:00
Luciano Ciccariello
c123b1819d
Sound test: add 42398.c (#799)
In Italian we say "Abbiamo fatto 30, facciamo 31". It literally means
"We come all the way up to 30, it wouldn't cost nothing to push a little
further for 31".

I moved `log.h` into `include/` to easily logging stuff elsewhere and
without weird `#ifdef VERSION_*` by adding `-DNO_LOGS` in the main
Makefile.

This is a bit hacky, but it does the trick. I had to expand `D_80138784`
otherwise I would get a segfault.

Both `SoundInit` and `func_801361F8` are called in the main, so I
removed them. To restore the previous behaviour and isolate the sound
engine, just replace `MainGame()` in the `main.c` with a `SoundInit()`
and then a loop that calls `UpdateGame()`.

Last, but not least, SDL2 is in.
2023-11-20 18:17:51 -08:00
Luciano Ciccariello
574c23f981
TT_000 func_80172120 (#791)
Second last function of TT_000 decompiled.

* Add `printf '\x00' | dd of=$@ bs=1 seek=40959 count=1 conv=notrunc` to
ensure the output binary is 40960 bytes long (useful once BSS is
migrated)
* Add `$(BUILD_DIR)/tt_%.ld` so every time the splat YAML changes, doing
`make tt_000` triggers the extract again
* All the variables from BSS are now in `10E8` and in order. Removing
`extern` from will properly create the BSS section
* Fix 7A4D0.c to not use the familiar entity extension
* Delete the fake symbols `D_80170660` and  `D_8017065C`
* Replace all the instances of `ext.generic` as `ext.bat` when possible

NOTES:
* For `SquareRoot12((...) << 0xC) >> 0xC;` I was not able to use the
macro `FLT`
*
2023-11-19 10:14:32 -08:00
bismurphy
b3c0524bc8
Change assets.py to output assembly (#752)
Issue was raised on discord, this should correct for it.

Previously assets.py would output raw binary data. This makes it more
difficult for the build chain to understand.

Now, it outputs an assembly file (which is just a big list of `.byte`
entries) which will represent the parsed data, and that assembly data
then gets assembled. Therefore, the symbols now exist.

In order to make this work, I adjusted the format of the splat, so now
we have lines like `[0x7718, assets, accessory, g_AccessoryDefs]` where
the entries are [Address, assets, configuration json file to use, name
of the output symbol].

I think this addresses the issue, but I am more than happy to iterate on
this if needed.
2023-11-07 22:05:07 +00:00
Luciano Ciccariello
81f85b195a
Extract and document all functions from main.exe (#736)
This took me a while. I used a [throw-away
script](https://gist.github.com/Xeeynamo/58da1ff8f3831d0ba5d23da27cbca025)
to help me with the quest, but I still had to manually check every
single YAML subsegment.

I transported over what we were able to decompile from main. For a few
functions I added the signatures and documented existing DRA code,
especially on the sound department (got inspired by the recent @sozud
PRs).

All the `.text` part from `main.exe` is now completely extracted as
C/ASM where appropiate. Almost all the functions have their original
PSY-Q names but 7 of them: `func_80012DBC`, `func_80012F84`,
`func_80017008`, `func_80017078`, `func_8001929C`, `func_80021F0C` and
`func_800286E0`. I did not feel confident enough to rename them, so I
left them be. The rest of the functions I am 99% sure they are all
accurate.

I am now excluding the
[asm/](https://github.com/Xeeynamo/sotn-decomp/tree/master/asm/us/main/psxsdk)
folder from the repo. It was useless.

This research confirms me the game uses the PSY-Q 3.5 libraries, with
the exception of `LIBGPU.LIB`, which is from PSY-Q 3.0 for some unknown
reason.

EDIT: `make format` was not taking care of duplicated symbols in our
symbol list. To speed-up my work, all the duplicate symbols (duplicates
= name AND offset) are now removed.
2023-11-06 14:53:27 -08:00
Luciano Ciccariello
28b44193c5
Replace RoomDimensions with Tilemap (#730)
Failing to decompile [RenderTilemap](https://decomp.me/scratch/WigVS)
made me realise there were a few fields and structures I have always
been suspicious to be part of the same structure. After
cross-referencing overlapping fields from different structures, I was
able to merge `D_80073088`, `g_Camera`, `D_8007309C`, `g_CurrentRoom`
and `g_CurrentRoomTileLayout` into the new `g_Tilemap`.

I was forced to touch the majority of the code-base, which gave me the
opportunity to standardise some field names (e.g. from
`currentRoomTileLayout`, `roomLayout`, `layout`, `t` into `tilemap`),
remove some fake code, redundant code and adjust some symbols.
2023-11-04 16:43:45 +00:00
Luciano Ciccariello
b75b6c3777
SEL import data/rodata (part 1) (#663)
I started importing the data in SEL also as a proof of concept we can
also import the SOTN-specific formatted strings. For that I integrated
the new tool `tools/sotn-str.py` into the build chain. It works like the
following:

* `./tools/sotn-str.py parse disks/us/ST/SEL/SEL.BIN 27598`: read the
binary content in a specific offset and parse it as a readable Unicode
string.
* `cat src/st/sel/2C048.c | ./tools/sotn-str.py process` converts the
Unicode string into a byte sequence that can be understood by the SOTN
engine.

There is a dirty hack I did to make this work:
```c
#ifndef SOTN_STR
#define _S(x) (x) // Strings processed by tools/sotn-str.py
#endif
```
When invoking `mipsel-linux-gnu-cpp` I am passing -DSOTN_STR,
effectively ignoring `_S` in the pre-processor. Then I am passing it
into `sotn-str` before `iconv` to resolve any string surrounded by `_S`.
I am a bit of afraid this might be annoying when using the permuter. I
tried doing `$(SOTN_STR) -f $< | $(CPP)` but apparently `cpp` does not
work well with content passed via stdin.
2023-10-10 21:29:30 +01:00
bismurphy
42d1ed319a
Rework extraction of equipment and accessory (#664)
When extracting equipment and accessories, we use scripts in
tools/splat_ext called accessory.py and equipment.py, each called with a
different makefile rule. These files have a lot of "data in code", with
function calls matching the structure of the equipment and accessory
structs. They also involve repeating the structure both in the
extracting from the game, and the rebuilding into the compiled
executable.

This PR takes the equipment and accessories, and turns them into a
single generic makefile rule, which calls a new script called
`assets.py`. We change the splat yaml to match. To control the
extraction, we add new _config.json files in the splat_ext. This means
we only need to define the structure of each of these in a single,
localized place.

If we like this new approach (which should be more flexible), I will see
whether it can work for more of the assets in that same area of the
makefile. I will then see about adding extraction of enemies (in
g_EnemyDefs) next.

assets.py is adapted from the old equipment.py with a lot of changes to
make it work more flexibly. I'm hoping it will be a nicer path forward
into the future.
2023-10-06 23:05:10 +01:00
Luciano Ciccariello
fa21ce1442
Enhance make extract on PSX (#662)
Previously if either the YAML or the symbol list were changed, we were
forced to either manually delete the `asm` folder of the specific
overlay or to invoke `make clean`, where the latter forced us to
re-extract all the overlays via `make -j extract`.

That changes with this PR. How `make` works is `target: dep1 dep2` where
if one of the dependencies has the last modified date greater than the
target, the rule is triggered again. Previously we were extracting
overlays doing `make extract_stcen`. But since every overlay extraction
generates a linker script I now changed the rules to do `make
build/us/stcen.ld` to extract the same overlay. As I explained above, if
either the YAML or one of the related symbols changes, the linker
modified date will be older and the extraction for that overlay to be
trigger again.

The above allowed me to stop polluting the repo root with the linker
scripts as they are now moved into `build/us` or `build/hd` depending of
what you're trying to build. If you do `make extract` twice in a row you
will be welcomed with a `make: Nothing to be done for 'extract'.`.

There are still some instances where you **might** need `make clean`
beforehand, especially when modifying the `segment` section in the YAML
or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I
want to see if the current solution will be enough.

### Please try removing `make clean` from your workflow, once this is
merged, to quickly detect possible problems with this new approach.

I am also planning to make more substantial changes on our build-chain
like this PR or #660 in the future. I am aiming to small incremental
changes over time in case I break someone's flow or detect design flaws.
I am also considering the breaking changes introduced in make 4.4, which
will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a
deadline to finish all the new build-chain work.

---------

Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-10-05 09:26:29 +01:00
Luciano Ciccariello
45a71c03ac
Rewrite Saturn make rules (#660)
## What

Enhance the existing `Makefile` to build new rules for the Saturn side
of the decomp. This should allow more flexibility when adding new
overlays or when tuning existing rules.

## Changes

I separated part of the Saturn build process in the separate file
`Makefile.saturn.mk`. I realise that naming it `saturn.mk` would have
been enough, but I pre-pended `Makefile.` so it can be found right below
the main `Makefile` when listing files in an alphabetic order. I plan to
do the same with the psx and psp toolchain, therefore you will find
`include Makefile.*.mk` in the main `Makefile`.

I deleted all the game building process done with Docker. Now that we
have an established way to do it natively I think it is no longer
required. We can always run the entire buildchain within a Docker
container instead of having `_native` and `_docker`. Now all the
`_native` references are removed. `build_saturn_native` is now
`build_saturn`.

`check_saturn` is no longer responsible of stripping the ELF into a
binary. That is now part of `build_saturn`.

I removed the references to `_li.o` (e.g. `alucard_li.o`) and used
`.elf` instead, which is closer to how the PSX build chain works. If
`_li.o` was a better preference, please let me know.

I am no longer using `./compile_dosemu.sh`. Instead I am using the new
`$(DOSEMU)` to directly invoke the tool within the Makefile. I have done
that to reduce the amount of dependent files.

I tried minimising duplication as much as possible. We now have a list
of overlays found in `SATURN_OVL_TARGETS`. Each expected output triggers
a series of dependencies so seamlessly build everything. `Makefile` is
smart enough to call `$(SATURN_TOOLCHAIN)` only once. If the game was
already built but just one source or symbol file changed, triggering a
new `build_saturn` will only compile the modified overlay.

The Saturn ADPCM files are now extracted in `assets/` instead of
`build/`. I think `assets/` should contain all the uncompressed and
uncooked files. The list of PCM file is not hardcoded. Instead I am now
using `$(wildcard disks/saturn/SD/*.PCM)`. This now means the tool tries
to convert PCMs from `SDD0.PCM` to `SDF0.PCM` with no success. As the
tool is silently failing I decided to leave it as I wrote it.

## Problems

I rewrote everything thinking about concurrency in mind. But `make -j
build_saturn` gives some unexpected output on `stdout`. I did not dig
too much into it. I suspect it might be dosemu. This is not a stopper as
we were not using `-j` when building the game anyway.

I also noticed doing `VERSION=saturn make build` calls
`mipsel-linux-gnu-ld` for `stage_02`. I suspect it is calling the rule
`$(MAIN_TARGET).elf: $(MAIN_O_FILES)` and simply moving `include
Makefile.*.mk` above it should fix it. But then it would cause the same
problem if I split the PSX rules into their own separate file. We never
used `make build` by setting the env variable `VERSION`, so this is not
either a breaking change or a stopper.

## Post thoughts

I am happy with what I achieved so far. I used the knowledge I
accumulated when maintaining the PSX counterpart. Since I now better
understand how `make` works, I was able to make some better decisions in
the Saturn counterpart. For example triggering a new build when the
symbol list changes is something the PSX build chain lacks of. I think
in the future it would be nice to trigger `make extract` when either the
YAML or the symbol list changes.
2023-10-03 21:42:08 +01:00
sozud
db74ae3b91
Saturn: Decompile entrypoint, don't strip elfs (#659)
I removed the stripping so these can be loaded into Ghidra with symbols.
2023-10-02 06:47:44 -07:00
sozud
0e81d9a723
Saturn: Add Alucard overlay (#658) 2023-10-01 16:26:24 -07:00
Luciano Ciccariello
ff1c3dc64c
Regenerate orphaned symbols in the CI (#625)
[As
mentioned](https://discord.com/channels/1079389589950705684/1135205782703570994/1154948845638254672),
since #595 some tools are not able to pick up some function names as the
symbol names are gone. This PR adds a quick tool to re-generate that
list from a map file and integrates it in the CI flow.

EDIT: CI is failing because the running Linter is from `master`, where
`mapfile-parser` is not installed.
2023-09-24 12:55:33 -04:00
Luciano Ciccariello
ee4a372906
Remove orphan symbols (#595)
Allows to not hard code the location in-memory of decompiled functions
and imported data if not required. This allows to relocate the
referenced symbols when editing the original code or game data. If you
were getting the offset of those symbols from the symbol list in
`config/` I suggest to use the built `build/us/*.map` file instead.
2023-09-16 10:40:40 +01:00
Luciano Ciccariello
acafc3414f
Decompile StoreSaveData (#571)
This took me a monumental effort. The PR does:
* Decompile `StoreSaveData` for DRA (US and HD)
* Decompile `StoreSaveData` for SEL
* Match `MemcardFormat` in SEL
* Cross-reference symbols between DRA and SEL
* Bring common code between DRA and SEL to parity
* Discover `save_mgr.c`, `save_mgr2.c` and `save_mgr3.c` have the same
layout in both DRA and SEL
* Deduplicate `save_mgr3.c`
* Import `.data` and `.rodata` from both DRA and SEL
* Convert UTF-8 Japanese characters into Shift-JIS while building
* Minor refactor with less hardcoded magic numbers

I made the following compromises:
* In DRA US `MakeMemcardPath` there is some dirty data at the end of the
string. I had to create a byte array called `HACK_SaveName`. This does
not occur on DRA HD or SEL
* I did not de-duplciate `save_mgr` as there are a good amount of
functions and symbols I have no knowledge about
* I did not decompile `save_mgr2` and it will come as a separate PR.
2023-09-09 21:09:46 +01:00
Luciano Ciccariello
0461c80c33
Start sharing some common stage code (#464)
Following up #454 , #453 and related from @sozud , I started to convert
some previously defined headers into proper shared C code. The advantage
from the header is the C file is completely independent, it will have
its own `.data` and `.rodata` and it will be compiled once.

I also took the opportunity to move `include/st/AnimateEntity.h` to
`src/st/animate_entity.h` for consistency. I think the change is small
enough to be included as part of this PR.
2023-09-02 23:55:48 -07:00
sozud
4417bdc0b0 Fix Saturn CI 2023-08-28 11:38:44 -07:00
Luciano Ciccariello
764914c3c1
Deduplicate shared code between weapon overlays (#506)
As per title.

Interestingly, I could get w_029 matching only with `-O1`. I added an
override in the Makefile. There were also a very few overlays that
needed some `#if !defined(W_0xx)` as few functions were absent in some
of them.

I could not rename `shared.h` into `shared.c` and compile it for
technical reasons. Compiling it into an object `shared.o` means I could
not use `#if !defined(W_0xx)` to exclude some functions. And compiling
it as `shared_0xx.o` would kinda defeat the purpose of having it shared
in the first place.
2023-08-24 01:32:11 +01:00
Luciano Ciccariello
e27ed2b093
Import w_000 data (#505)
The first overlay 100% decompiled. All the functions and data is
imported in `w_000.c`, symbols are all names and assets are extracted
into their appropiate files.

So far the `animset` always comes at the beginning of the data. This
overlay also has CLUTs and the animation definitions. I am not yet sure
if these resources can be extracted as assets. I am waiting to decompile
more weapon overlays to understand if this is a pattern that can be used
or the developers really baked this information as byte arrays. If this
data can be extracted as assets we might want to come up with a simple
viewer/editor in the future.

I could not yet get rid of `data` and `sbss` due to how the Makefile is
structured. Therefore I flagged the empty sections in the YAML. Once
they are all empty they can be removed all at once.
2023-08-22 19:56:54 -07:00
Luciano Ciccariello
7bc78922bf
Animation set extraction (#495)
I rewrote the logic on how `spriteparts` and `spritepartslist` were
extracted. They are now extracted under the new module `animset`, which
handles a list of Sprite Parts in the same JSON file. It is extracting
the information as a JSON and it is rebuilding it back into a binary
usable in-game. The extracted assets will have the `.animset.json`
extension.

It took me a lot of time to find a pattern that would work with the WRP
and the WEAPON overlays. I made myself sure to create a solution that
would work in all the case scenario I found so far. What I wrote will
work across all the overlays. It takes a while to find where the animset
banks are located and I think I will deal with it in a separate PR.

With this it would be possible to preview animations without using #235
but we would need to either convert it into something other programs can
read (HTML5+CSS, Aseprite, DarkFunction) or parse it with a custom-made
GUI tool.
2023-08-21 19:24:44 -07:00
Mark Street
9793d3d383
Update maspsx (#482) 2023-08-19 13:40:24 +01:00
Luciano Ciccariello
e92e9f4f4d
w_000 matching (#465)
![f_000](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/abcc5bd5-ed0c-4fc7-88fa-aa6e796aac1b)

Matches w_000, effectively opening the doors to understand how hand
equippables work. That includes weapons, shields, consumable; all of
them internally named as Weapons). This overlay is used by:

* Tyrfing
* Namakura
* Gladius
* Scimitar
* Cutlass
* Saber
* Falchion
* Broadsword
* Bekatowa
* Damascus sword
* Hunter sword
* Bastard sword
* Talwar
* Sword of Hador
* Luminus
* Harper
* Gram
* Mormegil
* Terminus Est
* Dark Blade
* Mourneblade
* Badelaire

Due to the nature of splat, renaming the symbols is hella hard but
possible once we start importing the `.data` section in it. As far I
know this overlay does not contain any `.rodata`.

Checklist to convert it from Draft to Ready:

- [x] Decompile [func_4000_8017B1D8](https://decomp.me/scratch/HLPI0)
- [x] Confirm name of the functions
- [X] ~~Put shareable code in the appropriate folder (either as `.h` or
`.c`~~ not all overlays have identical functions (see w_029). Needs more
research
- [X] ~~Import `.data` in the C file~~ needs substancial re-work in the
spriteparts asset extractor
- [x] Move `AnimSoundEvent` to the appropriate header
- [X] ~~Rename `Entity::unk58` and `Entity::`unk6A`~~ more appropriate
as a separate PR
2023-08-14 09:21:59 -07:00
Luciano Ciccariello
88aafd5fcd
Enhance Makefile for WEAPON0.BIN (#466)
## What is this

Required for #465 . I re-wrote how the build process works for the
`BIN/WEAPON0.BIN` ~~overlay~~ file.

## A bit of a background

On the PSP build I found `WEAPON0.BIN` is just a blob with `f0_000.bin`,
`w0_000.bin`, `f0_001.bin` and so on concatenated as a single file. `f0`
contains the graphics (or CHR) and `w0` contains the actual overlay (or
PRG). The individual overlays share a great amount of functions I
documented in `config/symbols.us.weapon.txt`.

## The problem

Splat creates its own `weapon.ld` linker script, insisting to link all
the overlays as a single binary file. While this is not technically
correct, it creates a bigger problem: symbols cannot be shared. In 58
overlays there are 58 `DestroyEntity`. Other decomps are
circumnavigating the issue by adding a prefix. Following the same
pattern we could have `W000_DestroyEntity`, `W001_DestroyEntity` and so
on. But this is not maintainable and it does not reflect our long-term
vision of having shared code between overlays.

## The solution

I created the dedicated linker script `weapon0.ld` to output the
individual overlay `w0_000.bin` and similar. `f0_000.bin` is also
created from the correspondent PNG. Once all the overlays and graphics
are created I am simply concatenating them back to `WEAPON0.BIN`. This
is much more efficient in terms of separation of concerns,
maintainability, build speed and it also allows to use the same symbol
names across overlays. Also very important, the generated individual
overlays matches the structure of what is found in the PSP build.

## Side note

I thought it was wise to prepare the Makefile and the linker script to
also allow rebasing and compiling for WEAPON1.BIN. The generated file
does not currently match and it will probably need a bit more research.
I left the build infrastructure there. To compile the non-matching
WEAPON1 just modify `$(BUILD_DIR)/WEAPON0.BIN: $(WEAPON0_FILES)` to
`$(BUILD_DIR)/WEAPON0.BIN: $(WEAPON0_FILES) $(WEAPON1_FILES)`.
2023-08-14 02:03:54 +01:00
Luciano Ciccariello
06afe4f9ba
Split BIN/WEAPON0.BIN (#154)
Splits `BIN/WEAPON0.BIN` in its own individual C files and assets. There
are 59 overlays 59 PNGs extracted as part of it. The `.data`, `.text`
and `.sbss` section are correctly split. I did not yet split the
`.rodata` but I assume it will be so small we can do it on-the-go.

This is fully integrated in the build process, it gives an 🆗 once
built back and it is fully integrated in the CI. We did not yet know how
weapons work in-game and we already have a few instances in the code
where we call `D_8017A000();` and similar. I think it is time to start
documenting them.

Thanks to @bismurphy <bismurphy@users.noreply.github.com> for addressing `analyze_calls.py` for this specific
PR.
2023-08-10 23:56:40 +01:00
Luciano Ciccariello
9b9b3f649f
Custom debugging module (#108)
# What

Adds a custom debug menu. It is written in C and it is meant to replace
the Bat familar `SERVANT/TT_000.BIN`. Once loaded you can long-press
SELECT+START to soft-reset the game and keep using the debug menu
everywhere, including when playing with Richter or during the credits.

One key requirement to run this is to have an emulator that emulates the
8MB of RAM. This is a key requirement to have the debug module surviving
soft-resets or accessing to the in-game menu. I personally used [PCSX
Redux](https://github.com/grumpycoders/pcsx-redux) to build this module.
I am not sure about the compatibility with other emulators. This does
**NOT** work on real hardware and it is a choice by design. The debug
module is intended to test different areas of the game and help
decompiling. It is not intended to be used in normal gameplay.

# Build

Simply invoke `make disk_debug` to create a disk image of the game in
`build/` with the debug module replacing the Bat familiar.

# Usage


![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/15a040b6-6191-41c4-b2b8-a4a906ed59eb)


On PCSX Redux go to Configuration, Emulation and tick the box `8MB`.

## Loading the module


![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/65b7ccb3-800e-4b66-84f5-5703fc91babe)

You need to enable the Bat Card from the menu. This will load the debug
module from the disk. If you want to re-load the module you need to
select another Familar Card, un-pause, pause again and select the Bat
Card once more.

## The main screen


![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/b528425a-ea6c-4c10-9c19-522612d0ad2a)

You will know you have loaded the debug module when you see the blue
rectangle on the top right. You can press R2 to cycle between the menus.
Some menus will temporarily freeze the game, some not. To quickly return
in-game you can either press TRIANGLE or START. To bring back the paused
debug menu just press R2 once again.

## Debug Mode


![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/12cac1ee-725b-484f-8564-89c03b6a755b)

### Stage

Teleports the player to a different stage. It is not stable and it can
crash often.

### Player

Switches between Alucard and Richter. Currently switching from Richter
to Alucard consistently crashes the game.

### No Clip

Allows to freely move the player within the room and without the
collisions on. Once the flag is enabled from the debug menu, pressing L2
while in-game will temporarily freeze the player movement and make it
immune to collision checking. You can then press the directional buttons
to slowly move the player or you can hold CROSS to move it faster. You
can also use SQUARE or CIRCLE to cycle between the player frames. Press
L2 again to deactivate the NoClip mode.

### Frame by frame

Freezes the game outside the debug module. Press L1 to advance by 1
frame. Hold L2 to put the game in slow-motion.

### Show hitboxes


![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/ea8dd918-cf37-4be1-bb87-541a66ac7f16)

As shown in the image

### Show debug messages


![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/5c9bc7b2-6bc7-4831-949a-bf73cdf910e8)

When the debug menu is un-paused, prints on the top left all the debug
messages from the game itself.

### Show collision layer


![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/959ad1ab-dfe0-42fe-a8d0-65c619469280)

This prints the internal collision value for every 16x16 tile on the
screen. Look the CheckCollision function for more information on how
each printed value is used.

### Show draw calls


![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/a52970fc-bf30-430d-98e3-5fca33c9f6f7)

Shows the maximum GPU resource usage since the game started. Currently
only the `max` option works. The `current` option will not show
anything.

### Show HBlank


![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/81759019-6bf9-4cf2-b88c-aa60eaa5ddcb)

Prints the current horizontal blank interrupt count.

## Entity Spawn


![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/f0fdc024-4537-4c07-a80d-816d605a3583)

Allows to immediately spawn new entities in the current map

### Mode

There are three list of entities, each one with their own ID: DRA, Stage
and RIC. The option RIC is hidden if Richter is not the current playing
character. As the list of entities per stage is maintained manually,
stage entities might not be available for all stages. The `Alloc` shows
how many entities are reserved or actually used. Pressing the SQUARE
button here will destroy all the entities within that range.

### ID

Press Left or Right to cycle between the different IDs available. Some
of them might crash the game immediately once spawned. Press CROSS to
immediately spawn the entity.

### Params

Each entity might have its own parameters. Sometimes the flag 0x8000 is
used, which can be toggled with the SQUARE button. Press CROSS to
immediately spawn the entity.

### Entity preview

Shows the entity before spawning it. This is turned off by default as it
can immediately crash by cycling through the available entity IDs.

### Place entity

Pressing CROSS will allow to move the entity across the screen before
placing it. Press CROSS again to place the entity and return to the
previous screen. Press SQUARE to quickly place multiple entities of the
same type.

## Sound player


![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/9d8ad3a5-bc78-4674-abd0-94537bbfae98)

There are three macro categories the sound player is split into. For
what is currently known only the sounds within the Kind 3 changes based
on the loaded stage.

### Stop all sounds

This will also disable the SPU IRQ, effectively unlocking the frame
rate.

### Load Stage

Loads a different sound font than the current loaded stage. This can
help to quickly preview and test SFXs from other stages without
necessarily moving the player there.

### Load Servant

Loads the sound font of a specific servant without necessarily equipping
the Familiar Card.

## Castle flags

Preview all the flags used to modify the behaviour of different parts of
the two castles.

### Edit mode


![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/b03ada2f-0fb4-4345-ad50-81209702032c)

You can move the cursor with the directional buttons and flip the flag
with CROSS. Press L1 or R1 to cycle between the pages. The cursor warps
when reaching the border of the flag grid, allowing a faster navigation.

### View mode

Allows to move between the flags more flexibly.

### Listen mode


![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/4f9ed118-d09d-4658-ba25-5865dbebbf3c)

Listens for the modified flags while playing. Every time a flag is
modified the offset and its value is registered on the top left up to 4
rows. When all the rows are occupied, new values will just remove the
oldest one. The last modified flag will always be displayed at the
bottom.

---------

Co-authored-by: Alejandro Asenjo Nitti <sonicdcer@users.noreply.github.com>
Co-authored-by: sozud <sozud@users.noreply.github.com>
2023-08-10 18:55:24 +01:00
sozud
26030a4692
Rewrite lints, lint more FIX (#439)
I noticed that my lints are already pretty slow and hard to maintain so
I moved them to rust and optimized them. All the files are now linted in
parallel, and we can do as many transforms to a single line as we want.
I think this provides a good baseline where we can have a big suite of
lints if we want to.

I added more cases to the FIX linter as well. Rust always puts a newline
at the end of a file so we picked up some changes due to that. I think
git wants you to do that anyway so it seems like an improvement to me.
2023-08-05 11:36:00 -07:00
sozud
3448b7f48b
Revert "Revert "Splat libsnd out of main (#423)"" (#428)
This reverts commit ca528e1172
2023-08-01 20:46:21 -07:00
Luciano Ciccariello
ca528e1172 Revert "Splat libsnd out of main (#423)"
This reverts commit f31eb3b406 due to a problem with the report generation.
2023-08-02 00:41:05 +01:00
Luciano Ciccariello
f31eb3b406
Splat libsnd out of main (#423)
Fully splits `libsnd` into individual files following the original PSY-Q
SDK file names. The same is done with `libcd` and `libspu` just to
define the boundaries of `libsnd`. I also correctly names **all** the
function symbols from the sound library with their right offset. We
previously had some wrong symbol offsets, leading to the inability to
create their C counterparts with `INCLUDE_ASM`. This PR will start to
open the doors to decompiling `main` or understand better how `DRA`
communicates with `main`.

This took an incredible amount of time as I had to do it manually. I
would have written a script, but there are some variants I would prefer
to handle manually. To achieve this I used the [PSY-Q
signatures](e9e46e7e13/350/LIBSND.LIB.json).
I also discovered the game uses PSY-Q 3.5, which brings some piece of
mind to what they used to develop the game.

After this PR my plan is to:
* Finish to split the other PSY-Q libs
* Move `asm/main/*.s` into their own C files
* Include `main` progress in our reporting
2023-08-01 22:53:40 +01:00
Luciano Ciccariello
c7578ccf73
Function Analyzer CI integration (#417)
Following the incredible work of @bismurphy in #396, this will integrate
the tool in the CI. The markdown file `gh-duplicates/functions.md` will
have clickable links to navigate right to the SVG call tree of the
correspondent functions.

---------

Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-07-31 12:36:35 -07:00