Commit Graph

187 Commits

Author SHA1 Message Date
Jonathan Hohle
11631e1475
Add CEN Overlay to HD (#1705)
Adds the CEN overaly to the HD version of the game.

Resolves #1638.

This uses most, but not all, matching functions. Some were excluded
because due to symbols mapping that has not bee completed yet.

Not all data has been imported, and mapping symbols, and data should
make many of the remaining functions easy to import.
2024-10-01 22:31:18 -07:00
bismurphy
2e4ebb994e
Add NO0 (Marble Gallery) overlay (#1691)
This adds this overlay to the project.

Importantly, we use a new tool (tools/auto_dedupe_new_overlay) to
process the new overlay and automatically split its C files according to
known duplicate files. As of now, this does not do any decompiling (the
whole overlay is kept as INCLUDE_ASM), but it automatically does all the
file splits and code copying needed. Hopefully this reduces some amount
of duplicated work.

To be clear, this new script is not in any build chain, but is meant to
be used after `make-config` to take the overlay from being a single
giant lump of `us.c` to being individual files, for the sake of easier
deduplicating. It will likely need some revision for future overlays,
but at least it should be a good start toward reducing tedious work.

I'll leave this overlay in place like this for about a week in case
there are any newcomers who would like to try de-duplicating some of
these files and decompiling the new functions; after that week I'll get
into doing things myself.
2024-09-30 00:53:54 +01:00
Luciano Ciccariello
f86e111fc5
Extract cutscene scripts, portraits and normalize code (#1686)
More research on how cutscenes work.

I normalized all the various C files as `cutscene.c`, marked all the
isolated function as `static` and renamed the main entity as
`{STAGE}_CutsceneExec` (e.g. `CEN_CutsceneExec`). I am using the
`OVL_EXPORT` to automate the names.

TO-DO:

- [x] Rename entity as `{STAGE}_EntityCutscene` for consistency
- [x] CEN
- [x] DRE
- [x] NO3
- [x] NZ0
- [x] ST0
- [x] MAR
~~SEL~~

The offset of the portrait data seems to be hardcoded. I have no idea
how to resolve these offsets at compilation time. The entire cutscene
script thing is very sketchy and horribly designed by the original
developers. What a nightmare to integrate into our project.

This is how a cutscene script gets decompiled:

```
LOAD_PORTRAIT(0x80188D8C, 0),
SET_PORTRAIT(1, 0),
SCRIPT_UNKNOWN_11(),
PLAY_SOUND(0x37B),
WAIT_FOR_SOUND(),
SET_SPEED(4),
'T','h','a','t',' ','v','o','i','c','e','!',' ',
SET_WAIT(16),
SET_SPEED(3),
'A','l','u','c','a','r','d',',',
LINE_BREAK(),
SET_WAIT(16),
SET_FLAG(2),
'i','t','\'','s',' ','y','o','u','!',
SET_WAIT(48),
NEXT_DIALOG(),
```
2024-09-28 19:56:35 -07:00
Aleksandr Shabelnikov
c2669ae583
#1635: RBO3 overlay (#1671) 2024-09-25 10:02:18 -07:00
Jonathan Hohle
6706a45bf9
Fix Check Output, Executable make-config.py (#1668)
Fixed a tab matching issue with the `check` target. These were matching
'\' and 't' as separator characters rather than '↹' (horizontal tab)

Made `tools/make-config.py` executable.
2024-09-24 19:10:38 +01:00
Josh Schreuder
05d8eab3db
#1637: Add servant/tt_001.bin (#1659)
Followed [the
guide](https://github.com/Xeeynamo/sotn-decomp/wiki/Decompilation#add-new-overlay)
which seemed to work relatively painlessly for this overlay.

I hit an issue in `make-config.py` where no duplicates were found and
have added a separate commit which fixed that. Otherwise no issues with
guide.
```
josh@JoshsPC:/mnt/c/dev/sotn-decomp$ tools/make-config.py tt_001
✔ generating psx splat config
✔ splitting config/splat.us.tt_001.yaml
✔ adjusting files at src/servant/tt_001
✔ disassembling matched functions
✖ finding duplicates across overlays
Traceback (most recent call last):
  File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1132, in <module>
    raise e
  File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1126, in <module>
    make_config(args.input, args.version)
  File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1106, in make_config
    if found > 0:
TypeError: '>' not supported between instances of 'NoneType' and 'int'
```
2024-09-23 18:39:35 -07:00
Luciano Ciccariello
31abdd439e
Add HD version of the RIC overlay (#1660)
I was also able to remove the weird `goto block` on `RicMain`.

The HD version seems to be an older version. I am not entirely sure of
the US changes, but it seem to fix a series of bugs when Richter gets
grabbed by Gaibon? There's a new debug string which ints
`g_Player.unk7A` being named as `run_disable_f` but I did not yet rename
it.
2024-09-23 23:39:39 +01:00
Luciano Ciccariello
148691da5a
Allow make-config to cross-reference symbols from data and bss (#1650)
Some checks failed
Format code / format (push) Failing after 0s
Build C code / extract-assets (push) Failing after 0s
Build Saturn version / build-and-test-saturn (push) Has been skipped
Build C code / build-linux (push) Has been skipped
Build C code / build-macos (push) Has been skipped
Build C code / build-windows (push) Has been skipped
Build C code / build-linux-lle (push) Has been skipped
Build Saturn version / function-finder-saturn (push) Has been skipped
Build Debug Module tool / build (push) Failing after 0s
Build PSX and PSP version / build-and-test (pspeu, hd) (push) Has been skipped
Build PSX and PSP version / build-and-test (pspeu, pspeu) (push) Has been skipped
Build PSX and PSP version / build-and-test (us, us) (push) Has been skipped
Build PSX and PSP version / generate-progress-report (pspeu, hd) (push) Has been skipped
Build PSX and PSP version / generate-progress-report (pspeu, pspeu) (push) Has been skipped
Build PSX and PSP version / generate-progress-report (us, us) (push) Has been skipped
Build PSX and PSP version / generate-duplicates-report (us, us) (push) Has been skipped
Build PSX and PSP version / generate-duplicates-report-psp (pspeu, pspeu) (push) Has been skipped
Improve `make-config.py` to internally use the `symbols.py cross` and
cross-reference symbols from duplicate functions. This allows to add
even more symbols for new overlays with a third pass.

This was one of my biggest pain points when I was de-duplicating the
boss `MAR` overlay as I was not able to directly copy&paste duplicate
functions or include shared ones without having to individually fix the
symbols coming from `.data` and `.bss`. This is now automated.

```
$ python3 ./tools/make-config.py ric --version hd
✔ generating psx splat config
✔ splitting config/splat.hd.ric.yaml
✔ adjusting files at src/ric
✔ disassembling matched functions
✔ finding duplicates across overlays
✔ adding cross-referenced function names
✔ renamed 141 functions, splitting again
✔ cross-referencing 141 functions
✔ adding cross-referenced symbol names
✔ renamed 145 data/bss symbols, splitting again
```
2024-09-21 20:59:55 +01:00
Jonathan Hohle
ca857ea4e4
Build QoL (#1632)
Several quality of life changes for building:

* The Makefile has a `help` target which will show common targets.
Targets with comments starting with `##@` will be included in the help
output.

* The `check` target now has colored output.

* Added `check_disk` target which will check hashes of extracted disk
contents (useful for those who dump their own discs)

* Added `dump_disk` target which will create a bin/cue pair from an
original disc.
2024-09-20 23:56:34 +01:00
Jonathan Hohle
8287841ccd
Python virtualenv Support (#1620)
Ubuntu/Debian and Python recommend using virtual environments for
project-specific Python dependencies and as of Ubuntu 24 this is lightly
enforced when installing packages via pip. This is due to pip and the
system package manager installing files to the same location which may
cause either's internal state to no longer reflect what is actually
installed.

This updates the project to use a Python `virtualenv` for project
dependencies and updates internal scripts to support both global and
virtualenvs, but favors virtualenvs for new workspaces.

All tools that hardcode `/usr/bin/python3` now use `env(1)` to find the
first `python3` in the path. For those with a virtualenv configured,
this will be the Python managed there. For everyone else, this should be
the system Python or whatever other scheme they may have used previously
(assuming `python3` already existed in their `PATH`).

The `Makefile` has been updated to prepend `.venv/bin` to the `PATH` and
use `python3` from there if it exists and has been configured. It also
has a new `python-dependencies` target which will configure the venv and
install all python dependnecies there.

The `Dockerfile` has been updated to create an external `.venv` outside
of the project directory. Python's `virtualenv`s are not relocatable and
may hardcode paths which are mounted differently in the container and
host. To deal with differences in paths between the container (which
mounts the host's project directory to `/sotn`) host which may be at an
arbitarary directory the `VENV_PATH` environment variable is used to
override paths in the `Makefile`.

GitHub workflows have been updated to use `.venv`.
2024-09-17 23:19:20 -07:00
Luciano Ciccariello
fffa28c7f7
Add BOSS/MAR.BIN (#1608)
Small PR to enable #557 and to let be used as an example for [our
wiki](https://github.com/Xeeynamo/sotn-decomp/wiki/Decompilation#add-new-overlay).

This was done with the new `make-config.py` tool.
2024-09-15 21:49:56 +01:00
Luciano Ciccariello
9bc0f5f39f
Add tool to generate splat configs (#1603)
As promised, a new tool to automatically generate a splat config for PSP
and PSX overlays for multiple game versions. This first version will
perform the following tasks:

* Create the splat config with data, rodata, text and bss sections to
their right offsets
* Generate a symbol list by cross-referencing duplicate symbols
* Split the overlay by creating the correspondent C file

This tool is not currently:

* Splitting C files
* Importing shared code already decompiled

There is a lot of tech involved, especially to detect the various
sections for the PSX overlays. But I will leave it to you to discover
how the implementation work if you're interested. Hopefully the code is
self-documented enough.

Usage examples:

* `python3 tools/make-config.py mar`
* `VERSION=jp12 python3 tools/make-config.py no2`
* `python3 tools/make-config.py tt_001 --version pspko`

With this tool I aim to add in the project multiple game versions of
TT_001, MAR and RBO3 from #557 and NO0.
2024-09-15 21:16:58 +01:00
Jonathan Hohle
aff02911f9
Enchanced Draw Mode Lint (#1589)
A few updates that didn't make it into the first round:

* masks are now identified and expanded
* AND-EQUALS and OR-EQUALS are supported
* `sotn-lint` is run before `clang-format` to avoid having to run the
`format-src` target twice to catch line breaks
* `.h` files are included in `sotn-lint` since they are being used for
shared code
* `sotn` lint ignores the `mednafen` directory
2024-09-07 13:00:44 -07:00
Luciano Ciccariello
487007194c
Add AGPL license (#1565)
From the [public
announcement](https://discord.com/channels/1079389589950705684/1079395108501331990/1276310695964835910)
on Discord:

> We've been progressing very far with the project and I started
wondering what would be the best course of action to protect the work of
existing and future contributors. I personally consider contributions to
a decompilation project as creative work. Therefore it is my desire to
protect this creative work from being re-claimed by other people. This
is especially true in the light of the ability of modding the existing
decompiled code and create derivative work.
> 
> I talked privately with the current most active contributors. And me,
@sozud , @bismurphy and @joshlory so far agreed on pushing through the
AGPLv3 license for all the decompiled code in `sotn-decomp/src`,
excluding any third party code and decompiled code from the PSX SDK. But
I've been wanting to ask all of you if you are fine your code to be
licensed and protected under the AGPLv3 terms.
>
>
https://raw.githubusercontent.com/Xeeynamo/sotn-decomp/89686514916cabd0dd88ae0387f749a889c19e05/src/LICENSE
> 
> I put and all the contributors, which can be accessed to the page
https://github.com/Xeeynamo/sotn-decomp/graphs/contributors . Again,
this will work for existing and future contributors. In the license I
put my full name as I will be responsible of any legality. And I
understand not every contributor wants to disclose their identity.
Please give an approval to my latest PR if you do not have any
objection. If not, please write here publicly.
>
> PS. I understand past contributors will either not see the message
above or decide to ignore it. If I do not hear from anyone of the
contributors for more than one week between those who did not interact
with me or the server, I will still go ahead and apply the license. It
is not in my intention to ignore past contributors. But I do not want
contributors who have been inactive for months or years to be a blocker.
Please do not mistake this as an act of malevolence.

---------

Co-authored-by: Joey Murphy <tjmurphy@mit.edu>
2024-09-06 16:14:55 -04:00
Luciano Ciccariello
f1faef333a
Weapon assets and statically linked (#1470)
Allow to compile weapon overlays as a single C files. The benefit is to
disallow symbols to be exported by statically compile them all but the
header. We would no longer need to use `OVL_EXPORT` as long as the
symbols are marked as `static`. The frames data is also generated as an
array of data that can be just included into their respective C files,
generated by JSON files on the fly.

I changed `animset` to `frameset` as the data does not represent a set
of animations but rather a set of frames, where every frame has multiple
sprites. The information on how the frames are played (e.g. animation)
is found elsewhere.

Following the full list of changes to achieve this project:
* The `animset` from splat is now dummied out
* The `frameset` JSON is now generated by the asset manager
* The `frameset` JSON is now compiled into `src/weapon/w_0xx_y.h`
* The `header.c` has been moved into `shared.h`
* For simplicity I added a `#define g_Animset w_000_1`. I did not want
to hack the asset manager too much to force the name to be `g_Animset`
and I did not want to change all the symbols yet.
* Simplified `weapon_pc.c` as we now only need the exported header
* Always try to build the most up-to-date asset manager: this should
avoid weird errors whenever people pull new changes

As the asset manager now can accept those `config` files, now the CLI
supports the following commands:
* `stage extract`
* `stage build`
* `config extract` (NEW)
* `config build` (NEW)

The config structure is heavily inspired to Splat to maintain
consistency and continuity.
2024-08-09 00:57:26 +01:00
Mark Street
20c3370f4f
Use allegrex assembler for PSP (#1424)
I moved *some* of the PSX-specific stuff out of the common `Makefile`
and into the `Makefile.psx.mk` -- there is still more that could be done
here, but this should be enough to have the correct rules for PSX and
PSP with the desired assembler.


Also threw in a bonus feature where I've removed the `move` macro from
`macro.inc` because `maspsx` does this automagically for you now.
2024-07-22 15:36:28 -04:00
Luciano Ciccariello
99bb97e859
HD WRP matching (#1421)
Mostly identical to MAD and PSP. There are four very different functions
from US that needs to be decompiled to be 100% complete:
* [x] `e_misc.c` `EntityMessageBox`
* [x] `e_misc.c` `EntityRelicOrb`
* [x] ~`e_stage_name.c` `StageNamePopupHelper`~ it does not exist
* [x] `e_stage_name.c` `EntityStageNamePopup`

These four very different functions are the only ones that use the JP
text instead of the US one. This overlay also lacks of `BlitChar`.

`EntityStageNamePopup` is very similar to the PSP counterpart. I used
@joshlory scratch to match the HD part.

`EntityRelicOrb` had some #ifdef between US and BETA (aka the MAD
overlay). I found some code that crossed between BETA and HD, so I just
put an `#else` after `VERSION_US`. We are starting to reconstruct how
the source code originally evolved across different game builds.
2024-07-19 19:05:26 -05:00
sozud
82d145ab72
More libgpu, maspsx update (#1391)
This file seems to use an older aspsx. Thanks to @mkst for the fixes
2024-07-08 08:19:21 +01:00
Luciano Ciccariello
de46f99e2e
New asset manager (#1343)
This aims to deprecate all the Splat tools in `tools/splat_ext` in
favour of a more centralised asset manager. This brings the following
advantages:

* Much faster extraction
* Faster build
* Automatically define `static` symbols or unique names whenever
`static` is not possible
* Allow to embed assets into the output binary
* Drastically simplify `Makefile` by removing all the asset build rules
* Avoid situations where it is not possible to extract and build assets
that is not 4-byte aligned

This is achieved by having the splat YAML targeting a normal C file as
data and have an external tool to take care of the following:

1. Extract asset files straight from the overlay binary file into human
readable file in `assets/st/STAGE_NAME`
2. Build assets as header files that go into `src/st/STAGE_NAME` to just
include them from any C file

This requires each stage header to have the following new format: please
see `src/st/nz0/header.c`

Built assets in `src/st` are ignored by Git.

As for now, for simplicity sake, the steps `make extract_assets` and
`make build_assets` are just executed within `make extract` exclusively
for the US version.

I plan to auto-generate files such as `src/st/nz0/tile_data.c`.

For a first iteration I am aiming to handle the following:

* [X] Extract rooms: `assets/st/*/rooms.json`
* [X] Extract room layers: `assets/st/*/entity_layouts.json`
* [X] Extract tilemap data: `assets/st/*/tilemap_*.bin`
* [X] Extract tilemap definitions: `assets/st/*/tiledef_*.json`
* [X] Extract sprites: `assets/st/*/sprites.json`
* [x] Extract entity layouts
* [X] Build rooms: `src/st/*/rooms.h`
* [X] Build room layers: `src/st/*/layers.h`
* [X] Build tilemap data: `src/st/*/tilemap_*.h`
* [X] Build tilemap definitions: `src/st/*/tiledef_*.h`
* [x] Build sprites (aka `g_SpriteBanks`)
* [x] Build entity layouts
* [x] Allow the tool to suggest how to adjust the Splat config for each
overlay

I want the tool to cover the following stages:
* [x] CEN
* [x] DRE
* ~MAD~ I do not think this can be done, it is way too different from
the other overlays
* [x] NO3
* [x] NP3
* [X] NZ0
* [x] ST0
* [X] WRP
* [x] RWRP
* ~WRP (PSP)~ Maybe in a follow-up PR

For a later iteration I plan on extracting and build:

* Entity GFX thingie
* The CLUT thingie in the header
* Uncompressed GFX data
* Cutscene data
* Blueprints
* The `src/config_us.h` thingie

---------

Co-authored-by: Josh Lory <josh.lory@outlook.com>
2024-07-02 21:38:36 +01:00
Jonathan Hohle
75752b8878
Speed Up make format (#1359)
After building for all supported architectures `make format` was taking
around 30s. This commit includes various changes which get that down to
around 3½s on the same machine.

This changes the `format` target in a few ways:

  * filters out files generated by CMake
  * runs `clang-format` in parallel
  * breaks `format` into 3 separate targets which can be run in parallel

When building the PC version cmake creates a file to test compiler
features. This file is picked up by the pattern which matches files for
`clang-format`. This file took around 20s, filtering it out brought
total format time to around 10s.

Instead of passing a list created by a `find` command directly as the
args to `clang-format` the list is now passed to `xargs` which runs
several processes in parallel, each looking at 10 files (determined by
experimentation). This has the added advantage of ensuring as the source
list grows `ARG_MAX` won't be a concern. This brought total format time
to around 6s.

`format` was broken up into three targets `-src`, `-tools`, and
`-symbols`. These targets can be run in parallel, bringing the total
time down to 3½s.

---------

Co-authored-by: Jonathan Hohle <jon@ttkb.co>
2024-06-30 09:26:53 +01:00
Jonathan Hohle
9354d7e5ac
Fix Directory Creation Dependencies in Makefile (#1353)
This updates the Makefile so that targets which depend on phony targets
(like `*_dirs`) have direct dependencies on those targets rather than
relying on transient dependencies from dependent targets. Other targets
explicitly create parent directories for their files.

This resolves #739, making it possible to using make-4.4 to build in
parallel.
2024-06-28 14:23:41 +01:00
Luciano Ciccariello
8abb09e3ff
Update splat to 0.24.4 (#1112)
Still work in progress. I removed splat as a submodule and started using
it as a pip package instead. Everything is matching but the memory card
icons part in both DRA and SEL. I still have no idea what the issue is.
Once this PR is good to be merged, we can get rid of the splat fork too.
2024-06-12 18:50:32 +01:00
Jonathan Hohle
312c8441f5
More Descriptive func_801B9744 (#1137)
This adds some additional context for func_801B9744 and renames globals
to fit their assumed purpose.

Migrated various stream metadata to C definitions.

The makefile changes add a dependency on `sel.h` to all the objects that
get built from the `st/sel` directory.

Co-authored-by: Jonathan Hohle <jon@ttkb.co>
2024-06-11 10:47:59 -07:00
Jonathan Hohle
4dff513033
Makefile Quality of Life Changes (#1275)
The `check` recipe now has dependencies on all of the files it checks
allowing them to be rebuilt if necessary.

Consistent use of `dd` and suppressing `dd`'s output which noise for the
replacement and padding it's doing.

Run 7z in non-interactive mode to avoid confirming overwrite of files
during extraction (equivalent behavior to `VERSION=us` `extract_disk`).

`patch` is a phony target.

Co-authored-by: Jonathan Hohle <jon@ttkb.co>
2024-06-08 18:13:03 +01:00
Luciano Ciccariello
151346d62c
Update clang-format as self-contained binary (#1257)
Version 18.1.6 pulled from
https://github.com/xeeynamo/sotn-decomp/releases and compiled with the
following:

```Dockerfile
FROM alpine:latest
RUN apk add --no-cache \
    build-base \
    cmake \
    git \
    ninja \
    libstdc++ \
    zlib-dev \
    python3
ENV LLVM_VERSION=llvmorg-18.1.6
RUN mkdir -p /opt/llvm \
    && cd /opt/llvm \
    && git clone --depth=1 --branch $LLVM_VERSION https://github.com/llvm/llvm-project.git \
    && mkdir -p llvm-project/build \
    && cd llvm-project/build \
    && cmake -G Ninja \
        -DLLVM_ENABLE_PROJECTS=clang \
        -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_CXX_FLAGS="-static-libstdc++ -static-libgcc" \
        -DCMAKE_EXE_LINKER_FLAGS="-static" \
        -DLLVM_ENABLE_RTTI=ON \
        ../llvm \
    && ninja clang-format \
    && strip bin/clang-format
FROM scratch
COPY --from=0 /opt/llvm/llvm-project/build/bin/clang-format /clang-format
ENTRYPOINT ["/usr/local/bin/clang-format"]
```
`docker build --output . .`

Original source code:
https://github.com/llvm/llvm-project/tree/llvmorg-18.1.6
commit: `1118c2e05e67a36ed8ca250524525cdb66a55256`
2024-06-04 21:07:50 +01:00
Luciano Ciccariello
5362c8b8da
Add ST/WRP from PSP (#1172)
Extract ST/WRP out of #1119 . All the function symbols should have been
cross-referenced. There as some PSX functions missing from PSP and some
new functions from PSP that are not present on PSX (e.g.
`st_init_wrp.c`).

The files `st_debug.c` and `e_breakable.c` are shared between WRP PSX
and WRP PSP. Everything else from PSP is isolated into its own folder. I
had to do some tricks on the YAML config to allow shared code.

`ST_WRP_MERGE = st_debug e_breakable` in the `Makefile` is a bit
annoying as MWCC complains about every single minute detail from the C
source that has been already decompiled for the PSX US build.

`EntityWarpSmallRocks` is matching on PSP but I couldn't extract the
rodata without having a bunch of linker errors. This might be a Splat
issue. I need to investigate further.

`func_psp_09244760` is soooo interesting. The values from `0x11` to
`0x17` matches the Entity IDs that are unique to the WRP overlay. This
aligns to what we have in `typedef enum EntityIDs`.

Overall I am very excited to the recent discoveries from the PSP build!
2024-05-25 11:56:06 -07:00
Mark Street
b6a0ddbeb6
Add mwccgap (#1131)
OK. Here goes. Version 0.0.1 alpha of the MWCC Global Assembly Processor
(mwccgap). It's currently very simple/limited, but it appears to work
for `src/servant/tt_000/10E8.c`.

There is lot more that can be done to improve mwccgap - i.e. supporting
.rodata migration would be a good addition, but let's see how far we can
get with it in it's current state.

Note that the Makefile could do with some improvements - we don't nede
to use mwccgap for any C file that *dont* have INCLUDE_ASM macros (it's
a waste of time) so these could be ignored, i.e. for SSSV I do something
like this to find the files that need fixing up:
```
GLOBAL_ASM_C_FILES := $(shell $(GREP) GLOBAL_ASM $(SRC_DIR) </dev/null 2>/dev/null)
```
.. although this is perhaps too simple given that SOTN has a mix of PSP
and PSX functions (and therefore there may be INCLUDE_ASM for a PSX
function but none for PSP functions...
2024-05-18 11:41:00 +01:00
Jonathan Hohle
76d1584c89
Replace BIOS Trampolines (#1133)
This replaces several dozen BIOS trampolines from the extracted ASM to
"decompiled" source. These are modeled on the `INCLUDE_ASM` macro, but
generate the instructions necessary for each trampoline directly instead
of importing an extracted source file.

Because these trampolines never return, and GCC 2.6 doesnt appear to
have builtins for leaving off the return jump postamble, these will
likely need to remain assembly.

This also changes the `main.elf` target to depend on `main.ld`, and
undefined symbols files, allowing `make build` to regenerate those files
if necessary.

Co-authored-by: Jonathan Hohle <jon@ttkb.co>
2024-05-17 08:45:26 +01:00
Jonathan Hohle
20a1fb6998
Support Single BIN CUE Files (#1127)
In CUE files `FILE` is a stateful, global declaration that applies to
all following `TRACK`s. `sotn-disk` was treating `FILE` declarations as
`TRACK` delimiters which would result in incorrect parsing of CUE files
with a single `FILE`, but multiple tracks.

Now when reading a `FILE` declaration, the path is stored and processing
continues. A `TRACK` declaration will use the previously defined path,
and if a previous track had been started, append that previous track to
the list.

This also builds `sotn-disk` using the local repository instead of
pulling the latest commit from GitHub. The target depends on `sotn-disk`
sources and will rebuild as necessary (or with `make
~/go/bin/sotn-disk`).

As an aside,
[pull/232](https://github.com/Xeeynamo/sotn-decomp/pull/232) ran into
this same error, but fixes in a slightly different way that leaves side
effects that may make supporting things like `INDEX` (for extracting the
placeholder audio, for example) more error prone in the future.

Co-authored-by: Jonathan Hohle <jon@ttkb.co>
2024-05-14 19:59:56 +01:00
Luciano Ciccariello
15865c38ba
Completes w_051 (#1122)
All the stuff marked as `static` does not have the `g_` prepended as by
definition they are not global
2024-05-12 18:31:16 +01:00
Luciano Ciccariello
aa6c5692fb
More TT_000 PSP functions matching (#1115) 2024-04-23 20:06:05 +01:00
Luciano Ciccariello
9e7e87eaac
Add TT_000 from the HD version (#1111)
100% decompiled.

`US` adds some padding to have the file exactly 40KB long. The HD
version does not add any padding, hence why I ended up producing
`tt_000_raw.bin`. I would have used just `tt_000.bin` as a name as found
in `disks/pspeu/PSP_GAME/USRDIR/res/ps/hdbin/tt_000.bin`, but on
Windows-based file system it would collide with `TT_000.BIN` due to the
OS having case insensitive names.

I modified `make clean` as I found annoying that `VERSION=hd make clean`
would wipe out `us` build stuff.

The only different function in HD is `ProcessEvent`, which has a weaker
check. Another hint suggesting HD being older than US.

`s32 _unused[26];` added enough padding in the bss section to get an
🆗 . I am pretty sure it is unused data because the final binary is
not aligned by any power of 2.
2024-04-04 22:28:57 +01:00
Mark Street
a5ffd53cff
Update maspsx (#1100)
Also tweaked the Dockerfile since this is what I use to build the
project.
2024-03-15 14:04:57 -07:00
Mark Street
dfc74d4acd
Migrate .rodata to functions for main.exe (#1078)
I feel like getting all the rodata migrated will make it easier to
update splat (also makes it easier to match functions that use rodata).
2024-02-24 11:35:01 +00:00
Mark Street
41ad71e388
Pass undefined_syms files to linker (#1065)
Creating a PR for this, will look at updating splat in a separate PR
2024-02-18 14:57:03 +00:00
Luciano Ciccariello
78953c01aa
Allow spritesheet to be extracted from the CLI (#1035)
A side-project I've been working on that helped me understanding some
structures that are now a bit better documented in #1034 . Richter
sprites are extracted as usual. Doing `python3
tools/splat_ext/spritesheet.py decode disks/us/BIN/ARC_F.BIN assets/arc
alucard disks/us/BIN/F_GAME.BIN 1` can extract Alucard sprites too.
There is no repacking yet as I want to avoid using splat for this one.
2024-02-05 21:57:48 +00:00
Luciano Ciccariello
8eb7795b4f
Decompile some PSX SDK functions (#1002) 2024-01-21 17:58:32 +00:00
sozud
abca5d44d2
POC for JP sotn_strs (#986)
This is a proof-of-concept for Japanese sotn_strs. The macro `_SJ` is
used to designate them. I only converted one string in config_jp to
serve as a test for the pipeline. I moved sotn_str to a folder and added
some tests. I renamed it since python doesn't like modules with - in the
name. The code could be cleaner and the conversion table could use some
more work but I think this is a reasonable start.
2024-01-17 08:27:17 +00:00
sozud
23a143b295
Mark SEL handwritten functions as asm (#977) 2024-01-14 11:27:34 -08:00
Luciano Ciccariello
70d2a82688
Fix reporting for the RIC overlay (#880)
Re-generate symbols for `build/us/ric.map`. Also I moved the
`--no-default` as it was previously ignored.
2023-12-24 12:29:21 -05:00
Mark Street
fd49b82d1f
Update maspsx (--expand-li) (#877)
Once decomp.me is updated I will update the SOTN preset to include
`--expand-li`. Let me know if you run into any issues but this builds an
🆗 binary for me.
2023-12-20 21:13:37 +00:00
ser-pounce
ce30caa958
Makefile refactoring (#866)
Needed a break from all the asm. I realize that Makefile changes aren't
always appreciated as they can break things in unexpected ways,
especially if they come from newbies to the project (ahem), so feel free
to close this if it's not needed.

One comment about removing `/**/*.s` and `/**/*.c`: as far as I could
see there were no subdirectories, so didn't seem needed.
2023-12-18 21:22:42 +00:00
Luciano Ciccariello
21f158a7c7
Load assets json (#851)
Uses a small JSON parsing library called cJSON (MIT license) to parse
the assets `equipment.json` and `accessory.json` and load them in memory
rather than hard coding it or pre-building it.
2023-12-17 14:10:11 +00:00
Luciano Ciccariello
b851d5e4d2
Import 4A538, demo and 5087C data (#835)
As per title. I also added a new tool called `dirt_patch`. As I
mentioned in our Discord server there is some left-over data from
previous dev builds in DRA.BIN and potentially elsewhere too. The tool
uses the new file `config/dirt.us.json` which stores a list of patches
to avoid crazy hacks and `#ifdef` for the sake of getting a match. I
hope this tool will not be abused.
2023-12-12 10:36:34 -08:00
Luciano Ciccariello
e9bed91931
Extract memory card icons (#166)
Extract the memory card icons out of DRA.BIN and SEL.BIN.


![image](https://user-images.githubusercontent.com/6128729/227794798-4dd071f9-512b-4c72-8f5d-fcbf7a615519.png)

I also took the opportunity to extract `g_MemcardPortMask` into its own
source file `save_mgr_pre.c`. Unfortunately this is required to keep the
original data order. This suggests that most likely the icon palette and
bitmap were baked into the original `save_mgr.c` as byte arrays. I
decided to take a different approach and extracted them as PNGs for
better moddability.

I had to spin-off Splat into a new fork due to some breaking changes on
0.18.0.
2023-12-11 23:12:13 -08:00
Luciano Ciccariello
c123b1819d
Sound test: add 42398.c (#799)
In Italian we say "Abbiamo fatto 30, facciamo 31". It literally means
"We come all the way up to 30, it wouldn't cost nothing to push a little
further for 31".

I moved `log.h` into `include/` to easily logging stuff elsewhere and
without weird `#ifdef VERSION_*` by adding `-DNO_LOGS` in the main
Makefile.

This is a bit hacky, but it does the trick. I had to expand `D_80138784`
otherwise I would get a segfault.

Both `SoundInit` and `func_801361F8` are called in the main, so I
removed them. To restore the previous behaviour and isolate the sound
engine, just replace `MainGame()` in the `main.c` with a `SoundInit()`
and then a loop that calls `UpdateGame()`.

Last, but not least, SDL2 is in.
2023-11-20 18:17:51 -08:00
Luciano Ciccariello
574c23f981
TT_000 func_80172120 (#791)
Second last function of TT_000 decompiled.

* Add `printf '\x00' | dd of=$@ bs=1 seek=40959 count=1 conv=notrunc` to
ensure the output binary is 40960 bytes long (useful once BSS is
migrated)
* Add `$(BUILD_DIR)/tt_%.ld` so every time the splat YAML changes, doing
`make tt_000` triggers the extract again
* All the variables from BSS are now in `10E8` and in order. Removing
`extern` from will properly create the BSS section
* Fix 7A4D0.c to not use the familiar entity extension
* Delete the fake symbols `D_80170660` and  `D_8017065C`
* Replace all the instances of `ext.generic` as `ext.bat` when possible

NOTES:
* For `SquareRoot12((...) << 0xC) >> 0xC;` I was not able to use the
macro `FLT`
*
2023-11-19 10:14:32 -08:00
bismurphy
b3c0524bc8
Change assets.py to output assembly (#752)
Issue was raised on discord, this should correct for it.

Previously assets.py would output raw binary data. This makes it more
difficult for the build chain to understand.

Now, it outputs an assembly file (which is just a big list of `.byte`
entries) which will represent the parsed data, and that assembly data
then gets assembled. Therefore, the symbols now exist.

In order to make this work, I adjusted the format of the splat, so now
we have lines like `[0x7718, assets, accessory, g_AccessoryDefs]` where
the entries are [Address, assets, configuration json file to use, name
of the output symbol].

I think this addresses the issue, but I am more than happy to iterate on
this if needed.
2023-11-07 22:05:07 +00:00
Luciano Ciccariello
81f85b195a
Extract and document all functions from main.exe (#736)
This took me a while. I used a [throw-away
script](https://gist.github.com/Xeeynamo/58da1ff8f3831d0ba5d23da27cbca025)
to help me with the quest, but I still had to manually check every
single YAML subsegment.

I transported over what we were able to decompile from main. For a few
functions I added the signatures and documented existing DRA code,
especially on the sound department (got inspired by the recent @sozud
PRs).

All the `.text` part from `main.exe` is now completely extracted as
C/ASM where appropiate. Almost all the functions have their original
PSY-Q names but 7 of them: `func_80012DBC`, `func_80012F84`,
`func_80017008`, `func_80017078`, `func_8001929C`, `func_80021F0C` and
`func_800286E0`. I did not feel confident enough to rename them, so I
left them be. The rest of the functions I am 99% sure they are all
accurate.

I am now excluding the
[asm/](https://github.com/Xeeynamo/sotn-decomp/tree/master/asm/us/main/psxsdk)
folder from the repo. It was useless.

This research confirms me the game uses the PSY-Q 3.5 libraries, with
the exception of `LIBGPU.LIB`, which is from PSY-Q 3.0 for some unknown
reason.

EDIT: `make format` was not taking care of duplicated symbols in our
symbol list. To speed-up my work, all the duplicate symbols (duplicates
= name AND offset) are now removed.
2023-11-06 14:53:27 -08:00
Luciano Ciccariello
28b44193c5
Replace RoomDimensions with Tilemap (#730)
Failing to decompile [RenderTilemap](https://decomp.me/scratch/WigVS)
made me realise there were a few fields and structures I have always
been suspicious to be part of the same structure. After
cross-referencing overlapping fields from different structures, I was
able to merge `D_80073088`, `g_Camera`, `D_8007309C`, `g_CurrentRoom`
and `g_CurrentRoomTileLayout` into the new `g_Tilemap`.

I was forced to touch the majority of the code-base, which gave me the
opportunity to standardise some field names (e.g. from
`currentRoomTileLayout`, `roomLayout`, `layout`, `t` into `tilemap`),
remove some fake code, redundant code and adjust some symbols.
2023-11-04 16:43:45 +00:00