This aims to deprecate all the Splat tools in `tools/splat_ext` in
favour of a more centralised asset manager. This brings the following
advantages:
* Much faster extraction
* Faster build
* Automatically define `static` symbols or unique names whenever
`static` is not possible
* Allow to embed assets into the output binary
* Drastically simplify `Makefile` by removing all the asset build rules
* Avoid situations where it is not possible to extract and build assets
that is not 4-byte aligned
This is achieved by having the splat YAML targeting a normal C file as
data and have an external tool to take care of the following:
1. Extract asset files straight from the overlay binary file into human
readable file in `assets/st/STAGE_NAME`
2. Build assets as header files that go into `src/st/STAGE_NAME` to just
include them from any C file
This requires each stage header to have the following new format: please
see `src/st/nz0/header.c`
Built assets in `src/st` are ignored by Git.
As for now, for simplicity sake, the steps `make extract_assets` and
`make build_assets` are just executed within `make extract` exclusively
for the US version.
I plan to auto-generate files such as `src/st/nz0/tile_data.c`.
For a first iteration I am aiming to handle the following:
* [X] Extract rooms: `assets/st/*/rooms.json`
* [X] Extract room layers: `assets/st/*/entity_layouts.json`
* [X] Extract tilemap data: `assets/st/*/tilemap_*.bin`
* [X] Extract tilemap definitions: `assets/st/*/tiledef_*.json`
* [X] Extract sprites: `assets/st/*/sprites.json`
* [x] Extract entity layouts
* [X] Build rooms: `src/st/*/rooms.h`
* [X] Build room layers: `src/st/*/layers.h`
* [X] Build tilemap data: `src/st/*/tilemap_*.h`
* [X] Build tilemap definitions: `src/st/*/tiledef_*.h`
* [x] Build sprites (aka `g_SpriteBanks`)
* [x] Build entity layouts
* [x] Allow the tool to suggest how to adjust the Splat config for each
overlay
I want the tool to cover the following stages:
* [x] CEN
* [x] DRE
* ~MAD~ I do not think this can be done, it is way too different from
the other overlays
* [x] NO3
* [x] NP3
* [X] NZ0
* [x] ST0
* [X] WRP
* [x] RWRP
* ~WRP (PSP)~ Maybe in a follow-up PR
For a later iteration I plan on extracting and build:
* Entity GFX thingie
* The CLUT thingie in the header
* Uncompressed GFX data
* Cutscene data
* Blueprints
* The `src/config_us.h` thingie
---------
Co-authored-by: Josh Lory <josh.lory@outlook.com>
Decompiles stage header data for all in progress stages.
Of note is that NO3 and NP3 (and likely other future stages) use an
`AbbreviatedHeader` because their `SpriteBanks` start immediately after
`UpdateStageEntities`. There are several areas which use
`sizeof(Overlay)` to copy this data over, in those cases sprite bank
data is copied into the tail fields of the overlay but are never used.
What is a SOTN? A nasty, miserable little pile of pointer manipulations!
Well here we are, the last one! Solving the previous function helped me
figure out the patterns here, but wow, this is really a miracle.
Let me know how you want to handle the de-duplication. We can either do
that here, or merge this and do all the others in a second PR.
Fairly straightforward de-duplication. If anyone has better names for
either the function or the array it uses, feel free to suggest them, but
with this thing being so weird and its purpose being unknown, I had to
keep the names pretty generic.
As far as the duplicate-finder can tell, this is the last of the
cutscenes, but who knows - we may find more. Luckily, the pattern is
pretty recognizable.
Named it due to what we see at the very end in `case 7`, with the
TimeAttackController getting triggered.
Wow. This is perhaps one of the hardest functions I've ever done. It
does disgusting things with pointers that I've never seen anything like
before - some of them are the fault of the programmers, some are the
compiler.
Once I was like 98% done, I found func_801B69F8 which is very similar
(and decompiled), but wasn't picked up by the duplicate finder, so that
was a little sad. Once I found that though, the last of my mysteries
were quickly solved.
This is a huge function and it's nice to get it cracked.
Given the fact that sharing compiled C objects is not exactly possible
(code heavily copy&pasted maybe? this initiative is now abandoned 👉53a566f075)
I decided to keep pressing forward with shared headers. Thanks a lot to
@hohle for making our life much easier by cross-referencing symbols.
The file split on WRP seems to be the closest file split we might have
compared to the original source code (still speculating here). I think
it would be a good idea to start splitting other overlays too with the
same approach.
My idea is to have a file split like the following:
```
st/
cen/
e_particles.c
e_misc.c
st_common.c
nz0/
e_particles.c
e_misc.c
st_common.c
wrp/
e_particles.c
e_misc.c
st_common.c
e_particles.h
e_misc.h
st_common.h
```
each of those C files will just be a one-line `#include
"../the_shared_code.h"` as usual. Right now we create individual headers
for single functions, sometimes for more than one function when we think
grouping makes sense. But I think we can start merging some of those
headers and consolidate the code. This can be done gradually. For
example `src/st/e_particles.h` is still importing function headers under
the hood. That is okay for now, but later on I wish to import those
headers functions into their respective parent headers.
Another important aspect to consider that will validate a correct file
split is to start importing the data inside these new C files. Right now
we have floating data such as `src/st/wrp_psp/wrp_data_EA00.c` or
monstrosities such as `src/st/wrp/6FD0.c`. An example of a (possibly)
correct migrated data is what this PR does with WRP PSP and NZ0 PSX,
with data pointing in `src/st/*/e_particles.c`.
Pulls out `EntityExplosion` into a common file.
`cen` and `rwrp` appear to have not imported a declaration for
`AnimateEntity` and call it as an undeclared function with a int-sized
return value. Since `AnimateEntity` is declared in the same compilation
unit a workaround is done to fix how the return value is interpreted.
In the interest of reducing uses of `generic`, and especially the
`entityPtr` member of it, I found this function which had several
duplicates already, all of which used `generic`. Rather than cleaning it
up in every instance, I decided instead to de-duplicate it, and then
clean it up in the .h file.
I needed to make a new entity extension for this, and noticed that we
had `ET_Entity16`, but then I realized that this was being used for
`g_Entities[16]`, so I renamed that to be `ET_EntitySlot16`, leaving
`ET_EntityXX` available, for XX being the ID of an unknown entity.
Quite a big one. I merged `EntitySoulStealOrb` across all overlays by
normalising the symbol names. The symbol types was all over the place,
so I had to fix that as well.
I solved the s16/u16 ifdef between PSX and PSP. The only part I couldn't
match without a ifdef was `angle = (angle - 1) * 8;` and `angle = (angle
* 8) - 8;`.
I also decompiled the exclusive PSP functions `func_psp_0923AD68` and
`func_psp_0923B2F0` and merged the PSX and PSP `e_particles.c` code. The
exclusive functions required a file split.
This one had some small differences in MAD and ST0, so I worked them in
as ifdef.
Also, I cleaned up the function a bit, by removing the unneeded
pointers.
The GoldSizeIndex local variable is unneeded. It is possible to just use
the goldSize argument, and do `-=2`, but I think using the local
variable makes it more readable, so I left it in place.
This is an entity that is in every overlay, but appears to be unused, at
least so far.
It looks just like Gaibon's big fireball (in his second form), but that
is handled by a different entity. Presumably we will find its use in a
future overlay.
I also made some general improvements to the function, such as turning
the weird bit shifting into a simple division by 4, thus eliminating the
variables. I also created an entity extension for this entity.
Another victory for my automated deduplicator.
Another nice de-duplication. Cool to dig into these functions and see
what they do.
Continuing working on my automated de-duplication script. This one was
definitely much easier because of it. I didn't have to track down a
single symbol. In fact, most of my time putting this together was just
commenting in the .h file.
This spawns several copies of EntityUnkId14, very similarly to the
existing EntityUnkId15Spawner.
Most of the work for this PR (renaming all the functions,
cross-referencing symbols across overlays) was done by an automated
script I made. I am still testing this script, but it is nice to see
that it appears to be starting to work. I will test it on a few more
de-duplication PRs, and then consider adding it to the `tools`
directory.
Fairly standard function de-duplication PR. Starting to get kind of good
at these! I might look into automating parts of this process to make it
go faster...
Next I'll be doing the unkId14 spawner, which is just like the existing
unkid15 spawner.
Found this function which is in every overlay, and is a dependency of
another function I'd like to de-duplicate.
See the comment on the .h file for a description of what this function
appears to do. I think the fact that it's taking the entity's position,
offsetting it based on the provided array, and running CheckCollision
makes CheckColliderOffsets a reasonable name without being too verbose.
Happy to use a different name if anyone has opinions.
This function had been decompiled in a few overlays, but was still
INCLUDE_ASM in others, so this will also unify them to all be
decompiled.
I think I did this right, but this is my first time de-duplicating a
function, so please point out any mistakes :)
ST0 not included because it has different logic internally - will work
on decompiling that one next.
This change renames functions and global stage variables uniformly
across the stages so that these functions can be pulled out and shared
across all of the stages. Based on some other tests there are 12 or so
functions that this will allow to be pulled out of each stage. Since
these implementations are shared, an additional 12 asm functions can be
eliminated in a subsequent pass.
**Vars**
* `g_pStObjLayoutHorizontal` - a horizontally sorted array of stage
entities
* `g_pStObjLayoutVertical` - a vertically sorted array of stage entities
* `g_LayoutObjHorizontal` - a pointer to a `LayoutEntity` in
`g_pStObjLayoutHorizontal`
* `g_LayoutObjVertical` - a pointer to a `LayoutEntity` in
`g_pStObjLayoutVertical`
* `g_LayoutObjPosHorizontal` - the direction last traversed in
`g_LayoutObjHorizontal`
* `g_LayoutObjPosVertical` - the direction last traversed in
`g_pStObjLayoutVertical `
**Functions**
* `FindFirstEntityToTheRight` - given an `x` position, update
`g_LayoutObjHorizontal` with the first entity to the right of `x`
* `FindFirstEntityToTheLeft` - given a `x` position, update
`g_LayoutObjHorizontal` with the first entity to the left of `x`
(backwards)
* `CreateEntitiesToTheRight` - given an `x` position, create all
entities to the right (mutates `g_LayoutObjHorizontal`)
* `CreateEntitiesToTheLeft` - given an `x` position, create all entities
to the left (mutates `g_LayoutObjHorizontal`)
* `FindFirstEntityAbove` - given an `y` position, update
`g_LayoutObjVertical ` with the first entity to the above of `y`
* `FindFirstEntityBelow` - given an `y` position, update
`g_LayoutObjVertical ` with the first entity to the below of `y`
* `CreateEntitiesAbove` - given an `y` position, create all entities
above (mutates `g_LayoutObjVertical`)
* `CreateEntitiesBelow` - given an `y` position, create all entities
beneath (mutates `g_LayoutObjVertical`)
* `UpdateRoomPosition` - look at the current game loop scroll delta and
create any entities given the room layout
I believe all of these implementations are shared across all stages
(including `InitRoomEntities`, and two more `CreateEntity` functions)
(in my initial tests I had a small difference in `DER`, but I believe
that had to do with an incorrect symbol table change).
Co-authored-by: Jonathan Hohle <jon@ttkb.co>
Thanks a lot to @SestrenExsis for doing the majority of the function
matching! I only take the credit for cross-referencing it with the PSP
counterpart and massaging the code until I got a match on PSX.
[EquipKind](https://discord.com/channels/1079389589950705684/1087866009983139870/1196073673392652388)
group decision
[drawMode](https://discord.com/channels/1079389589950705684/1087866009983139870/1197935480428302476)
group decision
I was thinking a more soft PR, but in the light of `BLEND_VISIBLE` to
`DRAW_HIDE`, a mass rename would have happened anyway. So I decided to
rename everything since the majority of the code stands on the same line
of `BLEND_VISIBLE`.
The only changes other than the mass rename are:
```c
#define blendMode drawMode // maintained to easily migrate existing scratches
```
this might or not be useful
.
```c
#define DRAW_DEFAULT 0x00
#define DRAW_TRANSP 0x01 // make it semi transparent
#define DRAW_COLORS 0x04 // use color blending
#define DRAW_HIDE 0x08 // do not render the primitive
#define DRAW_TPAGE 0x10 // use custom tpage
#define DRAW_NOMENU 0x80 // do not render if D_800973EC is set
#define DRAW_ABSPOS 0x2000 // use absolute coordinates with DRAW_NOMENU
```
This is based on the research from the
[RenderPrimitives](https://decomp.me/scratch/geI8L) function.
Decompile and de-duplicate EntityStageNamePopup for all the overlays the
function is in.
I also detected a helper function I renamed as `PrimDecreaseBrightness`
that was originally accepting a completely wrong parameter type. This
helped me to get rid of `Unkstruct_80128BBC` too.
Some deduping and renaming that might help later
- Dedupe DestroyEntity
- Dedupe ST EntityIsNearPlayer
- Dedupe DestroyEntitiesFromIndex
- Dedupe ST CollectHeart
- Dedupe ST UnkEntityFunc0
- Decompile + rename rwrp EntityExplosionSpawn
Follow-up to #797 by decompiling the MAD counterpart and share all the
functions within the same file. I renamed the function as
`UnkPrimHelper` as I do not know what it does.
Failing to decompile [RenderTilemap](https://decomp.me/scratch/WigVS)
made me realise there were a few fields and structures I have always
been suspicious to be part of the same structure. After
cross-referencing overlapping fields from different structures, I was
able to merge `D_80073088`, `g_Camera`, `D_8007309C`, `g_CurrentRoom`
and `g_CurrentRoomTileLayout` into the new `g_Tilemap`.
I was forced to touch the majority of the code-base, which gave me the
opportunity to standardise some field names (e.g. from
`currentRoomTileLayout`, `roomLayout`, `layout`, `t` into `tilemap`),
remove some fake code, redundant code and adjust some symbols.
Allows to not hard code the location in-memory of decompiled functions
and imported data if not required. This allows to relocate the
referenced symbols when editing the original code or game data. If you
were getting the offset of those symbols from the symbol list in
`config/` I suggest to use the built `build/us/*.map` file instead.
Reviving [Dialogue
duplicates](https://github.com/Xeeynamo/sotn-decomp/pull/541) #541
@Xeeynamo I think it would be fine to merge this and follow up with
factoring these out to .h. What do you think?
---------
Co-authored-by: Alejandro Javier Asenjo Nitti <alejandro.asenjo88@gmail.com>
As I finished importing the data in WRP, I came with a possible new
pattern when de-duplicating functions. I kept using a header file
(easily indexable from VS Code) in `src/st`. But this time it gets
included by `entity_relic_orb.c`. All the sections `.data`, `.rodata`
and `.text` are migrated into `entity_relic_orb.c` but the file itself
just includes `src/st/entity_relic_orb.h`. Thinking forward, once we
will be able to fix the problem present in #464 it will just be a matter
of renaming `entity_relic_orb.h` into `entity_relic_orb.c` and remove
each individual `entity_relic_orb.c` that currently acts as a proxy.
~~This PR is a draft. If you agree this is a good pattern I will proceed
to do the same with the remaining overlays after #528 is good to be
merged.~~
Took me a while. I had to move a good chunk of the data from 6FD0 due to
`const char* D_80180F84[] = {"Obtained "};` as it declares data in both
`.data` and `.rodata`. Due to its strategic location, I did not yet find
a clean way to have it matching in other overlays without starting to
import data.
The function is not very clean and there are loads of temp variables and
fake stuff. I ask to spend a bit of time to make improvements where
possible. I want to get it in a good shape before committing myself to
de-duplicating it for all the other overlays.
https://decomp.me/scratch/kyBr8
This is the Entity that pushes the player through the castle door at the
entrance.
I decompiled this a long time ago, but it wasn't matching because of an
instruction reorder, oddly, it was matching in PSYQ 4.0 for some weird
reason. Today i discovered why, the problem was the function prototype.
https://decomp.me/scratch/a1jYQ
This is an unknown function. I don't know what it does. It is duplicated
across most overlays. It is called in CollectGold, EntityEquipItemDrop,
and TestCollisions.
It seems to take in a string, and then do a bunch of things with
primitives, but I don't know what. We don't directly reference that
array, we only read and write to it based on a pointer which moves
through the array. I tried to remove the pointer but didn't get
anywhere.
Fully-matching scratch is here: https://decomp.me/scratch/UV2K2
I am starting with only this one version of the function, so that if
changes need to be made (variable names, control flow, etc), I can
update them once here, and then copy them to the other places, instead
of making changes in every place. Once we are happy with the state of
this function, I will add the duplicates to this PR prior to merging.
---------
Co-authored-by: bismurphy <bismurphy@users.noreply.github.com>
After attempting this de-duplication process I see that it's pretty
slow, tedious and error-prone. I think it would make sense to have an
intermediate phase where we split stuff into #includes like this PR
does. Eventually this will be combined back into C files with proper
file splits. I think a process that is incremental and would get us
there eventually could be like this:
Split single function into a header, #include it.
Rename any D_XXXXXXX variables to UNK_Whatever#, add to symbols.*.txt
Once the above is done for all instances of the function, perform file
splits (if we know there's a sequential block of functions, wait to
split until the whole block is done)
Combine individual.h files into a c file, integrate building
st/shared_#.c into Makefile
I'm probably just going to do a little bit on this each day to try and
chip away at it, it's too tedious to do a big push.
