Had an idea, not sure if this is useful or not but I figured it might
help give an indication for newer people or those not in the Discord
loop if code is already duplicated when looking at the function report.
Here's an example output from the current state (minus the scratches
which didn't seem to load for me locally so I skipped them)
[functions.md](https://github.com/user-attachments/files/17162454/functions.md)
@bismurphy guessing this would be your area of expertise 😄
`CutsceneUnk2` uses 8 prims but only 7 are allocated. It looks like at
some point the game writes to the address `0x00000000` but I did not
confirm it with a PS1 debugger. This is a potential bug. `CutsceneUnk2`
signature was also wrong.
I had to get rid of `GfxBank` in `src/st/cen/header.c` and I probably
need to do the same for the other header files. The code reads chunks of
WORDS at the time and a structure would be too large to include
`GFX_TERMINATE` in it.
CEN is there but not linked because it conflicts with some WRP symbols.
I decided to take a much simpler approach compared to what I did with
WRP (which needs to be refactored later on).
There's a cutscene parser. The asset manager is exporting it to
`src/st/cen/cutscene_data.h` if you want to have a look. I know I am
still using `config/assets.us.weapon.yaml` and it probably needs to be
renamed as `config/assets.us.yaml` later on. I am still deciding.
* `ProcessEvent`
* `CreateEventEntity`
* `CalculateAngleToEntity` (the function formerly known as
func_80173F30)
* `GetTargetPositionWithDistanceBuffer` (the function formerly known as
func_80173F74)
There's 4 more duplicates which I'll start on separately so as to not
make this PR too big.
Decompiled libgpu SetDrawEnv
This function actually packs some of the short length fields from a
DRAWENV struct into DR_ENV struct code fields, which are longs.
IE dr_env->code[7] gets packed with env->clip.x in the hi 2 bytes and
env->clip.y in the low 2 bytes.
Perhaps DR_ENV should actually be PACKED_DRAWENV.
Co-authored-by: sozud <sozud@users.noreply.github.com> (main
decompilation)
Co-authored-by: dezgeg <dezgeg@users.noreply.github.com> (getting it
across the finish line)
Did my best to give contextual names where possible based on surrounding
code.
There's still some remaining ones where it wasn't clear what the entity
was referring to so have left some as is.
Export the symbol list from the compiled elf and call `splat split
TARGET --disassemble-all`, so `symbols cross` can correctly
cross-reference the symbols.
Now there is also a warning when symbols cannot be cross-referenced
correctly.
cc. @hohle I noticed this issue on your stream
Before:
```
$ tools/make-config.py --version hd cen
✔ generating psx splat config
✔ splitting config/splat.hd.stcen.yaml
✔ adjusting files at src/st/cen
✔ disassembling matched functions
✔ finding duplicates across overlays
✔ adding cross-referenced function names
✔ renamed 99 functions, splitting again
✔ cross-referencing 0 functions
✔ adding cross-referenced symbol names
```
Now:
```
$ tools/make-config.py --version hd cen
✔ generating psx splat config
✔ splitting config/splat.hd.stcen.yaml
✔ adjusting files at src/st/cen
✔ disassembling matched functions
✔ finding duplicates across overlays
✔ adding cross-referenced function names
✔ renamed 99 functions, splitting again
✔ splitting config/splat.us.stcen.yaml 👈 NEW!
✔ cross-referencing 98 functions 👈 FIXED!
✔ adding cross-referenced symbol names
✔ renamed 56 data/bss symbols, splitting again 👈 56 instead of 0!
```
- Decompiled duplicate function func_us_801714F4
- Decompiled duplicate function func_us_80172904
- Decompiled duplicate function func_us_8017293C (renamed
Tt001UpdateAnim per tt_000 duplicate)
- Decompiled duplicate function func_us_80172B50
- Decompiled duplicate function func_us_80172B88
- Added EntitySearch D_80170EE4[] to servant.h to be used by
func_us_80172B88 similar to duplicate in tt_000
A lot of these are in the same order in tt_000, so I would assume they
could be moved to shared static functions at some point.
this one was trickier and larger than my previous contributions. I'm not
sure what the best way to rename the symbols are, so I expect to receive
some comments on that.
- `column` util was missing from the Docker image, added it and
organised the requirements file so it's more human friendly
- rearranged the USER argument in the Dockerfile to avoid the scenario
where you didnt specify the user and so lose the layers where apt
installed all the packages
- add the python deps at the end to ensure docker can reuse cached
layers if a python dep changes.
- update maspsx (I wanted to test latest maspsx against sotn, so this is
what led to the other changes!)
In analyze_calls, we need to analyze a function's calls, and be able to
track where they go. Given that a `jal` call only has a function name,
it's possible for it to go to a function in any overlay. Therefore, we
need to disambiguate across overlays.
We do this by identifying what overlay every function is in. This comes
from the filename of the assembly file.
Since the overlay might be in, for example, asm/us/dra (a "level 3
overlay" because it has 3 directory names) or asm/us/st/cen (a "level 4
overlay") we had special logic for testing for "st".
Now that we can also have level 4 overlays in "boss" (with Maria and BO3
coming up) we need to test for those too.
While we're at it, we also only run on the "us" version's assembly
files.
The decompilation script did not like to decompile anything from tt_001
because it ended in us.c. This would cause it to find muliple (5) paths
and would halt.
Change moves the code to F84.c in line with the code split in tt_000
Fixed a tab matching issue with the `check` target. These were matching
'\' and 't' as separator characters rather than '↹' (horizontal tab)
Made `tools/make-config.py` executable.
`w_030` expects an `s16` at entity offset 0x90. All of the existing uses
of `unk90` were already using specific extension structs which declare
that offset to be an `s16`. `unk90` was changed from an `s32` to an
`s16` a new `s16 unk92` field was added.
It turned out that `ET_WeaponUnk044` was only split because it was
expecting an `s16` at offset 0x90 and could use `ET_Weapon` now.
A few other extension structures were modified for clarity while trying
to fit them into one of the more generic structs.
Followed [the
guide](https://github.com/Xeeynamo/sotn-decomp/wiki/Decompilation#add-new-overlay)
which seemed to work relatively painlessly for this overlay.
I hit an issue in `make-config.py` where no duplicates were found and
have added a separate commit which fixed that. Otherwise no issues with
guide.
```
josh@JoshsPC:/mnt/c/dev/sotn-decomp$ tools/make-config.py tt_001
✔ generating psx splat config
✔ splitting config/splat.us.tt_001.yaml
✔ adjusting files at src/servant/tt_001
✔ disassembling matched functions
✖ finding duplicates across overlays
Traceback (most recent call last):
File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1132, in <module>
raise e
File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1126, in <module>
make_config(args.input, args.version)
File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1106, in make_config
if found > 0:
TypeError: '>' not supported between instances of 'NoneType' and 'int'
```
I was also able to remove the weird `goto block` on `RicMain`.
The HD version seems to be an older version. I am not entirely sure of
the US changes, but it seem to fix a series of bugs when Richter gets
grabbed by Gaibon? There's a new debug string which ints
`g_Player.unk7A` being named as `run_disable_f` but I did not yet rename
it.
Fixes a few issues found by @bismurphy where certain symbols that were
not supposed to be added, were added.
The major fix is when the second pass found a meaningful symbol name
like `MakeExplosion`, the third pass could add a new symbol with the
same offset but with the default splat name `func_8019055C`. With the
new `add_symbol_unique` the symbol table will stay unique. On top of
that when adding a new symbol `func_8019055C` can be replaced by
`MakeExplosion` but not the other way around. This will ensure only
meaningful names will be resolved.
I went through all of CEN, cross-referencing it with WRP to make as many
files match as possible. This involved pulling in data (which had been
pulled in on WRP but not on CEN yet), renaming shared variables to not
have placeholder names, adjusting file splits, and more.
EntityBreakable seems to be very different here, so I did not do any
deduplication on that one yet. I think we need to get a robust system
for deduplicating across stages so we can share .h files even when the
stages are different. Right now there's no way for an included .h file
to know it's part of CEN (at least, that I know of).
This is similar to other cutscenes but with a few minor differences. Of
note is that the PSX version uses an uninitialized pointer.
Pulls in the remaining duplicates and `func_us_8018C90C` as well
finishing off the remaining functions in `boss/mar`.
not sure where to put these externs, I didnt see a header file for this.
also, the `D_us_8019AF2A[0] ^= 1;` construct is sus for two reasons:
1) this is A bytes away from g_CutsceneFlags, and
2) I could only get a matching decompile by making these arrays and
accessing the first element, even though a ptr would make more sense to
me.
Improve `make-config.py` to internally use the `symbols.py cross` and
cross-reference symbols from duplicate functions. This allows to add
even more symbols for new overlays with a third pass.
This was one of my biggest pain points when I was de-duplicating the
boss `MAR` overlay as I was not able to directly copy&paste duplicate
functions or include shared ones without having to individually fix the
symbols coming from `.data` and `.bss`. This is now automated.
```
$ python3 ./tools/make-config.py ric --version hd
✔ generating psx splat config
✔ splitting config/splat.hd.ric.yaml
✔ adjusting files at src/ric
✔ disassembling matched functions
✔ finding duplicates across overlays
✔ adding cross-referenced function names
✔ renamed 141 functions, splitting again
✔ cross-referencing 141 functions
✔ adding cross-referenced symbol names
✔ renamed 145 data/bss symbols, splitting again
```
I found `estimate_gnu_c_function_begin` might not be reliable all the
times. This happened with RIC HD. I found a way to circumnavigate the
issue for the time being.
Adds a spinner used as an indicator that the tool is working. I broke
down the duplicate function finder into multiple steps to give faster
feedback to the user.
I am now checking the duplicates only between overlays that make sense
to search to. Splitting `ric` will only cross-reference it with `ric
us`. Same for the servants, stages and the `sel` stage. On my machine
this change trimmed down the execution time from 1m25s to just 10s.
The spinner is very easy to use. Just use `spinner_start` instead of
`print`. Use `spinner_stop(True)` to terminate it and go to a new line,
but a new `spinner_start` will do that for you. Use
`spinner_stop(False)` when an operation fails.
This can be merged independently from #1643 and #1644.
I was experimenting with `python3 tools/symbols.py cross
asm/us/ric/matchings/1AC60/RicInit.s
asm/hd/ric/nonmatchings/hd/RicInit.s` and I found the tool was giving no
matches.
With this fix the output produced is now:
```
PLAYER_rotY = 0x800733F4;
PLAYER_rotX = 0x800733F2;
ric_anim_stand_relax = 0x80155480;
g_IsPrologueStage = 0x80154570;
D_80175958 = 0x801758AC;
D_801759D8 = 0x8017592C;
D_801530AC = 0x801530AC;
D_80153AA0 = 0x80153AA0;
D_80153D24 = 0x80153D24;
D_801541A8 = 0x801541A8;
```
Without this patch, the tool was looking at
`disks/hd/PSP_GAME/USRDIR/res/ps/hdbin/ric.bin`, which does not exist.
Now it is possible to successfully create configs for HD overlays.
This completes the migration for all stages into e_misc.h. They are now
all using it in a unified manner.
It's very satisfying to have everything coming together across the
stages like this!
After decompiling this function for ST0, I've now gone ahead and added
all the necessary ifdef in order to make the .h file match for ST0 as
well.
This version is also very close to matching for PSP, so if/when we
figure out the final adjustments needed, it should be easy to integrate
that.
Next I will make ST0 use e_misc.h, but for now since there are already
so many changes here I wanted to split it up so the PR didn't get too
major.
First working version of this.
Next steps will involve doing #ifdef to unify the function, but wanted
to start by getting something in that matches by itself.
Several quality of life changes for building:
* The Makefile has a `help` target which will show common targets.
Targets with comments starting with `##@` will be included in the help
output.
* The `check` target now has colored output.
* Added `check_disk` target which will check hashes of extracted disk
contents (useful for those who dump their own discs)
* Added `dump_disk` target which will create a bin/cue pair from an
original disc.
Good amount of deduplication here.
The only issue I ran into is that NZ0 has slightly different data for
one array (it's missing two zeros at the end). I tried doing `#undef
STAGE #define STAGE STAGE_NZ0` in `nz0.h` and testing `#if STAGE ==
STAGE_NZ0`, but for some reason that was failing. I went with `#define
STAGE_IS_NZ0` and `#if !defined(STAGE_IS_NZ0)` for now; not sure what I
was doing wrong.
Pulls this function into a .h file.
Since each overlay uses a different offset into g_CastleFlags, we use
`#define HEART_DROP_CASTLE_FLAG` before each `#include` statement in
order to set these up correctly.
Next we will hopefully be able to dedupe e_misc.c, but that will take a
bit longer especially since PSP still needs EntityRelicOrb done, but I'm
working on that one now.
