- Decompiled duplicate function func_us_801714F4
- Decompiled duplicate function func_us_80172904
- Decompiled duplicate function func_us_8017293C (renamed
Tt001UpdateAnim per tt_000 duplicate)
- Decompiled duplicate function func_us_80172B50
- Decompiled duplicate function func_us_80172B88
- Added EntitySearch D_80170EE4[] to servant.h to be used by
func_us_80172B88 similar to duplicate in tt_000
A lot of these are in the same order in tt_000, so I would assume they
could be moved to shared static functions at some point.
this one was trickier and larger than my previous contributions. I'm not
sure what the best way to rename the symbols are, so I expect to receive
some comments on that.
- `column` util was missing from the Docker image, added it and
organised the requirements file so it's more human friendly
- rearranged the USER argument in the Dockerfile to avoid the scenario
where you didnt specify the user and so lose the layers where apt
installed all the packages
- add the python deps at the end to ensure docker can reuse cached
layers if a python dep changes.
- update maspsx (I wanted to test latest maspsx against sotn, so this is
what led to the other changes!)
In analyze_calls, we need to analyze a function's calls, and be able to
track where they go. Given that a `jal` call only has a function name,
it's possible for it to go to a function in any overlay. Therefore, we
need to disambiguate across overlays.
We do this by identifying what overlay every function is in. This comes
from the filename of the assembly file.
Since the overlay might be in, for example, asm/us/dra (a "level 3
overlay" because it has 3 directory names) or asm/us/st/cen (a "level 4
overlay") we had special logic for testing for "st".
Now that we can also have level 4 overlays in "boss" (with Maria and BO3
coming up) we need to test for those too.
While we're at it, we also only run on the "us" version's assembly
files.
The decompilation script did not like to decompile anything from tt_001
because it ended in us.c. This would cause it to find muliple (5) paths
and would halt.
Change moves the code to F84.c in line with the code split in tt_000
Fixed a tab matching issue with the `check` target. These were matching
'\' and 't' as separator characters rather than '↹' (horizontal tab)
Made `tools/make-config.py` executable.
`w_030` expects an `s16` at entity offset 0x90. All of the existing uses
of `unk90` were already using specific extension structs which declare
that offset to be an `s16`. `unk90` was changed from an `s32` to an
`s16` a new `s16 unk92` field was added.
It turned out that `ET_WeaponUnk044` was only split because it was
expecting an `s16` at offset 0x90 and could use `ET_Weapon` now.
A few other extension structures were modified for clarity while trying
to fit them into one of the more generic structs.
Followed [the
guide](https://github.com/Xeeynamo/sotn-decomp/wiki/Decompilation#add-new-overlay)
which seemed to work relatively painlessly for this overlay.
I hit an issue in `make-config.py` where no duplicates were found and
have added a separate commit which fixed that. Otherwise no issues with
guide.
```
josh@JoshsPC:/mnt/c/dev/sotn-decomp$ tools/make-config.py tt_001
✔ generating psx splat config
✔ splitting config/splat.us.tt_001.yaml
✔ adjusting files at src/servant/tt_001
✔ disassembling matched functions
✖ finding duplicates across overlays
Traceback (most recent call last):
File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1132, in <module>
raise e
File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1126, in <module>
make_config(args.input, args.version)
File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1106, in make_config
if found > 0:
TypeError: '>' not supported between instances of 'NoneType' and 'int'
```
I was also able to remove the weird `goto block` on `RicMain`.
The HD version seems to be an older version. I am not entirely sure of
the US changes, but it seem to fix a series of bugs when Richter gets
grabbed by Gaibon? There's a new debug string which ints
`g_Player.unk7A` being named as `run_disable_f` but I did not yet rename
it.
Fixes a few issues found by @bismurphy where certain symbols that were
not supposed to be added, were added.
The major fix is when the second pass found a meaningful symbol name
like `MakeExplosion`, the third pass could add a new symbol with the
same offset but with the default splat name `func_8019055C`. With the
new `add_symbol_unique` the symbol table will stay unique. On top of
that when adding a new symbol `func_8019055C` can be replaced by
`MakeExplosion` but not the other way around. This will ensure only
meaningful names will be resolved.
I went through all of CEN, cross-referencing it with WRP to make as many
files match as possible. This involved pulling in data (which had been
pulled in on WRP but not on CEN yet), renaming shared variables to not
have placeholder names, adjusting file splits, and more.
EntityBreakable seems to be very different here, so I did not do any
deduplication on that one yet. I think we need to get a robust system
for deduplicating across stages so we can share .h files even when the
stages are different. Right now there's no way for an included .h file
to know it's part of CEN (at least, that I know of).
This is similar to other cutscenes but with a few minor differences. Of
note is that the PSX version uses an uninitialized pointer.
Pulls in the remaining duplicates and `func_us_8018C90C` as well
finishing off the remaining functions in `boss/mar`.
not sure where to put these externs, I didnt see a header file for this.
also, the `D_us_8019AF2A[0] ^= 1;` construct is sus for two reasons:
1) this is A bytes away from g_CutsceneFlags, and
2) I could only get a matching decompile by making these arrays and
accessing the first element, even though a ptr would make more sense to
me.
Improve `make-config.py` to internally use the `symbols.py cross` and
cross-reference symbols from duplicate functions. This allows to add
even more symbols for new overlays with a third pass.
This was one of my biggest pain points when I was de-duplicating the
boss `MAR` overlay as I was not able to directly copy&paste duplicate
functions or include shared ones without having to individually fix the
symbols coming from `.data` and `.bss`. This is now automated.
```
$ python3 ./tools/make-config.py ric --version hd
✔ generating psx splat config
✔ splitting config/splat.hd.ric.yaml
✔ adjusting files at src/ric
✔ disassembling matched functions
✔ finding duplicates across overlays
✔ adding cross-referenced function names
✔ renamed 141 functions, splitting again
✔ cross-referencing 141 functions
✔ adding cross-referenced symbol names
✔ renamed 145 data/bss symbols, splitting again
```
I found `estimate_gnu_c_function_begin` might not be reliable all the
times. This happened with RIC HD. I found a way to circumnavigate the
issue for the time being.
Adds a spinner used as an indicator that the tool is working. I broke
down the duplicate function finder into multiple steps to give faster
feedback to the user.
I am now checking the duplicates only between overlays that make sense
to search to. Splitting `ric` will only cross-reference it with `ric
us`. Same for the servants, stages and the `sel` stage. On my machine
this change trimmed down the execution time from 1m25s to just 10s.
The spinner is very easy to use. Just use `spinner_start` instead of
`print`. Use `spinner_stop(True)` to terminate it and go to a new line,
but a new `spinner_start` will do that for you. Use
`spinner_stop(False)` when an operation fails.
This can be merged independently from #1643 and #1644.
I was experimenting with `python3 tools/symbols.py cross
asm/us/ric/matchings/1AC60/RicInit.s
asm/hd/ric/nonmatchings/hd/RicInit.s` and I found the tool was giving no
matches.
With this fix the output produced is now:
```
PLAYER_rotY = 0x800733F4;
PLAYER_rotX = 0x800733F2;
ric_anim_stand_relax = 0x80155480;
g_IsPrologueStage = 0x80154570;
D_80175958 = 0x801758AC;
D_801759D8 = 0x8017592C;
D_801530AC = 0x801530AC;
D_80153AA0 = 0x80153AA0;
D_80153D24 = 0x80153D24;
D_801541A8 = 0x801541A8;
```
Without this patch, the tool was looking at
`disks/hd/PSP_GAME/USRDIR/res/ps/hdbin/ric.bin`, which does not exist.
Now it is possible to successfully create configs for HD overlays.
This completes the migration for all stages into e_misc.h. They are now
all using it in a unified manner.
It's very satisfying to have everything coming together across the
stages like this!
After decompiling this function for ST0, I've now gone ahead and added
all the necessary ifdef in order to make the .h file match for ST0 as
well.
This version is also very close to matching for PSP, so if/when we
figure out the final adjustments needed, it should be easy to integrate
that.
Next I will make ST0 use e_misc.h, but for now since there are already
so many changes here I wanted to split it up so the PR didn't get too
major.
First working version of this.
Next steps will involve doing #ifdef to unify the function, but wanted
to start by getting something in that matches by itself.
Several quality of life changes for building:
* The Makefile has a `help` target which will show common targets.
Targets with comments starting with `##@` will be included in the help
output.
* The `check` target now has colored output.
* Added `check_disk` target which will check hashes of extracted disk
contents (useful for those who dump their own discs)
* Added `dump_disk` target which will create a bin/cue pair from an
original disc.
Good amount of deduplication here.
The only issue I ran into is that NZ0 has slightly different data for
one array (it's missing two zeros at the end). I tried doing `#undef
STAGE #define STAGE STAGE_NZ0` in `nz0.h` and testing `#if STAGE ==
STAGE_NZ0`, but for some reason that was failing. I went with `#define
STAGE_IS_NZ0` and `#if !defined(STAGE_IS_NZ0)` for now; not sure what I
was doing wrong.
Pulls this function into a .h file.
Since each overlay uses a different offset into g_CastleFlags, we use
`#define HEART_DROP_CASTLE_FLAG` before each `#include` statement in
order to set these up correctly.
Next we will hopefully be able to dedupe e_misc.c, but that will take a
bit longer especially since PSP still needs EntityRelicOrb done, but I'm
working on that one now.
I went through every use of g_CastleFlags and created enum members for
the different flags. This will allow us to better keep track of what
flags exist and how they are used. This will also aid in tracking the
arrangement of the overlays in the flags, as there seems to be some
level of organization. As we add more gameplay overlays we can hopefully
learn how these all work together.
Some functions were using local `const int` declarations to give names
to flags. For example, there was `const int jewelSwordRoomUnlock = 51;`.
For these ones, which had flag purposes identified in code, I went ahead
and gave their global castle flag a similar name (in this case,
`JEWEL_SWORD_ROOM`). For all other flags, I kept the name as a
placeholder of `CASTLE_FLAG_###`; later someone else can identify what
the individual flags are.
This should be a good step in making the castle flags more
understandable, as well as to connect the different functions across the
game and properly see which ones are touching the same flags. When a
castle flag is written as 0x32 in some places, and 50 in others, it
makes it harder to search. Now that we can use CASTLE_FLAG_50, hopefully
we will have better understanding of all the uses of the flag.
Ubuntu/Debian and Python recommend using virtual environments for
project-specific Python dependencies and as of Ubuntu 24 this is lightly
enforced when installing packages via pip. This is due to pip and the
system package manager installing files to the same location which may
cause either's internal state to no longer reflect what is actually
installed.
This updates the project to use a Python `virtualenv` for project
dependencies and updates internal scripts to support both global and
virtualenvs, but favors virtualenvs for new workspaces.
All tools that hardcode `/usr/bin/python3` now use `env(1)` to find the
first `python3` in the path. For those with a virtualenv configured,
this will be the Python managed there. For everyone else, this should be
the system Python or whatever other scheme they may have used previously
(assuming `python3` already existed in their `PATH`).
The `Makefile` has been updated to prepend `.venv/bin` to the `PATH` and
use `python3` from there if it exists and has been configured. It also
has a new `python-dependencies` target which will configure the venv and
install all python dependnecies there.
The `Dockerfile` has been updated to create an external `.venv` outside
of the project directory. Python's `virtualenv`s are not relocatable and
may hardcode paths which are mounted differently in the container and
host. To deal with differences in paths between the container (which
mounts the host's project directory to `/sotn`) host which may be at an
arbitarary directory the `VENV_PATH` environment variable is used to
override paths in the `Makefile`.
GitHub workflows have been updated to use `.venv`.
PSX: https://decomp.me/scratch/RL97B
PSP: https://decomp.me/scratch/3WKmX
`func_us_8018B4A0` needed to be decompiled before importing all the
assets due to GCC compiling `PfnEntityUpdates[entity->entityId - 1]`
into a hardcoded offset landing to the `header.c` data as a fake symbol.
The decompiled function seem to be some kind of object spawner for the
cutscenes controller. I have yet to confirm that, so I did not rename
any symbols yet.
EDIT: `func_us_8018B4A0` turned out to be a shared function I renamed as
`CutsceneRun`. DRE and CEN use slightly different variants and they will
not use the new `cutscene.h`. I plan on putting more stuff into
`cutscene.h` in a separate PR and maybe rename all the C files that
handle cutscenes accordingly.
I added some of the correct labels to the unknown field on the
RoomTeleports. The unknown field is used by the CD Rooms to reload the
tileset if you move half way through the room but then decide to turn
back. When not used in a CD Room I don't know if it has any effect. But
some warps that aren't used in a CD Room do assign this field.
Matches 72% of the entire overlay in one go. Around 80% of the overlay
code is made of duplicate function.
A few observations:
* `func_us_8018AC0C` seems to be an older, buggy version of
`func_801B246C` and it seem to indicate `ObjInit2` was either a `u16`
array or there were macros involved.
* `e_breakable` is the same function, but the data changes in most
overlays; I had to pull the data out. The SFX ID is also different.
Data handled through the asset manager and BSS will come as part of a
separate PR.
