Game-Decompilation/sotn-decomp
1
0
Fork 0
mirror of https://github.com/Xeeynamo/sotn-decomp.git synced 2024-10-06 18:13:30 +00:00
Code Issues Actions 8 Packages Projects Releases Wiki Activity
master
sotn-decomp/Makefile.psx.mk

136 lines
7.5 KiB
Makefile
Raw Permalink Normal View History

Use allegrex assembler for PSP (#1424) I moved *some* of the PSX-specific stuff out of the common `Makefile` and into the `Makefile.psx.mk` -- there is still more that could be done here, but this should be enough to have the correct rules for PSX and PSP with the desired assembler. Also threw in a bonus feature where I've removed the `move` macro from `macro.inc` because `maspsx` does this automagically for you now.
2024-07-22 19:36:28 +00:00
# flags
AS_FLAGS += -Iinclude -march=r3000 -mtune=r3000 -no-pad-sections -O1 -G0
PSXCC_FLAGS := -quiet -mcpu=3000 -fgnu-linker -mgas -gcoff
Enhance make extract on PSX (#662) Previously if either the YAML or the symbol list were changed, we were forced to either manually delete the `asm` folder of the specific overlay or to invoke `make clean`, where the latter forced us to re-extract all the overlays via `make -j extract`. That changes with this PR. How `make` works is `target: dep1 dep2` where if one of the dependencies has the last modified date greater than the target, the rule is triggered again. Previously we were extracting overlays doing `make extract_stcen`. But since every overlay extraction generates a linker script I now changed the rules to do `make build/us/stcen.ld` to extract the same overlay. As I explained above, if either the YAML or one of the related symbols changes, the linker modified date will be older and the extraction for that overlay to be trigger again. The above allowed me to stop polluting the repo root with the linker scripts as they are now moved into `build/us` or `build/hd` depending of what you're trying to build. If you do `make extract` twice in a row you will be welcomed with a `make: Nothing to be done for 'extract'.`. There are still some instances where you **might** need `make clean` beforehand, especially when modifying the `segment` section in the YAML or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I want to see if the current solution will be enough. ### Please try removing `make clean` from your workflow, once this is merged, to quickly detect possible problems with this new approach. I am also planning to make more substantial changes on our build-chain like this PR or #660 in the future. I am aiming to small incremental changes over time in case I break someone's flow or detect design flaws. I am also considering the breaking changes introduced in make 4.4, which will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a deadline to finish all the new build-chain work. --------- Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-10-05 08:26:29 +00:00
# configuration
Pass undefined_syms files to linker (#1065) Creating a PR for this, will look at updating splat in a separate PR
2024-02-18 14:57:03 +00:00
PSX_OVLS := dra ric weapon
Add NO0 (Marble Gallery) overlay (#1691) This adds this overlay to the project. Importantly, we use a new tool (tools/auto_dedupe_new_overlay) to process the new overlay and automatically split its C files according to known duplicate files. As of now, this does not do any decompiling (the whole overlay is kept as INCLUDE_ASM), but it automatically does all the file splits and code copying needed. Hopefully this reduces some amount of duplicated work. To be clear, this new script is not in any build chain, but is meant to be used after `make-config` to take the overlay from being a single giant lump of `us.c` to being individual files, for the sake of easier deduplicating. It will likely need some revision for future overlays, but at least it should be a good start toward reducing tedious work. I'll leave this overlay in place like this for about a week in case there are any newcomers who would like to try de-duplicating some of these files and decompiling the new functions; after that week I'll get into doing things myself.
2024-09-29 23:53:54 +00:00
PSX_KSTAGES := cen dre mad no0 no3 np3 nz0 sel st0 wrp
Enhance make extract on PSX (#662) Previously if either the YAML or the symbol list were changed, we were forced to either manually delete the `asm` folder of the specific overlay or to invoke `make clean`, where the latter forced us to re-extract all the overlays via `make -j extract`. That changes with this PR. How `make` works is `target: dep1 dep2` where if one of the dependencies has the last modified date greater than the target, the rule is triggered again. Previously we were extracting overlays doing `make extract_stcen`. But since every overlay extraction generates a linker script I now changed the rules to do `make build/us/stcen.ld` to extract the same overlay. As I explained above, if either the YAML or one of the related symbols changes, the linker modified date will be older and the extraction for that overlay to be trigger again. The above allowed me to stop polluting the repo root with the linker scripts as they are now moved into `build/us` or `build/hd` depending of what you're trying to build. If you do `make extract` twice in a row you will be welcomed with a `make: Nothing to be done for 'extract'.`. There are still some instances where you **might** need `make clean` beforehand, especially when modifying the `segment` section in the YAML or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I want to see if the current solution will be enough. ### Please try removing `make clean` from your workflow, once this is merged, to quickly detect possible problems with this new approach. I am also planning to make more substantial changes on our build-chain like this PR or #660 in the future. I am aiming to small incremental changes over time in case I break someone's flow or detect design flaws. I am also considering the breaking changes introduced in make 4.4, which will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a deadline to finish all the new build-chain work. --------- Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-10-05 08:26:29 +00:00
PSX_RSTAGES := rwrp
Add BOSS/MAR.BIN (#1608) Small PR to enable #557 and to let be used as an example for [our wiki](https://github.com/Xeeynamo/sotn-decomp/wiki/Decompilation#add-new-overlay). This was done with the new `make-config.py` tool.
2024-09-15 20:49:56 +00:00
PSX_BOSTAGES := mar
#1635: RBO3 overlay (#1671)
2024-09-25 17:02:18 +00:00
PSX_RBOSTAGES := rbo3
PSX_ALLSTAGES := $(addprefix st,$(PSX_KSTAGES)) $(addprefix st,$(PSX_RSTAGES)) $(addprefix bo,$(PSX_BOSTAGES)) $(addprefix bo,$(PSX_RBOSTAGES))
#1637: Add servant/tt_001.bin (#1659) Followed [the guide](https://github.com/Xeeynamo/sotn-decomp/wiki/Decompilation#add-new-overlay) which seemed to work relatively painlessly for this overlay. I hit an issue in `make-config.py` where no duplicates were found and have added a separate commit which fixed that. Otherwise no issues with guide. ``` josh@JoshsPC:/mnt/c/dev/sotn-decomp$ tools/make-config.py tt_001 ✔ generating psx splat config ✔ splitting config/splat.us.tt_001.yaml ✔ adjusting files at src/servant/tt_001 ✔ disassembling matched functions ✖ finding duplicates across overlays Traceback (most recent call last): File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1132, in <module> raise e File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1126, in <module> make_config(args.input, args.version) File "/mnt/c/dev/sotn-decomp/tools/make-config.py", line 1106, in make_config if found > 0: TypeError: '>' not supported between instances of 'NoneType' and 'int' ```
2024-09-24 01:39:35 +00:00
PSX_SERVANTS := tt_000 tt_001
Enhance make extract on PSX (#662) Previously if either the YAML or the symbol list were changed, we were forced to either manually delete the `asm` folder of the specific overlay or to invoke `make clean`, where the latter forced us to re-extract all the overlays via `make -j extract`. That changes with this PR. How `make` works is `target: dep1 dep2` where if one of the dependencies has the last modified date greater than the target, the rule is triggered again. Previously we were extracting overlays doing `make extract_stcen`. But since every overlay extraction generates a linker script I now changed the rules to do `make build/us/stcen.ld` to extract the same overlay. As I explained above, if either the YAML or one of the related symbols changes, the linker modified date will be older and the extraction for that overlay to be trigger again. The above allowed me to stop polluting the repo root with the linker scripts as they are now moved into `build/us` or `build/hd` depending of what you're trying to build. If you do `make extract` twice in a row you will be welcomed with a `make: Nothing to be done for 'extract'.`. There are still some instances where you **might** need `make clean` beforehand, especially when modifying the `segment` section in the YAML or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I want to see if the current solution will be enough. ### Please try removing `make clean` from your workflow, once this is merged, to quickly detect possible problems with this new approach. I am also planning to make more substantial changes on our build-chain like this PR or #660 in the future. I am aiming to small incremental changes over time in case I break someone's flow or detect design flaws. I am also considering the breaking changes introduced in make 4.4, which will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a deadline to finish all the new build-chain work. --------- Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-10-05 08:26:29 +00:00
PSX_US_TARGETS := main $(PSX_OVLS) $(PSX_ALLSTAGES) $(PSX_SERVANTS)
Add CEN Overlay to HD (#1705) Adds the CEN overaly to the HD version of the game. Resolves #1638. This uses most, but not all, matching functions. Some were excluded because due to symbols mapping that has not bee completed yet. Not all data has been imported, and mapping symbols, and data should make many of the remaining functions easy to import.
2024-10-02 05:31:18 +00:00
PSX_HD_TARGETS := dra ric tt_000 stcen stwrp
Enhance make extract on PSX (#662) Previously if either the YAML or the symbol list were changed, we were forced to either manually delete the `asm` folder of the specific overlay or to invoke `make clean`, where the latter forced us to re-extract all the overlays via `make -j extract`. That changes with this PR. How `make` works is `target: dep1 dep2` where if one of the dependencies has the last modified date greater than the target, the rule is triggered again. Previously we were extracting overlays doing `make extract_stcen`. But since every overlay extraction generates a linker script I now changed the rules to do `make build/us/stcen.ld` to extract the same overlay. As I explained above, if either the YAML or one of the related symbols changes, the linker modified date will be older and the extraction for that overlay to be trigger again. The above allowed me to stop polluting the repo root with the linker scripts as they are now moved into `build/us` or `build/hd` depending of what you're trying to build. If you do `make extract` twice in a row you will be welcomed with a `make: Nothing to be done for 'extract'.`. There are still some instances where you **might** need `make clean` beforehand, especially when modifying the `segment` section in the YAML or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I want to see if the current solution will be enough. ### Please try removing `make clean` from your workflow, once this is merged, to quickly detect possible problems with this new approach. I am also planning to make more substantial changes on our build-chain like this PR or #660 in the future. I am aiming to small incremental changes over time in case I break someone's flow or detect design flaws. I am also considering the breaking changes introduced in make 4.4, which will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a deadline to finish all the new build-chain work. --------- Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-10-05 08:26:29 +00:00
# immovable
PSX_BASE_SYMS := $(CONFIG_DIR)/symbols.$(VERSION).txt
extract_us: $(addprefix $(BUILD_DIR)/,$(addsuffix .ld,$(PSX_US_TARGETS)))
Extract BIN/F_GAME.BIN (#886) Mostly a proof of concept to extract resources from `BIN/F_GAME.BIN`. There are tons of hacks to make this work: * Tweaked png2s to decode binaries into PNGs * Add a `config/gfx.game.json` that describes how to extract different sprites out of a binary file * Coded the palette location by reading the `clut` parameter in the source code. A clut value of `0x196` corresponds to `"palette": "0x96"` or `"palette": 150`. * Coded the bits per pixel for a couple of 8-bit images * Coded the palette location of those graphics that expects the palette from DRA.BIN instead of F_GAME.BIN * For all the undiscovered palette, I called the files `unk` and defaulted their palette to grey The expectation is that sprites will slowly be documented and added into `config/gfx.game.json` by modders and enthusiasts. This does not yet pack back these PNGs into the binary file. The way these assets are extracted is probably not final or perfect, but we have to start from somewhere. ![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/ce12ad2d-d95d-41e8-ac31-186cd36ac965)
2023-12-25 19:23:25 +00:00
$(PNG2S) bdecode config/gfx.game.json disks/us assets/game
New asset manager (#1343) This aims to deprecate all the Splat tools in `tools/splat_ext` in favour of a more centralised asset manager. This brings the following advantages: * Much faster extraction * Faster build * Automatically define `static` symbols or unique names whenever `static` is not possible * Allow to embed assets into the output binary * Drastically simplify `Makefile` by removing all the asset build rules * Avoid situations where it is not possible to extract and build assets that is not 4-byte aligned This is achieved by having the splat YAML targeting a normal C file as data and have an external tool to take care of the following: 1. Extract asset files straight from the overlay binary file into human readable file in `assets/st/STAGE_NAME` 2. Build assets as header files that go into `src/st/STAGE_NAME` to just include them from any C file This requires each stage header to have the following new format: please see `src/st/nz0/header.c` Built assets in `src/st` are ignored by Git. As for now, for simplicity sake, the steps `make extract_assets` and `make build_assets` are just executed within `make extract` exclusively for the US version. I plan to auto-generate files such as `src/st/nz0/tile_data.c`. For a first iteration I am aiming to handle the following: * [X] Extract rooms: `assets/st/*/rooms.json` * [X] Extract room layers: `assets/st/*/entity_layouts.json` * [X] Extract tilemap data: `assets/st/*/tilemap_*.bin` * [X] Extract tilemap definitions: `assets/st/*/tiledef_*.json` * [X] Extract sprites: `assets/st/*/sprites.json` * [x] Extract entity layouts * [X] Build rooms: `src/st/*/rooms.h` * [X] Build room layers: `src/st/*/layers.h` * [X] Build tilemap data: `src/st/*/tilemap_*.h` * [X] Build tilemap definitions: `src/st/*/tiledef_*.h` * [x] Build sprites (aka `g_SpriteBanks`) * [x] Build entity layouts * [x] Allow the tool to suggest how to adjust the Splat config for each overlay I want the tool to cover the following stages: * [x] CEN * [x] DRE * ~MAD~ I do not think this can be done, it is way too different from the other overlays * [x] NO3 * [x] NP3 * [X] NZ0 * [x] ST0 * [X] WRP * [x] RWRP * ~WRP (PSP)~ Maybe in a follow-up PR For a later iteration I plan on extracting and build: * Entity GFX thingie * The CLUT thingie in the header * Uncompressed GFX data * Cutscene data * Blueprints * The `src/config_us.h` thingie --------- Co-authored-by: Josh Lory <josh.lory@outlook.com>
2024-07-02 20:38:36 +00:00
make extract_assets
make build_assets
Enhance make extract on PSX (#662) Previously if either the YAML or the symbol list were changed, we were forced to either manually delete the `asm` folder of the specific overlay or to invoke `make clean`, where the latter forced us to re-extract all the overlays via `make -j extract`. That changes with this PR. How `make` works is `target: dep1 dep2` where if one of the dependencies has the last modified date greater than the target, the rule is triggered again. Previously we were extracting overlays doing `make extract_stcen`. But since every overlay extraction generates a linker script I now changed the rules to do `make build/us/stcen.ld` to extract the same overlay. As I explained above, if either the YAML or one of the related symbols changes, the linker modified date will be older and the extraction for that overlay to be trigger again. The above allowed me to stop polluting the repo root with the linker scripts as they are now moved into `build/us` or `build/hd` depending of what you're trying to build. If you do `make extract` twice in a row you will be welcomed with a `make: Nothing to be done for 'extract'.`. There are still some instances where you **might** need `make clean` beforehand, especially when modifying the `segment` section in the YAML or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I want to see if the current solution will be enough. ### Please try removing `make clean` from your workflow, once this is merged, to quickly detect possible problems with this new approach. I am also planning to make more substantial changes on our build-chain like this PR or #660 in the future. I am aiming to small incremental changes over time in case I break someone's flow or detect design flaws. I am also considering the breaking changes introduced in make 4.4, which will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a deadline to finish all the new build-chain work. --------- Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-10-05 08:26:29 +00:00
extract_hd: $(addprefix $(BUILD_DIR)/,$(addsuffix .ld,$(PSX_HD_TARGETS)))
HD WRP matching (#1421) Mostly identical to MAD and PSP. There are four very different functions from US that needs to be decompiled to be 100% complete: * [x] `e_misc.c` `EntityMessageBox` * [x] `e_misc.c` `EntityRelicOrb` * [x] ~`e_stage_name.c` `StageNamePopupHelper`~ it does not exist * [x] `e_stage_name.c` `EntityStageNamePopup` These four very different functions are the only ones that use the JP text instead of the US one. This overlay also lacks of `BlitChar`. `EntityStageNamePopup` is very similar to the PSP counterpart. I used @joshlory scratch to match the HD part. `EntityRelicOrb` had some #ifdef between US and BETA (aka the MAD overlay). I found some code that crossed between BETA and HD, so I just put an `#else` after `VERSION_US`. We are starting to reconstruct how the source code originally evolved across different game builds.
2024-07-20 00:05:26 +00:00
make extract_assets_hd
Weapon assets and statically linked (#1470) Allow to compile weapon overlays as a single C files. The benefit is to disallow symbols to be exported by statically compile them all but the header. We would no longer need to use `OVL_EXPORT` as long as the symbols are marked as `static`. The frames data is also generated as an array of data that can be just included into their respective C files, generated by JSON files on the fly. I changed `animset` to `frameset` as the data does not represent a set of animations but rather a set of frames, where every frame has multiple sprites. The information on how the frames are played (e.g. animation) is found elsewhere. Following the full list of changes to achieve this project: * The `animset` from splat is now dummied out * The `frameset` JSON is now generated by the asset manager * The `frameset` JSON is now compiled into `src/weapon/w_0xx_y.h` * The `header.c` has been moved into `shared.h` * For simplicity I added a `#define g_Animset w_000_1`. I did not want to hack the asset manager too much to force the name to be `g_Animset` and I did not want to change all the symbols yet. * Simplified `weapon_pc.c` as we now only need the exported header * Always try to build the most up-to-date asset manager: this should avoid weird errors whenever people pull new changes As the asset manager now can accept those `config` files, now the CLI supports the following commands: * `stage extract` * `stage build` * `config extract` (NEW) * `config build` (NEW) The config structure is heavily inspired to Splat to maintain consistency and continuity.
2024-08-08 23:57:26 +00:00
make build_assets_hd
Enhance make extract on PSX (#662) Previously if either the YAML or the symbol list were changed, we were forced to either manually delete the `asm` folder of the specific overlay or to invoke `make clean`, where the latter forced us to re-extract all the overlays via `make -j extract`. That changes with this PR. How `make` works is `target: dep1 dep2` where if one of the dependencies has the last modified date greater than the target, the rule is triggered again. Previously we were extracting overlays doing `make extract_stcen`. But since every overlay extraction generates a linker script I now changed the rules to do `make build/us/stcen.ld` to extract the same overlay. As I explained above, if either the YAML or one of the related symbols changes, the linker modified date will be older and the extraction for that overlay to be trigger again. The above allowed me to stop polluting the repo root with the linker scripts as they are now moved into `build/us` or `build/hd` depending of what you're trying to build. If you do `make extract` twice in a row you will be welcomed with a `make: Nothing to be done for 'extract'.`. There are still some instances where you **might** need `make clean` beforehand, especially when modifying the `segment` section in the YAML or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I want to see if the current solution will be enough. ### Please try removing `make clean` from your workflow, once this is merged, to quickly detect possible problems with this new approach. I am also planning to make more substantial changes on our build-chain like this PR or #660 in the future. I am aiming to small incremental changes over time in case I break someone's flow or detect design flaws. I am also considering the breaking changes introduced in make 4.4, which will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a deadline to finish all the new build-chain work. --------- Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-10-05 08:26:29 +00:00
extract_disk_us: extract_disk_psxus
extract_disk_hd: extract_disk_pspeu
extract_disk_psx%: $(SOTNDISK)
$(SOTNDISK) extract disks/sotn.$*.cue disks/$*
Fix Directory Creation Dependencies in Makefile (#1353) This updates the Makefile so that targets which depend on phony targets (like `*_dirs`) have direct dependencies on those targets rather than relying on transient dependencies from dependent targets. Other targets explicitly create parent directories for their files. This resolves #739, making it possible to using make-4.4 to build in parallel.
2024-06-28 13:23:41 +00:00
# todo: these should have an explicit dependency on extract disk
$(BUILD_DIR)/main.ld: $(CONFIG_DIR)/splat.$(VERSION).main.yaml | main_dirs
Enhance make extract on PSX (#662) Previously if either the YAML or the symbol list were changed, we were forced to either manually delete the `asm` folder of the specific overlay or to invoke `make clean`, where the latter forced us to re-extract all the overlays via `make -j extract`. That changes with this PR. How `make` works is `target: dep1 dep2` where if one of the dependencies has the last modified date greater than the target, the rule is triggered again. Previously we were extracting overlays doing `make extract_stcen`. But since every overlay extraction generates a linker script I now changed the rules to do `make build/us/stcen.ld` to extract the same overlay. As I explained above, if either the YAML or one of the related symbols changes, the linker modified date will be older and the extraction for that overlay to be trigger again. The above allowed me to stop polluting the repo root with the linker scripts as they are now moved into `build/us` or `build/hd` depending of what you're trying to build. If you do `make extract` twice in a row you will be welcomed with a `make: Nothing to be done for 'extract'.`. There are still some instances where you **might** need `make clean` beforehand, especially when modifying the `segment` section in the YAML or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I want to see if the current solution will be enough. ### Please try removing `make clean` from your workflow, once this is merged, to quickly detect possible problems with this new approach. I am also planning to make more substantial changes on our build-chain like this PR or #660 in the future. I am aiming to small incremental changes over time in case I break someone's flow or detect design flaws. I am also considering the breaking changes introduced in make 4.4, which will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a deadline to finish all the new build-chain work. --------- Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-10-05 08:26:29 +00:00
$(SPLAT) $<
Fix Directory Creation Dependencies in Makefile (#1353) This updates the Makefile so that targets which depend on phony targets (like `*_dirs`) have direct dependencies on those targets rather than relying on transient dependencies from dependent targets. Other targets explicitly create parent directories for their files. This resolves #739, making it possible to using make-4.4 to build in parallel.
2024-06-28 13:23:41 +00:00
touch $@
# todo: these should have an explicit dependency on extract disk
$(BUILD_DIR)/dra.ld: $(CONFIG_DIR)/splat.$(VERSION).dra.yaml $(PSX_BASE_SYMS) $(CONFIG_DIR)/symbols.$(VERSION).dra.txt | dra_dirs
Enhance make extract on PSX (#662) Previously if either the YAML or the symbol list were changed, we were forced to either manually delete the `asm` folder of the specific overlay or to invoke `make clean`, where the latter forced us to re-extract all the overlays via `make -j extract`. That changes with this PR. How `make` works is `target: dep1 dep2` where if one of the dependencies has the last modified date greater than the target, the rule is triggered again. Previously we were extracting overlays doing `make extract_stcen`. But since every overlay extraction generates a linker script I now changed the rules to do `make build/us/stcen.ld` to extract the same overlay. As I explained above, if either the YAML or one of the related symbols changes, the linker modified date will be older and the extraction for that overlay to be trigger again. The above allowed me to stop polluting the repo root with the linker scripts as they are now moved into `build/us` or `build/hd` depending of what you're trying to build. If you do `make extract` twice in a row you will be welcomed with a `make: Nothing to be done for 'extract'.`. There are still some instances where you **might** need `make clean` beforehand, especially when modifying the `segment` section in the YAML or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I want to see if the current solution will be enough. ### Please try removing `make clean` from your workflow, once this is merged, to quickly detect possible problems with this new approach. I am also planning to make more substantial changes on our build-chain like this PR or #660 in the future. I am aiming to small incremental changes over time in case I break someone's flow or detect design flaws. I am also considering the breaking changes introduced in make 4.4, which will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a deadline to finish all the new build-chain work. --------- Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-10-05 08:26:29 +00:00
$(SPLAT) $<
Fix Directory Creation Dependencies in Makefile (#1353) This updates the Makefile so that targets which depend on phony targets (like `*_dirs`) have direct dependencies on those targets rather than relying on transient dependencies from dependent targets. Other targets explicitly create parent directories for their files. This resolves #739, making it possible to using make-4.4 to build in parallel.
2024-06-28 13:23:41 +00:00
touch $@
# todo: these should have an explicit dependency on extract disk
$(BUILD_DIR)/ric.ld: $(CONFIG_DIR)/splat.$(VERSION).ric.yaml $(PSX_BASE_SYMS) $(CONFIG_DIR)/symbols.$(VERSION).ric.txt | ric_dirs
Enhance make extract on PSX (#662) Previously if either the YAML or the symbol list were changed, we were forced to either manually delete the `asm` folder of the specific overlay or to invoke `make clean`, where the latter forced us to re-extract all the overlays via `make -j extract`. That changes with this PR. How `make` works is `target: dep1 dep2` where if one of the dependencies has the last modified date greater than the target, the rule is triggered again. Previously we were extracting overlays doing `make extract_stcen`. But since every overlay extraction generates a linker script I now changed the rules to do `make build/us/stcen.ld` to extract the same overlay. As I explained above, if either the YAML or one of the related symbols changes, the linker modified date will be older and the extraction for that overlay to be trigger again. The above allowed me to stop polluting the repo root with the linker scripts as they are now moved into `build/us` or `build/hd` depending of what you're trying to build. If you do `make extract` twice in a row you will be welcomed with a `make: Nothing to be done for 'extract'.`. There are still some instances where you **might** need `make clean` beforehand, especially when modifying the `segment` section in the YAML or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I want to see if the current solution will be enough. ### Please try removing `make clean` from your workflow, once this is merged, to quickly detect possible problems with this new approach. I am also planning to make more substantial changes on our build-chain like this PR or #660 in the future. I am aiming to small incremental changes over time in case I break someone's flow or detect design flaws. I am also considering the breaking changes introduced in make 4.4, which will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a deadline to finish all the new build-chain work. --------- Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-10-05 08:26:29 +00:00
$(SPLAT) $<
Fix Directory Creation Dependencies in Makefile (#1353) This updates the Makefile so that targets which depend on phony targets (like `*_dirs`) have direct dependencies on those targets rather than relying on transient dependencies from dependent targets. Other targets explicitly create parent directories for their files. This resolves #739, making it possible to using make-4.4 to build in parallel.
2024-06-28 13:23:41 +00:00
touch $@
# todo: these should have an explicit dependency on extract disk
$(BUILD_DIR)/stmad.ld: $(CONFIG_DIR)/splat.$(VERSION).stmad.yaml $(CONFIG_DIR)/symbols.beta.txt $(CONFIG_DIR)/symbols.stmad.txt | stmad_dirs
Enhance make extract on PSX (#662) Previously if either the YAML or the symbol list were changed, we were forced to either manually delete the `asm` folder of the specific overlay or to invoke `make clean`, where the latter forced us to re-extract all the overlays via `make -j extract`. That changes with this PR. How `make` works is `target: dep1 dep2` where if one of the dependencies has the last modified date greater than the target, the rule is triggered again. Previously we were extracting overlays doing `make extract_stcen`. But since every overlay extraction generates a linker script I now changed the rules to do `make build/us/stcen.ld` to extract the same overlay. As I explained above, if either the YAML or one of the related symbols changes, the linker modified date will be older and the extraction for that overlay to be trigger again. The above allowed me to stop polluting the repo root with the linker scripts as they are now moved into `build/us` or `build/hd` depending of what you're trying to build. If you do `make extract` twice in a row you will be welcomed with a `make: Nothing to be done for 'extract'.`. There are still some instances where you **might** need `make clean` beforehand, especially when modifying the `segment` section in the YAML or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I want to see if the current solution will be enough. ### Please try removing `make clean` from your workflow, once this is merged, to quickly detect possible problems with this new approach. I am also planning to make more substantial changes on our build-chain like this PR or #660 in the future. I am aiming to small incremental changes over time in case I break someone's flow or detect design flaws. I am also considering the breaking changes introduced in make 4.4, which will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a deadline to finish all the new build-chain work. --------- Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-10-05 08:26:29 +00:00
$(SPLAT) $<
$(GFXSTAGE) d disks/$(VERSION)/ST/MAD/F_MAD.BIN $(ASSETS_DIR)/st/mad
Fix Directory Creation Dependencies in Makefile (#1353) This updates the Makefile so that targets which depend on phony targets (like `*_dirs`) have direct dependencies on those targets rather than relying on transient dependencies from dependent targets. Other targets explicitly create parent directories for their files. This resolves #739, making it possible to using make-4.4 to build in parallel.
2024-06-28 13:23:41 +00:00
# todo: these should have an explicit dependency on extract disk
$(BUILD_DIR)/st%.ld: $(CONFIG_DIR)/splat.$(VERSION).st%.yaml $(PSX_BASE_SYMS) $(CONFIG_DIR)/symbols.$(VERSION).st%.txt | st%_dirs
Enhance make extract on PSX (#662) Previously if either the YAML or the symbol list were changed, we were forced to either manually delete the `asm` folder of the specific overlay or to invoke `make clean`, where the latter forced us to re-extract all the overlays via `make -j extract`. That changes with this PR. How `make` works is `target: dep1 dep2` where if one of the dependencies has the last modified date greater than the target, the rule is triggered again. Previously we were extracting overlays doing `make extract_stcen`. But since every overlay extraction generates a linker script I now changed the rules to do `make build/us/stcen.ld` to extract the same overlay. As I explained above, if either the YAML or one of the related symbols changes, the linker modified date will be older and the extraction for that overlay to be trigger again. The above allowed me to stop polluting the repo root with the linker scripts as they are now moved into `build/us` or `build/hd` depending of what you're trying to build. If you do `make extract` twice in a row you will be welcomed with a `make: Nothing to be done for 'extract'.`. There are still some instances where you **might** need `make clean` beforehand, especially when modifying the `segment` section in the YAML or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I want to see if the current solution will be enough. ### Please try removing `make clean` from your workflow, once this is merged, to quickly detect possible problems with this new approach. I am also planning to make more substantial changes on our build-chain like this PR or #660 in the future. I am aiming to small incremental changes over time in case I break someone's flow or detect design flaws. I am also considering the breaking changes introduced in make 4.4, which will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a deadline to finish all the new build-chain work. --------- Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-10-05 08:26:29 +00:00
$(SPLAT) $<
$(GFXSTAGE) d disks/$(VERSION)/ST/$$(echo '$*' | tr '[:lower:]' '[:upper:]')/F_$$(echo '$*' | tr '[:lower:]' '[:upper:]').BIN $(ASSETS_DIR)/st/$*
Add BOSS/MAR.BIN (#1608) Small PR to enable #557 and to let be used as an example for [our wiki](https://github.com/Xeeynamo/sotn-decomp/wiki/Decompilation#add-new-overlay). This was done with the new `make-config.py` tool.
2024-09-15 20:49:56 +00:00
$(BUILD_DIR)/bo%.ld: $(CONFIG_DIR)/splat.$(VERSION).bo%.yaml $(PSX_BASE_SYMS) $(CONFIG_DIR)/symbols.$(VERSION).bo%.txt | bo%_dirs
$(SPLAT) $<
$(GFXSTAGE) d disks/$(VERSION)/BOSS/$$(echo '$*' | tr '[:lower:]' '[:upper:]')/F_$$(echo '$*' | tr '[:lower:]' '[:upper:]').BIN $(ASSETS_DIR)/boss/$*
HD WRP matching (#1421) Mostly identical to MAD and PSP. There are four very different functions from US that needs to be decompiled to be 100% complete: * [x] `e_misc.c` `EntityMessageBox` * [x] `e_misc.c` `EntityRelicOrb` * [x] ~`e_stage_name.c` `StageNamePopupHelper`~ it does not exist * [x] `e_stage_name.c` `EntityStageNamePopup` These four very different functions are the only ones that use the JP text instead of the US one. This overlay also lacks of `BlitChar`. `EntityStageNamePopup` is very similar to the PSP counterpart. I used @joshlory scratch to match the HD part. `EntityRelicOrb` had some #ifdef between US and BETA (aka the MAD overlay). I found some code that crossed between BETA and HD, so I just put an `#else` after `VERSION_US`. We are starting to reconstruct how the source code originally evolved across different game builds.
2024-07-20 00:05:26 +00:00
build/hd/st%.ld: $(CONFIG_DIR)/splat.$(VERSION).st%.yaml $(PSX_BASE_SYMS) $(CONFIG_DIR)/symbols.$(VERSION).st%.txt | st%_dirs
$(SPLAT) $<
$(GFXSTAGE) d disks/pspeu/PSP_GAME/USRDIR/res/ps/hdbin/f_$*.bin $(ASSETS_DIR)/st/$*
Fix Directory Creation Dependencies in Makefile (#1353) This updates the Makefile so that targets which depend on phony targets (like `*_dirs`) have direct dependencies on those targets rather than relying on transient dependencies from dependent targets. Other targets explicitly create parent directories for their files. This resolves #739, making it possible to using make-4.4 to build in parallel.
2024-06-28 13:23:41 +00:00
$(BUILD_DIR)/tt_%.ld: $(CONFIG_DIR)/splat.$(VERSION).tt_%.yaml $(PSX_BASE_SYMS) $(CONFIG_DIR)/symbols.$(VERSION).tt_%.txt | tt_%_dirs
Enhance make extract on PSX (#662) Previously if either the YAML or the symbol list were changed, we were forced to either manually delete the `asm` folder of the specific overlay or to invoke `make clean`, where the latter forced us to re-extract all the overlays via `make -j extract`. That changes with this PR. How `make` works is `target: dep1 dep2` where if one of the dependencies has the last modified date greater than the target, the rule is triggered again. Previously we were extracting overlays doing `make extract_stcen`. But since every overlay extraction generates a linker script I now changed the rules to do `make build/us/stcen.ld` to extract the same overlay. As I explained above, if either the YAML or one of the related symbols changes, the linker modified date will be older and the extraction for that overlay to be trigger again. The above allowed me to stop polluting the repo root with the linker scripts as they are now moved into `build/us` or `build/hd` depending of what you're trying to build. If you do `make extract` twice in a row you will be welcomed with a `make: Nothing to be done for 'extract'.`. There are still some instances where you **might** need `make clean` beforehand, especially when modifying the `segment` section in the YAML or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I want to see if the current solution will be enough. ### Please try removing `make clean` from your workflow, once this is merged, to quickly detect possible problems with this new approach. I am also planning to make more substantial changes on our build-chain like this PR or #660 in the future. I am aiming to small incremental changes over time in case I break someone's flow or detect design flaws. I am also considering the breaking changes introduced in make 4.4, which will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a deadline to finish all the new build-chain work. --------- Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-10-05 08:26:29 +00:00
$(SPLAT) $<
Fix Directory Creation Dependencies in Makefile (#1353) This updates the Makefile so that targets which depend on phony targets (like `*_dirs`) have direct dependencies on those targets rather than relying on transient dependencies from dependent targets. Other targets explicitly create parent directories for their files. This resolves #739, making it possible to using make-4.4 to build in parallel.
2024-06-28 13:23:41 +00:00
touch $@
$(BUILD_DIR)/weapon.ld: $(CONFIG_DIR)/splat.$(VERSION).weapon.yaml $(PSX_BASE_SYMS) $(CONFIG_DIR)/symbols.$(VERSION).weapon.txt $(CONFIG_DIR)/symbols.$(VERSION).weapon.txt.in
Enhance make extract on PSX (#662) Previously if either the YAML or the symbol list were changed, we were forced to either manually delete the `asm` folder of the specific overlay or to invoke `make clean`, where the latter forced us to re-extract all the overlays via `make -j extract`. That changes with this PR. How `make` works is `target: dep1 dep2` where if one of the dependencies has the last modified date greater than the target, the rule is triggered again. Previously we were extracting overlays doing `make extract_stcen`. But since every overlay extraction generates a linker script I now changed the rules to do `make build/us/stcen.ld` to extract the same overlay. As I explained above, if either the YAML or one of the related symbols changes, the linker modified date will be older and the extraction for that overlay to be trigger again. The above allowed me to stop polluting the repo root with the linker scripts as they are now moved into `build/us` or `build/hd` depending of what you're trying to build. If you do `make extract` twice in a row you will be welcomed with a `make: Nothing to be done for 'extract'.`. There are still some instances where you **might** need `make clean` beforehand, especially when modifying the `segment` section in the YAML or renaming the symbols. A `rm -rf asm/$(VERSION)/$*` can help but I want to see if the current solution will be enough. ### Please try removing `make clean` from your workflow, once this is merged, to quickly detect possible problems with this new approach. I am also planning to make more substantial changes on our build-chain like this PR or #660 in the future. I am aiming to small incremental changes over time in case I break someone's flow or detect design flaws. I am also considering the breaking changes introduced in make 4.4, which will most likely be included in Ubuntu 24.04 LTS. I set March 2024 as a deadline to finish all the new build-chain work. --------- Co-authored-by: sozud <122322823+sozud@users.noreply.github.com>
2023-10-05 08:26:29 +00:00
$(SPLAT) $<
Fix Directory Creation Dependencies in Makefile (#1353) This updates the Makefile so that targets which depend on phony targets (like `*_dirs`) have direct dependencies on those targets rather than relying on transient dependencies from dependent targets. Other targets explicitly create parent directories for their files. This resolves #739, making it possible to using make-4.4 to build in parallel.
2024-06-28 13:23:41 +00:00
touch $@
Pass undefined_syms files to linker (#1065) Creating a PR for this, will look at updating splat in a separate PR
2024-02-18 14:57:03 +00:00
Use allegrex assembler for PSP (#1424) I moved *some* of the PSX-specific stuff out of the common `Makefile` and into the `Makefile.psx.mk` -- there is still more that could be done here, but this should be enough to have the correct rules for PSX and PSP with the desired assembler. Also threw in a bonus feature where I've removed the `move` macro from `macro.inc` because `maspsx` does this automagically for you now.
2024-07-22 19:36:28 +00:00
$(BUILD_DIR)/src/st/sel/%.c.o: src/st/sel/%.c $(MASPSX_APP) $(CC1PSX) src/st/sel/sel.h | stsel_dirs
$(CPP) $(CPP_FLAGS) -lang-c $< | $(SOTNSTR) | $(ICONV) | $(CC) $(CC_FLAGS) $(PSXCC_FLAGS) | $(MASPSX) | $(AS) $(AS_FLAGS) -o $@
$(BUILD_DIR)/$(ASSETS_DIR)/weapon/%_1.animset.o: $(ASSETS_DIR)/weapon/%_1.animset.json
./tools/splat_ext/animset.py gen-asm $< $(BUILD_DIR)/$(ASSETS_DIR)/weapon/$*_1.animset.s -s g_Animset
$(AS) $(AS_FLAGS) -o $@ $(BUILD_DIR)/$(ASSETS_DIR)/weapon/$*_1.animset.s
$(BUILD_DIR)/$(ASSETS_DIR)/weapon/%_2.animset.o: $(ASSETS_DIR)/weapon/%_2.animset.json
./tools/splat_ext/animset.py gen-asm $< $(BUILD_DIR)/$(ASSETS_DIR)/weapon/$*_2.animset.s -s g_Animset2
$(AS) $(AS_FLAGS) -o $@ $(BUILD_DIR)/$(ASSETS_DIR)/weapon/$*_2.animset.s
# assembly and c files
$(BUILD_DIR)/%.s.o: %.s
mkdir -p $(dir $@)
$(AS) $(AS_FLAGS) -o $@ $<
$(BUILD_DIR)/%.c.o: %.c $(MASPSX_APP) $(CC1PSX)
mkdir -p $(dir $@)
$(CPP) $(CPP_FLAGS) -lang-c $< | $(SOTNSTR) | $(ICONV) | $(CC) $(CC_FLAGS) $(PSXCC_FLAGS) | $(MASPSX) | $(AS) $(AS_FLAGS) -o $@
$(BUILD_DIR)/$(SRC_DIR)/main/psxsdk/libgpu/sys.c.o: $(SRC_DIR)/main/psxsdk/libgpu/sys.c $(MASPSX_APP) $(CC1PSX)
$(CPP) $(CPP_FLAGS) -lang-c $< | $(SOTNSTR) | $(ICONV) | $(CC) $(CC_FLAGS) $(PSXCC_FLAGS) | $(MASPSX_21) | $(AS) $(AS_FLAGS) -o $@
New asset manager (#1343) This aims to deprecate all the Splat tools in `tools/splat_ext` in favour of a more centralised asset manager. This brings the following advantages: * Much faster extraction * Faster build * Automatically define `static` symbols or unique names whenever `static` is not possible * Allow to embed assets into the output binary * Drastically simplify `Makefile` by removing all the asset build rules * Avoid situations where it is not possible to extract and build assets that is not 4-byte aligned This is achieved by having the splat YAML targeting a normal C file as data and have an external tool to take care of the following: 1. Extract asset files straight from the overlay binary file into human readable file in `assets/st/STAGE_NAME` 2. Build assets as header files that go into `src/st/STAGE_NAME` to just include them from any C file This requires each stage header to have the following new format: please see `src/st/nz0/header.c` Built assets in `src/st` are ignored by Git. As for now, for simplicity sake, the steps `make extract_assets` and `make build_assets` are just executed within `make extract` exclusively for the US version. I plan to auto-generate files such as `src/st/nz0/tile_data.c`. For a first iteration I am aiming to handle the following: * [X] Extract rooms: `assets/st/*/rooms.json` * [X] Extract room layers: `assets/st/*/entity_layouts.json` * [X] Extract tilemap data: `assets/st/*/tilemap_*.bin` * [X] Extract tilemap definitions: `assets/st/*/tiledef_*.json` * [X] Extract sprites: `assets/st/*/sprites.json` * [x] Extract entity layouts * [X] Build rooms: `src/st/*/rooms.h` * [X] Build room layers: `src/st/*/layers.h` * [X] Build tilemap data: `src/st/*/tilemap_*.h` * [X] Build tilemap definitions: `src/st/*/tiledef_*.h` * [x] Build sprites (aka `g_SpriteBanks`) * [x] Build entity layouts * [x] Allow the tool to suggest how to adjust the Splat config for each overlay I want the tool to cover the following stages: * [x] CEN * [x] DRE * ~MAD~ I do not think this can be done, it is way too different from the other overlays * [x] NO3 * [x] NP3 * [X] NZ0 * [x] ST0 * [X] WRP * [x] RWRP * ~WRP (PSP)~ Maybe in a follow-up PR For a later iteration I plan on extracting and build: * Entity GFX thingie * The CLUT thingie in the header * Uncompressed GFX data * Cutscene data * Blueprints * The `src/config_us.h` thingie --------- Co-authored-by: Josh Lory <josh.lory@outlook.com>
2024-07-02 20:38:36 +00:00
extract_assets: $(SOTNASSETS)
Weapon assets and statically linked (#1470) Allow to compile weapon overlays as a single C files. The benefit is to disallow symbols to be exported by statically compile them all but the header. We would no longer need to use `OVL_EXPORT` as long as the symbols are marked as `static`. The frames data is also generated as an array of data that can be just included into their respective C files, generated by JSON files on the fly. I changed `animset` to `frameset` as the data does not represent a set of animations but rather a set of frames, where every frame has multiple sprites. The information on how the frames are played (e.g. animation) is found elsewhere. Following the full list of changes to achieve this project: * The `animset` from splat is now dummied out * The `frameset` JSON is now generated by the asset manager * The `frameset` JSON is now compiled into `src/weapon/w_0xx_y.h` * The `header.c` has been moved into `shared.h` * For simplicity I added a `#define g_Animset w_000_1`. I did not want to hack the asset manager too much to force the name to be `g_Animset` and I did not want to change all the symbols yet. * Simplified `weapon_pc.c` as we now only need the exported header * Always try to build the most up-to-date asset manager: this should avoid weird errors whenever people pull new changes As the asset manager now can accept those `config` files, now the CLI supports the following commands: * `stage extract` * `stage build` * `config extract` (NEW) * `config build` (NEW) The config structure is heavily inspired to Splat to maintain consistency and continuity.
2024-08-08 23:57:26 +00:00
cd tools/sotn-assets; $(GO) install
$(SOTNASSETS) stage extract -stage_ovl disks/$(VERSION)/ST/CEN/CEN.BIN -o assets/st/cen
$(SOTNASSETS) stage extract -stage_ovl disks/$(VERSION)/ST/DRE/DRE.BIN -o assets/st/dre
$(SOTNASSETS) stage extract -stage_ovl disks/$(VERSION)/ST/NO3/NO3.BIN -o assets/st/no3
$(SOTNASSETS) stage extract -stage_ovl disks/$(VERSION)/ST/NP3/NP3.BIN -o assets/st/np3
$(SOTNASSETS) stage extract -stage_ovl disks/$(VERSION)/ST/NZ0/NZ0.BIN -o assets/st/nz0
$(SOTNASSETS) stage extract -stage_ovl disks/$(VERSION)/ST/ST0/ST0.BIN -o assets/st/st0
$(SOTNASSETS) stage extract -stage_ovl disks/$(VERSION)/ST/WRP/WRP.BIN -o assets/st/wrp
$(SOTNASSETS) stage extract -stage_ovl disks/$(VERSION)/ST/RWRP/RWRP.BIN -o assets/st/rwrp
Add BOSS/MAR.BIN (#1608) Small PR to enable #557 and to let be used as an example for [our wiki](https://github.com/Xeeynamo/sotn-decomp/wiki/Decompilation#add-new-overlay). This was done with the new `make-config.py` tool.
2024-09-15 20:49:56 +00:00
$(SOTNASSETS) stage extract -stage_ovl disks/$(VERSION)/BOSS/MAR/MAR.BIN -o assets/boss/mar
Asset cutscene two stages (#1738) I completely rewrote the cutscene asset handler. Now instead of parsing the data from the original overlay into a C-like header file, it instead follows a two-stage process. This works by extracting it in `asset/` with `make extract_assets`, to then allow modders to modify the file and build it as a C-like header with `make build_assets`. This also aims to fix #1701 as the build process takes account of the two-stage process. I created a framework where each asset type should only make available the two methods `Extract` and `Build`. The entire transformation process should be isolated to not create cognitive overload like what we can find in `build.go`. I would need to migrate all the existing asset types to properly use this new framework. The old code served well enough to understand how to build the entire infrastructure, but it needs to be migrated using the new pattern. Last, but not least, I renamed `config/assets.us.weapon.yaml` to `config/assets.us.yaml` as it is now used by all the overlays
2024-10-06 12:10:06 +00:00
$(SOTNASSETS) config extract config/assets.us.yaml
HD WRP matching (#1421) Mostly identical to MAD and PSP. There are four very different functions from US that needs to be decompiled to be 100% complete: * [x] `e_misc.c` `EntityMessageBox` * [x] `e_misc.c` `EntityRelicOrb` * [x] ~`e_stage_name.c` `StageNamePopupHelper`~ it does not exist * [x] `e_stage_name.c` `EntityStageNamePopup` These four very different functions are the only ones that use the JP text instead of the US one. This overlay also lacks of `BlitChar`. `EntityStageNamePopup` is very similar to the PSP counterpart. I used @joshlory scratch to match the HD part. `EntityRelicOrb` had some #ifdef between US and BETA (aka the MAD overlay). I found some code that crossed between BETA and HD, so I just put an `#else` after `VERSION_US`. We are starting to reconstruct how the source code originally evolved across different game builds.
2024-07-20 00:05:26 +00:00
extract_assets_hd: $(SOTNASSETS)
Weapon assets and statically linked (#1470) Allow to compile weapon overlays as a single C files. The benefit is to disallow symbols to be exported by statically compile them all but the header. We would no longer need to use `OVL_EXPORT` as long as the symbols are marked as `static`. The frames data is also generated as an array of data that can be just included into their respective C files, generated by JSON files on the fly. I changed `animset` to `frameset` as the data does not represent a set of animations but rather a set of frames, where every frame has multiple sprites. The information on how the frames are played (e.g. animation) is found elsewhere. Following the full list of changes to achieve this project: * The `animset` from splat is now dummied out * The `frameset` JSON is now generated by the asset manager * The `frameset` JSON is now compiled into `src/weapon/w_0xx_y.h` * The `header.c` has been moved into `shared.h` * For simplicity I added a `#define g_Animset w_000_1`. I did not want to hack the asset manager too much to force the name to be `g_Animset` and I did not want to change all the symbols yet. * Simplified `weapon_pc.c` as we now only need the exported header * Always try to build the most up-to-date asset manager: this should avoid weird errors whenever people pull new changes As the asset manager now can accept those `config` files, now the CLI supports the following commands: * `stage extract` * `stage build` * `config extract` (NEW) * `config build` (NEW) The config structure is heavily inspired to Splat to maintain consistency and continuity.
2024-08-08 23:57:26 +00:00
cd tools/sotn-assets; $(GO) install
Add CEN Overlay to HD (#1705) Adds the CEN overaly to the HD version of the game. Resolves #1638. This uses most, but not all, matching functions. Some were excluded because due to symbols mapping that has not bee completed yet. Not all data has been imported, and mapping symbols, and data should make many of the remaining functions easy to import.
2024-10-02 05:31:18 +00:00
$(SOTNASSETS) stage extract -stage_ovl disks/pspeu/PSP_GAME/USRDIR/res/ps/hdbin/cen.bin -o assets/st/cen
Weapon assets and statically linked (#1470) Allow to compile weapon overlays as a single C files. The benefit is to disallow symbols to be exported by statically compile them all but the header. We would no longer need to use `OVL_EXPORT` as long as the symbols are marked as `static`. The frames data is also generated as an array of data that can be just included into their respective C files, generated by JSON files on the fly. I changed `animset` to `frameset` as the data does not represent a set of animations but rather a set of frames, where every frame has multiple sprites. The information on how the frames are played (e.g. animation) is found elsewhere. Following the full list of changes to achieve this project: * The `animset` from splat is now dummied out * The `frameset` JSON is now generated by the asset manager * The `frameset` JSON is now compiled into `src/weapon/w_0xx_y.h` * The `header.c` has been moved into `shared.h` * For simplicity I added a `#define g_Animset w_000_1`. I did not want to hack the asset manager too much to force the name to be `g_Animset` and I did not want to change all the symbols yet. * Simplified `weapon_pc.c` as we now only need the exported header * Always try to build the most up-to-date asset manager: this should avoid weird errors whenever people pull new changes As the asset manager now can accept those `config` files, now the CLI supports the following commands: * `stage extract` * `stage build` * `config extract` (NEW) * `config build` (NEW) The config structure is heavily inspired to Splat to maintain consistency and continuity.
2024-08-08 23:57:26 +00:00
$(SOTNASSETS) stage extract -stage_ovl disks/pspeu/PSP_GAME/USRDIR/res/ps/hdbin/wrp.bin -o assets/st/wrp
New asset manager (#1343) This aims to deprecate all the Splat tools in `tools/splat_ext` in favour of a more centralised asset manager. This brings the following advantages: * Much faster extraction * Faster build * Automatically define `static` symbols or unique names whenever `static` is not possible * Allow to embed assets into the output binary * Drastically simplify `Makefile` by removing all the asset build rules * Avoid situations where it is not possible to extract and build assets that is not 4-byte aligned This is achieved by having the splat YAML targeting a normal C file as data and have an external tool to take care of the following: 1. Extract asset files straight from the overlay binary file into human readable file in `assets/st/STAGE_NAME` 2. Build assets as header files that go into `src/st/STAGE_NAME` to just include them from any C file This requires each stage header to have the following new format: please see `src/st/nz0/header.c` Built assets in `src/st` are ignored by Git. As for now, for simplicity sake, the steps `make extract_assets` and `make build_assets` are just executed within `make extract` exclusively for the US version. I plan to auto-generate files such as `src/st/nz0/tile_data.c`. For a first iteration I am aiming to handle the following: * [X] Extract rooms: `assets/st/*/rooms.json` * [X] Extract room layers: `assets/st/*/entity_layouts.json` * [X] Extract tilemap data: `assets/st/*/tilemap_*.bin` * [X] Extract tilemap definitions: `assets/st/*/tiledef_*.json` * [X] Extract sprites: `assets/st/*/sprites.json` * [x] Extract entity layouts * [X] Build rooms: `src/st/*/rooms.h` * [X] Build room layers: `src/st/*/layers.h` * [X] Build tilemap data: `src/st/*/tilemap_*.h` * [X] Build tilemap definitions: `src/st/*/tiledef_*.h` * [x] Build sprites (aka `g_SpriteBanks`) * [x] Build entity layouts * [x] Allow the tool to suggest how to adjust the Splat config for each overlay I want the tool to cover the following stages: * [x] CEN * [x] DRE * ~MAD~ I do not think this can be done, it is way too different from the other overlays * [x] NO3 * [x] NP3 * [X] NZ0 * [x] ST0 * [X] WRP * [x] RWRP * ~WRP (PSP)~ Maybe in a follow-up PR For a later iteration I plan on extracting and build: * Entity GFX thingie * The CLUT thingie in the header * Uncompressed GFX data * Cutscene data * Blueprints * The `src/config_us.h` thingie --------- Co-authored-by: Josh Lory <josh.lory@outlook.com>
2024-07-02 20:38:36 +00:00
build_assets: $(SOTNASSETS)
Weapon assets and statically linked (#1470) Allow to compile weapon overlays as a single C files. The benefit is to disallow symbols to be exported by statically compile them all but the header. We would no longer need to use `OVL_EXPORT` as long as the symbols are marked as `static`. The frames data is also generated as an array of data that can be just included into their respective C files, generated by JSON files on the fly. I changed `animset` to `frameset` as the data does not represent a set of animations but rather a set of frames, where every frame has multiple sprites. The information on how the frames are played (e.g. animation) is found elsewhere. Following the full list of changes to achieve this project: * The `animset` from splat is now dummied out * The `frameset` JSON is now generated by the asset manager * The `frameset` JSON is now compiled into `src/weapon/w_0xx_y.h` * The `header.c` has been moved into `shared.h` * For simplicity I added a `#define g_Animset w_000_1`. I did not want to hack the asset manager too much to force the name to be `g_Animset` and I did not want to change all the symbols yet. * Simplified `weapon_pc.c` as we now only need the exported header * Always try to build the most up-to-date asset manager: this should avoid weird errors whenever people pull new changes As the asset manager now can accept those `config` files, now the CLI supports the following commands: * `stage extract` * `stage build` * `config extract` (NEW) * `config build` (NEW) The config structure is heavily inspired to Splat to maintain consistency and continuity.
2024-08-08 23:57:26 +00:00
$(SOTNASSETS) stage build_all -i assets/st/cen -o src/st/cen/
$(SOTNASSETS) stage build_all -i assets/st/dre -o src/st/dre/
$(SOTNASSETS) stage build_all -i assets/st/no3 -o src/st/no3/
$(SOTNASSETS) stage build_all -i assets/st/np3 -o src/st/np3/
$(SOTNASSETS) stage build_all -i assets/st/nz0 -o src/st/nz0/
$(SOTNASSETS) stage build_all -i assets/st/st0 -o src/st/st0/
$(SOTNASSETS) stage build_all -i assets/st/wrp -o src/st/wrp/
$(SOTNASSETS) stage build_all -i assets/st/rwrp -o src/st/rwrp/
Add BOSS/MAR.BIN (#1608) Small PR to enable #557 and to let be used as an example for [our wiki](https://github.com/Xeeynamo/sotn-decomp/wiki/Decompilation#add-new-overlay). This was done with the new `make-config.py` tool.
2024-09-15 20:49:56 +00:00
$(SOTNASSETS) stage build_all -i assets/boss/mar -o src/boss/mar/
Asset cutscene two stages (#1738) I completely rewrote the cutscene asset handler. Now instead of parsing the data from the original overlay into a C-like header file, it instead follows a two-stage process. This works by extracting it in `asset/` with `make extract_assets`, to then allow modders to modify the file and build it as a C-like header with `make build_assets`. This also aims to fix #1701 as the build process takes account of the two-stage process. I created a framework where each asset type should only make available the two methods `Extract` and `Build`. The entire transformation process should be isolated to not create cognitive overload like what we can find in `build.go`. I would need to migrate all the existing asset types to properly use this new framework. The old code served well enough to understand how to build the entire infrastructure, but it needs to be migrated using the new pattern. Last, but not least, I renamed `config/assets.us.weapon.yaml` to `config/assets.us.yaml` as it is now used by all the overlays
2024-10-06 12:10:06 +00:00
$(SOTNASSETS) config build config/assets.$(VERSION).yaml
Weapon assets and statically linked (#1470) Allow to compile weapon overlays as a single C files. The benefit is to disallow symbols to be exported by statically compile them all but the header. We would no longer need to use `OVL_EXPORT` as long as the symbols are marked as `static`. The frames data is also generated as an array of data that can be just included into their respective C files, generated by JSON files on the fly. I changed `animset` to `frameset` as the data does not represent a set of animations but rather a set of frames, where every frame has multiple sprites. The information on how the frames are played (e.g. animation) is found elsewhere. Following the full list of changes to achieve this project: * The `animset` from splat is now dummied out * The `frameset` JSON is now generated by the asset manager * The `frameset` JSON is now compiled into `src/weapon/w_0xx_y.h` * The `header.c` has been moved into `shared.h` * For simplicity I added a `#define g_Animset w_000_1`. I did not want to hack the asset manager too much to force the name to be `g_Animset` and I did not want to change all the symbols yet. * Simplified `weapon_pc.c` as we now only need the exported header * Always try to build the most up-to-date asset manager: this should avoid weird errors whenever people pull new changes As the asset manager now can accept those `config` files, now the CLI supports the following commands: * `stage extract` * `stage build` * `config extract` (NEW) * `config build` (NEW) The config structure is heavily inspired to Splat to maintain consistency and continuity.
2024-08-08 23:57:26 +00:00
build_assets_hd: $(SOTNASSETS)
Add CEN Overlay to HD (#1705) Adds the CEN overaly to the HD version of the game. Resolves #1638. This uses most, but not all, matching functions. Some were excluded because due to symbols mapping that has not bee completed yet. Not all data has been imported, and mapping symbols, and data should make many of the remaining functions easy to import.
2024-10-02 05:31:18 +00:00
$(SOTNASSETS) stage build_all -i assets/st/cen -o src/st/cen/
Weapon assets and statically linked (#1470) Allow to compile weapon overlays as a single C files. The benefit is to disallow symbols to be exported by statically compile them all but the header. We would no longer need to use `OVL_EXPORT` as long as the symbols are marked as `static`. The frames data is also generated as an array of data that can be just included into their respective C files, generated by JSON files on the fly. I changed `animset` to `frameset` as the data does not represent a set of animations but rather a set of frames, where every frame has multiple sprites. The information on how the frames are played (e.g. animation) is found elsewhere. Following the full list of changes to achieve this project: * The `animset` from splat is now dummied out * The `frameset` JSON is now generated by the asset manager * The `frameset` JSON is now compiled into `src/weapon/w_0xx_y.h` * The `header.c` has been moved into `shared.h` * For simplicity I added a `#define g_Animset w_000_1`. I did not want to hack the asset manager too much to force the name to be `g_Animset` and I did not want to change all the symbols yet. * Simplified `weapon_pc.c` as we now only need the exported header * Always try to build the most up-to-date asset manager: this should avoid weird errors whenever people pull new changes As the asset manager now can accept those `config` files, now the CLI supports the following commands: * `stage extract` * `stage build` * `config extract` (NEW) * `config build` (NEW) The config structure is heavily inspired to Splat to maintain consistency and continuity.
2024-08-08 23:57:26 +00:00
$(SOTNASSETS) stage build_all -i assets/st/wrp -o src/st/wrp/
New asset manager (#1343) This aims to deprecate all the Splat tools in `tools/splat_ext` in favour of a more centralised asset manager. This brings the following advantages: * Much faster extraction * Faster build * Automatically define `static` symbols or unique names whenever `static` is not possible * Allow to embed assets into the output binary * Drastically simplify `Makefile` by removing all the asset build rules * Avoid situations where it is not possible to extract and build assets that is not 4-byte aligned This is achieved by having the splat YAML targeting a normal C file as data and have an external tool to take care of the following: 1. Extract asset files straight from the overlay binary file into human readable file in `assets/st/STAGE_NAME` 2. Build assets as header files that go into `src/st/STAGE_NAME` to just include them from any C file This requires each stage header to have the following new format: please see `src/st/nz0/header.c` Built assets in `src/st` are ignored by Git. As for now, for simplicity sake, the steps `make extract_assets` and `make build_assets` are just executed within `make extract` exclusively for the US version. I plan to auto-generate files such as `src/st/nz0/tile_data.c`. For a first iteration I am aiming to handle the following: * [X] Extract rooms: `assets/st/*/rooms.json` * [X] Extract room layers: `assets/st/*/entity_layouts.json` * [X] Extract tilemap data: `assets/st/*/tilemap_*.bin` * [X] Extract tilemap definitions: `assets/st/*/tiledef_*.json` * [X] Extract sprites: `assets/st/*/sprites.json` * [x] Extract entity layouts * [X] Build rooms: `src/st/*/rooms.h` * [X] Build room layers: `src/st/*/layers.h` * [X] Build tilemap data: `src/st/*/tilemap_*.h` * [X] Build tilemap definitions: `src/st/*/tiledef_*.h` * [x] Build sprites (aka `g_SpriteBanks`) * [x] Build entity layouts * [x] Allow the tool to suggest how to adjust the Splat config for each overlay I want the tool to cover the following stages: * [x] CEN * [x] DRE * ~MAD~ I do not think this can be done, it is way too different from the other overlays * [x] NO3 * [x] NP3 * [X] NZ0 * [x] ST0 * [X] WRP * [x] RWRP * ~WRP (PSP)~ Maybe in a follow-up PR For a later iteration I plan on extracting and build: * Entity GFX thingie * The CLUT thingie in the header * Uncompressed GFX data * Cutscene data * Blueprints * The `src/config_us.h` thingie --------- Co-authored-by: Josh Lory <josh.lory@outlook.com>
2024-07-02 20:38:36 +00:00
Extract memory card icons (#166) Extract the memory card icons out of DRA.BIN and SEL.BIN. ![image](https://user-images.githubusercontent.com/6128729/227794798-4dd071f9-512b-4c72-8f5d-fcbf7a615519.png) I also took the opportunity to extract `g_MemcardPortMask` into its own source file `save_mgr_pre.c`. Unfortunately this is required to keep the original data order. This suggests that most likely the icon palette and bitmap were baked into the original `save_mgr.c` as byte arrays. I decided to take a different approach and extracted them as PNGs for better moddability. I had to spin-off Splat into a new fork due to some breaking changes on 0.18.0.
2023-12-12 07:12:13 +00:00
$(BUILD_DIR)/assets/dra/memcard_%.png.o: assets/dra/memcard_%.png
mkdir -p $(dir $@)
Extract BIN/F_GAME.BIN (#886) Mostly a proof of concept to extract resources from `BIN/F_GAME.BIN`. There are tons of hacks to make this work: * Tweaked png2s to decode binaries into PNGs * Add a `config/gfx.game.json` that describes how to extract different sprites out of a binary file * Coded the palette location by reading the `clut` parameter in the source code. A clut value of `0x196` corresponds to `"palette": "0x96"` or `"palette": 150`. * Coded the bits per pixel for a couple of 8-bit images * Coded the palette location of those graphics that expects the palette from DRA.BIN instead of F_GAME.BIN * For all the undiscovered palette, I called the files `unk` and defaulted their palette to grey The expectation is that sprites will slowly be documented and added into `config/gfx.game.json` by modders and enthusiasts. This does not yet pack back these PNGs into the binary file. The way these assets are extracted is probably not final or perfect, but we have to start from somewhere. ![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/ce12ad2d-d95d-41e8-ac31-186cd36ac965)
2023-12-25 19:23:25 +00:00
$(PNG2S) encode $< \
Extract memory card icons (#166) Extract the memory card icons out of DRA.BIN and SEL.BIN. ![image](https://user-images.githubusercontent.com/6128729/227794798-4dd071f9-512b-4c72-8f5d-fcbf7a615519.png) I also took the opportunity to extract `g_MemcardPortMask` into its own source file `save_mgr_pre.c`. Unfortunately this is required to keep the original data order. This suggests that most likely the icon palette and bitmap were baked into the original `save_mgr.c` as byte arrays. I decided to take a different approach and extracted them as PNGs for better moddability. I had to spin-off Splat into a new fork due to some breaking changes on 0.18.0.
2023-12-12 07:12:13 +00:00
$(BUILD_DIR)/assets/dra/memcard_$*.png.s g_saveIcon$* \
$(BUILD_DIR)/assets/dra/memcard_$*.pal.s g_saveIconPal$*
$(AS) $(AS_FLAGS) -o $(BUILD_DIR)/assets/dra/memcard_$*.png.o $(BUILD_DIR)/assets/dra/memcard_$*.png.s
rm $(BUILD_DIR)/assets/dra/memcard_$*.png.s
$(AS) $(AS_FLAGS) -o $(BUILD_DIR)/assets/dra/memcard_$*.pal.o $(BUILD_DIR)/assets/dra/memcard_$*.pal.s
rm $(BUILD_DIR)/assets/dra/memcard_$*.pal.s
$(BUILD_DIR)/assets/st/sel/memcard_%.png.o: assets/st/sel/memcard_%.png
mkdir -p $(dir $@)
Extract BIN/F_GAME.BIN (#886) Mostly a proof of concept to extract resources from `BIN/F_GAME.BIN`. There are tons of hacks to make this work: * Tweaked png2s to decode binaries into PNGs * Add a `config/gfx.game.json` that describes how to extract different sprites out of a binary file * Coded the palette location by reading the `clut` parameter in the source code. A clut value of `0x196` corresponds to `"palette": "0x96"` or `"palette": 150`. * Coded the bits per pixel for a couple of 8-bit images * Coded the palette location of those graphics that expects the palette from DRA.BIN instead of F_GAME.BIN * For all the undiscovered palette, I called the files `unk` and defaulted their palette to grey The expectation is that sprites will slowly be documented and added into `config/gfx.game.json` by modders and enthusiasts. This does not yet pack back these PNGs into the binary file. The way these assets are extracted is probably not final or perfect, but we have to start from somewhere. ![image](https://github.com/Xeeynamo/sotn-decomp/assets/6128729/ce12ad2d-d95d-41e8-ac31-186cd36ac965)
2023-12-25 19:23:25 +00:00
$(PNG2S) encode $< \
Extract memory card icons (#166) Extract the memory card icons out of DRA.BIN and SEL.BIN. ![image](https://user-images.githubusercontent.com/6128729/227794798-4dd071f9-512b-4c72-8f5d-fcbf7a615519.png) I also took the opportunity to extract `g_MemcardPortMask` into its own source file `save_mgr_pre.c`. Unfortunately this is required to keep the original data order. This suggests that most likely the icon palette and bitmap were baked into the original `save_mgr.c` as byte arrays. I decided to take a different approach and extracted them as PNGs for better moddability. I had to spin-off Splat into a new fork due to some breaking changes on 0.18.0.
2023-12-12 07:12:13 +00:00
$(BUILD_DIR)/assets/st/sel/memcard_$*.png.s g_saveIcon$* \
$(BUILD_DIR)/assets/st/sel/memcard_$*.pal.s g_saveIconPal$*
$(AS) $(AS_FLAGS) -o $(BUILD_DIR)/assets/st/sel/memcard_$*.png.o $(BUILD_DIR)/assets/st/sel/memcard_$*.png.s
rm $(BUILD_DIR)/assets/st/sel/memcard_$*.png.s
$(AS) $(AS_FLAGS) -o $(BUILD_DIR)/assets/st/sel/memcard_$*.pal.o $(BUILD_DIR)/assets/st/sel/memcard_$*.pal.s
rm $(BUILD_DIR)/assets/st/sel/memcard_$*.pal.s
Reference in New Issue Copy Permalink