sotn-decomp/.github/workflows/saturn-build.yml
Luciano Ciccariello 45a71c03ac
Rewrite Saturn make rules (#660)
## What

Enhance the existing `Makefile` to build new rules for the Saturn side
of the decomp. This should allow more flexibility when adding new
overlays or when tuning existing rules.

## Changes

I separated part of the Saturn build process in the separate file
`Makefile.saturn.mk`. I realise that naming it `saturn.mk` would have
been enough, but I pre-pended `Makefile.` so it can be found right below
the main `Makefile` when listing files in an alphabetic order. I plan to
do the same with the psx and psp toolchain, therefore you will find
`include Makefile.*.mk` in the main `Makefile`.

I deleted all the game building process done with Docker. Now that we
have an established way to do it natively I think it is no longer
required. We can always run the entire buildchain within a Docker
container instead of having `_native` and `_docker`. Now all the
`_native` references are removed. `build_saturn_native` is now
`build_saturn`.

`check_saturn` is no longer responsible of stripping the ELF into a
binary. That is now part of `build_saturn`.

I removed the references to `_li.o` (e.g. `alucard_li.o`) and used
`.elf` instead, which is closer to how the PSX build chain works. If
`_li.o` was a better preference, please let me know.

I am no longer using `./compile_dosemu.sh`. Instead I am using the new
`$(DOSEMU)` to directly invoke the tool within the Makefile. I have done
that to reduce the amount of dependent files.

I tried minimising duplication as much as possible. We now have a list
of overlays found in `SATURN_OVL_TARGETS`. Each expected output triggers
a series of dependencies so seamlessly build everything. `Makefile` is
smart enough to call `$(SATURN_TOOLCHAIN)` only once. If the game was
already built but just one source or symbol file changed, triggering a
new `build_saturn` will only compile the modified overlay.

The Saturn ADPCM files are now extracted in `assets/` instead of
`build/`. I think `assets/` should contain all the uncompressed and
uncooked files. The list of PCM file is not hardcoded. Instead I am now
using `$(wildcard disks/saturn/SD/*.PCM)`. This now means the tool tries
to convert PCMs from `SDD0.PCM` to `SDF0.PCM` with no success. As the
tool is silently failing I decided to leave it as I wrote it.

## Problems

I rewrote everything thinking about concurrency in mind. But `make -j
build_saturn` gives some unexpected output on `stdout`. I did not dig
too much into it. I suspect it might be dosemu. This is not a stopper as
we were not using `-j` when building the game anyway.

I also noticed doing `VERSION=saturn make build` calls
`mipsel-linux-gnu-ld` for `stage_02`. I suspect it is calling the rule
`$(MAIN_TARGET).elf: $(MAIN_O_FILES)` and simply moving `include
Makefile.*.mk` above it should fix it. But then it would cause the same
problem if I split the PSX rules into their own separate file. We never
used `make build` by setting the env variable `VERSION`, so this is not
either a breaking change or a stopper.

## Post thoughts

I am happy with what I achieved so far. I used the knowledge I
accumulated when maintaining the PSX counterpart. Since I now better
understand how `make` works, I was able to make some better decisions in
the Saturn counterpart. For example triggering a new build when the
symbol list changes is something the PSX build chain lacks of. I think
in the future it would be nice to trigger `make extract` when either the
YAML or the symbol list changes.
2023-10-03 21:42:08 +01:00

128 lines
4.3 KiB
YAML

name: Build Saturn version
on:
push:
branches:
- master
paths-ignore:
- '*.md'
- '**/*.md'
pull_request_target:
paths-ignore:
- '*.md'
- '**/*.md'
workflow_dispatch:
jobs:
build-and-test-saturn:
# Building and testing cannot work if the repository owner is not Xeeynamo
# due to the missing secrets to clone the game's data repository
if: github.repository == 'Xeeynamo/sotn-decomp'
runs-on: ubuntu-latest
steps:
- name: Clone main repo (PR)
if: github.event_name == 'pull_request_target'
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
submodules: false
- name: Clone main repo
if: github.event_name != 'pull_request_target'
uses: actions/checkout@v3
with:
ref: ${{ github.ref }}
submodules: false
- name: Clone dependencies
uses: actions/checkout@v3
with:
repository: xeeynamo/sotn-decomp-dependencies
ref: saturn
token: ${{ secrets.SOTN_DECOMP_DEPENDENCIES_TOKEN }}
path: 'disks/dependencies'
- name: Setting up dependencies
working-directory: disks
run: |
cat dependencies/* > dependencies.tar.gz
tar -xf dependencies.tar.gz
mv *.cue sotn.saturn.cue
mv *.bin sotn.saturn.bin
- name: Install bchunk 7zip
run: |
sudo apt-get install p7zip-full bchunk binutils-sh-elf
- name: Install dosemu
run: |
git clone https://github.com/sozud/dosemu-deb.git
cd dosemu-deb
sudo apt-get install xfonts-utils
sudo dpkg -i fdpp_1.6-1_amd64.deb
sudo dpkg -i fdpp-dev_1.6-1_amd64.deb
sudo dpkg -i comcom32_0.1~alpha3-1_all.deb
sudo dpkg -i dosemu2_2.0~pre9-1_amd64.deb
- name: Extract dependencies
run: VERSION=saturn make extract_disk
- name: Extract asm
run: VERSION=saturn make extract
- name: Build saturn binaries
run: make build_saturn
- name: Check saturn
run: VERSION=saturn make check
function-finder-saturn:
if: github.repository == 'Xeeynamo/sotn-decomp' && github.ref == 'refs/heads/master' && github.event_name == 'push'
runs-on: ubuntu-latest
steps:
- name: Clone main repo (PR)
if: github.event_name == 'pull_request_target'
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
submodules: false
- name: Clone main repo
if: github.event_name != 'pull_request_target'
uses: actions/checkout@v3
with:
ref: ${{ github.ref }}
submodules: false
- name: Clone dependencies
uses: actions/checkout@v3
with:
repository: xeeynamo/sotn-decomp-dependencies
ref: saturn
token: ${{ secrets.SOTN_DECOMP_DEPENDENCIES_TOKEN }}
path: 'disks/dependencies'
- name: Setting up dependencies
working-directory: disks
run: |
cat dependencies/* > dependencies.tar.gz
tar -xf dependencies.tar.gz
mv *.cue sotn.saturn.cue
mv *.bin sotn.saturn.bin
- name: Install bchunk 7zip
run: |
sudo apt-get install p7zip-full bchunk binutils-sh-elf
- name: Extract dependencies
run: VERSION=saturn make extract_disk
- name: Extract asm
run: VERSION=saturn make extract
- name: Clone asset repository
uses: actions/checkout@v3
with:
ref: 'gh-functions-saturn'
path: 'gh-functions-saturn'
- name: Set-up Python
uses: actions/setup-python@v4
with:
python-version: '3.x'
- name: Install secondary pre-requirements
run: pip3 install requests tabulate
- name: Generate duplicates and function report
run: python3 tools/function_finder/function_finder_saturn.py > gh-functions-saturn/functions_saturn.md
- name: Commit all reports
run: |
git config --global user.name 'GitHub Action'
git config --global user.email '41898282+github-actions[bot]@users.noreply.github.com'
git add -A
git commit -m 'Update reports' || true
git push
working-directory: gh-functions-saturn