mirror of
https://github.com/codewhale-hq/Codewhale.git
synced 2026-11-09 09:59:45 -10:00
ci: give the Ubuntu workspace test build disk and memory headroom
On every wave/0.10.1-next PR run, Test (ubuntu-latest) was terminated about
4.5 minutes into compiling the workspace test targets ("The runner has
received a shutdown signal", exit 143) at 04:30, 05:58, 06:02 and 07:26 UTC.
The 07:26 kill had no newer push to cancel it, so it was not the PR
concurrency group. Single-fix PRs finish the same --all-features build in
about 10 minutes; the combined wave is the largest tree.
For heavy Ubuntu PR/dispatch runs, add a step that removes preinstalled SDKs
the job never uses (dotnet, Android, GHC, CodeQL bundle) and adds 8 GiB of
swap, printing df/free before and after. The Run tests step now starts a
30-second memory/swap/disk trace on Linux and stops it after nextest,
preserving nextest's exit status, so a further termination records whether
memory or disk ran out. macOS and Windows are unchanged.
Validation: actionlint reports the same 12 pre-existing findings before and
after (none on the new lines); node .github/scripts/release-workflows.test.js
passes. Hosted proof is the next #6782 Ubuntu run.
Refs #6698
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: CodeWhale Bot <[email protected]>
This commit is contained in:
1 parent
4e277d6c7d
commit
a3a19dea57
1 file changed
+33
-1
@@ -822,6 +822,25 @@ jobs:
|
||||
sleep 15
|
||||
done
|
||||
sudo apt-get install -y libdbus-1-dev pkg-config
|
||||
- name: Ubuntu build headroom
|
||||
# The combined wave/0.10.1-next workspace build was terminated
|
||||
# mid-compile on every PR run ("runner has received a shutdown
|
||||
# signal", exit 143, no newer push to cancel it) while single-fix PRs
|
||||
# finished the same build in ~10 minutes. Drop preinstalled SDKs this
|
||||
# job never uses and add swap so the full --all-features test build
|
||||
# has disk and memory headroom; the Run tests step logs both.
|
||||
if: needs.changes.outputs.heavy == 'true' && matrix.os == 'ubuntu-latest' && (github.event_name == 'workflow_dispatch' || github.event_name == 'pull_request')
|
||||
shell: bash
|
||||
run: |
|
||||
df -h / /mnt; free -m
|
||||
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /usr/local/.ghcup /opt/hostedtoolcache/CodeQL
|
||||
if sudo fallocate -l 8G /mnt/cw-swapfile && sudo chmod 600 /mnt/cw-swapfile \
|
||||
&& sudo mkswap /mnt/cw-swapfile >/dev/null && sudo swapon /mnt/cw-swapfile; then
|
||||
echo "Added 8 GiB swap at /mnt/cw-swapfile"
|
||||
else
|
||||
echo "::warning::Could not add swap; continuing with the runner default"
|
||||
fi
|
||||
df -h / /mnt; free -m
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
if: needs.changes.outputs.heavy == 'true' && (matrix.os != 'ubuntu-latest' || github.event_name == 'workflow_dispatch' || github.event_name == 'pull_request')
|
||||
with:
|
||||
@@ -879,7 +898,20 @@ jobs:
|
||||
# one. nextest does not run doctests — the next step keeps them.
|
||||
if: needs.changes.outputs.heavy == 'true' && (matrix.os != 'ubuntu-latest' || github.event_name == 'workflow_dispatch' || github.event_name == 'pull_request') && !(github.event_name == 'workflow_dispatch' && inputs.workspace_test_mode == 'shared-process-twice')
|
||||
shell: bash
|
||||
run: sh scripts/with-hermetic-test-home.sh cargo nextest run --workspace --all-features --locked --profile ci
|
||||
run: |
|
||||
monitor=
|
||||
if [ "$RUNNER_OS" = Linux ]; then
|
||||
# Headroom trace: if the runner is terminated again, the last lines
|
||||
# say whether memory or disk ran out.
|
||||
( while sleep 30; do
|
||||
echo "[headroom] $(free -m | awk '/^Mem:/{print "mem used " $3 "/" $2 " MiB"} /^Swap:/{print "swap used " $3 "/" $2 " MiB"}' | paste -sd ' ') disk free $(df -h / | awk 'NR==2{print $4}')"
|
||||
done ) &
|
||||
monitor=$!
|
||||
fi
|
||||
status=0
|
||||
sh scripts/with-hermetic-test-home.sh cargo nextest run --workspace --all-features --locked --profile ci || status=$?
|
||||
if [ -n "$monitor" ]; then kill "$monitor" 2>/dev/null || true; fi
|
||||
exit "$status"
|
||||
env:
|
||||
CODEWHALE_EXT_HOST_TESTS: '1'
|
||||
# sccache 0.17 panics resolving its config directory under the
|
||||
|
||||
Reference in new issue
Block a user