Both Linux CUDA jobs installed the cuda-toolkit-12-9 metapackage, which
apt reports as 3935 MB of archives per job. That is 16 jobs per nightly
run on the x64 matrix plus the arm64 leg, and it made "Install CUDA
Toolkit" the second-largest cost in those jobs at 36.5 min across the
x64 legs.
Most of it is never used. The metapackage pulls Nsight Systems and
Compute, cuFFT, cuSPARSE, cuSOLVER and NPP; ggml-cuda links only
CUDA::cudart, CUDA::cublas and CUDA::cuda_driver plus CCCL for
thrust/cub, and stable-diffusion.cpp links only ggml. Install that
subset instead, ~595 MB of direct archives, and add a guard that names
the missing file if NVIDIA ever repackages one of them rather than
failing partway through the compile.
Package names verified present in both developer.download.nvidia.com
repos the jobs use, ubuntu2204/x86_64 and ubuntu2404/sbsa.
The Windows CUDA installs already select sub-packages through
Jimver/cuda-toolkit and were left alone.
Co-Authored-By: Claude Opus 5 <[email protected]>
Two changes, both measured against run 30332616539.
Cache the windows-latest-cmake build. Its cuda12 leg compiles nine CUDA
architectures in a single job and had no ccache step at all, making it
the second-longest job in the run at 90.6 min of Build. windows-latest-cuda
runs the same MSVC/Ninja/Release setup with ccache and averages 1.6 min
per leg at 97.96% hit rate. No cmake flags are needed: ggml's
src/CMakeLists.txt sets RULE_LAUNCH_COMPILE when it finds ccache on PATH
(GGML_CCACHE defaults to ON), which is how windows-latest-cuda already
gets its hits. The cuda12 leg gets max-size 2G because nine
architectures plus GGML_CPU_ALL_VARIANTS will not fit the 500M default
that suits the cpu and vulkan legs.
Parallelize xz when packing the Linux CUDA artifacts. tar -cJf drives a
single-threaded xz and was the largest cost in those jobs, 50.9 min
across the eight x64 legs plus 5.9 min on arm64, against only 12.8 min
of Build. Measured on a 2.9 GB binary payload: 12m41s single-threaded
versus 47.7s on 32 cores, with the archive 1.8% larger because threaded
mode compresses independent blocks. The runners are 4-core, so expect
roughly 3-4x there.
ccache itself was already healthy everywhere it was configured, at
98-100% hit rates, and the release job's artifact download is fine on a
full ubuntu-latest VM. Neither needed changing.
Co-Authored-By: Claude Opus 5 <[email protected]>
The release is reused across runs against the same tag (idempotent
create-or-get). If a family (e.g. rocm-therock) was complete and
published in an earlier run but the current run finds it incomplete and
drops it locally, its assets from that earlier run were never removed —
the release would keep looking complete for a family this run couldn't
actually reproduce.
The upload step now reconciles existing release assets against the
current ./artifact file set: anything not fully uploaded (a broken
partial upload, as before) or not present in this run's file set (stale
leftovers from a dropped family) gets deleted before uploading.
Addresses review feedback on lemonade-sdk/stable-diffusion.cpp#14. The
always() fix requested in the same review was already applied in
03e249d.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Without always(), GitHub Actions' default skip-if-a-needed-job-failed
behavior meant a single failed build job (e.g. one CUDA sm_* matrix leg,
several of which use fail-fast: false) could skip the release job
outright, before the per-family completeness check ever ran — silently
producing no release instead of a correct partial one.
The other issue found in the equivalent llama.cpp fix (a hard-fail step
that could block unrelated families) doesn't apply here: this workflow
has no cross-job merge step comparable to llama.cpp's Windows CPU-into-
other-zips step, so there's nothing else that needs the same treatment.
Follow-up to #14.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Port the artifact-completeness gate from lemonade-sdk/llama.cpp#16 to
this repo's release job, adapted to its build matrix:
- cuda: 8 sm_* variants on ubuntu x64 + windows x64, plus sm_121-only
on ubuntu arm64 (GB10) — 17 files total.
- rocm-therock: TheRock nightly ROCm 7.13.0 on windows + ubuntu.
- rocm-hipsdk: official/legacy HIP SDK ROCm on windows (7.1.1) +
ubuntu (7.2.1) — kept separate from rocm-therock since they're
different SDKs, not just different platforms of the same backend.
- cpu: ubuntu + windows avx2 builds.
- metal: macos-14 + macos-latest builds.
Each family is all-or-nothing: if any artifact in a family is
missing, every artifact for that family is dropped from the release
so a release never contains a partial/inconsistent set (e.g. CUDA
split across sm_* or platforms). Other families are unaffected by a
gap in one. A release with zero artifacts overall is never created.
Also brings the release/upload mechanism up to parity with
llama.cpp's: an idempotent create-or-get-release step (safe to
re-run after a partial failure) and a retrying, resumable asset
upload, replacing the old anzz1/action-create-release +
github-script@v3 flow.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
stable-diffusion.cpp's Windows CUDA build fails with:
fatal error C1128: number of sections exceeded object file format
limit: compile with /bigobj
src/stable-diffusion.cpp combined with the CUDA backend's generated
code exceeds MSVC's default COFF section limit per object file. The
windows-latest-cmake job already works around the same class of issue
with -DCMAKE_CXX_FLAGS='/bigobj'; apply the same flag to the CUDA
build.
v0.2.22 does not include CUDA 12.9.x; v0.2.24 was the first release to
add it. Bump to latest v0.2.35 and use 12.9.1 (latest 12.9 patch).
Co-authored-by: Copilot <[email protected]>
- Add sm_121 to ubuntu-latest-cuda and windows-latest-cuda matrices
- Add new ubuntu-arm64-cuda job targeting ubuntu-24.04-arm runner with sm_121
- Use CUDA sbsa (ARM64) package repo for arm64 CUDA Toolkit install
- Add ubuntu-arm64-cuda to release job needs
Co-authored-by: Copilot <[email protected]>
The Windows CUDA cmake invocation was missing -DGGML_CUDA=ON, which
meant the ggml CUDA backend was never compiled in. Without it, the
build silently falls back to CPU even when the CUDA toolkit is
installed. This is the same fix already applied to the Linux CUDA
build in commit 6a68a98.
Fixes: lemonade-sdk/lemonade#2171 (sdcpp:cuda falls back to CPU on Windows + RTX 4090)
Co-authored-by: Copilot <[email protected]>
Add -DGGML_CUDA=ON and -DCMAKE_CUDA_COMPILER explicitly to the Linux
CUDA cmake invocation so CUDA is actually compiled instead of silently
falling back to CPU when auto-detection fails.
Widen the ggml artifact collection from libggml-cuda.so only to
libggml*.so* (Linux) and ggml*.dll (Windows) to capture all ggml shared
libraries (core, base backend, cuda backend) regardless of where cmake
places them in the build tree.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
The ggml CUDA backend is built as a shared library plugin when
SD_BUILD_SHARED_LIBS=ON. Bundle it alongside the CUDA runtime libs so
that CUDA inference works out of the box without the plugin being
missing at runtime.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Add a pull_request trigger to build.yml so the release build is
validated on PRs, mirroring lemonade-sdk/llama.cpp. The pack/upload
and release jobs remain gated on schedule/create_release, so nothing
is published on a PR run.
Scope the trigger to the workflow file itself: the build jobs clone
source from upstream (leejet/stable-diffusion.cpp), so only changes
to this workflow can affect a PR run's outcome.
On PRs, the CUDA matrices build a single representative arch (sm_89)
instead of all seven; packaging logic is identical across sm_*, and
the full matrix still runs on schedule/dispatch.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Use multi-line array literal format for $runtimeDllPatterns and remove
the trailing backslash from the Copy-Item destination path, which was
causing an 'Unexpected token }' parser error at the foreach closing brace.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Without an explicit ref, actions/checkout makes a GitHub API call to
determine the default branch of leejet/stable-diffusion.cpp. With 14
parallel CUDA jobs all doing this simultaneously this call can fail,
presenting as a checkout error. Pinning to master skips the API call.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Linux:
- Add libcudart.so*, libnvJitLink.so* to bundled runtime libs
- Use cp -av (preserves symlinks) instead of cp -v
- RPATH step now patches all ELF files via file(1) instead of glob
Windows:
- Add curand and nvjitlink to CUDA toolkit sub-packages
- Copy cudart64, cublas64, cublasLt64, curand64, nvJitLink DLLs
from CUDA_PATH\bin before zipping
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Linux: add libcudart.so* to the bundled runtime libraries so binaries
can run on systems without a matching CUDA toolkit installed.
Windows: add id to cuda-toolkit step and robocopy cudart64/cublas64/
cublasLt64 DLLs into the zip so the build runs on machines without a
pre-installed CUDA runtime.
Addresses Copilot review comments on lemonade-sdk/stable-diffusion.cpp#7.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Lemonade's sd_server.cpp builds the Windows CUDA download URL with a
.7z extension. The CI was producing .zip, causing a 404 on download
which presented as an extraction failure.
Also adds .7z to the release upload script allowlist.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Linux: sd-master-{hash}-ubuntu-cuda-sm_XX-x64.tar.xz
Windows: sd-master-{hash}-windows-cuda-sm_XX-x64.zip
Also adds .tar.xz support to the release upload script.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Restores the filenames introduced in the initial CUDA PR:
sd-master-{hash}-bin-Linux-Ubuntu-22.04-x86_64-cuda-{sm}.zip
sd-master-{hash}-bin-win-cuda-{sm}-x64.zip
Reverts the intermediate ubuntu-cuda-* and windows-cuda-* name
experiments, and drops the .tar.xz branch from the release upload script.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Renames CUDA artifacts to match the format Lemonade SDK expects:
sd-master-148f69f-ubuntu-cuda-sm_89-x64.tar.xz
sd-master-148f69f-windows-cuda-sm_89-x64.zip
Also restores the sd-* download pattern in the release job and keeps
.tar.xz support in the upload script.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Linux CUDA artifacts are now named ubuntu-cuda-sm_XX-x64.tar.xz and
Windows CUDA artifacts are named windows-cuda-sm_XX-x64.zip, matching
the filenames Lemonade SDK constructs when downloading from releases.
Also removes the sd-* pattern filter from the release download step so
CUDA artifacts (which don't carry the sd- prefix) are included, and
adds .tar.xz support to the release upload script.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Adds ubuntu-latest-cuda and windows-latest-cuda jobs covering sm_75
through sm_120, modeled after the llama.cpp CUDA CI. Linux installs
CUDA 12.8 via apt, bundles cublas runtime libs, and patches RPATHs.
Windows uses Jimver/[email protected] with MSVC + Ninja Multi-Config.
Both jobs are wired into the release job.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
ggml's CMake defaults GGML_NATIVE=ON, which adds -march=native and silently
enables every extension the build runner's CPU advertises. Recent Azure
ubuntu-latest hosts are Xeon Platinum 8370C (AVX-512 capable), so the
master-593 release of libstable-diffusion.so was built with 5,396 zmm /
1,582 k-mask AVX-512 instructions and SIGILLs immediately on any consumer
or CI host without AVX-512.
Pin the Linux x86_64 cpu and ROCm builds to GGML_NATIVE=OFF +
GGML_AVX2=ON + GGML_FMA=ON + GGML_F16C=ON — a portable Haswell-era
baseline that downstream consumers (Lemonade, end users) can rely on.
The Windows AVX2 cpu build and Windows ROCm builds already use
GGML_NATIVE=OFF; this brings Linux into line.
Verified by inspecting libstable-diffusion.so:
master-569 (no AVX-512): 0 zmm refs, 0 k-mask refs
master-593 (AVX-512): 5,396 zmm refs, 1,582 k-mask refs
The crash this fixes manifests in lemonade-sdk/lemonade CI as
sd-server "exit code: -1" within 200ms on the GitHub-hosted ollama
test job, while the self-hosted ROCm runner (full AVX-512) succeeds.
* ci: add macOS arm64 (Metal) build for stable-diffusion.cpp
Adds a macos-arm64-cmake job covering [macos-14, macos-latest] that builds
leejet/stable-diffusion.cpp with Metal enabled and produces a release-ready
sd-*-bin-Darwin-macOS-<ver>-arm64.zip artifact, mirroring the existing Linux
job's naming convention. Wired into the release job so macOS arm64 binaries
are attached alongside Linux/Windows on scheduled and manual releases.
* ci(macos): bake @loader_path rpath so dylib loads next to binary
Without this, sd-cli ships with the GitHub runner's absolute build path
hardcoded in its rpath and fails at runtime with 'Library not loaded:
@rpath/libstable-diffusion.dylib'. Mirrors llama.cpp's release.yml flags.
* ci(macos): use stable artifact filename without OS version
Drop the dynamic Darwin-macOS-{sw_vers}-arm64 suffix in favor of a stable
sd-{branch}-{hash}-bin-Darwin-arm64.zip name (with -latest variant for
macos-latest). The previous naming embedded the runner image's macOS
point release (e.g. 14.8.5, 15.7.4), which churns silently as GitHub
patches the image and would break downstream consumers (Lemonade C++)
that download by exact filename. Matches the fixed-name pattern already
used by the Windows builds.
* ci: grant contents:write so the release job can publish artifacts
Without this, the workflow's GITHUB_TOKEN defaults to read-only on forks
and the 'Create release' / 'Upload release' steps fail with HTTP 403
'Resource not accessible by integration'. Same fix the Linux/Windows
release flow needs to function on Geramy's fork; safe on upstream where
the org-default token already has write access (this just makes it
explicit and forks-portable).
* ci(macos): name artifact …Darwin-arm64-metal.zip to align with backend ID
Lemonade's recipe matrix is adopting a 'metal' backend identifier for
macOS arm64 (mirroring the existing llamacpp/metal pattern). Renaming
the artifact's suffix to match keeps Lemonade C++'s download URL
construction trivial (filename = sd-{ver}-bin-Darwin-arm64-metal.zip)
and signals to consumers that this build has Metal acceleration enabled
(GGML_METAL=ON GGML_METAL_USE_BF16=ON GGML_METAL_EMBED_LIBRARY=ON).