## Summary
The AMD and Intel GPU runners are currently unavailable, so those CI
jobs fail whenever they run — including inside the **merge queue**,
which blocks all merges for some time. This PR makes the merge queue run
**only Checkstyle** for now.
## Change
Reduce the **AMD GPU** and **Intel GPU** workflow triggers to
`workflow_dispatch` only, mirroring the **NVIDIA GPU** workflow's `on:`
(manual trigger) **minus the schedule** — we don't maintain a schedule
for AMD/Intel.
This removes their `push`, `pull_request`, and `merge_group` triggers,
so:
- **Only Checkstyle runs in the merge queue** (it keeps its
`merge_group` trigger).
- AMD/Intel no longer run automatically on PRs or pushes.
- AMD/Intel can still be launched on demand via the Actions **"Run
workflow"** button once the runners are healthy again.
`checkstyle.yml` is unchanged.
## Notes
- Observed (not changed here): `merge-queue-failure.yml` listens for the
**NVIDIA GPU** workflow with `event == 'merge_group'`, but NVIDIA GPU
has no `merge_group` trigger, so that handler is currently a no-op. Can
be addressed separately.
Co-authored-by: Copilot <[email protected]>
## Summary
Allow the Python `liger-kernel` wheel to be published from corrected
`main` without creating another GitHub release event or changing the
existing release tag.
- Add `workflow_dispatch` with a boolean `publish` input, defaulting to
`false`.
- Preserve publication on release events; manual runs upload only when
`publish=true`.
- Reject release tags that disagree with the source project version
before building.
- Check distribution metadata and the wheel version before upload.
After merge, publish the corrected 0.8.4 source with:
```bash
gh workflow run publish-release.yml --repo linkedin/Liger-Kernel --ref main -f publish=true
```
## Validation
Built the 0.8.4 sdist and wheel, passed `twine check`, exercised the
manual and matching-tag cases, rejected a mismatched tag, and ran the
wheel metadata check.
This PR changes only the publishing workflow. Experimental-package
changes are intentionally excluded at the maintainer's request;
distribution metadata validation does not claim that all runtime imports
work.
Co-authored-by: kolehma8 <[email protected]>
Co-authored-by: Copilot <[email protected]>
## Summary
Make the LCK NVSHMEM dependency selectable from the pip command while
keeping the combined Hopper/Blackwell release wheel on **CUDA 12.9.1**.
```bash
pip install "liger-cute-kernels[cu12]"
# For a locally built CUDA 13 Hopper wheel:
pip install "/path/to/liger_cute_kernels-<version>-py3-none-linux_x86_64.whl[cu13]"
```
- Move NVSHMEM out of unconditional dependencies into `cu12` and `cu13`
extras, each pinned to `3.6.5`. Plain LCK leaves NVSHMEM provisioning to
the caller.
- Keep the existing digest-pinned CUDA 12.9.1 release image,
SM90a/SM100f targets, and `nvidia-nvshmem-cu12` build prerequisite.
Explicitly package `".[cu12]"` and validate that both optional
dependencies activate only for their corresponding extra.
- Fix the Hopper non-RDC cubin command to prepend `-I` to every CUDA
include directory. CUDA 13 exposes an additional CCCL include directory;
previously NVCC interpreted that bare path as a second input file.
- Update installation documentation and missing-runtime guidance.
Extras select runtime dependencies, not different compiled binaries.
Select exactly one matching the core's build toolkit; the two NVSHMEM
distributions share an installation namespace. Source builds still
require NVSHMEM to be installed before compilation.
### Why the release stays on CUDA 12.9
The complete Hopper core and non-RDC cubin build with CUDA 13.0.88 after
the include-path fix. However, the complete SM100 build fails in three
MoE backward variants because `ptxas` rejects mixed `.cta_group::1` and
`.cta_group::2` instructions in one kernel. This PR does not remove
those variants or attempt a kernel redesign.
NVIDIA reports the same newer-toolkit restriction in [CCCL
#8297](https://github.com/NVIDIA/cccl/pull/8297). The CUDA 13 extra
supports appropriately built source artifacts; it does not claim that
the combined release wheel builds with CUDA 13.
## Testing Done
- Hardware Type: **4 x NVIDIA H200**, CUDA compiler **13.0.88**, Torch
**2.11.0.1+cu130**.
- **21 focused build/loader tests passed**, including optional
dependency metadata, missing-runtime guidance, and CMake include-list
expansion with single/multiple CUDA include paths.
- Ruff lint and formatting checks passed for the changed Python files.
- Parsed the release workflow's embedded Python, checked the packaging
shell syntax, and exercised its optional-dependency validation against
metadata from the built wheel.
- Built, packaged, and installed the complete Hopper core plus non-RDC
cubin. The test wheel was 46.97 MiB and used `linux_x86_64`, not a
manylinux claim, because that test image uses glibc 2.38.
- Ran six numerical cases: Qwen3-30B-A3B, Qwen3-235B-A22B, and
Llama-4-Scout at 1,024/2,048 tokens per GPU, EP=4, seed 42. Compared
CUDA 13 default and non-RDC-enabled execution against the existing CUDA
12 wheel on the same GPUs and with the same Python frontend and inputs.
- Covered forward output, input gradients, all three expert-weight
gradients, and selected routing-weight gradients against Torch FP32. All
36 maximum-error metrics matched the CUDA 12 baseline exactly; the
change in the equally weighted average mean error was below 0.000008% in
both CUDA 13 modes. This is a single-seed comparison, not a claim of
bitwise-identical tensors or exhaustive coverage.
- Complete CUDA 13 SM100 compilation was attempted and failed as
described above. No combined CUDA 13 release wheel was produced. A fresh
full CUDA 12.9 release build was not run for this PR; its build image
and native compilation settings remain unchanged apart from the
compatible include-list fix.
The full suites below were not run; the targeted checks and numerical
coverage are listed above.
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
Co-authored-by: jkolehma <[email protected]>
Co-authored-by: Copilot <[email protected]>
## Summary
- publish `liger-cute-kernels` through PyPI Trusted Publishing instead
of long-lived username/password secrets
- bind the publish job to the `liger-cute-kernels` GitHub environment
configured in the pending publisher
- grant `id-token: write` only to the publish job
- use the PyPA-supported `pypa/gh-action-pypi-publish` action
- retain the guarded manual `publish=true` retry added by #1473
## Pending publisher configuration
The PyPI pending publisher must exactly match:
```text
Project: liger-cute-kernels
GitHub owner: linkedin
Repository: Liger-Kernel
Workflow: publish-liger-cute-kernels.yml
Environment: liger-cute-kernels
```
A pending publisher intentionally returns 404 for the project until its
first successful OIDC upload. That first upload creates the project and
associates it with the PyPI account that configured the pending
publisher.
## Publishing 0.8.3 after merge
Run **Build and Publish Liger Cute Kernels** manually on `main` with
`publish=true`. The workflow rebuilds and validates both cores, packages
the combined wheel, obtains a short-lived token through GitHub OIDC, and
uploads it to PyPI.
`PYPI_LCK_USERNAME` and `PYPI_LCK_PASSWORD` are not used by this
workflow and can be removed after the OIDC path succeeds.
## Validation
- workflow YAML parses successfully
- publish job uses environment `liger-cute-kernels`
- `id-token: write` is scoped to the publish job
- no Twine username/password secrets are referenced
- nightly and default manual builds remain non-publishing
Co-authored-by: jkolehma <[email protected]>
Co-authored-by: Copilot <[email protected]>
## Summary
- publish `liger-cute-kernels` with dedicated `PYPI_LCK_USERNAME` and
`PYPI_LCK_PASSWORD` repository secrets
- add an explicit `workflow_dispatch` `publish` input, defaulting to
`false`
- allow a manual publish retry after credentials are configured, while
preserving non-publishing nightly and default manual builds
- validate that both credentials exist before invoking Twine
- run Twine in verbose mode so future PyPI authorization errors include
the server response
## Background
The v0.8.3 release built and packaged the dual-architecture wheel
successfully, but public PyPI upload failed with HTTP 403. The existing
credentials can publish `liger-kernel` but are not authorized for the
separate `liger-cute-kernels` project.
Failed release run:
https://github.com/linkedin/Liger-Kernel/actions/runs/35137571890
## Required repository configuration
Create and verify the separate PyPI account, enable 2FA, and create an
**entire-account API token** for its first project upload. Add these
secrets to the main repository:
```text
PYPI_LCK_USERNAME=__token__
PYPI_LCK_PASSWORD=pypi-...
```
Do not include either value in source, PR comments, or workflow inputs.
After the first upload creates `liger-cute-kernels`, `PYPI_LCK_PASSWORD`
can be replaced with a project-scoped token from that account while
retaining the same username secret.
## Publishing 0.8.3 after merge
Run **Build and Publish Liger Cute Kernels** manually on `main` with
`publish=true`. The workflow rebuilds and validates both architecture
cores before the dedicated publish job runs.
## Validation
- workflow YAML parses successfully
- manual `publish` input is boolean and defaults to `false`
- release events still publish automatically
- main pushes and default manual dispatches still skip publication
- both dedicated credentials are checked before upload
---------
Co-authored-by: jkolehma <[email protected]>
Co-authored-by: Copilot <[email protected]>
## Summary
- replace GitHub toolcache Python with Ubuntu 22.04's native Python 3.10
inside the CUDA build containers
- install `libibverbs-dev` so the PyPI NVSHMEM IBGDA headers can include
`infiniband/mlx5dv.h`
- retain GitHub-hosted Python 3.12 and pip caching for the non-container
wheel packaging job
## Failures addressed
PR #1466 fixed the first `setup-python` cache-hook failure, but the
subsequent pre-merge workflow runs exposed two additional container
dependencies:
1. GitHub's Python 3.12 toolcache binary requires glibc 2.38, while the
digest-pinned CUDA Ubuntu 22.04 image provides glibc 2.35.
2. Both native architectures require `infiniband/mlx5dv.h` through
NVSHMEM's IBGDA headers; Ubuntu supplies it in `libibverbs-dev`.
The native core is Python-ABI-independent, and `apache-tvm-ffi==0.1.14`
provides a CPython 3.10 manylinux wheel.
## Validation
Pre-merge fork dispatch:
https://github.com/kolehma8/Liger-Kernel/actions/runs/35032379309
- dependency installation and CUDA 12.9 configuration pass for both
architectures
- SM100f core build and artifact upload pass in 6m34s
- SM90a all-config core/cubin build is still running
- the manual dispatch does not execute the Twine publication job
---------
Co-authored-by: jkolehma <[email protected]>
Co-authored-by: Copilot <[email protected]>
## Summary
- disable `actions/setup-python`'s pip cache hook in the CUDA container
matrix jobs
- retain Python 3.12 setup and all dependency installation through
`python -m pip`
- keep pip caching enabled for the non-container wheel-packaging job
## Root cause
Post-merge run
[35030952199](https://github.com/linkedin/Liger-Kernel/actions/runs/35030952199)
failed before compilation in both architecture jobs. `setup-python`
installed CPython successfully, then its cache hook tried to execute
`/__t/Python/3.12.14/x64/bin/pip` inside the job container and failed
with `ENOENT`.
The cache is only an optimization and is incompatible with this
container/toolcache path layout. Removing `cache: pip` avoids the
failing hook; subsequent steps already use `python -m pip`.
## Validation
- workflow YAML parses with the container cache disabled and the host
packaging cache retained
- fork workflow run:
https://github.com/kolehma8/Liger-Kernel/actions/runs/35031510256
Co-authored-by: jkolehma <[email protected]>
Co-authored-by: Copilot <[email protected]>
## Summary
- add an independent release workflow for `liger-cute-kernels` using the
pinned CUDA 12.9.1 Ubuntu 22.04 build image and CUTLASS v4.4.1
- build SM90a and SM100f cores in parallel with four compiler jobs, then
package both into one Python-ABI-independent wheel
- use the same public version as `liger-kernel` and publish only on
release events; manual dispatches build artifacts without publishing
- stop bundling NVSHMEM shared libraries and depend on
`nvidia-nvshmem-cu12==3.6.5`
- select the SM90a or SM100f core at runtime from the active GPU
capability while retaining legacy single-core wheel compatibility
- validate FSLCE forward/backward exports, wheel contents, version,
dependency metadata, manylinux tag, and the 100 MiB PyPI limit
## Testing done
- `ruff check` and `ruff format --check` on the changed Python files
- 17 targeted packaging/loader tests passed
- workflow YAML parsed successfully and `git diff --check` passed
- built both native cores on H200 pods with `LIGER_CUTE_BUILD_JOBS=4`:
- SM90a: 45m09s, peak compiler RSS 8.69 GiB
- SM100f: 2m42s, peak compiler RSS 7.42 GiB
- confirmed both cores export fused linear scaled cross-entropy forward
and backward APIs
- packaged one dual-architecture wheel: 87,726,555 bytes (83.66 MiB),
below PyPI's 100 MiB limit
- installed the wheel in an isolated target directory with
`nvidia-nvshmem-cu12==3.6.5` and verified on H200:
- selected `libliger_cute_kernels_sm90a.so`
- no NVSHMEM libraries were bundled in the LCK package
- NVSHMEM unique-ID initialization reported rank 0 / world size 1 and
finalized successfully
- `twine check` passed without warnings
The H200 measurement pod uses glibc 2.38, so its validation artifact was
tagged `py3-none-manylinux_2_38_x86_64`. The release workflow builds
inside the digest-pinned Ubuntu 22.04 image, asserts glibc 2.35, and
emits `py3-none-manylinux_2_35_x86_64`.
---------
Co-authored-by: jkolehma <[email protected]>
Co-authored-by: Copilot <[email protected]>
## Summary
Adds a B200 (SM100) counterpart to the daily H100 cuTile CI job, so the
cuTile suite runs on **both** archs — mirroring cuTeDSL, which already
runs on H100 and B200. Arch-specific cuTile regressions (e.g.
SM100/tcgen05 codegen issues) would otherwise go uncaught by CI.
## Changes
- **`dev/modal/tests.py`** — new `liger_cutile_tests_b200` Modal
function, identical to `liger_cutile_tests` but on B200. Installs
`.[dev,cutile-tileiras]` (so `import cuda.tile` resolves and the tests
run rather than `importorskip`) and runs `make test-cutile` (which sets
`LIGER_KERNEL_IMPL=cutile`).
- **`.github/workflows/nvi-ci.yml`** — adds a `cuTile (B200)` entry to
the `daily` matrix and refreshes the surrounding comments.
## Strict `B200!` GPU spec
Both B200 jobs now use the strict `gpu="B200!"` form, matching the H100
jobs' `H100!`. The `!` tells Modal not to substitute a different GPU
family. This matters for arch-specific SM100 tests: without it, a
fallback/substituted GPU could produce a **misleading pass**. This PR
also flips the pre-existing `liger_cutedsl_tests_b200` from `"B200"` →
`"B200!"` for consistency (small, related change — easy to drop if
undesired).
`max-parallel: 3` and the workflow-level concurrency group are
unchanged, so the 5th daily matrix entry still runs at most 3 GPU jobs
at once.
## Verification
Locally on a B200: `LIGER_KERNEL_IMPL=cutile pytest test/cutile/` →
**199 passed, 12 skipped**. The new job is meaningful and currently
green on B200.
Co-authored-by: GitHub Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
## Summary
Reorganizes the **NVIDIA GPU** workflow into a cadence- and GPU-aware
schedule, and adds the Modal functions to back it. Follows up #1410
(which added the per-DSL test folders and `make test-cutile` /
`test-cutedsl` / `test-cute` targets).
### Schedule
**Daily** (cron `24 9 * * *`):
| Suite | GPU | Modal fn |
|---|---|---|
| general correctness | H100 | `liger_correctness_tests` |
| cuTile | H100 | `liger_cutile_tests` (new) |
| cuTeDSL — SM90 half | H100 | `liger_cutedsl_tests_h100` (new) |
| cuTeDSL — SM100 half | B200 | `liger_cutedsl_tests_b200` (new) |
cuTeDSL runs on **both** archs on purpose: its SM90 tests only run on
H100 and its SM100 tests only run on B200 — each arch self-skips the
other half, so both are needed for full coverage.
**Weekly** (cron `24 13 * * 0`, Sundays, H100): `convergence` (latest
tf) + oldest-supported-tf **correctness** and **convergence**.
**Not scheduled:** `cute` (fused-MoE) tests, for now.
### The ≤3-concurrent fix
A 4th simultaneous Modal GPU call was being rejected (one job died
instantly at ~0.7 min). This is now capped two ways:
- **`strategy.max-parallel: 3`** on each matrix job (daily's 4 entries
run 3-then-1; weekly's 3 run together).
- **Workflow-level `concurrency: { group: <workflow>,
cancel-in-progress: false }`** serializes whole runs, so the daily and
weekly runs never overlap.
Only **one** cadence job activates per trigger (the job-level `if`
selects daily vs weekly from `github.event.schedule` / the
`workflow_dispatch` `suite` input), so at most **3** GPU jobs ever run
together. The `if` deliberately uses only the `github` context — **not**
the `matrix` context, which isn't available in a job-level `if`.
### Manual runs
`workflow_dispatch` gains a `suite` input (`daily` / `weekly`, default
`daily`).
## Notes / caveats
- **Convergence is currently red** on `main`'s scheduled runs
(pre-existing; correctness passes). Weekly scheduling is fine but expect
failures until that's fixed separately.
- **DSL suites run in CI for the first time** — the cuTeDSL-B200 (SM100)
path and the cuTile routing test may surface latent failures on first
run.
- **B200 access**: uses `gpu="B200"` on Modal — confirm the token/plan
has B200 capacity.
- `cuTile` installs the `cutile-tileiras` extra so `import cuda.tile`
resolves (otherwise those tests importorskip).
- Validated locally with a YAML parser + a gating simulation (exactly
one job per trigger) + modal-fn-name cross-check; `actionlint` wasn't
installable in the dev sandbox, and GPU suites can't run there.
Co-authored-by: Copilot <[email protected]>
## Summary
The three opt-in GPU DSL backends — **cutedsl**, **cutile**, and
**cute** (fused-MoE) — had their tests inconsistently placed: a partial
`test/cutedsl/` folder existed, but most DSL tests were scattered in
`test/transformers/`. This PR gives each DSL its **own test folder**,
mirroring the existing `test/cutedsl/` convention (drop the DSL prefix
from filenames), and wires up matching `make` targets.
One minor change in cron schedule for nvidia-ci.
## Changes
### cutedsl → `test/cutedsl/`
- Moved `test_cutedsl_fused_linear_cross_entropy_sm90` →
`test_fused_linear_cross_entropy_sm90`
- Moved `test_cutedsl_rms_norm_fastpath` → `test_rms_norm_fastpath`
- Moved `test_cutedsl_rope` → `test_rope`
- Moved `test_swiglu_cutedsl` → `test_swiglu`
- **Deleted** `test_cutedsl_rms_norm` — it was a duplicate *subset* of
the canonical `test/cutedsl/test_rms_norm.py` (same two test-function
names; the canonical file is a strict superset with the extra Blackwell
fast-path tests and stricter mode-specific bf16 dW tolerances). Deleting
avoids a filename + test-name collision; the canonical file is left
untouched.
- Fixed a stale docstring path in `test/cutedsl/test_backend.py`.
### cutile → new `test/cutile/`
- Moved `test_cutile_backend` → `test_backend`,
`test_cutile_fused_linear_cross_entropy_sm90` →
`test_fused_linear_cross_entropy_sm90`, `test_cutile_rms_norm` →
`test_rms_norm`, `test_cutile_rope` → `test_rope`, `test_cutile_swiglu`
→ `test_swiglu`.
- **Split** the 7 env-gated `test_cutile_*` tests out of the shared
`test/transformers/test_fused_linear_scaled_cross_entropy.py` into
`test/cutile/test_fused_linear_scaled_cross_entropy.py`. The
backend-agnostic frontend/dispatcher tests stay in `test/transformers/`.
### cute (fused-MoE) → new `test/cute/`
- Moved `test_cute_moe_autograd` → `test_moe_autograd` and `test_moe` →
`test_moe`.
- `test_fused_moe.py` is the general Triton MoE test and **stays** in
`test/transformers/`.
### Makefile
- `test:` and `test-verbose:` now `--ignore=test/cutile` and
`--ignore=test/cute` (and `test-verbose` also ignores `test/cutedsl` for
consistency), so DSL tests run only via their dedicated targets.
- New **`test-cutile`** (`LIGER_KERNEL_IMPL=cutile`, mirrors
`test-cutedsl`).
- New **`test-cute`** — intentionally does **not** set
`LIGER_KERNEL_IMPL=cute`, because selecting it would force `import
liger_kernel.ops` to load the separate `liger_cute_kernels` (lck) wheel
and break the always-on registration/import-guard tests.
## Notes / decisions
- All moves use `git mv` to preserve history. The `test.utils` import
blocks in the moved rope/swiglu files were re-sorted by ruff because
`test` is now first-party inside the packaged DSL folders.
- **Coverage shift (intended):** a few always-on pure-Python tests
(cutedsl rms_norm fastpath, cute registration/import-guard) previously
ran under `make test`; they now run via the dedicated `test-*` targets,
consistent with the one-folder-per-DSL model.
- No new basename collisions: the DSL folders all have `__init__.py`
(unique package-qualified module names) and are ignored by `make test`.
## Validation
- `ruff check` and `ruff format --check` pass across the whole `test/`
tree.
- All affected files pass `py_compile`; `make -n` dry-runs verified for
the new targets.
- ⚠️ GPU/Modal test execution (`make test-cutedsl` / `test-cutile` /
`test-cute`) was **not** run in this environment (no CUDA/torch
available). These should be validated on a GPU host with the
corresponding extras installed.
---------
Co-authored-by: Copilot <[email protected]>
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
Remove push, pull_request, and merge_group triggers from the NVIDIA GPU
workflow so it no longer runs on every PR/merge. It now runs only via
workflow_dispatch (manual) and a daily schedule (00:00 UTC). Also drop
the now-dead merge_group if-guards on all jobs and simplify the
concurrency key.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
CI test to be done
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
Co-authored-by: Copilot <[email protected]>
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
Add -m flag for modal runs. Otherwise modal runs fail with
```
Run modal run dev.modal.tests::liger_oldest_v4_correctness_tests
╭─ Error ──────────────────────────────────────────────────────────────────────╮
│ Python module paths must be specified with the -m flag. Use `modal run -m │
│ dev.modal.tests` instead. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
Also, updated secrets with a temporary modal secrets to test things out.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
Update ROCm source install instructions to install ROCm PyTorch wheels
before installing Liger, so `pip install -e .` and `pip install -e
".[dev]"` work without replacing the ROCm torch stack. Align AMD CI with
the same install order and ROCm wheel index.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
1. Removing checkstyle from GPU tests as they are removed from
workflows.
2. Renaming Nvidia GPU tests to they can be configured from branch
protection rules.
3. Adding if clause to run the GPU tests only in merge_group.
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
Bug fix for merge queue.
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
Refactoring checkstyle as independent workflow and removing it from
other workflows.
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
Checkstyle run as part of this PR.
- Hardware Type: <BLANK>
- [x] run `make test` to ensure correctness
- [x] run `make checkstyle` to ensure code style
- [x] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
We have too many tests right now. It costs too much.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [x] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
>[!IMPORTANT]
>Do not merge this PR before all issues are resolved!
>
>Testing with three versions: `4.52.0` , `4.57.6` and the latest stable
version
>[!NOTE]
>nvi-ci is split into correctness test ci and convergence test ci to
speed up testing in this PR
>, and more jobs for testing bc with transformers v4 (4.49.0 and 4.57.6)
>
>Whether keeping this change or not is yet to be discussed
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
This is a dev branch for aggregating PRs related to transformers v5
changes.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [x] run `make test` to ensure correctness
- [x] run `make checkstyle` to ensure code style
- [x] run `make test-convergence` to ensure convergence
---------
Signed-off-by: Tcc0403 <[email protected]>
Co-authored-by: Yunwei Dai <[email protected]>
Co-authored-by: Vaibhav Jindal <[email protected]>
Co-authored-by: Yuki Uehara <[email protected]>
Co-authored-by: Chun-Mao Lai <[email protected]>
Co-authored-by: Vaibhav Jindal <[email protected]>
Co-authored-by: Michael Lai <[email protected]>
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
Changes to the nvidia tests in merge queue.
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
n/a
- Hardware Type: <BLANK>
- [x] run `make test` to ensure correctness
- [x] run `make checkstyle` to ensure code style
- [x] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
Moving unit testing to a merge queue.
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
Not applicable, workflow change.
- Hardware Type: <BLANK>
- [x] run `make test` to ensure correctness
- [x] run `make checkstyle` to ensure code style
- [x] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
This PR fixes a command injection vulnerability in the benchmark
workflow by sanitizing user-controlled inputs.
### Security Issue:
The `commit_hash` and `overwrite` workflow dispatch inputs were used
directly in shell commands via `${{ github.event.inputs.* }}`
interpolation. This allowed an attacker to inject arbitrary shell
commands through the "Commit hash to benchmark" input field.
### Fix:
User-controlled inputs are now declared as environment variables in the
workflow's env section.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
Copy the benchmark data from gh-pages before every mkdocs deployment and
paste it after the deployment. This helps in keeping the benchmark data
after every mkdocs deployment.
Fixes: https://github.com/linkedin/Liger-Kernel/issues/875
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
Will have to fix the docs in a new PR.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
Daily tests have become costly in terms of Modal credits since we now
run them on H100s. At the same time, they don’t add much value because
we already run tests on every push to the main branch. The only real
benefit is catching regressions when upstream dependencies like
transformers change, but those issues will surface soon enough once new
changes are pushed to the main branch.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [x] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
This PR fixes#793 The AMD CI for `Liger-Kernel` was failing due to
PyTorch ROCm not being installed correctly. The CI was installing a CUDA
build instead of ROCm because PyPI packages were taking precedence over
the extraindexurl nightly ROCm index URL. This caused the error:
```
and getattr(torch, infer_device()).get_device_properties(0).total_memory < 36e9,
E AttributeError: module 'torch.cpu' has no attribute 'get_device_properties'
```
Since CUDA PyTorch couldn't detect the GPU on the AMD Device.
This change updates the README and setup so that AMD users explicitly
install the ROCm nightly PyTorch after the dev requirements, ensuring
the correct ROCm build is used and torch can detect GPU.
---
## Details
1. **Intel CI issues remain unresolved.**
2. AMD users now need to run one additional step to (as updated in
readme to downalod rocm torch, I tried using index-url instead of
extra-index-url that helped to downalod correct version but still
installed cuda version because of precedence so its better to manually
downalod as specified in readme):
```
pip install -e .[dev]
pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm6.3/
```
This ensures the ROCm build overrides any CUDA installations from PyPI.
---
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
Ran:
`python -m pytest test/transformers/test_tvd.py -v `
(was earlier giving AttributeError: module 'torch.cpu' has no attribute
'get_device_properties' , now all testcase passed)
<img width="1440" height="778" alt="testcasepassed"
src="https://github.com/user-attachments/assets/ec1f9d81-2afa-45d2-9b29-ccbacccf257f"
/>
<img width="1440" height="774" alt="alltestcaserun"
src="https://github.com/user-attachments/assets/5a427ebf-f8e4-4d38-bbf0-0bd9012504bc"
/>
All test cases passed on AMD MI300X.
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: AMD MI300X
- [x] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
Fixes#766
As described in the issue, this PR aims to enhance docs by enabling
auto-generate docs. Additionally, I've set the paths to only look for
changes in docs/, so the doc workflow is not executed on every commit.
Additionally, I've also exposed doc-strings to be visible as part of the
docs.
The site is here: https://paragekbote.github.io/Liger-Kernel/
Also, what do you think can be done about the low-level api page, can
something similar be performed?
cc: @shimizust
## Testing Done
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
Weekly runs were failing, fixed it.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
---------
Co-authored-by: Manan Shah <[email protected]>
Co-authored-by: Vaibhav Jindal <[email protected]>
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
This pr adds code for running benchmarks for every release.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
---------
Co-authored-by: Manan Shah <[email protected]>
Co-authored-by: Vaibhav Jindal <[email protected]>
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
This pr does two things:
1) Making H100 a strict requirement
2) It gives an option to run benchmarks for any commit manually but with
the current (latest ie from main branch) benchmark scripts. This will
help us get accurate benchmarks results for previous commits after the
benchmark scripts are updated (merge this after the new benchmark
scripts are merged).
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
---------
Co-authored-by: Manan Shah <[email protected]>
Co-authored-by: Vaibhav Jindal <[email protected]>
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
This PR sets the time limit for the modal to run to 90 minutes and also
the workflow would be triggered every Friday.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
---------
Co-authored-by: Manan Shah <[email protected]>
Co-authored-by: Vaibhav Jindal <[email protected]>
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
---------
Co-authored-by: Manan Shah <[email protected]>
Co-authored-by: Vaibhav Jindal <[email protected]>
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
---------
Co-authored-by: Manan Shah <[email protected]>
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
The docs workflow currently replaces the gh-pages branch with new files
due to which the benchmark data is lost. Currently I am disabling this
so that we can get the data appended in the gh-pages branch.
Also, modified the script to add commit hashes to a json file, so that
we do not call github api to fetch it due to API limitations
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
GH_TOKEN removed and used github config info to push to gh-pages branch.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
This is pr for adding the data generated from the benchmark scripts to
the gh-pages branch. It adds in this fashion:
latest_main_branch_commit_hash/benchmark.csv.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
Trying to run just the KTO loss for now for testing purposes. It will
run after any thing is pushed to the main branch
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
- I believe we need to publish docs on pushes to `main` branch, not
`gh-pages` (`mkdocs gh-deploy --force` itself pushes to `gh-pages`)
- To resolve https://github.com/linkedin/Liger-Kernel/issues/641
## Testing Done
None
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
This PR does 2 things
- skip 2 tests that fails on XPU
- update XPU yaml file to use docker container
- Hardware Type: Intel GPU
- [x] run `make test` to ensure correctness
- [x] run `make checkstyle` to ensure code style
- [x] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
This is a PR to enable ROCm 6.3 CI given that the pytorch nightly build
includes ROCm 6.3 now. However, this **PR should not be merged yet** as
there is unexplained reasons as to why one of the convergence tests is
passing on ROCm 6.2 but failed on ROCm 6.3.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [x] run `make test` to ensure correctness
- [x] run `make checkstyle` to ensure code style
- [x] run `make test-convergence` to ensure convergence
Additional Info 2025-1-2
- All the tests passed on ROCm 6.2
- Only one failed test in ROCm 6.3
- FAILED CONVERGENCE TEST
```
====================================================== short test
summary info =======================================================
FAILED
test/convergence/test_mini_models.py::test_mini_model[mini_mllama-32-0.0001-dtype2-1e-08-1e-05-0.005-1e-05-0.005-1e-05]
- AssertionError: Number of mismatched elements: 2
Mismatch at index (0, 7): tensor1[(0, 7)] = 3.0651497840881348,
tensor2[(0, 7)] = 3.0652356147766113
Mismatch at index (0, 9): tensor1[(0, 9)] = 1.470238447189331,
tensor2[(0, 9)] = 1.4702625274658203
======================================== 1 failed, 16 passed, 2 warnings
in 94.82s (0:01:34) ==================
```
Co-authored-by: Shao Tang <[email protected]>
## Summary
This PR adds the dependencies for using liger-kernel on XPU and adds the
workflow.yml.
## Testing Done
Test done on local XPU machine. To make all tests pass, we need to first
merge PR #535 first.
- Hardware Type: Intel(R) Data Center GPU Max 1550
- [x] run `make test` to ensure correctness
- [x] run `make checkstyle` to ensure code style
- [x] run `make test-convergence` to ensure convergence
---------
Co-authored-by: Byron Hsu <[email protected]>
Co-authored-by: Stefan He <[email protected]>
## Summary
Fixes#64
Instead of using Sphinx which I found to be cumbersome to set-up and
iterate upon, I have created the docs using Material for Markdown, which
uses markdown files for pages and does not change the original markdown
files to a greater extent.
To preview in your local system: run make serve in the terminal.
To build pages: run make build in the terminal.
Index page:
Examples page:
Please let me know if any styling or further corrections are needed and
I will make the necessary changes.
cc: @ByronHsu
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
Closes https://github.com/linkedin/Liger-Kernel/issues/451
This PR implements dynamic dependency management that automatically
detects the user's hardware platform (CUDA, ROCm, or CPU) and installs
the appropriate dependencies. This ensures a smoother installation
experience across different hardware configurations.
### Key Changes
- Added `_get_platform()` function to detect hardware platform using
`nvidia-smi` or `rocm-smi`
- Implemented `get_default_dependencies()` to return platform-specific
core dependencies
- Implemented `get_optional_dependencies()` to handle dev and
transformers extras
- Updated `pyproject.toml` to use dynamic dependencies from setup.py
### Platform-specific Dependencies
- **CUDA/CPU**:
- Core: torch>=2.1.2, triton>=2.3.1
- Full dev environment including transformers, testing tools, and
visualization libraries
- **ROCm**:
- Core: torch>=2.6.0.dev, triton>=3.0.0, plus ROCm-specific requirements
- Streamlined dev dependencies focused on essential tools
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
### setup.py vs pyproject.toml
#### setup.py
- Traditional Python package configuration
- Allows dynamic configuration via Python code
- Needed for complex/conditional dependencies
- Supports custom build steps
#### pyproject.toml
- Modern, static configuration (PEP 517/518)
- Cleaner, declarative format
- Works well with modern tools (poetry, hatch)
- Preferred for simple packages
#### Using Both
- `pyproject.toml`: Build system specs, basic metadata
- `setup.py`: Dynamic dependencies, custom build logic
- Common in projects needing both static config and dynamic install
-
## Testing Done (In Progress)
<!--- This is a required section; please describe how this change was
tested. --->
1. Clone the repository
2. Test installation on different platforms:
```bash
# Basic installation
pip install -e .
# With dev tools
pip install -e .[dev]
# With transformers
pip install -e .[dev,transformers]
```
- [x] CPU
- [x] CUDA - NVIDIA
Tested with following config
- NVIDIA-SMI 535.129.03
- Driver Version: 535.129.03
- CUDA Version: 12.2
- [ ] ROCm - AMD
- Need help from the community to test on ROCm platform
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
## Hardware Type:
- NVIDIA-SMI 535.129.03
- CUDA Version: 12.2
- [x] run `make test` to ensure correctness
- [x] run `make checkstyle` to ensure code style
- [x] run `make test-convergence` to ensure convergence
---------
Co-authored-by: ByronHsu <[email protected]>
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
This is to clean up the `amd-ci.yml` setup steps.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [x] run `make test` to ensure correctness
- [x] run `make checkstyle` to ensure code style
- [x] run `make test-convergence` to ensure convergence
---------
Co-authored-by: tjtanaa <[email protected]>
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->
## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->
<!--
Replace BLANK with your device type. For example, A100-80G-PCIe
Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them.
-->
- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [ ] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence
