mirror of
https://github.com/linkedin/Liger-Kernel.git
synced 2026-10-12 02:17:47 +00:00
Stop printing coverage report by default in make test / test-convergence and fix tolerance in polynorm (#1310)
## Summary `pytest-cov` was forced on for every pytest invocation via `addopts` in `pyproject.toml`, dumping a full per-file coverage table (~190 lines) to the terminal on every `make test` and `make test-convergence` run (and any ad-hoc `pytest` invocation), regardless of whether coverage was wanted. This bloated CI/local logs significantly. - Remove `--cov*` flags from `[tool.pytest.ini_options] addopts` in `pyproject.toml` so coverage is no longer collected/printed by default. - Remove the redundant `--cov` flags from the Makefile `test` target. - Add a new opt-in `test-verbose` Makefile target (renamed from an earlier `test-cov`) that runs the same correctness tests with both coverage reporting (`term-missing` + `html`) and a full `--durations=0` test-duration breakdown, for anyone who explicitly wants that diagnostic output. - Remove `--durations=0` from `[tool.pytest.ini_options] addopts` — this was forcing every test's duration (slowest-first) to print on every run, adding a large, mostly-noisy timing block to every `make test`/`make test-convergence` run. Anyone who wants timing info can still pass `--durations=N` (or `--durations=0` for all) manually. ### Follow-up: tightened `poly_norm` bf16 gradient tolerance Also includes a small follow-up from review on #1309: the bf16 `atol` used for `test_poly_norm.py::test_correctness`'s `weight.grad`/`bias.grad` checks was loosened to `1.0`, which was flagged as unnecessarily risky. The largest mismatch actually observed (catastrophic cancellation in a large `sum()` reduction over up to ~524k near-canceling terms, differing reduction order between Triton and PyTorch — see #1309 for full root cause) was `2.25` at a value of `~34.5`. `atol=0.5` (tolerance `2.225`) fails just under that; `atol=0.6` (tolerance `2.325`) gives a small, deliberate margin and passed 5/5 repeated local runs. Tightened from `1.0 → 0.6`. ## Testing Done 1. Verified plain `pytest` / `make test` / `make test-convergence` no longer print coverage tables. 2. Verified `make test-cov` (opt-in) still produces the full coverage report as before. 3. Ran `test_poly_norm.py::test_correctness[dtype1-...]` (bf16) 5x with `atol=0.6` — all passed. Ran the full `test_poly_norm.py` + `test_fused_add_rms_norm.py` suites 3x — consistently `55 passed, 2 xfailed`. - Hardware Type: H100 - [x] run `make test` to ensure correctness - [x] run `make checkstyle` to ensure code style - [ ] run `make test-convergence` to ensure convergence --------- Co-authored-by: Copilot <[email protected]>
This commit is contained in:
co-authored by
Copilot
parent
5eebfda273
commit
5b8476aa92
@@ -1,13 +1,22 @@
|
||||
.PHONY: test checkstyle test-convergence all serve build clean
|
||||
.PHONY: test test-verbose checkstyle test-convergence all serve build clean
|
||||
|
||||
|
||||
all: checkstyle test test-convergence
|
||||
|
||||
# Command to run pytest for correctness tests
|
||||
test:
|
||||
python -m pytest --disable-warnings \
|
||||
--ignore=test/convergence \
|
||||
test/
|
||||
|
||||
# Command to run pytest for correctness tests with coverage reporting and full test-duration breakdown
|
||||
test-verbose:
|
||||
python -m pytest --disable-warnings \
|
||||
--cov=src/liger_kernel \
|
||||
--cov-report=term-missing \
|
||||
--cov-report=html \
|
||||
--cov-config=pyproject.toml \
|
||||
--durations=0 \
|
||||
--ignore=test/convergence \
|
||||
test/
|
||||
|
||||
|
||||
@@ -24,13 +24,6 @@ pythonpath = ["src", "."]
|
||||
asyncio_mode = "auto"
|
||||
log_cli = true
|
||||
log_cli_level = "INFO"
|
||||
addopts = [
|
||||
"--cov=src/liger_kernel",
|
||||
"--cov-report=term-missing",
|
||||
"--cov-report=html",
|
||||
"--cov-config=pyproject.toml",
|
||||
"--durations=0"
|
||||
]
|
||||
python_files = "test_*.py"
|
||||
testpaths = ["test/"]
|
||||
|
||||
|
||||
@@ -99,7 +99,7 @@ class NaivePolyNorm(nn.Module):
|
||||
(torch.float32, 3e-4, 1e-6),
|
||||
pytest.param(
|
||||
torch.bfloat16,
|
||||
1.0,
|
||||
6e-1,
|
||||
5e-2,
|
||||
marks=pytest.mark.skipif(not supports_bfloat16(), reason="bfloat16 not supported on this GPU"),
|
||||
),
|
||||
|
||||
Reference in New Issue
Block a user