mirror of
https://github.com/linkedin/Liger-Kernel.git
synced 2026-10-12 02:17:47 +00:00
Vaibhav JindalandCopilot
d5f281789a
## Summary
`test/transformers/test_llama4_rope.py` constructs the rotary embedding
directly as
`Llama4TextRotaryEmbedding(config=config, device=device)`. Newer
HuggingFace `transformers`
deprecated the `device` kwarg on config-based
`*RotaryEmbedding.__init__`
(`@deprecate_kwarg("device", version="5.18")`) and **removes it at
5.18**. Because CI installs
the latest `transformers` (`setup.py` pins `transformers>=4.52.0`, no
upper bound), every test
in the file fails at setup with:
```
TypeError: Llama4TextRotaryEmbedding.__init__() got an unexpected keyword argument 'device'
```
This fixes all of the currently-failing `test_llama4_rope.py` cases by
dropping the removed kwarg.
## Why dropping `device=` is correct (not just silencing)
`Llama4TextRotaryEmbedding.forward` relocates the frequency buffer to
the input tensor's device:
```python
inv_freq_expanded = self.inv_freq[...].to(x.device)
```
and the test tensors are already created on the target device, so the
rotary module does not
need to be constructed on a specific device. Verified on
`transformers==5.15.0` that `cos`/`sin`
are **bitwise identical** (`torch.equal`) to the previous
`device=`-based construction.
## Relationship to #1514
#1514 fixed the same `device`-kwarg drift, but only for call sites
routed through
`transformers_version_dispatch`. `test_llama4_rope.py` constructs the
module **directly**, so it
was not covered. This PR is the direct-construction counterpart.
## Testing
- `pytest test/transformers/test_llama4_rope.py` → **19 passed**
(transformers 5.15.0, H100).
- Simulated transformers ≥5.18 (strict signature): the old `device=`
call raises the exact CI
`TypeError`; the new call succeeds.
- `ruff check` / `ruff format --check` clean.
## Compatibility
| transformers | Before this PR | After this PR |
|---|---|---|
| < 4.48.0 | n/a (test path unchanged) | n/a |
| 4.48.0 – <5.18 | `device=` accepted (deprecation warning) | no
`device=`, identical result, no warning |
| ≥ 5.18 | **`TypeError` at construction** | works |
Co-authored-by: Copilot <[email protected]>