linkedin/Liger-Kernel
1
0
Fork 0
mirror of https://github.com/linkedin/Liger-Kernel.git synced 2026-10-12 02:17:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

[Fix] Handle missing 'elementwise_affine' in RMSNorm extra_repr for patched models (#990)

Browse Source 
## Summary

Fixes an AttributeError encountered in `LigerRMSNorm.extra_repr` when
models are patched `in-place` (e.g., using
`apply_liger_kernel_to_model`).

In my previous PR (#989 ), I added `elementwise_affine` to `extra_repr`
to improve layer visibility.
However, when layers are replaced via monkey patching, the
`LigerRMSNorm` constructor is typically skipped, leaving the instance
without the `elementwise_affine` attribute.


## Testing Done

I verified the fix locally. The issue is reproducible via
`test/transformers/test_monkey_patch.py`. After applying this fix,
`test/transformers/test_monkey_patch.py` passes successfully.

- Hardware Type: NVIDIA A100-SXM4-80GB
- [x] run `make test` to ensure correctness
- [x] run `make checkstyle` to ensure code style
- [x] run `make test-convergence` to ensure convergence
This commit is contained in:
Yunsheng Ni
2025-12-25 13:20:53 +08:00
committed by GitHub
parent 87d7057ae3
commit 01a66f1329
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/liger_kernel/transformers/rms_norm.py
+1 -1
View File
@@ -46,7 +46,7 @@ class LigerRMSNorm(nn.Module):
)
def extra_repr(self):
return f"elementwise_affine={self.elementwise_affine}, weight_shape={tuple(self.weight.shape) if self.weight is not None else None}, eps={self.variance_epsilon}, offset={self.offset}, in_place={self.in_place}, row_mode={self.row_mode}"
return f"weight_shape={tuple(self.weight.shape) if self.weight is not None else None}, eps={self.variance_epsilon}, offset={self.offset}, in_place={self.in_place}, row_mode={self.row_mode}"
class LigerRMSNormForGemma(LigerRMSNorm):