mirror of https://github.com/linkedin/Liger-Kernel.git synced 2026-10-12 02:17:47 +00:00

Files

T

Shivam Sahni bb84c0d2cf fix qwen2vl and mllama test to pass failing tests (#571 )

## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
1. Fix Qwen2VL mrope tests. Transformers removed BC with
Qwen2VLRotaryEmbedding which is why we need this change
2. Fix mllama tests to not use cross entropy. Using Cross Entropy causes
logits mismatch which is expected as we do inplace operation in cross
entropy kernel.
3. Runs make checkstyle --- there seems to be a bug in checkstyle
workflow where unformatted checkstyle seems to pass tests. Changed the
makefile to throw error in such cases.
<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->

## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->

<!-- 
Replace BLANK with your device type. For example, A100-80G-PCIe

Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them. 
-->

- Hardware Type: <BLANK>
- [ ] run `make test` to ensure correctness
- [x] run `make checkstyle` to ensure code style
- [ ] run `make test-convergence` to ensure convergence

2025-02-20 11:54:47 -08:00

README.md

Switch Lightning 1 GPU example to Qwen2 0.5B instruct model with 1024 max seq length (#193 )

2024-09-03 11:03:58 -07:00

requirements.txt

Switch Lightning 1 GPU example to Qwen2 0.5B instruct model with 1024 max seq length (#193 )

2024-09-03 11:03:58 -07:00

training.py

fix qwen2vl and mllama test to pass failing tests (#571 )

2025-02-20 11:54:47 -08:00

README.md

Liger-Kernel Example with Lightning Trainer

How to Run

pip install -r requirements.txt

# For single L40 48GB GPU
python training.py --model Qwen/Qwen2-0.5B-Instruct --num_gpu 1 --max_length 1024

# For 8XA100 40GB
python training.py --model meta-llama/Meta-Llama-3-8B --strategy deepspeed

Notes

The example uses Llama3 model that requires community license agreement and HuggingFace Hub login. If you want to use Llama3 in this example, please make sure you have done the followings:
- Agree on the community license agreement https://huggingface.co/meta-llama/Meta-Llama-3-8B
- Run huggingface-cli login and enter your HuggingFace token
The default hyperparameters and configurations for gemma works on single L40 48GB GPU and config for llama work on single node with 8xA100 40GB GPUs. For running on device with less GPU RAM, please consider reducing the per-GPU batch size and/or enable CPUOffload in FSDP.