mirror of
https://github.com/linkedin/Liger-Kernel.git
synced 2026-10-12 02:17:47 +00:00
Vaibhav Jindal
99a90f7cc4
## Summary <!--- This is a required section; please describe the main purpose of this proposed code change. ---> Copy the benchmark data from gh-pages before every mkdocs deployment and paste it after the deployment. This helps in keeping the benchmark data after every mkdocs deployment. Fixes: https://github.com/linkedin/Liger-Kernel/issues/875 <!--- ## Details This is an optional section; is there anything specific that reviewers should be aware of? ---> ## Testing Done <!--- This is a required section; please describe how this change was tested. ---> <!-- Replace BLANK with your device type. For example, A100-80G-PCIe Complete the following tasks before sending your PR, and replace `[ ]` with `[x]` to indicate you have done them. --> - Hardware Type: <BLANK> - [ ] run `make test` to ensure correctness - [ ] run `make checkstyle` to ensure code style - [ ] run `make test-convergence` to ensure convergence
4.6 KiB
4.6 KiB
Design
- @claire_yishan for the LOGO design
- Wave Snippets for generating the animated code snippets
Code
We referenced or used the following projects:
| # | Project | Description | Location | License |
|---|---|---|---|---|
| 1 | Unsloth | calculate_settings to determine block size and warp; We reuse it for Norm and MLP |
Liger Kernel Utils | Apache |
| 2 | Unsloth | We modified and added dW calculation on top of Unsloth implementation | Liger Kernel RMS Norm | Apache |
| 3 | Triton tutorial | We modified on top of triton tutorials | Liger Kernel RMS Norm | MIT |
| 4 | tiny shakespeare dataset | We use tiny shakespeare dataset to conduct convergence test on mini model | Liger Kernel Convergence | N/A |
| 5 | Efficient Cross Entropy | We use the idea of gradient-in-forward and chunking | Liger Kernel Linear Cross Entropy | MIT |
| 6 | Flash attn | We take many optimization ideas from the work, such as tiling and recomputation | BSD | |
| 7 | AutoAWQ | We reference the design of automodel | Liger Kernel Auto Model | MIT |
| 8 | llm.c | We reference the design of end-to-end testing | Liger Kernel Convergence Tests | MIT |
Many thanks to the contributors to these projects for their invaluable work that helped make Liger possible.