Parag Ekbote
954eee2bb0
## Summary Fixes #766 As described in the issue, this PR aims to enhance docs by enabling auto-generate docs. Additionally, I've set the paths to only look for changes in docs/, so the doc workflow is not executed on every commit. Additionally, I've also exposed doc-strings to be visible as part of the docs. The site is here: https://paragekbote.github.io/Liger-Kernel/ Also, what do you think can be done about the low-level api page, can something similar be performed? cc: @shimizust ## Testing Done <!-- Replace BLANK with your device type. For example, A100-80G-PCIe Complete the following tasks before sending your PR, and replace `[ ]` with `[x]` to indicate you have done them. --> - Hardware Type: <BLANK> - [ ] run `make test` to ensure correctness - [ ] run `make checkstyle` to ensure code style - [ ] run `make test-convergence` to ensure convergence
3.7 KiB
High-Level APIs
AutoModel
| AutoModel Variant | API |
|---|---|
| AutoModelForCausalLM | liger_kernel.transformers.AutoLigerKernelForCausalLM |
This API extends the implementation of the AutoModelForCausalLM within the transformers library from Hugging Face.
::: liger_kernel.transformers.AutoLigerKernelForCausalLM options: extra: show_docstring: true show_signature: true show_source: true
!!! Example "Try it Out" You can experiment as shown in this example here.
Patching
You can also use the Patching APIs to use the kernels for a specific model architecture.
| Model | API | Supported Operations |
|---|---|---|
| LLaMA 2 & 3 | liger_kernel.transformers.apply_liger_kernel_to_llama |
RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
| LLaMA 3.2-Vision | liger_kernel.transformers.apply_liger_kernel_to_mllama |
RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
| Mistral | liger_kernel.transformers.apply_liger_kernel_to_mistral |
RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
| Mixtral | liger_kernel.transformers.apply_liger_kernel_to_mixtral |
RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
| Gemma1 | liger_kernel.transformers.apply_liger_kernel_to_gemma |
RoPE, RMSNorm, GeGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
| Gemma2 | liger_kernel.transformers.apply_liger_kernel_to_gemma2 |
RoPE, RMSNorm, GeGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
| Qwen2, Qwen2.5, & QwQ | liger_kernel.transformers.apply_liger_kernel_to_qwen2 |
RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
| Qwen2-VL | liger_kernel.transformers.apply_liger_kernel_to_qwen2_vl |
RMSNorm, LayerNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
| Phi3 & Phi3.5 | liger_kernel.transformers.apply_liger_kernel_to_phi3 |
RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
Function Signatures
::: liger_kernel.transformers.apply_liger_kernel_to_llama options: extra: show_docstring: true show_signature: true
::: liger_kernel.transformers.apply_liger_kernel_to_mllama options: extra: show_docstring: true show_signature: true
::: liger_kernel.transformers.apply_liger_kernel_to_mistral options: extra: show_docstring: true show_signature: true
::: liger_kernel.transformers.apply_liger_kernel_to_mixtral options: extra: show_docstring: true show_signature: true
::: liger_kernel.transformers.apply_liger_kernel_to_gemma options: extra: show_docstring: true show_signature: true
::: liger_kernel.transformers.apply_liger_kernel_to_gemma2 options: extra: show_docstring: true show_signature: true
::: liger_kernel.transformers.apply_liger_kernel_to_qwen2 options: extra: show_docstring: true show_signature: true
::: liger_kernel.transformers.apply_liger_kernel_to_qwen2_vl options: extra: show_docstring: true show_signature: true
::: liger_kernel.transformers.apply_liger_kernel_to_phi3 options: extra: show_docstring: true show_signature: true