This commit is contained in:
ByronHsu
2024-08-13 06:51:06 +00:00
parent e85611c96f
commit 04b954550a
+7
View File
@@ -78,6 +78,10 @@ apply_liger_kernel_to_llama()
model = transformers.AutoModelForCausalLM.from_pretrained("<some llama model>")
```
[TODO] huggingface trainer
[TODO] lightning trainer
### 2. 🧩 Compose Your Own Model
```python
@@ -97,6 +101,9 @@ loss = loss_fn(model.weight, input, target)
loss.backward()
```
[TODO] multi-head llama3 model
[TODO] large vocab model
## ⚙️ Note on ML Compiler
### 1. ⚡ Torch Compile