* Fix embedding LoRA backward under autocast
* Tie LoRA adapters on tied output embeddings to the embedding adapter
PEFT's ensure_weight_tying builds the lm_head adapter as new Parameters over
views of the embed_tokens adapter. Adapter autocast on a bf16 model replaces
each one's storage separately, silently untying them, and even when storage
survives the optimizer keeps two states for one buffer. Replace the head's
lora_A/lora_B with parameterless modules that read the transposed embedding
adapter, so there is one set of weights whose gradient sums both uses. Saved
adapters keep PEFT's key layout.
* Keep tied LoRA adapters in a shared FSDP ownership group
Wing Lian
2a13793a63