mirror of
https://github.com/invoke-ai/InvokeAI.git
synced 2026-10-12 02:16:43 +00:00
551363c70a
* feat(model-manager): support GGUF-quantized T5 text encoders Add loading support for single-file GGUF T5 encoders (e.g. city96/t5-v1_1-xxl-encoder-gguf, llama.cpp naming), mirroring the existing Qwen3 GGUF encoder path. - Add T5Encoder_GGUF_Config (single-file, detects enc.blk.* keys + GGML tensors) and register it in the AnyModelConfig union - Add T5EncoderGGUFModel loader: remaps llama.cpp T5 keys to transformers naming, infers T5Config from tensor shapes, dequantizes token/relative-attention-bias embeddings, ties embed_tokens to shared - Work around transformers T5DenseGatedActDense casting activations to the uint8 GGML weight dtype (int8 guard doesn't cover uint8), which would corrupt the feed-forward output - Reject T5 encoders in the Qwen3 GGUF/checkpoint configs so the two stay mutually exclusive (both carry token_embd.weight; the factory resolves multi-matches from a set, so this is not order-safe) Reuse the vendored T5-XXL tokenizer instead of downloading it: move it out of Anima into a neutral invokeai/backend/t5 module shared by Anima and the GGUF loader, and update the package-data path accordingly. * Chore Typegen + Openapi * Add T5 Recalling * Add 2 gguf T5 to the Starter Models * Chore Ruff * Chore Typegen * test(t5-gguf): add unit tests for GGUF T5 loader helpers + fail-loud FFN patch guard - Add unit coverage for the pure, high-risk parts of T5EncoderGGUFModel: key remapping (_convert_t5_gguf_to_transformers), config inference (_infer_t5_config_from_state_dict), and the wo-dtype workaround. - Make _make_feed_forward_gguf_safe raise if it patches no feed-forward modules, so a future transformers class rename fails loudly at load time instead of silently corrupting encoder output. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * refactor(t5-gguf): drop dead _shape_of branch, document T5 v1.1 XXL assumptions - Inline tensor.shape in _infer_t5_config_from_state_dict: GGMLTensor.shape already returns the dequantized (logical) shape, so the _shape_of helper's fallback branch was unreachable. Remove the helper. - Document that config inference targets the T5 v1.1 XXL family and that the hardcoded architectural constants (rel-attention max distance, layer-norm epsilon, gated-gelu) are that family's defaults. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> --------- Co-authored-by: Lincoln Stein <[email protected]> Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>