mirror of
https://github.com/invoke-ai/InvokeAI.git
synced 2026-10-12 02:16:43 +00:00
main
14
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Alexander Eichhorn
|
aabfc51794 |
fix(model-manager): identify standalone 16-channel VAEs as FLUX.1 or SD3
Folders are identified by their scaling/shift constants and single files by an explicit base, then the backbone their name or install source states; adds FLUX.1-diffusers and SD3 VAE configs, and an override now decides within a latent family instead of being overruled. VAELoader loads SD3 single files in either layout, FLUX.1 folders outside float16, and flat VAE folders requested as SubModelType.VAE. Reidentify passes the stored install source, so name evidence survives a re-probe. |
||
|
Lincoln Stein
|
8ee0be8274 |
feat(models): add LTX-2.5 as an installable model family
- Taxonomy, configs and loaders for the LTX-2.5 component folder, the single-file transformer (bf16 / int8-convrot / header-named nvfp4) and the Gemma-4 text tower; diffusers model classes with pinned configs and key maps verified against the release - Architecture definition with a fitted latent preview space, starter models from the ungated mirror, shared single-file quantization guards extracted from the H3 loader - Identification is structural (2.5 = no video feed-forward bias; four-stage VAE decoder); stock sharded Gemma-4 stays a generic text LLM |
||
|
Alexander Eichhorn
|
22c10070b4 |
Merge branch 'main' into refactor/invocations-by-architecture
Resolved the starter_models modify/delete by distributing upstream's two new MiniMax H3 models over the split package, keeping STARTER_MODELS' curated order. Repointed three upstream test imports and upstream's new minimax_h3_reference.py at their post-refactor paths, grew GenerationModeKind, MiniMax H3's ModalityFacet and its mode-set guard by ref2v, and regenerated openapi.json, which had auto-merged without any of upstream's new nodes. |
||
|
Lincoln SteinandClaude Fable 5
|
d5465cabaf |
feat(mm): recognize MiniMax H3 Ref2VA models
Adds the ref2va variant to MiniMaxH3VariantType and teaches identification to classify it: - Single-file checkpoints: the former hard rejection of ref2va filenames becomes filename-based variant detection (the two task transformers are key-for-key indistinguishable, so the filename is the classifier and the variant override field corrects renamed files). - Diffusers folders: a transformer_ref/-only folder now identifies as ref2va; a dual-transformer folder stays fl2va. Ref2VA folder weights are not folder-loadable in v1, so a ref2va folder is always marked components_only - the generation path is a components install plus a single-file transformer override. - Starter models: Ref2VA pruned int8 transformer (~21 GB) and the Ref2V 4-step Turbo LoRA join the MiniMax H3 bundle. - Identification fixtures: ref2va single file, transformer_ref-only slim folder, and a variant-override case. - Legacy frontend: ref2va in the zod variant enum + label map; openapi.json and schema.ts regenerated. - webv2: variant label/options, a temporary transformer-picker gate, and a ref2v-token exclusion in the Turbo accelerator auto-pick so the Ref2VA LoRA never auto-applies to FL2VA runs. - Model loader invocation: fails fast on a Ref2VA transformer (override or main) so hand-authored workflows cannot silently generate degraded output; TODO(ref2va) markers note what the follow-up removes. Generation support (packing, reference conditioning, UI mode) lands in follow-up PRs. Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01S4B5exWsbC2z2Uu2tA167A |
||
|
Alexander EichhornandClaude Opus 5
|
7c03da6d58 |
fix(api): authenticate the capabilities route, and correct two stale expectations
Two regressions the full suite caught, both mine. `test_every_route_is_authenticated_or_explicitly_public` failed: the new `GET /api/v2/models/capabilities` had no auth dependency. I had reasoned that the response holds nothing user- or install-specific, which is true and is not the question the test asks. Its message says the allowlist is for routes that *must* be public, and this one does not have to be. It now takes `CurrentUserOrDefault` like every other route in the router. That falsifies a claim the route's own tests made — "it needs no services". `CurrentUserOrDefault` resolves through `ApiDependencies.invoker`, so the route does need one now. The test that asserted otherwise is replaced by one that asserts what actually matters and remains true: the model manager service is never touched, so the response cannot have become per-model. `test_default_settings_main[sdxl-refiner-None]` failed because the refiner now declares SDXL's canvas. The expectation was correct until it did; updated, with a note on why the refiner has dimensions but no steps or CFG. `openapi.json` and `schema.ts` regenerated: the delta is this route's description and its `security` block, nothing else. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> |
||
|
Alexander EichhornandClaude Opus 5
|
b6f841c08e |
feat(architectures): declare the generation defaults each architecture recommends
`MainModelDefaultSettings.from_base` was a `match base:` block of twelve cases, four of which sub-dispatched on variant, ending in `case _: return None`. It decides what the generation sliders say when a model is selected — 30 steps for Wan TI2V-5B, CFG off for Krea-2 Turbo — and it is read once, at identification, then stored on the config. These are product decisions rather than derivable facts, so they move to the architecture that owns them and stay data: a mapping from variant to settings, with `None` as the fallback. The facet is deliberately *not* `REQUIRED`. Four architectures (SD 3.5, the SDXL refiner, CogView 4, FLUX.1) reach the old fallback and have no defaults, with a standing `TODO(psyche)` asking whether they should. So a forgotten architecture fails softly here — no crash, just sliders the user sets himself — which is milder than the other facets, and the boot check cannot enforce it. A test pins exactly which four declare nothing, so the set stays a decision rather than an accident. ERNIE-Image's Turbo detection stays name-based, as it must: Turbo and the base model share an architecture and a config, so there is nothing on disk to probe. It becomes a `by_name_hint` mapping rather than a code branch, which keeps ERNIE's two settings objects in `defs/ernie_image.py` with the rest of the product data instead of stranding them in the resolver. The existing behavioural tests — leaf directory counts, ancestor directory must not — now run through it unchanged. `MainModelDefaultSettings` moves to `configs/default_settings.py`, and this is structural rather than tidying. A facet holding instances of that class cannot import `configs/main.py` once `configs/main.py` is the module doing the looking up: main imports the registry, the registry imports the defs, the defs import the facet, and the cycle closes on a half-initialized module. Splitting the data from the behaviour breaks it without a lazy import. `configs/main.py` re-exports the name, so every existing caller is unaffected. Verified by sweeping the old `from_base` against the new resolver over every (base, variant, name, path) combination — 16 bases x 32 variants x 5 names x 4 paths: 10560 for 10560 identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> |
||
|
Lincoln SteinandClaude Fable 5
|
c64c1a49ed |
feat(installer): explicit-file entries in multi-subfolder HF sources; components-only MiniMax H3 installs
A slim MiniMax H3 install (tokenizer/processor/VAEs + the two config JSONs identification needs, ~11 GB) was previously inexpressible: '::a+b' subfolder sources kept only files inside the listed folders, so the root modular_model_index.json was dropped and the download landed as an unknown model. - filter_files(): an entry in a multi-entry subfolder list that names a repo file exactly is included verbatim, bypassing the extension prefilter, variant scoring and config-only-folder pruning. Single-entry sources and pure-folder lists behave exactly as before. - _multifile_download(): explicit file entries keep their repo-relative paths (relative_to() would have flattened transformer/config.json to the root) and contribute their stems to the synthesized folder name. - Main_Diffusers_MiniMaxH3_Config: new components_only field, true when the transformer folder has no weight shards, so the UI can require the single-file transformer/text-encoder selections up front. - MiniMax H3 diffusers loader: requesting the Transformer or TextEncoder submodel from a components-only install fails with an actionable message instead of a cryptic missing-shard error. - Linear UI readiness (Generate + Canvas): a components-only H3 main blocks enqueue until both single-file overrides are selected. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Lincoln SteinandClaude Fable 5
|
a52e5f96bc |
feat(minimax-h3): single-file truncated int8-convrot Qwen3-VL text encoder support
Loads Comfy-Org's qwen3vl_32b_minimax_h3_{bf16,int8_convrot} single files (25.3 GiB int8 vs
the folder's 62 GiB bf16) as a text-encoder override on the H3 model loader node. The files
are a 50-layer truncation matching H3's conditioning contract (unnormalized hidden state
after layer 50; no final norm, no LM head - the loader installs an Identity norm and ties
the unused head to the embeddings). Quantized layers reuse the PR-6 Int8ConvrotLinear
runtime; tokenizer/processor still come from the diffusers folder. nvfp4 repacks are
rejected from the header before any tensor read.
Probe keys on the file's explicit minimax_h3_te safetensors metadata (structural fallback
for stripped re-uploads); Krea-2's Qwen3VLEncoder pickers and default-encoder selection are
scoped to exclude the H3-truncated encoders. Params persist v6->v7 (adds
minimaxH3TextEncoderModel); loader node v1.2.0; metadata + recall wired.
Co-Authored-By: Claude Fable 5 <[email protected]>
|
||
|
Lincoln SteinandClaude Fable 5
|
7174cbe632 |
feat(minimax-h3): single-file pruned/int8-convrot transformer support
Loads MiniMax's single-file FL2VA transformer repacks (Comfy-Org layout): bf16 or Comfy int8_tensorwise(+convrot) quantized, full or AdaLN-pruned. The 19.5 GiB pruned-int8 file stays int8-resident in VRAM (fits a 45 GiB card outright, vs ~53% partial load for the 62 GiB bf16 folder transformer). - Int8ConvrotLinear: int8 weight + per-channel scale buffers, dequantized and derotated per forward. ConvRot is a grouped-256 regular-Hadamard rotation folded into the stored weights; H is symmetric orthonormal, so derotation is the same grouped matmul (semantics verified against Comfy-Org/comfy-quants + comfy-kitchen; reimplemented, not copied). - MiniMaxH3PrunedTransformer3DModel: 'adaln curves' variant - the timestep MLP and full-width AdaLN inputs are replaced by a [1025, 8] fp32 curve table (lerp lookup, SiLU-free fp32 AdaLN projections), matching ComfyUI's reference semantics. Vendored transformer file untouched. - State-dict converter: remote-code -> vendored diffusers key layout, incl. fused-qkv row splitting (scales split with their rows) and comfy_quant marker fan-out. Verified key-for-key + shape-for-shape against the real 19.5 GiB checkpoint header (dry run, no tensor data). - Probe: Main_Checkpoint_MiniMaxH3_Config fingerprints on the H3-unique audio+video patch projections; ref2va excluded by filename; pruned detected via adaln_t_table. Stripped identification fixture generated from the real file's header. - Loader: transformer-only (Wan-GGUF precedent); shape-inferred config; strict load after swapping quantized linears. - Node: minimax_h3_model_loader v1.1.0 gains an optional 'Transformer (single file)' override; folder main keeps serving encoders + VAEs. Default-workflow pins bumped to match. - UI: checkpoint-format H3 mains hidden from the main picker and offered in a MiniMax H3 advanced section combobox; params persist v5 -> v6 (minimaxH3TransformerModel), migration + fixtures updated; wired through buildMiniMaxH3Graph + metadata. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Lincoln SteinandClaude Fable 5
|
e33c50d69f |
feat(minimax-h3): model support - vendored classes, taxonomy, probe, loader
Adds installable model support for MiniMax H3 (Hailuo 3.0) FL2VA, the 33B omni-modal transformer that jointly denoises video and stereo audio. - Vendors MiniMaxH3Transformer3DModel, AutoencoderKLMiniMaxH3, AutoencoderKLMiniMaxH3Audio and MiniMaxH3Scheduler from diffusers PR #14355 at abc5e9bf71 (unreleased branch) into invokeai/backend/minimax_h3. Only local changes: absolute imports + ruff import sorting; all referenced symbols exist in the pinned diffusers 0.39.0. B905 is ignored for the vendored files to keep them upstream-identical. - Taxonomy: BaseModelType.MiniMaxH3, MiniMaxH3VariantType.FL2VA, SubModelType.AudioVAE + SubModelType.Processor (the H3 layout needs both; the Qwen3VLProcessor is used even for text-only encoding). - Probe: Main_Diffusers_MiniMaxH3_Config keys on the Modular Diffusers root config (modular_model_index.json, class MiniMaxH3ModularPipeline) plus the audio_vae config as a partial-download guard. The repo's FL2VA/-Ref2VA/ remote-code subtrees declare MiniMaxH3Pipeline and are deliberately not matched. Ref2VA-only installs fail identification rather than mislabeling. modular_model_index.json is now a recognized root config marker in the factory scan (H3 installs have hundreds of shard files and would otherwise be rejected as non-models). - Loader: explicit submodel dispatch (vendored classes bf16; Qwen3-VL-32B via Qwen3VLForConditionalGeneration with the shared rope-config fix, lifted from krea2.py into model_manager/util/qwen3_vl.py; audio VAE kept fp32 - half-precision audio artifacts are audible). - Default UI settings: steps=50, cfg 1.0 (checkpoints are guidance-distilled - no CFG), native 1344x768 canvas. - Identification fixture built from the real HF repo config files (stripped stub shards); tests/model_identification passes 88/88. Part of the MiniMax H3 stack (PR 2 of 5); depends on the audio-mux PR. Post-review additions: - ModelRecordChanges.variant accepts fl2va (editing an installed H3 model otherwise 422s on the PATCH route). - Regenerated openapi.json + schema.ts, with the matching type-level frontend registration (zBaseModelType/zMainModelBase/zSubModelType/ zAnyModelVariant entries and the exhaustive MODEL_BASE_TO_* records) so typegen-checks and tsc stay green. UI behavior lands in the linear-UI PR. - _is_known_model_marker guards non-str _class_name (a list-valued _class_name in modular_model_index.json raised TypeError instead of reading as not-a-marker). Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Lincoln Stein
|
b42274a57e | Feat[model support]: Qwen Image — full pipeline with edit, generate LoRA, GGUF, quantization, and UI (#9000) | ||
|
|
dfc66b7142 |
Feature: Add FLUX.2 LOKR model support (detection and loading) (#8909)
* Add FLUX.2 LOKR model support (detection and loading) (#88) Fix BFL LOKR models being misidentified as AIToolkit format Fix alpha key warning in LOKR QKV split layers Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: lstein <[email protected]> * Fix BFL→diffusers key mapping for non-block layers in FLUX.2 LoRA/LoKR BFL's FLUX.2 model uses different names than diffusers' Flux2Transformer2DModel for top-level modules (embedders, modulations, output layers). The existing conversion only handled block-level renames (double_blocks→transformer_blocks), causing "Failed to find module" warnings for non-block LoRA keys like img_in, txt_in, modulation.lin, time_in, and final_layer. --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: lstein <[email protected]> Co-authored-by: Alexander Eichhorn <[email protected]> |
||
|
psychedelicious
|
3126726abf | tidy: removing unused code paths 4 | ||
|
psychedelicious
|
454d05bbde |
refactor: model manager v3 (#8607)
* feat(mm): add UnknownModelConfig * refactor(ui): move model categorisation-ish logic to central location, simplify model manager models list * refactor(ui)refactor(ui): more cleanup of model categories * refactor(ui): remove unused excludeSubmodels I can't remember what this was for and don't see any reference to it. Maybe it's just remnants from a previous implementation? * feat(nodes): add unknown as model base * chore(ui): typegen * feat(ui): add unknown model base support in ui * feat(ui): allow changing model type in MM, fix up base and variant selects * feat(mm): omit model description instead of making it "base type filename model" * feat(app): add setting to allow unknown models * feat(ui): allow changing model format in MM * feat(app): add the installed model config to install complete events * chore(ui): typegen * feat(ui): toast warning when installed model is unidentified * docs: update config docstrings * chore(ui): typegen * tests(mm): fix test for MM, leave the UnknownModelConfig class in the list of configs * tidy(ui): prefer types from zod schemas for model attrs * chore(ui): lint * fix(ui): wrong translation string * feat(mm): normalized model storage Store models in a flat directory structure. Each model is in a dir named its unique key (a UUID). Inside that dir is either the model file or the model dir. * feat(mm): add migration to flat model storage * fix(mm): normalized multi-file/diffusers model installation no worky now worky * refactor: port MM probes to new api - Add concept of match certainty to new probe - Port CLIP Embed models to new API - Fiddle with stuff * feat(mm): port TIs to new API * tidy(mm): remove unused probes * feat(mm): port spandrel to new API * fix(mm): parsing for spandrel * fix(mm): loader for clip embed * fix(mm): tis use existing weight_files method * feat(mm): port vae to new API * fix(mm): vae class inheritance and config_path * tidy(mm): patcher types and import paths * feat(mm): better errors when invalid model config found in db * feat(mm): port t5 to new API * feat(mm): make config_path optional * refactor(mm): simplify model classification process Previously, we had a multi-phase strategy to identify models from their files on disk: 1. Run each model config classes' `matches()` method on the files. It checks if the model could possibly be an identified as the candidate model type. This was intended to be a quick check. Break on the first match. 2. If we have a match, run the config class's `parse()` method. It derive some additional model config attrs from the model files. This was intended to encapsulate heavier operations that may require loading the model into memory. 3. Derive the common model config attrs, like name, description, calculate the hash, etc. Some of these are also heavier operations. This strategy has some issues: - It is not clear how the pieces fit together. There is some back-and-forth between different methods and the config base class. It is hard to trace the flow of logic until you fully wrap your head around the system and therefore difficult to add a model architecture to the probe. - The assumption that we could do quick, lightweight checks before heavier checks is incorrect. We often _must_ load the model state dict in the `matches()` method. So there is no practical perf benefit to splitting up the responsibility of `matches()` and `parse()`. - Sometimes we need to do the same checks in `matches()` and `parse()`. In these cases, splitting the logic is has a negative perf impact because we are doing the same work twice. - As we introduce the concept of an "unknown" model config (i.e. a model that we cannot identify, but still record in the db; see #8582), we will _always_ run _all_ the checks for every model. Therefore we need not try to defer heavier checks or resource-intensive ops like hashing. We are going to do them anyways. - There are situations where a model may match multiple configs. One known case are SD pipeline models with merged LoRAs. In the old probe API, we relied on the implicit order of checks to know that if a model matched for pipeline _and_ LoRA, we prefer the pipeline match. But, in the new API, we do not have this implicit ordering of checks. To resolve this in a resilient way, we need to get all matches up front, then use tie-breaker logic to figure out which should win (or add "differential diagnosis" logic to the matchers). - Field overrides weren't handled well by this strategy. They were only applied at the very end, if a model matched successfully. This means we cannot tell the system "Hey, this model is type X with base Y. Trust me bro.". We cannot override the match logic. As we move towards letting users correct mis-identified models (see #8582), this is a requirement. We can simplify the process significantly and better support "unknown" models. Firstly, model config classes now have a single `from_model_on_disk()` method that attempts to construct an instance of the class from the model files. This replaces the `matches()` and `parse()` methods. If we fail to create the config instance, a special exception is raised that indicates why we think the files cannot be identified as the given model config class. Next, the flow for model identification is a bit simpler: - Derive all the common fields up-front (name, desc, hash, etc). - Merge in overrides. - Call `from_model_on_disk()` for every config class, passing in the fields. Overrides are handled in this method. - Record the results for each config class and choose the best one. The identification logic is a bit more verbose, with the special exceptions and handling of overrides, but it is very clear what is happening. The one downside I can think of for this strategy is we do need to check every model type, instead of stopping at the first match. It's a bit less efficient. In practice, however, this isn't a hot code path, and the improved clarity is worth far more than perf optimizations that the end user will likely never notice. * refactor(mm): remove unused methods in config.py * refactor(mm): add model config parsing utils * fix(mm): abstractmethod bork * tidy(mm): clarify that model id utils are private * fix(mm): fall back to UnknownModelConfig correctly * feat(mm): port CLIPVisionDiffusersConfig to new api * feat(mm): port SigLIPDiffusersConfig to new api * feat(mm): make match helpers more succint * feat(mm): port flux redux to new api * feat(mm): port ip adapter to new api * tidy(mm): skip optimistic override handling for now * refactor(mm): continue iterating on config * feat(mm): port flux "control lora" and t2i adapter to new api * tidy(ui): use Extract to get model config types * fix(mm): t2i base determination * feat(mm): port cnet to new api * refactor(mm): add config validation utils, make it all consistent and clean * feat(mm): wip port of main models to new api * feat(mm): wip port of main models to new api * feat(mm): wip port of main models to new api * docs(mm): add todos * tidy(mm): removed unused model merge class * feat(mm): wip port main models to new api * tidy(mm): clean up model heuristic utils * tidy(mm): clean up ModelOnDisk caching * tidy(mm): flux lora format util * refactor(mm): make config classes narrow Simpler logic to identify, less complexity to add new model, fewer useless attrs that do not relate to the model arch, etc * refactor(mm): diffusers loras w * feat(mm): consistent naming for all model config classes * fix(mm): tag generation & scattered probe fixes * tidy(mm): consistent class names * refactor(mm): split configs into separate files * docs(mm): add comments for identification utils * chore(ui): typegen * refactor(mm): remove legacy probe, new configs dir structure, update imports * fix(mm): inverted condition * docs(mm): update docsstrings in factory.py * docs(mm): document flux variant attr * feat(mm): add helper method for legacy configs * feat(mm): satisfy type checker in flux denoise * docs(mm): remove extraneous comment * fix(mm): ensure unknown model configs get unknown attrs * fix(mm): t5 identification * fix(mm): sdxl ip adapter identification * feat(mm): more flexible config matching utils * fix(mm): clip vision identification * feat(mm): add sanity checks before probing paths * docs(mm): add reminder for self for field migrations * feat(mm): clearer naming for main config class hierarchy * feat(mm): fix clip vision starter model bases, add ref to actual models * feat(mm): add model config schema migration logic * fix(mm): duplicate import * refactor(mm): split big migration into 3 Split the big migration that did all of these things into 3: - Migration 22: Remove unique contraint on base/name/type in models table - Migration 23: Migrate configs to v6.8.0 schemas - Migration 24: Normalize file storage * fix(mm): pop base/type/format when creating unknown model config * fix(db): migration 22 insert only real cols * fix(db): migration 23 fall back to unknown model when config change fails * feat(db): run migrations 23 and 24 * fix(mm): false negative on flux lora * fix(mm): vae checkpoint probe checking for dir instead of file * fix(mm): ModelOnDisk skips dirs when looking for weights Previously a path w/ any of the known weights suffixes would be seen as a weights file, even if it was a directory. We now check to ensure the candidate path is actually a file before adding it to the list of weights. * feat(mm): add method to get main model defaults from a base * feat(mm): do not log when multiple non-unknown model matches * refactor(mm): continued iteration on model identifcation * tests(mm): refactor model identification tests Overhaul of model identification (probing) tests. Previously we didn't test the correctness of probing except in a few narrow cases - now we do. See tests/model_identification/README.md for a detailed overview of the new test setup. It includes instructions for adding a new test case. In brief: - Download the model you want to add as a test case - Run a script against it to generate the test model files - Fill in the expected model type/format/base/etc in the generated test metadata JSON file Included test cases: - All starter models - A handful of other models that I had installed - Models present in the previous test cases as smoke tests, now also tested for correctness * fix(mm): omit type/format/base when creating unknown config instance * feat(mm): use ValueError for model id sanity checks * feat(mm): add flag for updating models to allow class changes * tests(mm): fix remaining MM tests * feat: allow users to edit models freely * feat(ui): add warning for model settings edit * tests(mm): flux state dict tests * tidy: remove unused file * fix(mm): lora state dict loading in model id * feat(ui): use translation string for model edit warning * docs(db): update version numbers in migration comments * chore: bump version to v6.9.0a1 * docs: update model id readme * tests(mm): attempt to fix windows model id tests * fix(mm): issue with deleting single file models * feat(mm): just delete the dir w/ rmtree when deleting model * tests(mm): windows CI issue * fix(ui): typegen schema sync * fix(mm): fixes for migration 23 - Handle CLIP Embed and Main SD models missing variant field - Handle errors when calling the discriminator function, previously only handled ValidationError but it could be a ValueError or something else - Better logging for config migration * chore: bump version to v6.9.0a2 * chore: bump version to v6.9.0a3 |