vladmandic/sdnext
1
0
Fork 0
mirror of https://github.com/vladmandic/sdnext.git synced 2026-10-12 02:24:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'dev' into feat/filter-sampler-upscaler-choices

Browse Source
This commit is contained in:
Vladimir Mandic
2026-09-13 07:41:53 +02:00
committed by GitHub
parent bae986e90b e6d5efa97f
commit da5d934f1f
99 changed files with 2858 additions and 394 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+58 -23
View File
@@ -1,18 +1,31 @@
# Change Log for SD.Next
## Update for 2026-09-07
## Update for 2026-09-12
### Highlights for 2026-09-07
### Highlights for 2026-09-12
All-about-optimizations:
- improved LoRA performance and quality, especially with quantized models
- newly structured attention mechanisms
- modular pipelines with new guidance methods
- support for different caching stacks
*What's New*? Well, code-wise, this is a big one...
First, a-lot-of-optimizations:
- updated core packages
- improved **LoRA** performance and quality, especially with quantized models
- newly structured **attention** mechanisms
- modular pipelines with new **guidance** methods
- support for different **caching** stacks
- compute updates across the board
- enhanced cloud model support
### Details for 2026-09-07
And some cool new stuff and models:
- **DLSSv5** integration
- New models:
**Anima 2.9B**, **LLaDa-Image**
- And few cloud models:
*Google's Gemini, NanoBanana, Veo, Omni* and *X.AI's Grok*
- Some (light) UI restyling
Plus inevitable bug-fixes...
[Home](https://vladmandic.github.io/sdnext/) | [ChangeLog](https://github.com/vladmandic/automatic/blob/master/CHANGELOG.md) | [Docs](https://vladmandic.github.io/sdnext-docs/) | [Discord](https://discord.com/invite/sd-next-federal-batch-inspectors-1101998836328697867) | [Sponsor](https://github.com/sponsors/vladmandic)
### Details for 2026-09-12
- **Models**
- [Anima 2.9B Preview v1](https://huggingface.co/yeoj34760/Anima-2.9B)
@@ -21,6 +34,7 @@ All-about-optimizations:
LLaDA-Image is a 6.5B transformer with massive 16.3B fully-custom MoE text-encoder and optional 1.3B SigVQ conditioning model
with support for text-to-image, vq-conditioned text-to-image and image-editing workflows
*note* model is extremely quantization sensitive so minimum allowed quant type is `uint8`
- [MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) update with pre-quantized `sdnq-uint8` *pruned* variants
- **LoRA**
- see [LoRA docs](https://vladmandic.github.io/sdnext-docs/LoRA) for all of the improvements and usage instructions
*note*: lora now has its own settings section in *settings -> lora*
@@ -34,6 +48,16 @@ All-about-optimizations:
can significantly improve lora quality when using multiple loras at once
- per-block strength
- native support for **MiniMax**
- **DLSS**
- add DLSS support for: *NeuralRender, SuperSample and FrameGen*
dlls 5 caused quite a stir, but combined with generative ai it becomes a nice tool
- available as part of image/video generate workflows via *extras -> dlss*
or as a standalone *processing* workflow
or via xyz grid
- *note*: requires nvidia rtx gpu, windows platform and compatible gpu drivers
but...it can be used from wsl2: unpack required package on windows host and you can access it from the wsl2 environment
- *install*: requires [DLSS 5 Visual Enhancer](https://github.com/Merserk/dlss5-visual-enhancer/releases/tag/v7.0)
- *diag*: enable `SD_DLSS_DEBUG=true` and monitor `dlss.log` in the package directory
- **Attention**
- see [Attention docs](https://vladmandic.github.io/sdnext-docs/Attention) for details and usage instructions
*note*: attention now has its own settings section in *settings -> cross attention*
@@ -73,6 +97,7 @@ All-about-optimizations:
- option to skip triton autotune and use default config for all triton kernels
in *settings -> compute settings*
*note*: this may improve initial generate time, but may also reduce performance on some models
- rocm: update `rocm` script and add detailed `miopen` logging, thanks @resonantsky
- new optional transformer hooks
in *settings -> compute add-ons*
*PAG: Perturbed attention guidance, PAB: Pyramid attention broadcast, FBC: First Block Cache, FC: Faster Cache, LS: Layer Skip, MC: Mag Cache, TS: TaylorSeer*
@@ -89,24 +114,34 @@ All-about-optimizations:
- new articles: *Attention, Modular-Pipelines*
- updated: *LoRA, MiniMax*
- **Fixes**
- prompt: unnecessary secondary prompt if same
- ui: js fetch exception handling
- detailer: handling of stop/skip/pause
- rife: cleanup dead code, thanks @Anai-Guo
- lumina-dimoo: attention-kwargs, thanks @Anai-Guo
- network: improve type/version lookup
- lora: cleanup tags
- xyz grid: apply bool values
- vdm scheduler: fix steps, thanks @zjn20030811
- openvino: optimize recompile checks and lora loading
- log: ansi color handling
- api: prompt enhance with vision
- autocomplete: skip disabled networks
- compile: keep model compiled state
- prompt: cache checks when cfg changes
- detailer: handling of stop/skip/pause
- framepack: correct device assignment, thanks @li-lizhe
- log: ansi color handling
- lora: cleanup tags
- lora: support transformer ref models
- lucida: handle requirements
- vae: fetch scale factor from the model
- todo: remove dead code, thanks @Anai-Guo
- lumina-dimoo: attention-kwargs, thanks @Anai-Guo
- metadata: fix wildcard info
- minimax: crop image to video aspect ratio
- modular: handle module with remote-code
- network: improve type/version lookup
- offline: honor offline mode for more models, thanks @ryanmeador
- openvino: optimize recompile checks and lora loading
- prompt enhance: cloud models use correct system prompt
- prompt enhance: use init image for video
- prompt: cache checks when cfg changes
- prompt: unnecessary secondary prompt if same
- prompt: clean prompt after network parsing
- rife: cleanup dead code, thanks @Anai-Guo
- todo: remove dead code, thanks @Anai-Guo
- ui: js fetch exception handling
- update: handle git errors gracefully
- vae: fetch scale factor from the model
- vdm scheduler: fix steps, thanks @zjn20030811
- xyz grid: apply bool values
## Update for 2026-08-26
TODO.md
+4 -1
View File
@@ -29,14 +29,17 @@
### Roadmap
- Automated testing and integrate models repo
- [MiniMax VDN](https://huggingface.co/OpenVDN/vdn-minimax-h3)
- [MiniMax VSA](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree)
- [MiniMax Nunchaku-Lite](https://huggingface.co/rootonchair/MiniMax-H3-nunchaku-lite-int4)
- Video upscaling: LTX-Upscaler
- Video upscaling: [MiniMax-Upscaler](https://huggingface.co/LBH-123-AI/Minimax_h3_latent_Upscaler)
- Video capabilities to processing tab, add RIFE, upscaling (once available)
- Distraction-free UI mode with prompt-only, chat-based interface
- Video models: support finetunes
- Incorporate [prompting guides](https://github.com/CalamitousFelicitousness/ai-prompting-guides)
- Video models: use Networks/Reference instead of custom
- UI Lite vs Expert mode
- Expand custom VAE support
- Remove obsolete code: `olive-ai`
### OnHold
cli/api-enhance.py
+2 -1
View File
@@ -59,6 +59,7 @@ def enhance(args): # pylint: disable=redefined-outer-name
options['model'] = str(args.model)
if args.image:
options['image'] = encode(args.image)
options['use_vision'] = True
response = post('/sdapi/v1/prompt-enhance', options)
return response
@@ -72,6 +73,6 @@ if __name__ == "__main__":
parser.add_argument('--image', type=str, default=None, required=False, help='optional input image')
parser.add_argument('--nsfw', type=bool, action=argparse.BooleanOptionalAction, required=False, help='nsfw allowed')
args = parser.parse_args()
log.info(f'api-upscale: {args}')
log.info(f'api-enhance: {args}')
result = enhance(args)
log.info(result)
data/reference-base.json
+1
View File
@@ -986,6 +986,7 @@
},
"inclusionAI LLaDA-Image": {
"path": "inclusionAI/LLaDA-Image",
"preview": "inclusionAI--LLaDA-Image.jpg",
"desc": "LLaDA-Image is a unified 6B image generation and editing model with text, VQ-conditioned, and reference-image workflows.",
"extras": "steps: 50, cfg_scale: 5.0",
"size": 37.15,
data/reference-distilled.json
+1
View File
@@ -253,6 +253,7 @@
},
"inclusionAI LLaDA-Image Turbo": {
"path": "inclusionAI/LLaDA-Image-Turbo",
"preview": "inclusionAI--LLaDA-Image-Turbo.jpg",
"desc": "LLaDA-Image-Turbo is the distilled fast-generation and editing variant of LLaDA-Image.",
"extras": "steps: 4, cfg_scale: 1.0",
"size": 37.15,
extensions-builtin/sdnext-kanvas
+1 -1
Submodule extensions-builtin/sdnext-kanvas updated: b5fb2c580f...d814462675
extensions-builtin/sdnext-modernui
+1 -1
Submodule extensions-builtin/sdnext-modernui updated: 8dec874118...e95c40585f
installer.py
+6 -4
View File
@@ -409,7 +409,7 @@ def branch(folder=None):
b = 'master'
else:
b = b.split('\n')[0].replace('*', '').strip()
log.debug(f'Git submodule: {folder} / {b}')
log.debug(f'Submodule: folder="{folder}" branch="{b}"')
git(f'checkout {b}', folder, ignore=True, optional=True)
ts('branch', t_start)
return b
@@ -1140,7 +1140,7 @@ def list_extensions_folder(folder, quiet=False):
disabled_extensions = opts.get('disabled_extensions', [])
enabled_extensions = [x for x in os.listdir(folder) if os.path.isdir(os.path.join(folder, x)) and x not in disabled_extensions and not x.startswith('.')]
if not quiet:
log.info(f'Extensions: path="{folder}" enabled={enabled_extensions}')
log.info(f'Extensions: path="{folder}" available={enabled_extensions}')
return enabled_extensions
@@ -1363,7 +1363,6 @@ def install_requirements():
log.info('Install requirements: this may take a while...')
pip('install -r requirements.txt')
if args.optional:
quick_allowed = False
install_optional()
log.info('Install: verifying requirements')
if args.new:
@@ -1386,7 +1385,10 @@ def install_requirements():
# set environment variables controlling the behavior of various libraries
def set_environment():
log.debug('Setting environment tuning')
from modules.logger import console
log.debug(f'Console: terminal={console.is_terminal} width={console.width} height={console.height} color={console.color_system} legacy={console.legacy_windows}')
os.environ.setdefault('ACCELERATE', 'True')
os.environ.setdefault('ATTN_PRECISION', 'fp16')
os.environ.setdefault('ClDeviceGlobalMemSizeAvailablePercent', '100')
@@ -1704,7 +1706,7 @@ def check_version(reset=True): # pylint: disable=unused-argument
else:
dt = commits["commit"]["commit"]["author"]["date"]
commit = commits["commit"]["sha"][:8]
log.info(f'Version: app=sd.next latest={dt} hash={commit} branch={branch_name}')
log.info(f'Version: app="sd.next" latest={dt} hash={commit} branch={branch_name}')
except Exception as e:
log.error(f'Repository failed to check version: {e} {commits}')
ts('latest', t_start)
launch.py
+1 -1
View File
@@ -238,7 +238,7 @@ def main():
init_args() # setup argparser and default folders
installer.args = args
installer.setup_logging(debug=args.debug, trace=args.trace, filename=args.log)
log.info('Starting SD.Next')
log.info('Starting: [bold cyan]SD.Next[/]')
installer.get_logfile()
try:
sys.excepthook = installer.custom_excepthook
models/Reference/inclusionAI--LLaDA-Image-Turbo.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 52 KiB

models/Reference/inclusionAI--LLaDA-Image.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 41 KiB

modules/api/helpers.py
+1
View File
@@ -46,6 +46,7 @@ def decode_base64_to_image(encoding, quiet=False):
decoded = base64.b64decode(encoding)
data = io.BytesIO(decoded)
image = Image.open(data)
image = image.convert('RGB')
return image
except Exception as e:
log.warning(f'API cannot decode image: {e}')
modules/api/models.py
+4 -4
View File
@@ -366,11 +366,11 @@ class ReqPromptEnhance(BaseModel):
repetition_penalty: Optional[float] = Field(title="Repetition penalty", default=None, description="Penalizes repeated tokens to reduce repetition (1.0=no penalty)")
top_k: Optional[int] = Field(title="Top K", default=None, description="Limits token selection to the K most likely candidates")
top_p: Optional[float] = Field(title="Top P", default=None, description="Nucleus sampling threshold (0-1)")
thinking: bool = Field(title="Thinking", default=False, description="Enable thinking/reasoning mode")
keep_thinking: bool = Field(title="Keep thinking", default=False, description="Keep thinking tokens in output")
use_vision: bool = Field(title="Use vision", default=True, description="Use vision if model supports it")
thinking: Optional[bool] = Field(title="Thinking", default=False, description="Enable thinking/reasoning mode")
keep_thinking: Optional[bool] = Field(title="Keep thinking", default=False, description="Keep thinking tokens in output")
use_vision: Optional[bool] = Field(title="Use vision", default=True, description="Use vision if model supports it")
prefill: Optional[str] = Field(title="Prefill", default=None, description="Text to prefill the model response with")
keep_prefill: bool = Field(title="Keep prefill", default=False, description="Keep prefill text in the output")
keep_prefill: Optional[bool] = Field(title="Keep prefill", default=False, description="Keep prefill text in the output")
custom_args: Optional[str] = Field(title="Custom args", default=None, description="Custom arguments for the model")
process_words: Optional[str] = Field(title="Banned words", default=None, description="List of words to process")
semantic_threshold: Optional[float] = Field(title="Semantic threshold", default=None, description="Semantic similarity threshold for processed words")
modules/api/process.py
+2 -1
View File
@@ -226,6 +226,7 @@ class APIProcess:
if len(instance) == 0:
raise HTTPException(status_code=500, detail="Prompt enhancement script not found")
instance = instance[0]
decoded = decode_base64_to_image(req.image) if req.image else None
prompt = instance.enhance(
model=model,
prompt=req.prompt,
@@ -244,7 +245,7 @@ class APIProcess:
use_vision=req.use_vision,
prefill=req.prefill or '',
keep_prefill=req.keep_prefill,
image=decode_base64_to_image(req.image) if req.image else None,
image=decoded,
seed=seed,
nsfw=req.nsfw,
custom_args=req.custom_args,
modules/api/validate.py
+20 -18
View File
@@ -7,35 +7,37 @@ request_cost = {
"/file": 0,
"/internal/progress": 0,
"/run/predict": 0,
"/sdapi/v1/control": 5,
"/sdapi/v1/img2img": 5,
"/sdapi/v1/txt2img": 5,
"/sdapi/v1/video": 5,
"/sdapi/v1/browser/thumb": 0,
"/sdapi/v1/network/thumb": 0,
"/sdapi/v1/txt2img": 5,
"/sdapi/v1/img2img": 5,
"/sdapi/v1/control": 5,
"/sdapi/v1/video": 5,
}
log_cost = {
"/.well-known/appspecific/com.chrome.devtools.json": -1,
"/info": -1,
"/file": -1,
"/token": -1,
"/theme.css": -1,
"/sdapi/v1/browser/thumb": -1,
"/sdapi/v1/network/thumb": -1,
"/run/predict": -1,
"/queue/join": -1,
"/info": -1,
"/icon": -1,
"/internal/progress": -1,
"/sdapi/v1/version": -1,
"/sdapi/v1/log": -1,
"/sdapi/v1/torch": -1,
"/queue/join": -1,
"/run/predict": -1,
"/theme.css": -1,
"/token": -1,
"/sdapi/v1/checkpoint": -1,
"/sdapi/v1/gpu-smi": -1,
"/sdapi/v1/gpu": -1,
"/sdapi/v1/loaded-loras": -1,
"/sdapi/v1/log": -1,
"/sdapi/v1/memory": -1,
"/sdapi/v1/platform": -1,
"/sdapi/v1/checkpoint": -1,
"/sdapi/v1/loaded-loras": -1,
"/sdapi/v1/gpu-smi": -1,
"/sdapi/v1/status": 60,
"/sdapi/v1/progress": 60,
"/sdapi/v1/start": -1,
"/sdapi/v1/status": 60,
"/sdapi/v1/torch": -1,
"/sdapi/v1/version": -1,
"/sdapi/v1/browser/thumb": -1,
"/sdapi/v1/network/thumb": -1,
}
log_exclude_suffix = ['.css', '.js', '.ico', '.svg']
log_exclude_prefix = ['/assets']
modules/extensions.py
+3 -1
View File
@@ -271,4 +271,6 @@ def list_extensions():
enabled = dirname.lower() not in disabled_extensions
extension = Extension(name=dirname, path=path, enabled=enabled, is_builtin=is_builtin)
extensions.append(extension)
log.debug(f'Extensions: disabled={[e.name for e in extensions if not e.enabled]}')
enabled = [e.name for e in extensions if e.enabled]
disabled = [e.name for e in extensions if not e.enabled]
log.debug(f'Extensions: enabled={enabled} disabled={disabled}')
modules/extra_networks.py
+1
View File
@@ -164,6 +164,7 @@ def parse_prompt(prompt: str | None) -> tuple[str, defaultdict[str, list[ExtraNe
return ""
updated_prompt = re.sub(re_extra_net, found, prompt)
updated_prompt = updated_prompt.strip(', ')
return updated_prompt, res
modules/framepack/pipeline/hunyuan_video_packed.py
+1 -1
View File
@@ -74,7 +74,7 @@ def get_cu_seqlens(text_mask, img_len):
text_len = text_mask.sum(dim=1)
max_len = text_mask.shape[1] + img_len
cu_seqlens = torch.zeros([2 * batch_size + 1], dtype=torch.int32, device="cuda")
cu_seqlens = torch.zeros([2 * batch_size + 1], dtype=torch.int32, device=text_mask.device)
for i in range(batch_size):
s = text_len[i] + img_len
modules/gr_hijack.py
+3 -3
View File
@@ -208,13 +208,13 @@ def patch_gradio():
return {"is_generating": False, "data": [], "error": "empty response"}
return response
except GeneratorExit as e:
log.error(f"Gradio queue: events={len(events)} batch={batch} error: {e}")
log.error(f"Gradio queue: events={len(events)} batch={batch} reason=GeneratorExit {e}")
return {"is_generating": False, "data": [None, None, None, None, "cancelled", ""], "error": None}
except Exception as e:
log.error(f"Gradio queue: events={len(events)} batch={batch} error: {e}")
log.error(f"Gradio queue: events={len(events)} batch={batch} reason=Exception {e}")
raise
except BaseException as e:
log.error(f"Gradio queue: events={len(events)} batch={batch} error: {e}")
log.error(f"Gradio queue: events={len(events)} batch={batch} reason=BaseException {e}")
raise
def wrap_blocks_preprocess_data(self, fn_index: int, inputs: list, state: dict):
modules/image/grid.py
+1
View File
@@ -80,6 +80,7 @@ def image_grid(imgs: list, batch_size=1, rows: int | None = None, cols: int | No
for i, img in enumerate(params.imgs):
if img is not None:
grid.paste(img, box=(i % params.cols * w, i // params.cols * h))
grid.is_grid = True # flag image as grid
return grid
except Exception as e:
log.error(f'Grid: images={imgs} {e}')
modules/image/sharpfin.py
+13 -11
View File
@@ -7,6 +7,7 @@ and Triton GPU acceleration when available.
Non-CUDA devices fall back to PIL/torch.nn.functional automatically.
"""
import os
import sys
import torch
from PIL import Image
@@ -17,6 +18,7 @@ from modules.image.convert import to_tensor, to_pil
_sharpfin_checked = False
_sharpfin_ok = False
_triton_ok = False
debug = log.trace if os.environ.get('SD_PROCESS_DEBUG', None) is not None else lambda *args, **kwargs: None
def check_sharpfin():
@@ -104,7 +106,7 @@ def _scale_pil(scale_fn, tensor, out_res, rk, dev, dt, do_linear, src_h, src_w,
return scale_fn(tensor, out_res, resize_kernel=rk, device=dev, dtype=dt, do_srgb_conversion=do_linear, use_sparse=True)
except Exception:
_triton_ok = False
log.info("Sharpfin: Triton sparse disabled, using dense path")
log.debug("Sharpfin: Triton sparse disabled, using dense path")
return scale_fn(tensor, out_res, resize_kernel=rk, device=dev, dtype=dt, do_srgb_conversion=do_linear, use_sparse=False)
# Mixed axis: split into two single-axis resizes
if h > src_h: # H up, W down
@@ -115,7 +117,7 @@ def _scale_pil(scale_fn, tensor, out_res, rk, dev, dt, do_linear, src_h, src_w,
return scale_fn(intermediate, (h, w), resize_kernel=rk, device=dev, dtype=dt, do_srgb_conversion=do_linear, use_sparse=True)
except Exception:
_triton_ok = False
log.info("Sharpfin: Triton sparse disabled, using dense path")
log.debug("Sharpfin: Triton sparse disabled, using dense path")
return scale_fn(intermediate, (h, w), resize_kernel=rk, device=dev, dtype=dt, do_srgb_conversion=do_linear, use_sparse=False)
# H down, W up
use_sparse = _want_sparse(dev, rk, True)
@@ -125,7 +127,7 @@ def _scale_pil(scale_fn, tensor, out_res, rk, dev, dt, do_linear, src_h, src_w,
return scale_fn(intermediate, (h, w), resize_kernel=rk, device=dev, dtype=dt, do_srgb_conversion=do_linear, use_sparse=False)
except Exception:
_triton_ok = False
log.info("Sharpfin: Triton sparse disabled, using dense path")
log.debug("Sharpfin: Triton sparse disabled, using dense path")
intermediate = scale_fn(tensor, (h, src_w), resize_kernel=rk, device=dev, dtype=dt, do_srgb_conversion=do_linear, use_sparse=False)
return scale_fn(intermediate, (h, w), resize_kernel=rk, device=dev, dtype=dt, do_srgb_conversion=do_linear, use_sparse=False)
@@ -137,24 +139,24 @@ def resize_pil(image: Image.Image, target_size: tuple[int, int], *, kernel=None,
is_mask = image.mode == 'L'
if (image.width == w) and (image.height == h):
log.debug(f'Resize image: skip={w}x{h} fn={fn}')
# log.debug(f'Resize image: skip={w}x{h} fn={fn}')
return image
from modules import devices
dev = device if device is not None else devices.device
if not allow_sharpfin(dev):
log.debug(f'Resize image: method=PIL source={image.width}x{image.height} target={w}x{h} device={dev} fn={fn}')
debug(f'Resize image: method=PIL source={image.width}x{image.height} target={w}x{h} device={dev} fn={fn}')
return image.resize((w, h), resample=Image.Resampling.LANCZOS)
rk = get_kernel(kernel)
if rk is None:
log.debug(f'Resize image: method=PIL source={image.width}x{image.height} target={w}x{h} kernel=None fn={fn}')
debug(f'Resize image: method=PIL source={image.width}x{image.height} target={w}x{h} kernel=None fn={fn}')
return image.resize((w, h), resample=Image.Resampling.LANCZOS)
from modules.sharpfin.functional import scale
dt = dtype or torch.float16
do_linear = get_linearize(linearize, is_mask=is_mask)
log.debug(f'Resize image: method=sharpfin source={image.width}x{image.height} target={w}x{h} kernel={rk} device={dev} linearize={do_linear} fn={fn}')
debug(f'Resize image: method=sharpfin source={image.width}x{image.height} target={w}x{h} kernel={rk} device={dev} linearize={do_linear} fn={fn}')
tensor = to_tensor(image)
if tensor.dim() == 3:
tensor = tensor.unsqueeze(0)
@@ -182,14 +184,14 @@ def resize_tensor(tensor: torch.Tensor, target_size: tuple[int, int], *, kernel=
dev = devices.device
if not allow_sharpfin(dev):
mode = 'bilinear' if (target_size[0] * target_size[1]) > (tensor.shape[-2] * tensor.shape[-1]) else 'area'
log.debug(f'Resize tensor: method=torch mode={mode} shape={tensor.shape} target={target_size} fn={fn}')
debug(f'Resize tensor: method=torch mode={mode} shape={tensor.shape} target={target_size} fn={fn}')
inp = tensor if tensor.dim() == 4 else tensor.unsqueeze(0)
result = torch.nn.functional.interpolate(inp, size=target_size, mode=mode, antialias=mode != 'area')
return result.squeeze(0) if tensor.dim() == 3 else result
rk = get_kernel(kernel)
if rk is None:
mode = 'bilinear' if (target_size[0] * target_size[1]) > (tensor.shape[-2] * tensor.shape[-1]) else 'area'
log.debug(f'Resize tensor: method=torch mode={mode} shape={tensor.shape} target={target_size} kernel=None fn={fn}')
debug(f'Resize tensor: method=torch mode={mode} shape={tensor.shape} target={target_size} kernel=None fn={fn}')
inp = tensor if tensor.dim() == 4 else tensor.unsqueeze(0)
result = torch.nn.functional.interpolate(inp, size=target_size, mode=mode, antialias=mode != 'area')
return result.squeeze(0) if tensor.dim() == 3 else result
@@ -206,10 +208,10 @@ def resize_tensor(tensor: torch.Tensor, target_size: tuple[int, int], *, kernel=
both_up = (th >= src_h and tw >= src_w)
if both_down or both_up:
use_sparse = _triton_ok and dev.type == 'cuda' and rk.value == 'magic_kernel_sharp_2021' and both_down
log.debug(f'Resize tensor: method=sharpfin shape={tensor.shape} target={target_size} direction={both_up}:{both_down} kernel={rk} sparse={use_sparse} fn={fn}')
debug(f'Resize tensor: method=sharpfin shape={tensor.shape} target={target_size} direction={both_up}:{both_down} kernel={rk} sparse={use_sparse} fn={fn}')
result = scale(tensor, target_size, resize_kernel=rk, device=dev, dtype=dt, do_srgb_conversion=linearize, use_sparse=use_sparse)
else:
log.debug(f'Resize tensor: method=sharpfin shape={tensor.shape} target={target_size} direction={both_up}:{both_down} kernel={rk} sparse=False fn={fn}')
debug(f'Resize tensor: method=sharpfin shape={tensor.shape} target={target_size} direction={both_up}:{both_down} kernel={rk} sparse=False fn={fn}')
intermediate = scale(tensor, (th, src_w), resize_kernel=rk, device=dev, dtype=dt, do_srgb_conversion=linearize, use_sparse=False)
result = scale(intermediate, (th, tw), resize_kernel=rk, device=dev, dtype=dt, do_srgb_conversion=linearize, use_sparse=False)
if squeezed:
modules/logger.py
+29 -21
View File
@@ -182,21 +182,14 @@ def setup_logging(debug=None, trace=None, filename=None):
render_options = render_options.update_height(height=render_options.height - self.top - self.bottom)
lines = console.render_lines(self.renderable, render_options, style=style, pad=False)
_Segment = Segment
left = _Segment(" " * self.left, style) if self.left else None
right = [_Segment.line()]
blank_line: list[Segment] | None = None
if self.top:
blank_line = [_Segment(f'{" " * width}\\n', style)]
yield from blank_line * self.top
if left:
for line in lines:
yield left
yield from line
yield from right
else:
for line in lines:
yield from line
yield from right
for line in lines: # self.left is forced to 0 above, so no left-padding segment is ever emitted
yield from line
yield from right
if self.bottom:
blank_line = blank_line or [_Segment(f'{" " * width}\\n', style)]
yield from blank_line * self.bottom
@@ -232,20 +225,26 @@ def setup_logging(debug=None, trace=None, filename=None):
log.setLevel(logging.DEBUG) # log to file is always at level debug for facility `sd`
log.print = rprint
# use only the 16 standard ANSI color names (+ dim/bold modifiers) so the theme renders correctly on basic 16-color terminals too
theme = Theme({
"traceback.border": "black",
"inspect.value.border": "black",
"traceback.border.syntax_error": "dark_red",
"logging.level.info": "blue_violet",
"logging.level.debug": "orchid",
"logging.level.trace": "dark_blue",
"repr.attrib_name": "bright_cyan",
"repr.attrib_value": "orchid",
"repr.str": "sandy_brown",
"repr.number": "bright_green",
"repr.bool_true": "bright_green",
"traceback.border.syntax_error": "red",
"logging.level.trace": "dim cyan",
"logging.level.debug": "cyan",
"logging.level.info": "bright_cyan",
"logging.level.warning": "yellow",
"logging.level.error": "red",
"logging.level.critical": "bold bright_red",
"repr.attrib_name": "bright_white",
"repr.attrib_value": "cyan",
"repr.str": "bright_cyan",
"repr.none": "yellow",
"repr.number": "bright_yellow",
"repr.bool_true": "green",
"repr.bool_false": "bright_red",
"repr.values": "bright_cyan",
})
Padding.__rich_console__ = override_padding
@@ -270,7 +269,7 @@ def setup_logging(debug=None, trace=None, filename=None):
log_filter = LogFilter()
# handlers
rh = RichHandler(show_time=True, omit_repeated_times=False, show_level=True, show_path=False, markup=False, rich_tracebacks=True, log_time_format='%H:%M:%S-%f', level=level, console=console)
rh = RichHandler(show_time=True, omit_repeated_times=False, show_level=True, show_path=False, markup=True, rich_tracebacks=True, log_time_format='%H:%M:%S-%f', level=level, console=console)
if trace:
rh.formatter = logging.Formatter('[%(module)s][%(pathname)s:%(lineno)d] %(message)s')
rh.addFilter(log_filter)
@@ -319,3 +318,12 @@ def setup_logging(debug=None, trace=None, filename=None):
logging.getLogger("torch").setLevel(logging.DEBUG)
else:
logging.getLogger("torch").setLevel(logging.WARNING)
if __name__ == "__main__":
setup_logging(debug=True, trace=False, filename=None)
for l in [logging.TRACE, logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR, logging.CRITICAL]:
log.log(l, f"Test log level: {logging.getLevelName(l)}")
values = [None, True, False, "yes", "no", "sd.next", 1, 0, 1.0, [1,2,3], {"key": "value"}, (1,2), {1,2}, object()]
for v in values:
log.info(f"Test values: {type(v).__name__}={v}")
modules/lora/extra_networks_lora.py
+7 -7
View File
@@ -27,7 +27,7 @@ def get_stepwise(param, step, steps): # from https://github.com/cheald/sd-webui-
if m[1][-1] <= 1.0:
step = step / (max_steps - step_offset) if max_steps > 0 else 1.0
v = np.interp(step, m[1], m[0])
debug_log(f"Network load: type=LoRA step={step} steps={max_steps} v={v}")
debug_log(f"LoRA: stepwise step={step} steps={max_steps} v={v}")
return v
else:
return m
@@ -184,7 +184,7 @@ class ExtraNetworkLora(extra_networks.ExtraNetwork):
from modules.lora import lora_sdnq, lora_stack
requested = requested + [f'stack={lora_stack.signature()}{lora_sdnq.signature()}'] # settings-only stack or mechanism changes must re-trigger activation
if shared.opts.lora_force_reload:
debug_log(f'Network check: type=LoRA requested={requested} status="forced"')
debug_log(f'LoRA check requested={requested} status="forced"')
return True, "forced"
sd_model = shared.sd_model.pipe if hasattr(shared.sd_model, 'pipe') else shared.sd_model
if sd_model is None:
@@ -200,15 +200,15 @@ class ExtraNetworkLora(extra_networks.ExtraNetwork):
if len(requested) != len(loaded):
sd_model.loaded_loras.clear() # single-entry cache: any activation invalidates state recorded under other filter keys
sd_model.loaded_loras[key] = requested
debug_log(f'Network check: type=LoRA key="{key}" requested={requested} loaded={loaded} status="num changed"')
debug_log(f'LoRA check key="{key}" requested={requested} loaded={loaded} status="num changed"')
return True, "num changed"
for req, load in zip(requested, loaded, strict=False):
if req != load:
sd_model.loaded_loras.clear()
sd_model.loaded_loras[key] = requested
debug_log(f'Network check: type=LoRA key="{key}" requested={requested} loaded={loaded} status="content changed"')
debug_log(f'LoRA check key="{key}" requested={requested} loaded={loaded} status="content changed"')
return True, "content changed"
debug_log(f'Network check: type=LoRA key="{key}" requested={requested} loaded={loaded} status="same"')
debug_log(f'LoRA check key="{key}" requested={requested} loaded={loaded} status="same"')
return False, "none"
def activate(self, p, params_list, step=0, include=None, exclude=None): # pylint: disable=arguments-differ
@@ -236,7 +236,7 @@ class ExtraNetworkLora(extra_networks.ExtraNetwork):
if debug:
import sys
fn = f'{sys._getframe(2).f_code.co_name}:{sys._getframe(1).f_code.co_name}' # pylint: disable=protected-access
debug_log(f'Network load: type=LoRA include={include} exclude={exclude} method={load_method} reason="{load_reason}" requested={requested} fn={fn}')
debug_log(f'LoRA load: include={include} exclude={exclude} method={load_method} reason="{load_reason}" requested={requested} fn={fn}')
if load_method == 'diffusers':
has_changed, reason = self.changed(requested)
@@ -267,7 +267,7 @@ class ExtraNetworkLora(extra_networks.ExtraNetwork):
log.info(f'Network unload: type=LoRA networks={[n.name for n in l.previously_loaded_networks]} mode={networks.effective_mode()}')
networks.network_deactivate(include, exclude)
networks.network_activate(include, exclude)
debug_log(f'Network change: type=LoRA previous={[n.name for n in l.previously_loaded_networks]} current={[n.name for n in l.loaded_networks]}')
debug_log(f'LoRA change: previous={[n.name for n in l.previously_loaded_networks]} current={[n.name for n in l.loaded_networks]}')
if len(include) == 0:
l.previously_loaded_networks = l.loaded_networks.copy()
shared.state.end(jobid)
modules/lora/lora_convert.py
+11 -4
View File
@@ -487,28 +487,35 @@ def assign_network_names_to_compvis_modules(sd_model):
sd_model = getattr(shared.sd_model, "pipe", shared.sd_model) # wrapped model compatibility
network_layer_mapping = {}
if hasattr(sd_model, 'text_encoder') and sd_model.text_encoder is not None:
for name, module in sd_model.text_encoder.named_modules():
for name, module in sd_model.text_encoder.named_modules() :
prefix = "lora_te1_" if hasattr(sd_model, 'text_encoder_2') else "lora_te_"
network_name = prefix + name.replace(".", "_")
network_layer_mapping[network_name] = module
module.network_layer_name = network_name
if hasattr(sd_model, 'text_encoder_2'):
if hasattr(sd_model, 'text_encoder_2') and sd_model.text_encoder_2 is not None:
for name, module in sd_model.text_encoder_2.named_modules():
network_name = "lora_te2_" + name.replace(".", "_")
network_layer_mapping[network_name] = module
module.network_layer_name = network_name
if hasattr(sd_model, 'unet'):
if hasattr(sd_model, 'unet') and sd_model.unet is not None:
for name, module in sd_model.unet.named_modules():
network_name = "lora_unet_" + name.replace(".", "_")
network_layer_mapping[network_name] = module
module.network_layer_name = network_name
if hasattr(sd_model, 'transformer'):
if hasattr(sd_model, 'transformer') and sd_model.transformer is not None:
for name, module in sd_model.transformer.named_modules():
network_name = "lora_transformer_" + name.replace(".", "_")
network_layer_mapping[network_name] = module
if "norm" in network_name and "linear" not in network_name and shared.sd_model_type != "sd3":
continue
module.network_layer_name = network_name
if hasattr(sd_model, 'transformer_ref') and sd_model.transformer_ref is not None:
for name, module in sd_model.transformer_ref.named_modules():
network_name = "lora_transformer_" + name.replace(".", "_")
network_layer_mapping[network_name] = module
if "norm" in network_name and "linear" not in network_name and shared.sd_model_type != "sd3":
continue
module.network_layer_name = network_name
if hasattr(sd_model, 'llm_adapter') and sd_model.llm_adapter is not None:
for name, module in sd_model.llm_adapter.named_modules():
network_name = "lora_llm_adapter_" + name.replace(".", "_")
modules/lora/lora_load.py
+20 -9
View File
@@ -56,12 +56,19 @@ def lora_dump(lora, dct):
def load_safetensors(name, network_on_disk: network.NetworkOnDisk) -> network.Network | None:
if not shared.sd_loaded:
return None
sd_model = getattr(shared.sd_model, "pipe", shared.sd_model)
# cached
cached = lora_cache.get(name, None)
if cached is not None:
if l.debug:
log.trace(f'LoRA: load name="{name}" fn="{network_on_disk.filename}" cache=True')
return cached
# native dispatch
native_module = NATIVE_DISPATCH.get(shared.sd_model_type)
if l.debug:
log.trace(f'LoRA: load name="{name}" fn="{network_on_disk.filename}" native={native_module}')
if native_module is not None:
import importlib
mod = importlib.import_module(native_module)
@@ -69,6 +76,10 @@ def load_safetensors(name, network_on_disk: network.NetworkOnDisk) -> network.Ne
if net is not None:
lora_cache[name] = net
return net
# fallback to standard network loading
if l.debug:
log.trace(f'LoRA: load name="{name}" network_on_disk="{network_on_disk.filename}" safetensors')
net = network.Network(name, network_on_disk)
net.mtime = os.path.getmtime(network_on_disk.filename)
state_dict = sd_models.read_state_dict(network_on_disk.filename, what='network')
@@ -96,7 +107,7 @@ def load_safetensors(name, network_on_disk: network.NetworkOnDisk) -> network.Ne
emb_dict[vec_name] = weight
bundle_embeddings[emb_name] = emb_dict
continue
if parts[0] in ["clip_l","clip_g","t5","unet","transformer"]:
if parts[0] in ["clip_l", "clip_g", "t5", "unet", "transformer", "transformer_2"]:
network_part = []
while parts and parts[-1] in ["alpha","weight","lora_up","lora_down"]:
network_part.insert(0,parts[-1])
@@ -147,7 +158,7 @@ def load_safetensors(name, network_on_disk: network.NetworkOnDisk) -> network.Ne
if len(keys_failed_to_match) > 0:
log.warning(f'Network load: type=LoRA name="{name}" type={set(network_types)} unmatched={len(keys_failed_to_match)} matched={len(matched_networks)}')
if l.debug:
log.debug(f'Network load: type=LoRA name="{name}" unmatched={keys_failed_to_match}')
log.trace(f'Network load: type=LoRA name="{name}" unmatched={keys_failed_to_match}')
else:
log.debug(f'Network load: type=LoRA name="{name}" type={set(network_types)} keys={len(matched_networks)} dtypes={dtypes} fuse={lora_overrides.fuse_native()}:{shared.opts.lora_fuse_diffusers}')
if len(matched_networks) == 0:
@@ -172,13 +183,13 @@ def maybe_recompile_model(names, te_multipliers):
if not recompile_model:
skip_lora_load = True
if len(l.loaded_networks) > 0 and l.debug:
log.debug('Model Compile: Skipping LoRa loading')
log.trace('LoRA: recompile required, skip loading')
return recompile_model, skip_lora_load
else:
recompile_model = True
shared.compiled_model_state.lora_model = []
if l.debug:
log.debug(f'Model recompile check: task={sd_models.get_diffusers_task(shared.sd_model)} recompile={recompile_model} load={skip_lora_load}')
log.trace(f'LoRA recompile check: task={sd_models.get_diffusers_task(shared.sd_model)} recompile={recompile_model} load={skip_lora_load}')
if recompile_model:
current_task = sd_models.get_diffusers_task(shared.sd_model)
log.debug(f'Compile: task={current_task} force model reload')
@@ -285,7 +296,7 @@ def network_load(names, te_multipliers=None, unet_multipliers=None, dyn_dims=Non
if network_on_disk is not None:
shorthash = getattr(network_on_disk, 'shorthash', '').lower()
if l.debug:
log.debug(f'Network load: type=LoRA name="{name}" file="{network_on_disk.filename}" hash="{shorthash}" cached={name in lora_cache}')
log.trace(f'LoRA: name="{name}" fn="{network_on_disk.filename}" hash="{shorthash}" cached={name in lora_cache}')
try:
lora_scale = te_multipliers[i] if te_multipliers else shared.opts.extra_networks_default_multiplier
lora_module = lora_modules[i] if lora_modules and len(lora_modules) > i else None
@@ -330,8 +341,8 @@ def network_load(names, te_multipliers=None, unet_multipliers=None, dyn_dims=Non
try:
t1 = time.time()
if l.debug:
log.trace(f'Network load: type=LoRA list={sd_model.get_list_adapters()}')
log.trace(f'Network load: type=LoRA active={sd_model.get_active_adapters()}')
log.trace(f'LoRA: list={sd_model.get_list_adapters()}')
log.trace(f'LoRA: active={sd_model.get_active_adapters()}')
sd_model.set_adapters(adapter_names=lora_diffusers.diffuser_loaded, adapter_weights=lora_diffusers.diffuser_scales)
sd_model.enable_lora() # set_adapters does not clear the disabled flag left by a prior removal
except Exception as e:
@@ -359,7 +370,7 @@ def network_load(names, te_multipliers=None, unet_multipliers=None, dyn_dims=Non
networks.network_activate()
if len(l.loaded_networks) > 0 and l.debug:
log.debug(f'Network load: type=LoRA loaded={[n.name for n in l.loaded_networks]} cache={list(lora_cache)} fuse={lora_overrides.fuse_native()}:{shared.opts.lora_fuse_diffusers}')
log.trace(f'LoRA: loaded={[n.name for n in l.loaded_networks]} cache={list(lora_cache)} fuse={lora_overrides.fuse_native()}:{shared.opts.lora_fuse_diffusers}')
if recompile_model:
log.info("Network load: type=LoRA model recompile required")
modules/lora/lora_overrides.py
+17 -2
View File
@@ -1,4 +1,9 @@
import os
from modules import shared
from modules.logger import log
debug_log = log.trace if os.environ.get('SD_LORA_DEBUG', None) is not None else lambda *args, **kwargs: None
force_hashes_diffusers = [ # forced always
@@ -116,15 +121,22 @@ def disable_fuse():
from modules.lora import lora_common as l
from modules.lora import lora_stack
if lora_stack.select_possible(len(l.loaded_networks)) or lora_stack.select_engaged():
debug_log('LoRA: fuse=False reason="active select mode"')
return True # select flips per-layer winners against the pristine backup; a dormant select mode leaves fuse alone
sd_model = getattr(shared.sd_model, 'pipe', shared.sd_model)
if is_quantized(sd_model):
debug_log('LoRA: fuse=False reason="model is quantized"')
return True
if any(is_quantized(getattr(sd_model, name, None)) for name in fuse_components(sd_model)):
debug_log('LoRA: fuse=False reason="component is quantized"')
return True
if hasattr(sd_model, '_lora_partial'):
debug_log('LoRA: fuse=False reason="partial lora applied"')
return True
return shared.sd_model_type in fuse_ignore
if shared.sd_model_type in fuse_ignore:
debug_log(f'LoRA: fuse=False reason="model type {shared.sd_model_type} in fuse_ignore"')
return True
return False
def fuse_native():
@@ -134,4 +146,7 @@ def fuse_native():
the backup, activate and deactivate passes, since backup mode restores from a
stored tensor while fuse mode restores by subtracting the delta.
"""
return shared.opts.lora_fuse_native and not disable_fuse()
result = shared.opts.lora_fuse_native and not disable_fuse()
force = os.environ.get('SD_LORA_FUSE', None) is not None
debug_log(f'LoRA: native fuse={result} force={force}')
return (result or force)
modules/lora/networks.py
+3 -3
View File
@@ -56,7 +56,7 @@ from modules.logger import log, console
applied_layers: list[str] = []
refused_writes: int = 0 # deltas the modules would not take on the last activate pass; infotext reports the network as partial
native_active: bool = False
default_components = ['text_encoder', 'text_encoder_2', 'text_encoder_3', 'text_encoder_4', 'unet', 'transformer', 'transformer_2', 'llm_adapter']
default_components = ['text_encoder', 'text_encoder_2', 'text_encoder_3', 'text_encoder_4', 'unet', 'transformer', 'transformer_2', 'transformer_ref', 'llm_adapter']
class ActivationPass:
@@ -72,10 +72,10 @@ class ActivationPass:
self.sd_model = getattr(shared.sd_model, "pipe", shared.sd_model)
self.fuse = fuse
self.elimit = None # the error limiter, bound for the duration of the walk
self.wanted_names = tuple((x.name, x.te_multiplier, x.unet_multiplier, x.dyn_dim) for x in l.loaded_networks) if len(l.loaded_networks) > 0 else ()
self.wanted_names: tuple[tuple[str, float, list, int | None], ...] = tuple((x.name, x.te_multiplier, x.unet_multiplier, x.dyn_dim) for x in l.loaded_networks) if len(l.loaded_networks) > 0 else ()
self.stack_sig = lora_stack.signature() + lora_blocks.signature() + lora_sdnq.signature() # tracked beside network_current_names so stack-setting, block-weight and mechanism changes re-apply
self.select_active = len(l.loaded_networks) > 0 and lora_stack.active_select(len(l.loaded_networks)) # restore-only walks have nothing to stack; the count warning would fire on every network-free generation
self.component_wanted = ()
self.component_wanted: tuple[tuple[str, float, list, int | None], ...] = ()
self.device = None
self.group_offload = shared.opts.diffusers_offload_mode == "group"
self.group_stripped = {}
modules/minimax/minimax_ui.py
+12 -1
View File
@@ -24,6 +24,7 @@ def create_ui(prompt, _negative, styles, overrides, script_inputs, mp4_fps, mp4_
with gr.Accordion(open=True, label='Parameters', elem_id='minimax_param_accordion') as _param_accordion:
with gr.Row():
width, height = ui_sections.create_resolution_inputs('minimax', default_width=1024, default_height=576, step=32)
btn_detect_image_size = ToolButton(value=ui_symbols.detect, elem_id="minimax_resize_detect_size")
with gr.Row():
steps = gr.Slider(minimum=2, maximum=100, step=1, label="MiniMax steps", elem_id='minimax_steps', value=30)
frames = gr.Slider(label='MiniMax frames', minimum=22, maximum=362, step=17, value=124, elem_id='minimax_frames')
@@ -57,19 +58,29 @@ def create_ui(prompt, _negative, styles, overrides, script_inputs, mp4_fps, mp4_
model_info = next((m for m in models['MiniMax'] if m.name == model_name), None)
if model_info is None or model_info.name is None or model_info.name == '' or model_info.name == 'None':
return gr.update(value='none'), gr.update(visible=False), gr.update(visible=False)
log.debug(f'Selected: name="{model_info.name}" repo="{model_info.repo}" cls={model_info.repo_cls}')
if model_info.workflow == 'fl2va':
workflow = 'fl2va' if init_image is not None else 't2va'
else:
workflow = model_info.workflow
log.debug(f'Video: workflow={workflow} name="{model_info.name}" repo="{model_info.repo}" cls={model_info.repo_cls} image={init_image} selected')
return gr.update(value=f'Workflow: {workflow}'), gr.update(visible=workflow != 'ref2va'), gr.update(visible=workflow == 'ref2va')
def on_load(model_name: str):
model_info = next((m for m in models['MiniMax'] if m.name == model_name), None)
minimax_video.load_model(model_info.name if model_info is not None else None)
def on_image_size(init_image):
if init_image is not None:
try:
width, height = init_image.size
return gr.update(value=width), gr.update(value=height)
except Exception:
pass
return gr.update(), gr.update()
model.change(fn=on_change, inputs=[model, init_image], outputs=[workflow, input_accordion, reference_accordion], show_progress='hidden')
init_image.change(fn=on_change, inputs=[model, init_image], outputs=[workflow, input_accordion, reference_accordion], show_progress='hidden')
btn_detect_image_size.click(fn=on_image_size, inputs=[init_image], outputs=[width, height])
btn_load.click(fn=on_load, inputs=[model], outputs=[])
task_id = gr.Textbox(visible=False, value='')
modules/minimax/minimax_video.py
+15 -4
View File
@@ -49,23 +49,28 @@ def unwrap_file(entry):
return entry
def prepare_inputs(workflow: str | None, init_image: Image.Image | None, last_image: Image.Image | None, reference_media: list | None) -> dict:
def prepare_inputs(workflow: str | None, init_image: Image.Image | None, last_image: Image.Image | None, reference_media: list | None, width: int | None = None, height: int | None = None) -> dict:
"""The task args a workflow conditions on, resolved before the model load so a rejected request costs nothing."""
t_inputs = time.time()
from modules.image.resize import resize_image
from modules.minimax import minimax_references
if minimax_references.get_reference_caps(workflow) is not None:
entries = [unwrap_file(entry) for entry in (reference_media or [])]
references = minimax_references.resolve(workflow, entries, init_image)
log.debug(f'Prepare inputs: workflow={workflow} references={len(references)}')
log.debug(f'Video inputs: workflow={workflow} references={len(references)}')
return {'references': references}
task_args = {}
if init_image is not None:
if width is not None and height is not None:
init_image = resize_image(2, init_image, width, height) # crop to aspect ratio
task_args['image'] = init_image
if last_image is not None:
if width is not None and height is not None:
last_image = resize_image(2, last_image, width, height) # crop to aspect ratio
task_args['last_image'] = last_image
if reference_media:
log.warning(f'Video: op=reference workflow={workflow} references not supported, ignoring: count={len(reference_media)}')
log.debug(f'Prepare inputs: workflow={workflow} first={init_image} last={last_image}')
log.debug(f'Video inputs: workflow={workflow} first={init_image} last={last_image}')
timer.video.ts('inputs', t_inputs)
return task_args
@@ -111,18 +116,24 @@ def generate(task_id, _ui_state,
# resolved off the registry row so a bad reference is rejected before the load, the same as on the api path
selected = models_def.find(engine, model)
workflow = getattr(selected, 'workflow', None)
task_args = prepare_inputs(workflow, init_image, last_image, reference_media)
task_args = prepare_inputs(workflow, init_image, last_image, reference_media, width=width, height=height)
workflow = load_model(model) # override workflow based on loaded model
if not workflow:
progress.finish_task(task_id)
log.error('Video: model not loaded')
return None, 'Model not loaded'
init_images = [] # only so they are available for inspection by rest of the processing
if init_image is not None:
init_images.append(init_image)
if last_image is not None:
init_images.append(last_image)
p = processing.StableDiffusionProcessingVideo(
sd_model=shared.sd_model,
video_engine=engine,
video_model=model,
prompt=prompt,
styles=styles,
init_images=init_images,
seed=int(seed) if seed is not None else -1,
steps=int(steps),
width=width,
modules/modelloader.py
+1 -1
View File
@@ -490,5 +490,5 @@ def load_upscalers(quiet=False):
shared.sd_upscalers = upscalers
t1 = time.time()
if not quiet:
log.info(f"Available Upscalers: items={len(shared.sd_upscalers)} downloaded={len([x for x in shared.sd_upscalers if x.data_path is not None and os.path.isfile(x.data_path)])} user={len([x for x in shared.sd_upscalers if x.custom])} time={t1-t0:.2f} types={upscaler_types}")
log.info(f"Available Upscalers: items={len(shared.sd_upscalers)} downloaded={len([x for x in shared.sd_upscalers if x.data_path is not None and os.path.isfile(x.data_path)])} user={len([x for x in shared.sd_upscalers if x.custom])} time={t1-t0:.2f}")
return [x.name for x in shared.sd_upscalers]
modules/modular_load.py
+6 -3
View File
@@ -59,18 +59,21 @@ def preload_components(pipe, workflow: str | None, load_config: dict | None = No
if spec is None or getattr(spec, 'default_creation_method', None) != 'from_pretrained':
continue
repo = getattr(spec, 'pretrained_model_name_or_path', None)
cls = getattr(spec, 'type_hint', None)
cls = getattr(spec, 'type_hint', None) or {}
if not repo or cls is None:
continue
origin = getattr(cls, '__module__', '') or ''
cls_name = getattr(cls, '__name__', '') or '' # TODO preload: components with remote code resolve to cls none
cls_name = getattr(cls, '__name__', '') or ''
subfolder = getattr(spec, 'subfolder', None) or name
component = None
if origin.startswith('diffusers') and ('Transformer' in cls_name or 'UNet' in cls_name):
component = generic.load_transformer(repo, cls_name=cls, load_config=load_config, subfolder=subfolder, trust_remote_code=True)
elif origin.startswith('transformers') and 'text_encoder' in name:
elif origin.startswith('transformers') and ('text_encoder' in name):
# shared substitution is on: the map matches class plus a substring of the repo name, so its entries have to run narrow before broad
component = generic.load_text_encoder(repo, cls_name=cls, load_config=load_config, subfolder=subfolder)
if 'transformer' in name:
# fallback for component with remote-code as it does not have resolvable cls
component = generic.load_transformer(repo, cls_name=None, load_config=load_config, subfolder=subfolder, trust_remote_code=True)
if component is not None:
loaded[name] = component
return loaded
modules/processing.py
+12 -3
View File
@@ -352,11 +352,18 @@ def process_samples(p: StableDiffusionProcessing, samples):
method = p.color_correction_method if p.color_correction_method is not None else getattr(shared.opts, 'color_correction_method', 'histogram')
image = apply_color_correction(p.color_corrections[i], image, method=method)
if p.scripts is not None and isinstance(p.scripts, scripts_manager.ScriptRunner):
if p.scripts is not None and isinstance(p.scripts, scripts_manager.ScriptRunner) and not getattr(p, 'is_grid', False):
pp = scripts_manager.PostprocessImageArgs(image)
p.scripts.postprocess_image(p, pp)
if pp.image is not None:
image = pp.image
if isinstance(pp.image, list) and len(pp.image) > 0: # post process image can return original+processed
for i, img in enumerate(pp.image):
if i+1 < len(pp.image):
out_images.append(img)
out_infotexts.append(f"Postprocess image {i+1}")
image = pp.image[-1]
else:
image = pp.image
grading_params = processing_grading.GradingParams(
brightness=getattr(p, 'grading_brightness', 0.0),
@@ -609,7 +616,9 @@ def process_images_inner(p: StableDiffusionProcessing) -> Processed:
audio=audio,
)
if p.scripts is not None and isinstance(p.scripts, scripts_manager.ScriptRunner) and not (shared.state.interrupted or shared.state.skipped):
p.scripts.postprocess(p, results)
_results = p.scripts.postprocess(p, results)
if _results is not None:
results = _results
timer.process.record('post')
p.ops = list(set(p.ops))
t3 = time.time()
modules/processing_args.py
+1 -1
View File
@@ -444,7 +444,7 @@ def set_pipeline_args(p, model, prompts:list, negative_prompts:list, prompts_2:l
# handle missing resolution
if args.get('image', None) is not None and ('width' not in args or 'height' not in args):
if 'width' in possible and 'height' in possible:
vae_scale_factor = sd_vae.get_vae_scale_factor(model)
vae_scale_factor = sd_vae.get_vae_scale_factor(model, init_image=True)
if isinstance(args['image'], torch.Tensor) or isinstance(args['image'], np.ndarray):
if args['image'].shape[-1] == 3: # nhwc
args['width'] = args['image'].shape[-2]
modules/processing_class.py
+1 -1
View File
@@ -738,7 +738,7 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
def init(self, all_prompts=None, all_seeds=None, all_subseeds=None):
if self.init_images is not None and len(self.init_images) > 0:
vae_scale_factor = sd_vae.get_vae_scale_factor()
vae_scale_factor = sd_vae.get_vae_scale_factor(init_image=True)
if self.width is None or self.width == 0:
self.width = int(vae_scale_factor * (self.init_images[0].width * self.scale_by // vae_scale_factor))
if self.height is None or self.height == 0:
modules/processing_helpers.py
+1 -1
View File
@@ -397,7 +397,7 @@ def resize_init_images(p):
p.init_images = [p.image]
if getattr(p, 'init_images', None) is not None and len(p.init_images) > 0:
p.init_images = decode_images(p.init_images)
vae_scale_factor = sd_vae.get_vae_scale_factor()
vae_scale_factor = sd_vae.get_vae_scale_factor(init_image=True)
tgt_width = vae_scale_factor * math.ceil(p.init_images[0].width / vae_scale_factor)
tgt_height = vae_scale_factor * math.ceil(p.init_images[0].height / vae_scale_factor)
if p.init_images[0].size != (tgt_width, tgt_height):
modules/processing_info.py
-5
View File
@@ -208,11 +208,6 @@ def create_infotext(p: StableDiffusionProcessing, all_prompts=None, all_seeds=No
args['Sampler shift'] = get_opt('schedulers_shift') if get_opt('schedulers_shift') != shared.opts.data_labels.get('schedulers_shift').default else None
args['Sampler dynamic shift'] = get_opt('schedulers_dynamic_shift') if get_opt('schedulers_dynamic_shift') != shared.opts.data_labels.get('schedulers_dynamic_shift').default else None
# model specific
if shared.sd_model_type == 'h1':
args['LLM'] = None if shared.opts.model_h1_llama_repo == 'Default' else shared.opts.model_h1_llama_repo
# args.update(p.extra_generation_params)
for k, v in p.extra_generation_params.items():
if isinstance(v, (list, tuple)) and (job_size > index) and (len(v) > 1) and (len(v) == job_size): # likely a per-job param
args[k] = v[index]
modules/scripts_manager.py
+5 -1
View File
@@ -743,15 +743,19 @@ class ScriptRunner:
def postprocess(self, p: StableDiffusionProcessing, processed):
s = ScriptSummary('postprocess')
_processed = processed
for script in self.alwayson_scripts:
try:
args = resolve_script_args(script, p.script_args, p.per_script_args)
if args is not None:
script.postprocess(p, processed, *args)
result = script.postprocess(p, _processed, *args)
if result is not None: # allow postprocessing script to optionally modify results
_processed = result
except Exception as e:
errors.display(e, f'Running script postprocess: {script.filename}')
s.record(script.title())
s.report()
return _processed
def postprocess_batch(self, p: StableDiffusionProcessing, images, **kwargs):
s = ScriptSummary('postprocess-batch')
modules/sd_hijack_te.py
-2
View File
@@ -22,7 +22,6 @@ class PromptCache:
self.cache.clear()
self.id = id(shared.sd_model)
log.debug(f'Encode: prompt cache activate id={self.id} depth={len(self.cache)}')
prompt = self._hashable(prompt)
negative_prompt = self._hashable(negative_prompt)
if (isinstance(prompt, list) and len(prompt) == 1 and isinstance(prompt[0], str)):
cached = self.cache.get((prompt[0], negative_prompt, cfg_enabled), None)
@@ -41,7 +40,6 @@ class PromptCache:
if len(self.cache) >= self.max:
oldest_key = next(iter(self.cache))
del self.cache[oldest_key]
prompt = self._hashable(prompt)
negative_prompt = self._hashable(negative_prompt)
if (isinstance(prompt, list) and len(prompt) == 1 and isinstance(prompt[0], str)):
self.cache[(prompt[0], negative_prompt, cfg_enabled)] = encoded
modules/sd_models.py
+13
View File
@@ -1147,6 +1147,17 @@ def get_diffusers_task(pipe: diffusers.DiffusionPipeline) -> DiffusersTaskType:
return DiffusersTaskType.TEXT_2_IMAGE
def pipe_serves_task(pipe: diffusers.DiffusionPipeline, task_type: DiffusersTaskType) -> bool:
"""True when the pipeline class is registered for the task in the diffusers auto-pipeline tables."""
mappings = {
DiffusersTaskType.TEXT_2_IMAGE: diffusers.pipelines.auto_pipeline.AUTO_TEXT2IMAGE_PIPELINES_MAPPING,
DiffusersTaskType.IMAGE_2_IMAGE: diffusers.pipelines.auto_pipeline.AUTO_IMAGE2IMAGE_PIPELINES_MAPPING,
DiffusersTaskType.INPAINTING: diffusers.pipelines.auto_pipeline.AUTO_INPAINT_PIPELINES_MAPPING,
}
mapping = mappings.get(task_type)
return mapping is not None and pipe.__class__ in mapping.values()
def switch_pipe(cls: type[diffusers.DiffusionPipeline] | str, pipeline: diffusers.DiffusionPipeline | None = None, force = False, args: dict | None = None):
"""
args:
@@ -1354,6 +1365,8 @@ def set_diffuser_pipe(pipe, new_pipe_type):
return pipe
if get_diffusers_task(pipe) == new_pipe_type:
return pipe
if pipe_serves_task(pipe, new_pipe_type): # a class registered for several tasks classifies as one of them
return pipe
if get_diffusers_task(pipe) == DiffusersTaskType.MODULAR:
return pipe
modules/sd_offload_group.py
+10 -13
View File
@@ -206,20 +206,17 @@ def offload_ondemand(sd_model, include=[], exclude=[], reason='', force=False):
def report_group_stats(sd_model, module_names):
"""Per-component stats block once per loaded model; balanced mode prints its own from the hook map."""
checkpoint_name = sd_model.sd_checkpoint_info.name if getattr(sd_model, "sd_checkpoint_info", None) is not None else sd_model.__class__.__name__
if checkpoint_name in s.group_stats_reported: # keyed by checkpoint since a task switch rebuilds the pipe object
"""Per-component stats block once per loaded component; balanced mode prints its own from the hook map."""
modules = {name: getattr(sd_model, name, None) for name in module_names}
modules = {name: module for name, module in modules.items() if isinstance(module, torch.nn.Module)}
pending = {name: module for name, module in modules.items() if not getattr(module, 'sdnext_stats_reported', False)} # a task switch reuses the modules, a reload brings new ones
if not pending:
return
s.group_stats_reported.add(checkpoint_name)
total = 0.0
counted = []
for module_name in module_names:
module = getattr(sd_model, module_name, None)
if isinstance(module, torch.nn.Module):
total += get_module_size(module)[0]
counted.append(module_name)
report_model_stats(module_name, module)
log.info(f'Model class={sd_model.__class__.__name__} modules={len(counted)} size={total:.3f}')
for module_name, module in pending.items():
module.sdnext_stats_reported = True
report_model_stats(module_name, module)
total = sum(get_module_size(module)[0] for module in modules.values())
log.info(f'Model class={sd_model.__class__.__name__} modules={len(modules)} size={total:.3f}')
def apply_group_offload(sd_model):
modules/sd_offload_state.py
-1
View File
@@ -30,5 +30,4 @@ no_split_module_classes = [
]
accelerate_dtype_byte_size = None # monkey-patch accelerate.utils.modeling.dtype_byte_size
group_stats_reported = set()
move_stream = None
modules/sd_offload_utils.py
+5 -1
View File
@@ -174,7 +174,11 @@ def report_model_stats(module_name, module):
size, _params = get_module_size(module)
quant = getattr(module, "quantization_method", None)
params = sum(p.numel() for p in module.parameters(recurse=True))
try:
dtype = next(module.parameters(), torch.tensor([])).dtype
except Exception:
dtype = None
logical = get_logical_param_count(module)
log.debug(f'Module: name={module_name} cls={module.__class__.__name__} size={size:.3f} params={params} logical={logical} quant={quant}')
log.debug(f'Module: name={module_name} cls={module.__class__.__name__} size={size:.3f} params={params} logical={logical} quant={quant} dtype={dtype}')
except Exception as e:
log.error(f'Module stats: name={module_name} {e}')
modules/sd_vae.py
+6 -3
View File
@@ -28,7 +28,7 @@ vae_scale_override = {
}
def get_vae_scale_factor(model: DiffusionPipeline | None = None):
def get_vae_scale_factor(model: DiffusionPipeline | None = None, init_image: bool = False):
if not shared.sd_loaded:
vae_scale_factor = 8
return vae_scale_factor
@@ -58,9 +58,12 @@ def get_vae_scale_factor(model: DiffusionPipeline | None = None):
patch_size = model.patch_size
if isinstance(patch_size, (tuple, list)): # 3d patch sizes are (t, h, w); spatial term is last
patch_size = patch_size[-1]
multiple = vae_scale_factor * patch_size
if init_image and model is not None and hasattr(model, 'init_image_multiple'): # a pipeline that downsamples its source image needs a larger multiple than its output
multiple = max(multiple, int(model.init_image_multiple))
if debug:
log.trace(f'VAE: cls={model.__class__.__name__ if model else "None"} scale={vae_scale_factor} patch={patch_size}')
return vae_scale_factor * patch_size
log.trace(f'VAE: cls={model.__class__.__name__ if model else "None"} scale={vae_scale_factor} patch={patch_size} multiple={multiple}')
return multiple
def load_vae_dict(filename: str):
modules/shared.py
+1 -1
View File
@@ -211,7 +211,7 @@ def restart_server(restart=True):
demo.server.should_exit = True
demo.server.force_exit = True
demo.close(verbose=False)
demo.server.close()
# demo.server.close()
demo.fns = []
time.sleep(1)
sys.tracebacklimit = 100
modules/shared_defaults.py
+3 -3
View File
@@ -46,13 +46,13 @@ def get_default_modes(cmd_opts, mem_stat):
if devices.backend == "zluda":
default_sdp_options = ['Math']
default_cross_attention = ['Dynamic attention']
default_cross_attention = 'Dynamic attention'
elif devices.backend == "rocm":
agent = devices.get_hip_agent()
if agent.gfx_version < 0x1100:
default_cross_attention = ['Dynamic attention'] # only RDNA2 and older GPUs needs this
default_cross_attention = 'Dynamic attention' # only RDNA2 and older GPUs needs this
elif devices.backend in {"cpu", "mps"}:
default_cross_attention = ['Dynamic attention']
default_cross_attention = 'Dynamic attention'
if devices.get_optimal_device_name() != "cpu":
os.environ.setdefault('SDNQ_USE_OPENVINO_MM', '0')
modules/styles.py
+10 -6
View File
@@ -200,14 +200,15 @@ def apply_wildcards_to_prompt(prompt, all_wildcards, seed=-1, silent=False, p: S
except Exception as e:
log.error(f'Wildcards: wildcard="{wildcard}" error={e}')
t1 = time.time()
prompt, replaced_file, not_found = apply_file_wildcards(prompt, [], [], recursion=0, seed=seed, p=p)
prompt, replaced_files, missing_files = apply_file_wildcards(prompt, [], [], recursion=0, seed=seed, p=p)
t2 = time.time()
if replaced and not silent:
log.debug(f'Apply wildcards: {replaced} path="{shared.opts.wildcards_dir}" type=style time={t1-t0:.2f}')
if (len(replaced_file) > 0 or len(not_found) > 0) and not silent:
log.debug(f'Apply wildcards: found={replaced_file} missing={not_found} path="{shared.opts.wildcards_dir}" type=file seed={seed} time={t2-t2:.2f}')
if (len(replaced_files) > 0 or len(missing_files) > 0) and not silent:
log.debug(f'Apply wildcards: found={replaced_files} missing={missing_files} path="{shared.opts.wildcards_dir}" type=file seed={seed} time={t2-t1:.2f}')
if p is not None:
p.extra_generation_params['Wildcards'] = p.extra_generation_params.get('Wildcards', []) + [replaced_file]
wildcards = p.extra_generation_params.get('Wildcards', []) + replaced_files
p.extra_generation_params['Wildcards'] = ', '.join(wildcards)
if old_state is not None:
random.setstate(old_state)
return prompt
@@ -243,12 +244,15 @@ def apply_styles_to_extra(p, style: Style):
'size',
]
reference_style = get_reference_style()
extra = infotext.parse(reference_style) if shared.opts.extra_network_reference_values else {}
reference = infotext.parse(reference_style) if shared.opts.extra_network_reference_values else {}
extra = reference.copy()
style_extra = apply_wildcards_to_prompt(style.extra, [style.wildcards], silent=True, p=p)
style_extra = ' ' + style_extra.lower()
extra.update(infotext.parse(style_extra))
extra.pop('Prompt', None)
extra.pop('Negative prompt', None)
has_prompt = (style.prompt is not None) and len(style.prompt) > 2
has_negative = (style.negative_prompt is not None) and len(style.negative_prompt) > 2
if debug_enabled:
log.trace(f'Apply style extra: {extra}')
@@ -284,7 +288,7 @@ def apply_styles_to_extra(p, style: Style):
if debug_enabled:
log.trace(f'Apply style skip: {k}={v}')
skipped.append(f'{k}={v}')
log.debug(f'Apply style: name="{style.name}" params={params} settings={settings} unknown={skipped} reference={True if reference_style else False}')
log.debug(f'Apply style: name="{style.name}" prompt={has_prompt} negative={has_negative} params={params} settings={settings} unknown={skipped} reference={reference}')
class StyleDatabase:
modules/ui_definitions.py
+3 -2
View File
@@ -504,6 +504,7 @@ def create_settings(cmd_opts):
"openvino_cache_path": OptionInfo('cache', "Folder for OpenVINO cache", folder=True),
"onnx_cached_models_path": OptionInfo(os.path.join(paths.models_path, 'ONNX', 'cache'), "Folder for ONNX cached models", folder=True),
"onnx_temp_dir": OptionInfo(os.path.join(paths.models_path, 'ONNX', 'temp'), "Folder for ONNX conversion", folder=True),
"dlss_pkg_path": OptionInfo('', "Folder with DLSS package", gr.Textbox, { "visible": False}),
}))
# --- Image Options ---
@@ -610,13 +611,13 @@ def create_settings(cmd_opts):
"ui_disabled": OptionInfo([], "Disabled UI tabs", gr.Dropdown, { 'visible': False }),
"cards_sep_ui": OptionInfo("<h2>Networks panel</h2>", "", gr.HTML),
"extra_networks_card_size": OptionInfo(140, "Network card size (px)", gr.Slider, {"minimum": 20, "maximum": 2000, "step": 1}),
"extra_networks_card_size": OptionInfo(130, "Network card size (px)", gr.Slider, {"minimum": 20, "maximum": 2000, "step": 1}),
"extra_networks_card_cover": OptionInfo("sidebar", "Network panel position", gr.Radio, {"choices": ["cover", "inline", "sidebar"]}),
"extra_networks_card_square": OptionInfo(True, "Disable variable aspect ratio"),
"other_sep_ui": OptionInfo("<h2>Other...</h2>", "", gr.HTML),
"ui_locale": OptionInfo("Auto", "UI locale", gr.Dropdown, lambda: {"choices": theme.list_locales()}),
"font_size": OptionInfo(14, "Font size", gr.Slider, {"minimum": 8, "maximum": 32, "step": 1}),
"font_size": OptionInfo(15, "Font size", gr.Slider, {"minimum": 8, "maximum": 32, "step": 1}),
"gpu_monitor": OptionInfo(3000, "GPU monitor interval", gr.Slider, {"minimum": 100, "maximum": 60000, "step": 100}),
"aspect_ratios": OptionInfo("1:1, 4:3, 3:2, 16:9, 16:10, 21:9, 2:3, 3:4, 9:16, 10:16, 9:21", "Allowed aspect ratios"),
"compact_view": OptionInfo(False, "Compact view"),
modules/ui_extra_networks.py
+2
View File
@@ -27,6 +27,8 @@ refresh_time = 0
extra_pages = shared.extra_networks
debug = log.trace if os.environ.get('SD_EN_DEBUG', None) is not None else lambda *args, **kwargs: None
debug('Trace: EN')
card_empty = '<div class="card"></div>'
card_full = '''
<div class='card' onclick={card_click} title='{name}' data-page='{page}' data-name='{name}' data-filename='{filename}' data-short='{short}' data-tags='{tags}' data-mtime='{mtime}' data-size='{size}' data-search='{search}' data-version='{version}' style='--data-color: {color}'>
<div class='overlay'>
modules/ui_extra_networks_checkpoints.py
+1
View File
@@ -111,6 +111,7 @@ class ExtraNetworksPageCheckpoints(ui_extra_networks.ExtraNetworksPage):
primary = tag.split(',')[0].strip() if len(tag) > 0 else ''
else:
primary = ''
primary = primary.lower()
if ('nunchaku' in tag) and (devices.backend != 'cuda' and not shared.cmd_opts.experimental):
count['hidden'] += 1
modules/ui_guidance.py
+3
View File
@@ -110,7 +110,10 @@ def create_guidance_inputs(tab):
standard_args = args_base + args_legacy
def update_stored(component, name):
if component is None or name is None:
return
_stored_args[name] = component
for component in modular_args:
label = getattr(component, 'label', None)
value = getattr(component, 'value', None)
modules/ui_settings.py
+2 -1
View File
@@ -259,7 +259,8 @@ def create_ui(disabled_tabs=None):
if item[1].section is not None and item[1].section[0] == section_id
] # find all items in this section
hidden = (section_id is None) or ('hidden' in section_id.lower()) or ('hidden' in section_text.lower()) or ('legacy' in section_id.lower()) or ('legacy' in section_text.lower())
# log.trace(f'Settings: section="{section_id}" title="{section_text}" items={len(items)} hidden={hidden}')
# for (key, _item) in items:
# log.trace(f'Settings: id={section_id} text={section_text} key={key} hidden={hidden}')
if hidden:
for (key, _item) in items:
hidden_list.append(key)
modules/ui_symbols.py
+1 -1
View File
@@ -23,13 +23,13 @@ sort = '⇕'
detect = '📐'
folder = '📂'
random = '🎲️'
reuse = '♻️'
info = 'ℹ' # noqa
reset = '🔄'
upload = '⬆️'
loading = '↺'
reuse = '⬅️'
search = '🔍'
tools = '🛠'
preview = '🖼️'
image = '🖌️'
resize = '⁜'
modules/update.py
+42 -25
View File
@@ -6,6 +6,7 @@ import installer as i
version = SimpleNamespace(**{
'url': '',
'branch': '',
'origin': '',
'current': '0000-00-00',
'chash': '0000000',
'latest': '0000-00-00',
@@ -14,34 +15,50 @@ version = SimpleNamespace(**{
def get_version():
# try:
origin = i.git('remote get-url origin')
origin = origin.splitlines()[0]
version.branch = i.git('rev-parse --abbrev-ref HEAD')
version.branch = version.branch.splitlines()[0]
version.url = origin.removesuffix('.git') + '/tree/' + version.branch
try:
origin = i.git('remote get-url origin')
origin = origin.splitlines()
if len(origin) > 0:
version.origin = origin[0]
version.url = version.origin.removesuffix('.git') + '/tree/' + version.branch
else:
version.origin = 'unknown'
i.log.warning('Version: origin URL not found')
ver = i.git('log --pretty=format:"%h %ad" -1 --date=short')
ver = ver.splitlines()[0]
version.chash, version.current = ver.split(' ')
branch = i.git('rev-parse --abbrev-ref HEAD')
branch = branch.splitlines()
if len(branch) > 0:
version.branch = branch[0]
else:
version.branch = 'unknown'
i.log.warning('Version: branch not found')
i.git('fetch')
ver = i.git(f'log origin/{version.branch} --pretty=format:"%h %ad" -1 --date=short')
ver = ver.splitlines()[0]
version.lhash, version.latest = ver.split(' ')
gitlog = i.git('log --pretty=format:"%h %ad" -1 --date=short')
gitlog = gitlog.splitlines()
if len(gitlog) > 0:
version.chash, version.current = gitlog[0].split(' ')
# except Exception as e:
# i.log.error(f'Version check failed: {e}')
i.log.info(f'Version: {vars(version)}')
latest = '<div style="color: var(--secondary-500)">You\'re up to date!</div>' if version.chash == version.lhash else '<div style="color: var(--secondary-500)">Update available!</div>'
html = f'''
<div>URL: <a href="{version.url}" target="_blank">{version.url}</a></div>
<div>Current branch: <span style="color: var(--highlight-color)">{version.branch}</span></div>
<div>Current version: <span style="color: var(--highlight-color)">{version.current}</span> hash <span style="color: var(--highlight-color)">{version.chash}</span></div>
<div>Latest version: <span style="color: var(--highlight-color)">{version.latest}</span> hash <span style="color: var(--highlight-color)">{version.lhash}</span></div>
{latest}
'''
return html
i.git('fetch')
ver = i.git(f'log origin/{version.branch} --pretty=format:"%h %ad" -1 --date=short')
ver = ver.splitlines()
if len(ver) > 0 and ' ' in ver[0]:
version.lhash, version.latest = ver[0].split(' ')
i.log.info(f'Version: {vars(version)}')
latest = '<div style="color: var(--secondary-500)">You\'re up to date!</div>' if version.chash == version.lhash else '<div style="color: var(--secondary-500)">Update available!</div>'
html = f'''
<div>URL: <a href="{version.url}" target="_blank">{version.url}</a></div>
<div>Origin: <span style="color: var(--highlight-color)">{version.origin}</span></div>
<div>Current branch: <span style="color: var(--highlight-color)">{version.branch}</span></div>
<div>Current version: <span style="color: var(--highlight-color)">{version.current}</span> hash <span style="color: var(--highlight-color)">{version.chash}</span></div>
<div>Latest version: <span style="color: var(--highlight-color)">{version.latest}</span> hash <span style="color: var(--highlight-color)">{version.lhash}</span></div>
{latest}
'''
return html
except Exception as e:
i.log.error(f'Version: {e}')
html = f'<div style="color: var(--error-color)">Error while detecting version<br>{str(e)}</div>'
return html
def apply_update(update_rebase, update_submodules, update_extensions):
modules/video_models/models_def.py
+36
View File
@@ -723,6 +723,30 @@ try:
image_hijack=False,
vae_hijack=False,
vae_remote=False),
Model(name='MiniMax H3 Pruned SDNQ uint8',
url='https://huggingface.co/MiniMaxAI/MiniMax-H3',
repo='OzzyGT/MiniMax_H3_sdnq_8bit_pruned',
repo_cls='MiniMaxH3ModularPipeline',
workflow='fl2va',
base=True,
te_cls=None,
dit_cls=None,
te_hijack=False,
image_hijack=False,
vae_hijack=False,
vae_remote=False),
Model(name='MiniMax H3 Pruned SDNQ uint8 Ref2VA',
url='https://huggingface.co/MiniMaxAI/MiniMax-H3',
repo='OzzyGT/MiniMax_H3_sdnq_8bit_pruned',
repo_cls='MiniMaxH3ModularPipeline',
workflow='ref2va',
base=True,
te_cls=None,
dit_cls=None,
te_hijack=False,
image_hijack=False,
vae_hijack=False,
vae_remote=False),
Model(name='MiniMax H3',
url='https://huggingface.co/MiniMaxAI/MiniMax-H3',
repo='MiniMaxAI/MiniMax-H3',
@@ -747,6 +771,18 @@ try:
image_hijack=False,
vae_hijack=False,
vae_remote=False),
Model(name='MiniMax H3 VDN',
url='https://huggingface.co/OpenVDN/vdn-minimax-h3',
repo='OpenVDN/vdn-minimax-h3',
repo_cls='MiniMaxH3ModularPipeline',
workflow='fl2va',
base=True,
te_cls=None,
dit_cls=None,
te_hijack=False,
image_hijack=False,
vae_hijack=False,
vae_remote=False),
],
'Google Veo': [
Model(name='Google Veo 3.1 T2V',
modules/video_models/video_minimax.py
+1 -1
View File
@@ -25,7 +25,7 @@ def apply_overrides(p, pipe, still: bool = False, audio: bool = True):
while frames > max_frames:
frames -= pipe.vae_frames_per_chunk
if frames != getattr(p, 'frames', None):
log.debug(f'Pipeline: cls={pipe.__class__.__name__} frames={getattr(p, "frames", None)} aligned={frames}')
log.debug(f'Pipeline: cls={pipe.__class__.__name__} frames requested={getattr(p, "frames", None)} aligned={frames}')
p.frames = frames
p.task_args['num_frames'] = frames
p.steps = max(2, p.steps)
pipelines/generic_text_encoder.py
+2
View File
@@ -21,6 +21,8 @@ def get_shared(cls, repo_id, subfolder=None, variant=None, shared_id: str | None
identifiers = []
if isinstance(identifiers, str):
identifiers = [identifiers]
if not isinstance(identifiers, list):
identifiers = []
identifiers = [identifier.lower() for identifier in identifiers if identifier is not None]
shared_id = shared_id or repo_id.lower()
if item['cls'] == cls and (not identifiers or any(identifier in shared_id for identifier in identifiers)):
pipelines/generic_transformer.py
+10 -7
View File
@@ -48,6 +48,9 @@ def load_transformer(
modules_to_not_convert = []
if modules_dtype_dict is None:
modules_dtype_dict = {}
if cls_name is None:
from diffusers import AutoModel
cls_name = AutoModel
offline_args = {'local_files_only': True} if shared.opts.offline_mode else {}
jobid = shared.state.begin('Load DiT')
try:
@@ -75,11 +78,14 @@ def load_transformer(
if trust_remote_code:
load_args['trust_remote_code'] = True
load_kwargs = {**load_args, **quant_args, **offline_args, **kwargs}
return cls_name.from_pretrained(
module = cls_name.from_pretrained(
repo_id,
cache_dir=shared.opts.hfcache_dir,
**load_kwargs,
)
if cls_name.__name__ == 'AutoModel':
log.debug(f'Load model: transformer="{repo_id}" cls={module.__class__.__name__}')
return module
local_file = None
override_name = None
@@ -158,13 +164,11 @@ def load_transformer(
**load_kwargs,
)
# 4. default loading from diffusers repo (also the fallback when an
# incompatible override is dropped above)
# 4. default loading from local file (also the fallback when an incompatible override is dropped above) # 5. default loading from diffusers repo (also the fallback when an incompatible override is dropped above)
else:
transformer = load_from_repo()
# mark the dropdown selection as loaded so the slot's onchange callback
# does not force a redundant full reload for an already-consumed override
# mark the dropdown selection as loaded so the slot's onchange callback, does not force a redundant full reload for an already-consumed override
if transformer is not None and override_name is not None and getattr(shared.opts, override_opt, None) == override_name:
setattr(sd_unet, tracker_attr, override_name)
@@ -192,8 +196,7 @@ def load_transformer(
log.debug(f'Load model: transformer="{repo_id}" quant="{quant_type}" size={module_size:.3f} params={param_num:.3f} memory={module_memory}')
try:
# quantized models legitimately report the storage dtype (e.g. fp8 comfy_quant
# adopted via SDNQ); the compute dtype lives in the dequantizers, not the params
# quantized models legitimately report the storage dtype (e.g. fp8 comfy_quant adopted via SDNQ); the compute dtype lives in the dequantizers, not the params
if getattr(transformer, 'quantization_config', None) is None:
actual_dtype = transformer.dtype
if isinstance(actual_dtype, torch.dtype) and isinstance(dtype, torch.dtype) and actual_dtype != dtype:
pipelines/llada/modeling_llada2uni_moe.py
+4
View File
@@ -91,6 +91,10 @@ class LLaDA2MoePreTrainedModel(PreTrainedModel):
module.weight.data.normal_(mean=0.0, std=std)
if module.padding_idx is not None:
module.weight.data[module.padding_idx].zero_()
elif isinstance(module, LLaDA2MoeRotaryEmbedding): # non-persistent buffers come back uninitialized from the meta-device load
inv_freq, module.attention_scaling = module.rope_init_fn(module.config, module.inv_freq.device)
module.inv_freq.copy_(inv_freq)
module.original_inv_freq = module.inv_freq
def rotate_half(hidden_states):
pipelines/llada/pipeline_llada_image.py
+3 -1
View File
@@ -90,6 +90,8 @@ class LLaDAImagePipeline(DiffusionPipeline):
self.vae_scale_factor = 2 ** (len(self.vae.config.block_out_channels) - 1) if self.vae is not None else 8
self.latent_scale_factor = self.vae_scale_factor * 2
self.patch_size = 2 # transformer patch size, read by the host to round sizes to the latent multiple
self.init_image_multiple = self.latent_scale_factor * 2 # editing feeds a half-resolution copy of the source image to the semantic encoder
self.image_processor = VaeImageProcessor(vae_scale_factor=self.latent_scale_factor)
@classmethod
@@ -383,7 +385,7 @@ class LLaDAImagePipeline(DiffusionPipeline):
if generation_mode == "vq" and (height % 16 != 0 or width % 16 != 0):
raise ValueError("`height` and `width` must be divisible by 16 in VQ mode.")
required_multiple = self.latent_scale_factor * (2 if generation_mode == "editing" else 1)
required_multiple = self.init_image_multiple if generation_mode == "editing" else self.latent_scale_factor
if height <= 0 or width <= 0 or height % required_multiple != 0 or width % required_multiple != 0:
raise ValueError(f"`height` and `width` must be divisible by {required_multiple}.")
if num_inference_steps < 1:
pipelines/minimax/minimax_latents.py
+38 -25
View File
@@ -1,7 +1,14 @@
import diffusers
from modules.logger import log
warned = False
def unpack_latents(latents, components: diffusers.modular_pipelines.ModularPipeline, state: diffusers.modular_pipelines.BlockState):
global warned # pylint: disable=global-statement
if warned:
return latents
from diffusers.modular_pipelines.minimax_h3.modular_pipeline import align_num_frames, video_latent_num_frames
from modules import processing_callbacks
frames = getattr(processing_callbacks.p, 'frames', 1)
@@ -9,29 +16,35 @@ def unpack_latents(latents, components: diffusers.modular_pipelines.ModularPipel
height = getattr(processing_callbacks.p, 'height', 1024)
if frames <= 0 or width <= 0 or height <= 0:
return latents
num_frames = align_num_frames(frames, components.vae_frames_per_chunk, components.vae_latents_per_chunk)
num_latent_frames = video_latent_num_frames(num_frames, components.vae_frames_per_chunk, components.vae_latents_per_chunk)
latent_height = height // components.vae_spatial_compression_ratio
latent_width = width // components.vae_spatial_compression_ratio
patch_t, patch_h, patch_w = components.patch_size
channels = components.vae_latent_channels
rows = state.latents[state.num_condition_video_rows :]
rows = rows.reshape(
-1,
num_latent_frames // patch_t,
latent_height // patch_h,
latent_width // patch_w,
channels,
patch_t,
patch_h,
patch_w,
)
rows = rows.permute(0, 4, 1, 5, 2, 6, 3, 7)
latents = rows.reshape(
-1,
channels,
num_latent_frames,
latent_height,
latent_width,
).contiguous()
try:
num_frames = align_num_frames(frames, components.vae_frames_per_chunk, components.vae_latents_per_chunk)
num_latent_frames = video_latent_num_frames(num_frames, components.vae_frames_per_chunk, components.vae_latents_per_chunk)
latent_height = height // components.vae_spatial_compression_ratio
latent_width = width // components.vae_spatial_compression_ratio
patch_t, patch_h, patch_w = components.patch_size
channels = components.vae_latent_channels
rows = state.latents[state.num_condition_video_rows :]
rows = rows.reshape(
-1,
num_latent_frames // patch_t,
latent_height // patch_h,
latent_width // patch_w,
channels,
patch_t,
patch_h,
patch_w,
)
rows = rows.permute(0, 4, 1, 5, 2, 6, 3, 7)
latents = rows.reshape(
-1,
channels,
num_latent_frames,
latent_height,
latent_width,
).contiguous()
except Exception as e:
# fails with sliced attention due to shape mismatch as state.latents only contains a subset of the full video latents
if not warned:
warned = True
log.warning(f'Video unpack latents: {e}')
return latents
pipelines/model_lens.py
+1 -2
View File
@@ -12,9 +12,9 @@ def load_lens(checkpoint_info, diffusers_load_config=None):
sd_models.hf_auth_check(checkpoint_info)
from pipelines import lens
load_args, _quant_args = model_quant.get_dit_args(diffusers_load_config, allow_quant=False)
log.debug(f'Load model: type=Lens repo="{repo_id}" config={diffusers_load_config} offload={shared.opts.diffusers_offload_mode} dtype={devices.dtype} reasoner={shared.opts.model_lens_enable_pe} args={load_args}')
generic.set_pipeline('Lens', diffusers.Krea2Pipeline)
if repo_id is None or repo_id.lower() == 'none':
return None
@@ -27,7 +27,6 @@ def load_lens(checkpoint_info, diffusers_load_config=None):
diffusers.pipelines.auto_pipeline.AUTO_TEXT2IMAGE_PIPELINES_MAPPING["lens"] = lens.LensPipeline
diffusers.pipelines.auto_pipeline.AUTO_IMAGE2IMAGE_PIPELINES_MAPPING["lens"] = lens.LensImg2ImgPipeline
diffusers.pipelines.auto_pipeline.AUTO_INPAINT_PIPELINES_MAPPING["lens"] = lens.LensInpaintPipeline
generic.set_pipeline('Lens', lens.LensPipeline)
pipe = lens.LensPipeline.from_pretrained(
repo_id,
transformer=transformer,
pipelines/model_llada.py
+14
View File
@@ -19,15 +19,21 @@ def load_llada_image(checkpoint_info, diffusers_load_config=None):
if repo_id is None or repo_id.lower() == 'none':
return None
sdnq_quantize_weights_mode = None
sdnq_quantize_weights_mode_te = None
sdnq_quantize_matmul_mode_te = None
if 'Model' in shared.opts.sdnq_quantize_weights:
if any(x in shared.opts.sdnq_quantize_weights_mode for x in ['2', '3', '4', '5', '6']):
sdnq_quantize_weights_mode = shared.opts.sdnq_quantize_weights_mode
shared.opts.sdnq_quantize_weights_mode = 'uint8'
log.warning('LLaDAImage: cls=LLaDAImageTransformer2DModel quant=uint8 override')
if 'TE' in shared.opts.sdnq_quantize_weights:
if any(x in shared.opts.sdnq_quantize_weights_mode_te for x in ['2', '3', '4', '5', '6']):
sdnq_quantize_weights_mode_te = shared.opts.sdnq_quantize_weights_mode_te
shared.opts.sdnq_quantize_weights_mode_te = 'uint8'
log.warning('LLaDAImage: cls=LLaDA2MoeModelLM quant=uint8 override')
if shared.opts.sdnq_quantize_matmul_mode_te != 'disabled':
sdnq_quantize_matmul_mode_te = shared.opts.sdnq_quantize_matmul_mode_te
shared.opts.sdnq_quantize_matmul_mode_te = 'disabled'
log.warning('LLaDAImage: cls=LLaDA2MoeModelLM matmul=disabled override')
@@ -72,6 +78,14 @@ def load_llada_image(checkpoint_info, diffusers_load_config=None):
}
# generation_mode = "text", "vq", "editing"
# restore settings post-load
if sdnq_quantize_weights_mode is not None:
shared.opts.sdnq_quantize_weights_mode = sdnq_quantize_weights_mode
if sdnq_quantize_weights_mode_te is not None:
shared.opts.sdnq_quantize_weights_mode_te = sdnq_quantize_weights_mode_te
if sdnq_quantize_matmul_mode_te is not None:
shared.opts.sdnq_quantize_matmul_mode_te = sdnq_quantize_matmul_mode_te
del transformer, text_encoder
sd_hijack_te.init_hijack(pipe)
sd_hijack_vae.init_hijack(pipe)
pipelines/native_transformer.py
+2 -2
View File
@@ -45,7 +45,7 @@ import os
import json
import time
from dataclasses import dataclass, field
from typing import Callable
from typing import Callable, cast
import huggingface_hub as hf
import torch
@@ -185,7 +185,7 @@ def auto_pickup_converter(cls: type) -> Callable[[dict], dict] | None:
fn = entry.get("checkpoint_mapping_fn")
if fn is None or is_noop_converter(fn):
return None
return fn
return cast('Callable[[dict], dict]', fn) # diffusers' mapping fns vary in signature (extra kwargs/config), all compatible at call sites
def is_noop_converter(fn: Callable) -> bool:
pyproject.toml
+5 -4
View File
@@ -384,9 +384,10 @@ python-version = "3.10"
[tool.ty.src]
include = [
"*.py",
"modules/*.py",
"pipelines/*.py",
"scripts/*.py",
"modules/**/*.py",
"pipelines/**/*.py",
"scripts/**/*.py",
"extensions-builtin/**/*.py"
]
exclude = [
@@ -412,8 +413,8 @@ exclude = [
"modules/teacache",
"modules/face/*.py",
"modules/sub_quadratic_attention.py",
"pipelines/**/*.py",
"scripts/**/*.py",
"pipelines/*/**/*.py",
"scripts/*/**/*.py",
]
[tool.ty.rules]
scripts/differential_diffusion.py
+1 -1
View File
@@ -869,7 +869,7 @@ class StableDiffusionXLDiffImg2ImgPipeline(DiffusionPipeline, FromSingleFileMixi
total_time_steps = num_inference_steps
#end diff diff change
timesteps, num_inference_steps = self.get_timesteps(
num_inference_steps, strength, device, denoising_start=denoising_start if denoising_value_valid else None # pylint: disable=missing-parentheses-for-call-in-test, using-constant-test
num_inference_steps, strength, device, denoising_start=denoising_start if denoising_value_valid(denoising_start) else None
)
latent_timestep = timesteps[:1].repeat(batch_size * num_images_per_prompt)
scripts/dlss/__init__.py
View File
scripts/dlss/controller_cli.py
+229
View File
@@ -0,0 +1,229 @@
import os
import json
import shutil
import base64
import select
import threading
import subprocess
import time
import uuid
import numpy as np
from PIL import Image
from modules.logger import log
from modules.errors import display
debug = os.environ.get('SD_DLSS_DEBUG', None) is not None
def image_to_nchw(image: Image.Image) -> np.ndarray:
arr = np.array(image.convert('RGB'), dtype=np.uint8) # HWC
return np.ascontiguousarray(arr.transpose(2, 0, 1))[np.newaxis, ...] # 1CHW
def images_to_nchw(images: list) -> np.ndarray:
# last image is reference, skip all images that do not have same dimensions as the last image
if len(images) > 1:
ref_size = images[-1].size
images = [image for image in images if image.size == ref_size]
return np.concatenate([image_to_nchw(image) for image in images], axis=0)
def nchw_to_images(arr) -> list:
arr = np.asarray(arr)
return [Image.fromarray(arr[i].transpose(1, 2, 0), 'RGB') for i in range(arr.shape[0])]
def _encode_value(value):
if isinstance(value, np.ndarray):
arr = np.ascontiguousarray(value)
return { '__ndarray__': True, 'dtype': str(arr.dtype), 'shape': list(arr.shape), 'data': base64.b64encode(arr.tobytes()).decode('ascii') }
return value
def _decode_value(value):
if isinstance(value, dict) and value.get('__ndarray__'):
data = base64.b64decode(value['data'])
return np.frombuffer(data, dtype=value['dtype']).reshape(value['shape'])
if isinstance(value, dict):
return { k: _decode_value(v) for k, v in value.items() }
if isinstance(value, list):
return [_decode_value(v) for v in value]
return value
class DLSSController:
"""Persistent stdio bridge to the DLSS package's long-lived controller worker (app/controller.py)."""
def __init__(self):
self.process: subprocess.Popen | None = None
self.pkg_path: str | None = None
self.lock = threading.Lock()
def get_python(self, pkg_path: str):
python_exe = os.path.join(pkg_path, 'bin', 'python-3.13.15-embed-amd64', 'python.exe')
if not os.path.exists(python_exe):
log.error(f'DLSS: path={pkg_path} python={python_exe} not found')
return None
return python_exe
def is_alive(self) -> bool:
return self.process is not None and self.process.poll() is None
def stop(self):
process = self.process
self.process = None
if process is None:
return
try:
if process.poll() is None and process.stdin is not None:
line = json.dumps({ 'request_id': str(uuid.uuid4()), 'command': 'shutdown', 'args': [], 'kwargs': {} }) + '\n'
process.stdin.write(line.encode('utf-8'))
process.stdin.flush()
except Exception:
pass
try:
if process.poll() is None:
process.terminate()
process.wait(timeout=5.0)
except Exception:
pass
def ensure_installed(self, pkg_path: str) -> bool:
if self.is_alive() and self.pkg_path == pkg_path:
return True
self.stop()
python_exe = self.get_python(pkg_path)
if not python_exe:
return False
# create _sdnext directory if it doesn't exist
sdnext_path = os.path.join(pkg_path, '_sdnext')
if not os.path.exists(sdnext_path):
if debug:
log.trace(f'DLSS install: create folder="{sdnext_path}"')
try:
os.makedirs(sdnext_path, exist_ok=True)
except Exception as e:
log.error(f'DLSS install: failed to create folder: {e}')
display(e, 'DLSS')
return False
files_to_copy = ['__init__.py', 'controller_srv.py', 'utils.py', 'verify.py', 'render.py', 'supersample.py', 'framegen.py']
for file_name in files_to_copy:
# src path is current path of this file
src = os.path.join(os.path.dirname(__file__), file_name)
dst = os.path.join(sdnext_path, file_name)
# not exist or newer
if not os.path.exists(dst) or os.path.getmtime(src) > os.path.getmtime(dst):
if debug:
log.trace(f'DLSS install: copy src="{src}" "{dst}"')
try:
shutil.copy2(src, dst)
except Exception as e:
log.error(f'DLSS install: failed to copy {file_name}: {e}')
display(e, 'DLSS')
return False
return True
def ensure_started(self, pkg_path: str) -> bool:
if self.is_alive() and self.pkg_path == pkg_path:
return True
self.stop()
python_exe = self.get_python(pkg_path)
if not python_exe:
return False
env = {
'GRADIO_ANALYTICS_ENABLED': 'False',
'PYTHONNOUSERSITE': '1',
'PYTHONIOENCODING': 'utf-8'
}
if debug:
env['SD_DLSS_DEBUG'] = 'True'
log.trace(f'DLSS controller start: env={env}')
try:
self.process = subprocess.Popen( # pylint: disable=consider-using-with
[python_exe, '-m', '_sdnext.controller_srv'],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
cwd=pkg_path,
env=env,
bufsize=0,
)
except Exception as e:
log.error(f'DLSS controller start: {e}')
display(e, 'DLSS')
self.process = None
return False
self.pkg_path = pkg_path
response = self._send({ 'request_id': str(uuid.uuid4()), 'command': 'status', 'args': [], 'kwargs': {} }, timeout=30.0)
if response is None or response.get('status') != 'ok':
log.error(f'DLSS controller start: response={response}')
self.stop()
return False
if debug:
log.trace(f'DLSS controller start: result={response.get("result")}')
return True
def _send(self, request: dict, timeout: float = 300.0):
process = self.process
if process is None or process.stdin is None or process.stdout is None:
return None
try:
process.stdin.write((json.dumps(request) + '\n').encode('utf-8'))
process.stdin.flush()
except Exception as e:
log.error(f'DLSS: failed to send request: {e}')
display(e, 'DLSS')
self.process = None
return None
deadline = time.time() + timeout
while True:
remaining = deadline - time.time()
if remaining <= 0:
log.error(f'DLSS controller: timeout={timeout}')
return None
try:
ready, _, _ = select.select([process.stdout], [], [], remaining)
except Exception:
ready = [process.stdout] # select() is not supported on pipes on some platforms: fall back to a blocking read
if not ready:
log.error(f'DLSS controller: timeout={timeout}')
return None
try:
raw = process.stdout.readline()
except Exception as e:
log.error(f'DLSS controller read: {e}')
display(e, 'DLSS')
self.process = None
return None
if not raw:
stderr = process.stderr.read().decode('utf-8', errors='ignore') if process.stderr else ''
log.error(f'DLSS controller process: stderr="{stderr.strip()}"')
self.process = None
return None
try:
return json.loads(raw.decode('utf-8'))
except Exception:
if debug:
log.trace(f'DLSS controller stray output: {raw!r}')
continue # skip any non-JSON noise emitted before the JSON response line
def call(self, pkg_path: str, command: str, kwargs: dict, timeout: float = 600.0) -> dict:
with self.lock:
if not self.ensure_installed(pkg_path):
return { 'status': 'error', 'result': None, 'error': { 'code': 'not_installed', 'message': 'controller is not installed' } }
if not self.ensure_started(pkg_path):
return { 'status': 'error', 'result': None, 'error': { 'code': 'not_ready', 'message': 'controller failed to start' } }
encoded_kwargs = { key: _encode_value(value) for key, value in kwargs.items() }
request = { 'request_id': str(uuid.uuid4()), 'command': command, 'args': [], 'kwargs': encoded_kwargs }
if debug:
log.trace(f'DLSS controller request: command={command} timeout={timeout}')
response = self._send(request, timeout=timeout)
if response is None:
return { 'status': 'error', 'result': None, 'error': { 'code': 'not_ready', 'message': 'controller is not responding' } }
if isinstance(response.get('result'), (dict, list)):
response['result'] = _decode_value(response['result'])
return response
controller = DLSSController()
scripts/dlss/controller_srv.py
+400
View File
@@ -0,0 +1,400 @@
from __future__ import annotations
import sys
from pathlib import Path
_ROOT = Path(__file__).resolve().parents[1]
if str(_ROOT) not in sys.path:
sys.path.insert(0, str(_ROOT))
# when launched directly as the stdio bridge subprocess, keep the real stdout clean of stray
# prints from native imports/libraries below so only explicit JSON response lines reach the pipe
_stdio_stdout = None
if __name__ == "__main__":
_stdio_stdout = sys.stdout
sys.stdout = sys.stderr
import base64
import ctypes
import dataclasses
import json
import multiprocessing as mp
import os
import threading
import uuid
from queue import Empty
from typing import Any
import numpy as np
from .utils import StandaloneError, log
from .framegen import DLSSFrameGen, InterpolationOptions
from .render import DLSSNeuralRenderer, RenderOptions
from .supersample import DLSSSuperSample, UpscaleOptions
from .verify import DLSSVerify, VerifyOptions
_SUPPORTED_COMMANDS = {
"status",
"verify",
"render",
"upscale",
"framegen",
"cancel",
"reset",
"shutdown",
}
def _response(request_id: str, *, status: str, result: Any = None, error: dict[str, str] | None = None, diagnostics: dict[str, Any] | None = None) -> dict[str, Any]:
return {
"request_id": request_id,
"status": status,
"result": result,
"error": error,
"diagnostics": diagnostics or {},
}
def _coerce_options(options: Any, *, default: Any, option_type: type[Any]) -> Any:
if options is None:
return default
if isinstance(options, option_type):
return options
if isinstance(options, dict):
return option_type(**options)
raise TypeError(f"Expected {option_type.__name__} or dict, got {type(options).__name__}")
def _set_shared_text(buffer: Any, value: str) -> None:
with buffer.get_lock():
for index in range(len(buffer)):
buffer[index] = "\0"
for index, character in enumerate(value[: len(buffer) - 1]):
buffer[index] = character
def _get_shared_text(buffer: Any) -> str:
with buffer.get_lock():
return "".join(buffer).split("\0", 1)[0]
def _dispatch_command(command: str, request_id: str, args: tuple[Any, ...], kwargs: dict[str, Any], *, busy: Any = None, current_request_id: Any = None, current_command: Any = None) -> dict[str, Any]: # pylint: disable=unused-argument
log.debug(f'DLSSController dispatch: command={command} id={request_id}')
if command == "status":
is_busy = bool(busy.value) if busy is not None else False
active_request_id = _get_shared_text(current_request_id) if current_request_id is not None else ""
active_command = _get_shared_text(current_command) if current_command is not None else ""
return _response(
request_id,
status="ok",
result={
"ready": True,
"busy": is_busy,
"pid": os.getpid(),
"id": active_request_id if is_busy else "",
"job": active_command if is_busy else "idle",
},
diagnostics={"controller": "ready"},
)
if command == "reset":
return _response(request_id, status="ok", result={"reset": True}, diagnostics={"controller": "reset"})
if command == "cancel":
target_id = str(kwargs.get("request_id") or "")
return _response(request_id, status="ok", result={"cancelled": bool(target_id), "target_request_id": target_id}, diagnostics={"controller": "cancelled"})
if command == "verify":
gpu_uuid = str(kwargs.get("gpu_uuid", "auto"))
options = _coerce_options(kwargs.get("options"), default=VerifyOptions(), option_type=VerifyOptions)
result = DLSSVerify()(gpu_uuid, options)
return _response(request_id, status="ok", result={"ok": result.ok, "report": result.to_dict()}, diagnostics={"gpu": result.gpu or {}})
if command == "render":
images = kwargs.get("images")
if images is None:
raise StandaloneError("invalid_arguments", "NeuralRender: missing required images")
options = _coerce_options(kwargs.get("options"), default=RenderOptions(), option_type=RenderOptions)
result = DLSSNeuralRenderer()(np.asarray(images), options)
return _response(request_id, status="ok", result=result, diagnostics={"shape": list(result.shape)})
if command == "upscale":
images = kwargs.get("images")
if images is None:
raise StandaloneError("invalid_arguments", "SuperSample: missing required images")
options = _coerce_options(kwargs.get("options"), default=UpscaleOptions(), option_type=UpscaleOptions)
result = DLSSSuperSample()(np.asarray(images), options)
return _response(request_id, status="ok", result=result, diagnostics={"shape": list(result.shape)})
if command == "framegen":
frames = kwargs.get("frames")
if frames is None:
raise StandaloneError("invalid_arguments", "FrameGen: missing required frames")
source_fps = kwargs.get("source_fps")
target_fps = kwargs.get("target_fps")
if source_fps is None or target_fps is None:
raise StandaloneError("invalid_arguments", "FrameGen: missing source/target FPS")
options = _coerce_options(kwargs.get("options"), default=InterpolationOptions(), option_type=InterpolationOptions)
result = DLSSFrameGen()(np.asarray(frames), source_fps, target_fps, options)
return _response(request_id, status="ok", result=result, diagnostics={"shape": list(result.shape)})
raise StandaloneError("invalid_arguments", f"Controller: unsupported command: {command!r}")
def _controller_worker(request_queue: mp.Queue, response_queue: mp.Queue, busy: Any, current_request_id: Any, current_command: Any) -> None:
worker_lock = threading.Lock()
while True:
try:
request = request_queue.get(timeout=0.25)
except Empty:
continue
if not isinstance(request, dict):
response_queue.put(_response(str(uuid.uuid4()), status="error", error={"code": "invalid_arguments", "message": "Controller request must be a dict."}))
continue
request_id = str(request.get("request_id") or uuid.uuid4())
command = str(request.get("command") or "").strip().lower()
args = tuple(request.get("args", ()))
kwargs = dict(request.get("kwargs", {}))
if command == "shutdown":
log.debug(f'DLSSController shutdown: id={request_id}')
response_queue.put(_response(request_id, status="ok", result={"shutdown": True}, diagnostics={"controller": "shutdown"}))
return
if command not in _SUPPORTED_COMMANDS:
log.warning(f'DLSSController: command={command} id={request_id} unsupported')
response_queue.put(_response(request_id, status="error", error={"code": "invalid_arguments", "message": f"Unsupported command: {command!r}"}, diagnostics={"controller": "invalid_command"}))
continue
try:
if command != "status":
busy.value = True
_set_shared_text(current_request_id, request_id)
_set_shared_text(current_command, command)
with worker_lock:
response = _dispatch_command(command, request_id, args, kwargs, busy=busy, current_request_id=current_request_id, current_command=current_command)
response_queue.put(response)
except StandaloneError as exc:
log.error(f'DLSSController: StandaloneError command={command} id={request_id} code={exc.code} message={exc.message}')
response_queue.put(_response(request_id, status="error", error={"code": exc.code, "message": exc.message}, diagnostics={"controller": "error"}))
except Exception as exc: # pragma: no cover - defensive catch for controller safety
log.error(f'DLSSController: unexpected exception command={command} id={request_id} error={exc}')
response_queue.put(_response(request_id, status="error", error={"code": "processing_failed", "message": str(exc)}, diagnostics={"controller": "error"}))
finally:
if command != "status":
busy.value = False
_set_shared_text(current_request_id, "")
_set_shared_text(current_command, "")
class ControllerProcess(mp.Process):
def __init__(self, request_queue: mp.Queue | None = None, response_queue: mp.Queue | None = None, *, busy: Any = None, current_request_id: Any = None, current_command: Any = None, ctx: mp.context.BaseContext | None = None) -> None:
self.ctx = ctx or mp.get_context("spawn")
self.request_queue = request_queue or self.ctx.Queue()
self.response_queue = response_queue or self.ctx.Queue()
self.busy = busy or self.ctx.Value("b", False)
self.current_request_id = current_request_id or self.ctx.Array(ctypes.c_wchar, 256)
self.current_command = current_command or self.ctx.Array(ctypes.c_wchar, 64)
super().__init__(target=_controller_worker, args=(self.request_queue, self.response_queue, self.busy, self.current_request_id, self.current_command))
class ControllerClient:
"""Simple client wrapper for callers that want a long-lived controller process."""
def __init__(self, request_queue: mp.Queue | None = None, response_queue: mp.Queue | None = None, *, process: ControllerProcess | None = None, timeout: float = 30.0, ctx: mp.context.BaseContext | None = None) -> None:
self.ctx = ctx or mp.get_context("spawn")
self.request_queue = request_queue or self.ctx.Queue()
self.response_queue = response_queue or self.ctx.Queue()
self.timeout = timeout
self.process = process
self.busy = process.busy if process is not None else self.ctx.Value("b", False)
self.current_request_id = process.current_request_id if process is not None else self.ctx.Array(ctypes.c_wchar, 256)
self.current_command = process.current_command if process is not None else self.ctx.Array(ctypes.c_wchar, 64)
self._pending: dict[str, dict[str, Any]] = {}
def start(self) -> "ControllerClient":
if self.process is None or not self.process.is_alive():
self.process = ControllerProcess(self.request_queue, self.response_queue, busy=self.busy, current_request_id=self.current_request_id, current_command=self.current_command, ctx=self.ctx)
self.process.start()
log.info(f'DLSSController: pid={self.process.pid} started')
return self
def _send_and_wait(self, command: str, *args: Any, **kwargs: Any) -> dict[str, Any]:
if self.process is None or not self.process.is_alive():
self.start()
request_id = str(uuid.uuid4())
request = {"request_id": request_id, "command": command, "args": list(args), "kwargs": kwargs}
self.request_queue.put(request)
while True:
try:
response = self.response_queue.get(timeout=self.timeout)
except Empty as exc:
raise TimeoutError(f"Controller request timed out for command {command!r}.") from exc
if response.get("request_id") == request_id:
return response
self._pending[response.get("request_id", str(uuid.uuid4()))] = response
def status(self) -> dict[str, Any]:
if self.process is None or not self.process.is_alive():
self.start()
is_busy = bool(self.busy.value)
active_request_id = _get_shared_text(self.current_request_id)
active_command = _get_shared_text(self.current_command)
return _response(
str(uuid.uuid4()),
status="ok",
result={
"ready": self.process.is_alive(),
"busy": is_busy,
"pid": self.process.pid,
"id": active_request_id if is_busy else "",
"job": active_command if is_busy else "idle",
},
diagnostics={"controller": "busy" if is_busy else "ready"},
)
def verify(self, *, gpu_uuid: str = "auto", options: VerifyOptions | dict[str, Any] | None = None) -> dict[str, Any]:
return self._send_and_wait("verify", gpu_uuid=gpu_uuid, options=options)
def render(self, *, images: np.ndarray, options: RenderOptions | dict[str, Any] | None = None) -> dict[str, Any]:
return self._send_and_wait("render", images=images, options=options)
def upscale(self, *, images: np.ndarray, options: UpscaleOptions | dict[str, Any] | None = None) -> dict[str, Any]:
return self._send_and_wait("upscale", images=images, options=options)
def framegen(self, *, frames: np.ndarray, source_fps: float | str, target_fps: float | str, options: InterpolationOptions | dict[str, Any] | None = None) -> dict[str, Any]:
return self._send_and_wait("framegen", frames=frames, source_fps=source_fps, target_fps=target_fps, options=options)
def cancel(self, request_id: str) -> dict[str, Any]:
return self._send_and_wait("cancel", request_id=request_id)
def reset(self) -> dict[str, Any]:
return self._send_and_wait("reset")
def shutdown(self) -> dict[str, Any]:
if self.process is None or not self.process.is_alive():
return {"request_id": "shutdown", "status": "ok", "result": {"shutdown": True}, "error": None, "diagnostics": {}}
response = self._send_and_wait("shutdown")
if self.process.is_alive():
self.process.join(timeout=10.0)
return response
def close(self) -> None:
try:
self.shutdown()
except Exception:
pass
if self.process is not None and self.process.is_alive():
self.process.terminate()
self.process.join(timeout=10.0)
def __enter__(self) -> "ControllerClient":
return self.start()
def __exit__(self, exc_type: Any, exc: Any, tb: Any) -> None:
self.close()
def start_controller(*, request_queue: mp.Queue | None = None, response_queue: mp.Queue | None = None, timeout: float = 30.0, ctx: mp.context.BaseContext | None = None) -> ControllerClient:
client = ControllerClient(request_queue=request_queue, response_queue=response_queue, timeout=timeout, ctx=ctx)
return client.start()
# ---- stdio bridge -----------------------------------------------------------------
# Used only when this module is launched directly as a subprocess, e.g.
# `python.exe app/controller.py`, to drive the same dispatch logic over a single
# stdin/stdout JSON-lines protocol instead of multiprocessing queues. This allows an
# external caller running a different Python interpreter (for example WSL/Linux Python
# invoking the packaged Windows embedded python.exe) to reuse one long-lived worker.
def _stdio_encode(value: Any) -> Any:
if isinstance(value, np.ndarray):
arr = np.ascontiguousarray(value)
return {"__ndarray__": True, "dtype": str(arr.dtype), "shape": list(arr.shape), "data": base64.b64encode(arr.tobytes()).decode("ascii")}
if hasattr(value, "to_dict") and callable(value.to_dict):
return _stdio_encode(value.to_dict())
if dataclasses.is_dataclass(value) and not isinstance(value, type):
return {k: _stdio_encode(v) for k, v in dataclasses.asdict(value).items()}
if isinstance(value, dict):
return {k: _stdio_encode(v) for k, v in value.items()}
if isinstance(value, (list, tuple)):
return [_stdio_encode(v) for v in value]
return value
def _stdio_decode(value: Any) -> Any:
if isinstance(value, dict) and value.get("__ndarray__"):
data = base64.b64decode(value["data"])
return np.frombuffer(data, dtype=value["dtype"]).reshape(value["shape"])
if isinstance(value, dict):
return {k: _stdio_decode(v) for k, v in value.items()}
if isinstance(value, list):
return [_stdio_decode(v) for v in value]
return value
def _stdio_write(payload: dict[str, Any]) -> None:
_stdio_stdout.write(json.dumps(payload) + "\n")
_stdio_stdout.flush()
def _stdio_error(request_id: str, code: str, message: str) -> dict[str, Any]:
return {"request_id": request_id, "status": "error", "result": None, "error": {"code": code, "message": message}, "diagnostics": {}}
def _stdio_main() -> None:
# long-lived worker: one JSON request per line on stdin, one JSON response per line on the real stdout
for raw_line in sys.stdin:
line = raw_line.strip()
if not line:
continue
try:
request = json.loads(line)
except json.JSONDecodeError as exc:
_stdio_write(_stdio_error("", "invalid_arguments", f"Malformed request: {exc}"))
continue
request_id = str(request.get("request_id") or "")
command = str(request.get("command") or "").strip().lower()
args = tuple(request.get("args", ()))
try:
kwargs = _stdio_decode(dict(request.get("kwargs", {})))
except Exception as exc:
_stdio_write(_stdio_error(request_id, "invalid_arguments", f"Failed to decode request payload: {exc}"))
continue
if command == "shutdown":
_stdio_write({"request_id": request_id, "status": "ok", "result": {"shutdown": True}, "error": None, "diagnostics": {}})
return
if command not in _SUPPORTED_COMMANDS:
_stdio_write(_stdio_error(request_id, "invalid_arguments", f"Unsupported command: {command!r}"))
continue
try:
response = _dispatch_command(command, request_id, args, kwargs)
except StandaloneError as exc:
response = _stdio_error(request_id, exc.code, exc.message)
except Exception as exc: # pragma: no cover - defensive catch for controller safety
response = _stdio_error(request_id, "processing_failed", str(exc))
_stdio_write(_stdio_encode(response))
if __name__ == "__main__":
_stdio_main()
def __enter__(self) -> "ControllerClient":
return self.start()
def __exit__(self, exc_type: Any, exc: Any, tb: Any) -> None: # pylint: disable=unused-argument
self.close()
__all__ = [
"ControllerClient",
"ControllerProcess",
"start_controller",
]
scripts/dlss/framegen.py
+204
View File
@@ -0,0 +1,204 @@
from __future__ import annotations
from dataclasses import dataclass
from fractions import Fraction
from typing import Any
import numpy as np
from src.core.jobs import JobController, active_job
from src.frame_interpolation.capabilities import probe_frame_interpolation_capabilities
from src.frame_interpolation.guides import DLSSGGuideGenerator
from src.frame_interpolation.models import ENGINE_CHOICES, resolve_target_rate
from src.frame_interpolation.native import DirectDLSSGSession
from src.frame_interpolation.scheduler import choose_interpolation_plan, output_frame_count
from .utils import StandaloneError, nchw_image_to_hwc, rgba_to_rgb_nchw, validate_nchw, rgb_to_rgba, log
@dataclass(frozen=True, slots=True)
class InterpolationOptions:
ai_gpu_uuid: str = "auto"
engine: str = "Auto"
def validate(self) -> None:
if self.engine not in ENGINE_CHOICES:
raise ValueError(f"Unknown frame interpolation engine: {self.engine!r}.")
@dataclass(frozen=True, slots=True)
class _TimedFrame:
rgba: np.ndarray
timestamp: Fraction
class _Stage:
def __init__(self, session: DirectDLSSGSession, width: int, height: int) -> None:
self.session = session
self.guides = DLSSGGuideGenerator(width, height)
self.previous: _TimedFrame | None = None
def push(self, frame: _TimedFrame) -> list[_TimedFrame]:
previous = self.previous
guide = self.guides.process(frame.rgba, force_reset=previous is not None and frame.timestamp <= previous.timestamp)
self.previous = frame
generated = self.session.process_frame(
frame.rgba,
guide.motion,
frame.timestamp,
reset=previous is None or guide.reset,
)
result: list[_TimedFrame] = []
if previous is not None and not guide.reset:
interval = frame.timestamp - previous.timestamp
count = len(generated)
for index, rgba in enumerate(generated, start=1):
result.append(_TimedFrame(
rgba,
previous.timestamp + interval * Fraction(index, count + 1),
))
result.append(frame)
return result
class DLSSFrameGen:
"""In-memory RGB NCHW constant-frame-rate DLSS frame interpolation."""
def __init__(self) -> None:
log.info('DLSSFrameGen: init')
self.diagnostics: dict[str, Any] = {}
self.last_report: dict[str, Any] = {}
def __call__(
self,
frames: np.ndarray,
source_fps: str | int | float | Fraction,
target_fps: str | int | float | Fraction,
options: InterpolationOptions | None = None,
*,
controller: JobController | None = None,
) -> np.ndarray:
log.info('DLSSFrameGen: call')
options = options or InterpolationOptions()
options.validate()
batch, _, height, width = validate_nchw(frames, name="frames")
if width < 64 or height < 64:
raise StandaloneError("invalid_dimensions", "FrameGen: invalid resolution")
source_rate = resolve_target_rate(source_fps)
target_rate = resolve_target_rate(target_fps)
own_controller = controller or JobController()
log.debug(f'DLSSFrameGen: controller={own_controller}')
try:
with active_job(own_controller) as active_controller:
capabilities = probe_frame_interpolation_capabilities(options.ai_gpu_uuid)
log.debug(f'DLSSFrameGen: capabilities={capabilities}')
if not capabilities.available:
raise StandaloneError("feature_unavailable", "FrameGen: unavailable. " + capabilities.detail,
)
plan = choose_interpolation_plan(
source_rate,
target_rate,
options.engine,
capabilities.native_multiplier,
cfr=True,
)
source_frames = [
_TimedFrame(rgb_to_rgba(nchw_image_to_hwc(frames, index, name="frames")), Fraction(index, 1) / source_rate)
for index in range(batch)
]
if plan.generated_per_interval == 0:
result = self._resample_source(source_frames, target_rate, source_rate)
else:
result = self._generate(source_frames, plan, active_controller, width, height)
expected = output_frame_count(Fraction(batch, 1) / source_rate, target_rate)
if len(result) != expected:
log.error(f'DLSSFrameGen: result length={len(result)} expected={expected}')
raise StandaloneError("invalid_native_output", f"FrameGen: interpolation produced {len(result)} frames; expected {expected}.")
output = np.stack([rgba_to_rgb_nchw(item.rgba)[0] for item in result], axis=0)
log.debug(f'DLSSFrameGen: output={output.shape}')
self.diagnostics = {
"gpu": capabilities.gpu,
"driver": capabilities.driver,
"runtime_version": capabilities.runtime_version,
"worker_version": capabilities.worker_version,
"selected_path": plan.path,
"native_multiplier": plan.native_multiplier,
"cascade_stages": plan.cascade_stages,
}
except StandaloneError:
raise
except Exception as exc:
log.error(f'DLSSFrameGen: unexpected exception {exc}')
raise StandaloneError("processing_failed", f"FrameGen: failed: {exc}") from exc
self.last_report = {
"input_shape": tuple(frames.shape),
"output_shape": tuple(output.shape),
"source_fps": str(source_rate),
"target_fps": str(target_rate),
}
return np.ascontiguousarray(output)
@staticmethod
def _resample_source(frames: list[_TimedFrame], target_rate: Fraction, source_rate: Fraction) -> list[_TimedFrame]:
count = output_frame_count(Fraction(len(frames), 1) / source_rate, target_rate)
result: list[_TimedFrame] = []
for index in range(count):
ideal = Fraction(index, 1) / target_rate
selected = min(frames, key=lambda frame, target=ideal: abs(frame.timestamp - target))
result.append(_TimedFrame(selected.rgba.copy(), ideal))
return result
@staticmethod
def _generate(source_frames, plan, controller, width, height) -> list[_TimedFrame]:
sessions: list[DirectDLSSGSession] = []
stages: list[_Stage] = []
try:
stage_count = plan.cascade_stages or 1
for stage_index in range(stage_count):
generated_count = (
plan.generated_per_interval
if plan.path == "Native DLSSG"
else 1
)
expected_frames = (
len(source_frames)
if stage_index == 0
else max(1, (len(source_frames) - 1) * (1 << stage_index) + 1)
)
session = DirectDLSSGSession(
width,
height,
expected_frames,
generated_count,
controller,
)
sessions.append(session)
stages.append(_Stage(session, width, height))
candidates: list[_TimedFrame] = []
for source in source_frames:
if controller.cancel.is_set():
raise StandaloneError("cancelled", "FrameGen: cancelled")
items = [source]
for stage in stages:
next_items: list[_TimedFrame] = []
for item in items:
next_items.extend(stage.push(item))
items = next_items
candidates.extend(items)
duration = Fraction(len(source_frames), 1) / plan.source_rate
count = output_frame_count(duration, plan.target_rate)
result: list[_TimedFrame] = []
for index in range(count):
ideal = Fraction(index, 1) / plan.target_rate
selected = min(candidates, key=lambda frame, target=ideal: abs(frame.timestamp - target))
result.append(_TimedFrame(selected.rgba.copy(), ideal))
return result
finally:
for session in reversed(sessions):
try:
session.close()
except (OSError, RuntimeError, ValueError):
session.abort()
__all__ = ["DLSSFrameGen", "InterpolationOptions"]
scripts/dlss/render.py
+164
View File
@@ -0,0 +1,164 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Any
import numpy as np
from src.core.jobs import JobController, active_job
from src.core.runtime import (
DLSSFrameSession,
prepare_runtime,
resolve_native_settings,
resolve_output_size,
resolve_runtime_ai_gpu,
resolve_upscaling_mode,
resize_fit,
)
from src.neural_rendering.image.models import ImageConversionOptions
from .utils import StandaloneError, nchw_image_to_hwc, rgba_to_rgb_nchw, validate_nchw, rgb_to_rgba, log
@dataclass(frozen=True, slots=True)
class RenderOptions:
ai_gpu_uuid: str = "auto"
nr_style: str = "Default"
nr_intensity: float = 1.0
local_tone_strength: float = 1.0
local_structure_strength: float = 1.0
skin_structure_strength: float = -1.0
upscaling_factor: float = 1.0
warmup_frames: int = 0
nr_preset: str = "Default"
automatic_mask: bool = False
dlss_model_preset: str = "Default"
def source_options(self) -> ImageConversionOptions:
return ImageConversionOptions(
ai_gpu_uuid=self.ai_gpu_uuid,
nr_style=self.nr_style,
nr_intensity=self.nr_intensity,
local_tone_strength=self.local_tone_strength,
local_structure_strength=self.local_structure_strength,
skin_structure_strength=self.skin_structure_strength,
upscaling_factor=self.upscaling_factor,
warmup_frames=self.warmup_frames,
nr_preset=self.nr_preset,
automatic_mask=self.automatic_mask,
dlss_model_preset=self.dlss_model_preset,
)
def validate(self) -> None:
if isinstance(self.warmup_frames, bool) or not isinstance(self.warmup_frames, int) or self.warmup_frames < 0:
raise ValueError("warmup_frames must be a non-negative integer.")
if not isinstance(self.automatic_mask, bool):
raise ValueError("automatic_mask must be a boolean.")
options = self.source_options()
resolve_upscaling_mode(options.upscaling_factor)
resolve_native_settings(options)
class DLSSNeuralRenderer:
"""RGB NCHW adapter for still-image DLSS Neural Rendering."""
def __init__(self) -> None:
log.info('DLSSNeuralRenderer: init')
self.diagnostics: dict[str, Any] = {}
self.last_report: dict[str, Any] = {}
def __call__(
self,
images: np.ndarray,
options: RenderOptions | None = None,
*,
controller: JobController | None = None,
) -> np.ndarray:
log.info('DLSSNeuralRenderer: call')
options = options or RenderOptions()
options.validate()
batch, _channels, height, width = validate_nchw(images, name="images")
log.debug(f'DLSSNeuralRenderer: input={images.shape}')
if width < 64 or height < 64:
raise StandaloneError("invalid_dimensions", "NeuralRender: invalid resolution")
output_width, output_height = resolve_output_size(width, height, options.upscaling_factor)
own_controller = controller or JobController()
log.debug(f'DLSSNeuralRenderer: controller={own_controller}')
outputs: list[np.ndarray] = []
try:
with active_job(own_controller) as active_controller:
prepared = prepare_runtime()
log.debug(f'DLSSNeuralRenderer: runtime={prepared}')
gpu = resolve_runtime_ai_gpu(prepared.gpus, prepared.runtime_bundle, options.ai_gpu_uuid)
log.debug(f'DLSSNeuralRenderer: gpu={gpu}')
factor, mode = resolve_upscaling_mode(options.upscaling_factor)
native_settings = resolve_native_settings(options.source_options())
session_diagnostics: list[dict[str, Any]] = []
session = DLSSFrameSession(
input_width=width,
input_height=height,
output_width=output_width,
output_height=output_height,
frame_count=batch,
warmup_frames=options.warmup_frames,
factor=factor,
mode=mode,
native_settings=native_settings,
gpu=gpu,
runtime_bundle=prepared.runtime_bundle,
controller=active_controller,
)
log.debug(f'DLSSNeuralRenderer: session={session}')
for index in range(batch):
if active_controller.cancel.is_set():
raise StandaloneError("cancelled", "NeuralRender: cancelled.")
try:
rgb = nchw_image_to_hwc(images, index, name="images")
rgba = rgb_to_rgba(rgb)
render_input = resize_fit(rgba, session.render_width, session.render_height)
motion = np.zeros((session.render_height, session.render_width, 2), dtype=np.float16)
log.debug(f'DLSSNeuralRenderer: index={index} processes={render_input.shape}')
processed, _ = session.process(
index=index,
rgba=render_input,
motion=motion,
reset=True,
pts=0,
)
log.debug(f'DLSSNeuralRenderer: index={index} processed={processed.shape}')
outputs.append(rgba_to_rgb_nchw(processed)[0])
session_diagnostics.append({
"render_width": session.render_width,
"render_height": session.render_height,
"applied_dlss_model_preset": session.applied_dlss_model_preset,
"worker_logs": session.worker_logs,
"completed_frames": session.completed_frames,
})
for l in session.worker_logs or []:
log.debug(f'DLSSNeuralRenderer worker: {l}')
except Exception as e:
log.error(f'DLSSNeuralRenderer: exception {e}')
if session is not None and not session.closed:
session.abort()
raise
session.close()
self.diagnostics = {
"gpu": dict(gpu),
"runtime_bundle": prepared.runtime_bundle,
"sessions": session_diagnostics,
}
except StandaloneError:
raise
except Exception as exc:
log.error(f'DLSSNeuralRenderer: unexpected exception {exc}')
raise StandaloneError("processing_failed", f"NeuralRender failed: {exc}") from exc
result = np.ascontiguousarray(np.stack(outputs, axis=0))
self.last_report = {
"input_shape": tuple(images.shape),
"output_shape": tuple(result.shape),
"completed_images": batch,
}
return result
__all__ = ["DLSSNeuralRenderer", "RenderOptions"]
scripts/dlss/supersample.py
+117
View File
@@ -0,0 +1,117 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Any
import numpy as np
from src.core.jobs import JobController, active_job
from src.upscale.image.models import ImageUpscaleOptions, output_size as source_output_size
from src.upscale.video.models import UpscaleOptions as NativeUpscaleOptions
from src.upscale.video.native import RTXVideoSession, probe_capabilities
from .utils import StandaloneError, nchw_image_to_hwc, hwc_to_nchw, validate_nchw, srgb_to_worker, worker_to_srgb_rgb, log
@dataclass(frozen=True, slots=True)
class UpscaleOptions:
vsr_quality: int = 4
size_mode: str = "Scale factor"
scale_factor: float = 2.0
width: int = 3840
height: int = 2160
aspect_lock: bool = True
ai_gpu_uuid: str = "auto"
def source_options(self) -> ImageUpscaleOptions:
return ImageUpscaleOptions(
vsr_quality=self.vsr_quality,
size_mode=self.size_mode,
scale_factor=self.scale_factor,
width=self.width,
height=self.height,
aspect_lock=self.aspect_lock,
ai_gpu_uuid=self.ai_gpu_uuid,
)
def validate(self) -> None:
source = self.source_options()
source.validate()
class DLSSSuperSample:
"""RGB NCHW adapter for the native RTX Video Super Resolution worker."""
def __init__(self) -> None:
log.info('DLSSSuperSample: init')
self.diagnostics: dict[str, Any] = {}
self.last_report: dict[str, Any] = {}
def __call__(
self,
images: np.ndarray,
options: UpscaleOptions | None = None,
*,
controller: JobController | None = None,
) -> np.ndarray:
log.info('DLSSSuperSample: call')
options = options or UpscaleOptions()
options.validate()
batch, _, height, width = validate_nchw(images, name="images")
source = options.source_options()
output_width, output_height = source_output_size(width, height, source)
own_controller = controller or JobController()
log.debug(f'DLSSSuperSample: controller={own_controller}')
outputs: list[np.ndarray] = []
try:
with active_job(own_controller) as active_controller:
capabilities = probe_capabilities(options.ai_gpu_uuid, controller=active_controller)
log.debug(f'DLSSSuperSample: capabilities={capabilities}')
native_options = NativeUpscaleOptions(
vsr_enabled=True,
vsr_quality=int(options.vsr_quality),
ai_gpu_uuid=options.ai_gpu_uuid,
)
native_options.validate(for_render=False)
with RTXVideoSession(
width,
height,
output_width,
output_height,
native_options,
1,
capabilities,
active_controller,
) as session:
log.debug(f'DLSSSuperSample: session={session}')
for index in range(batch):
if active_controller.cancel.is_set():
raise StandaloneError("cancelled", "SuperSample: cancelled.")
frame = nchw_image_to_hwc(images, index, name="images")
worker_input = srgb_to_worker(frame)
worker_output = session.process_frame(worker_input)
rgb = worker_to_srgb_rgb(worker_output, output_width, output_height)
log.debug(f'DLSSSuperSample: processed={rgb.shape}')
outputs.append(hwc_to_nchw(rgb, name="upscaled RGB output")[0])
self.diagnostics = {
"gpu": dict(capabilities.gpu),
"sdk_version": capabilities.sdk_version,
"worker_version": capabilities.worker_version,
"completed_frames": session.completed_frames,
"last_results": session.last_results,
}
except StandaloneError:
raise
except Exception as exc:
log.error(f'DLSSSuperSample: unexpected exception {exc}')
raise StandaloneError("processing_failed", f"SuperSample: failed: {exc}") from exc
result = np.ascontiguousarray(np.stack(outputs, axis=0))
self.last_report = {
"input_shape": tuple(images.shape),
"output_shape": tuple(result.shape),
"completed_images": batch,
}
return result
__all__ = ["DLSSSuperSample", "UpscaleOptions"]
scripts/dlss/utils.py
+118
View File
@@ -0,0 +1,118 @@
from __future__ import annotations
import logging
import numpy as np
MAX_DIMENSION = 16_384
logging.getLogger().handlers.clear()
logging.basicConfig(
level=logging.DEBUG,
filename='dlss.log',
encoding='utf-8',
filemode='a',
format='%(asctime)s %(levelname)s %(message)s',
# datefmt='%Y-%m-%d %H:%M:%S-%f',
force=True,
)
log = logging.getLogger(__name__)
log.debug('DLSSInit')
class StandaloneError(RuntimeError):
"""Base error with a stable machine-readable code."""
def __init__(self, code: str, message: str) -> None:
super().__init__(message)
self.code = code
self.message = message
class InvalidArrayError(StandaloneError):
def __init__(self, message: str) -> None:
super().__init__("invalid_array", message)
class VerificationError(StandaloneError):
def __init__(self, message: str) -> None:
super().__init__("verification_failed", message)
class ProcessingError(StandaloneError):
def __init__(self, message: str, *, code: str = "processing_failed") -> None:
super().__init__(code, message)
def validate_nchw(array: np.ndarray, *, name: str = "array") -> tuple[int, int, int, int]:
if not isinstance(array, np.ndarray):
raise InvalidArrayError(f"{name} must be a NumPy array.")
if array.ndim != 4:
raise InvalidArrayError(f"{name} must have shape (N, 3, H, W); got {array.shape}.")
batch, channels, height, width = array.shape
if batch < 1:
raise InvalidArrayError(f"{name} must contain at least one image.")
if channels != 3:
raise InvalidArrayError(f"{name} must contain RGB data with C=3; got C={channels}.")
if not 1 <= height <= MAX_DIMENSION or not 1 <= width <= MAX_DIMENSION:
raise InvalidArrayError(
f"{name} dimensions must be between 1 and {MAX_DIMENSION}; got {width}x{height}."
)
if array.dtype != np.uint8:
raise InvalidArrayError(f"{name} must use dtype uint8; got {array.dtype}.")
return batch, channels, height, width
def copy_nchw(array: np.ndarray, *, name: str = "array") -> np.ndarray:
validate_nchw(array, name=name)
return np.ascontiguousarray(array.copy())
def nchw_image_to_hwc(array: np.ndarray, index: int = 0, *, name: str = "array") -> np.ndarray:
batch, _, _, _ = validate_nchw(array, name=name)
if not 0 <= index < batch:
raise InvalidArrayError(f"{name} image index {index} is outside batch size {batch}.")
return np.ascontiguousarray(array[index].transpose(1, 2, 0))
def hwc_to_nchw(array: np.ndarray, *, name: str = "image") -> np.ndarray:
if not isinstance(array, np.ndarray) or array.ndim != 3 or array.shape[2] != 3:
raise InvalidArrayError(f"{name} must have HWC RGB shape (H, W, 3); got {getattr(array, 'shape', None)}.")
if array.dtype != np.uint8:
raise InvalidArrayError(f"{name} must use dtype uint8; got {array.dtype}.")
return np.ascontiguousarray(array.transpose(2, 0, 1)[None, ...])
def rgb_to_rgba(array: np.ndarray) -> np.ndarray:
"""Add opaque alpha only at the private native-worker boundary."""
if array.ndim != 3 or array.shape[2] != 3 or array.dtype != np.uint8:
raise InvalidArrayError("Native RGB input must have HWC uint8 shape with three channels.")
result = np.empty((*array.shape[:2], 4), dtype=np.uint8)
result[..., :3] = array
result[..., 3] = 255
return np.ascontiguousarray(result)
def rgba_to_rgb_nchw(array: np.ndarray) -> np.ndarray:
if array.ndim != 3 or array.shape[2] != 4 or array.dtype != np.uint8:
raise InvalidArrayError("Native RGBA output must have HWC uint8 shape with four channels.")
return hwc_to_nchw(np.ascontiguousarray(array[..., :3]), name="native RGB output")
def srgb_to_worker(rgb: np.ndarray) -> np.ndarray:
"""Convert HWC sRGB RGB data to the RTX Video worker's gamma-2.2 RGBA data."""
if rgb.ndim != 3 or rgb.shape[2] != 3 or rgb.dtype != np.uint8:
raise InvalidArrayError("RGB input must have HWC uint8 shape with three channels.")
rgba = rgb_to_rgba(rgb)
values = rgba[..., :3].astype(np.float32) / 255.0
linear = np.where(values <= 0.04045, values / 12.92, ((values + 0.055) / 1.055) ** 2.4)
rgba[..., :3] = np.rint(np.clip(linear, 0.0, 1.0) ** (1.0 / 2.2) * 255.0).astype(np.uint8) # pylint: disable=unsupported-assignment-operation
return np.ascontiguousarray(rgba)
def worker_to_srgb_rgb(data: bytes | bytearray | memoryview, width: int, height: int) -> np.ndarray:
"""Convert packed worker RGBA output to an HWC sRGB RGB array."""
rgba = np.frombuffer(data, dtype=np.uint8).reshape(height, width, 4).copy()
values = (rgba[..., :3].astype(np.float32) / 255.0) ** 2.2
rgb = np.where(values <= 0.0031308, values * 12.92, 1.055 * values ** (1.0 / 2.4) - 0.055)
rgba[..., :3] = np.rint(np.clip(rgb, 0.0, 1.0) * 255.0).astype(np.uint8)
return np.ascontiguousarray(rgba[..., :3])
scripts/dlss/verify.py
+201
View File
@@ -0,0 +1,201 @@
from __future__ import annotations
import importlib.util
import platform
import sys
from dataclasses import dataclass, field
from typing import Any
from src.core.gpu_selection import resolve_ai_gpu
from src.core.gpu_detection import detect_gpus
from src.core.paths import ADDON, DLSS_SUPERRES, FFMPEG, FFPROBE, HOST_DXGI, NEURAL_RUNTIME, WORKER
from src.core.runtime import validate_runtime_files
from .utils import log
@dataclass(frozen=True, slots=True)
class VerifyOptions:
level: str = "basic"
check_neural: bool = True
check_upscale: bool = True
check_interpolation: bool = True
def validate(self) -> None:
if self.level not in {"basic", "deep"}:
raise ValueError("Verification level must be 'basic' or 'deep'.")
for name in ("check_neural", "check_upscale", "check_interpolation"):
if not isinstance(getattr(self, name), bool):
raise ValueError(f"{name} must be a boolean.")
@dataclass(frozen=True, slots=True)
class VerificationCheck:
name: str
passed: bool
detail: str = ""
def to_dict(self) -> dict[str, Any]:
log.debug(f'DLSSVerify: check="{self.name}" passed={self.passed} detail="{self.detail}"')
result = {"name": self.name, "passed": self.passed, "detail": self.detail}
return result
@dataclass(frozen=True, slots=True)
class VerificationReport:
ok: bool
level: str
python: str
platform: str
gpu: dict[str, Any] | None
paths: dict[str, str]
checks: tuple[VerificationCheck, ...]
diagnostics: tuple[str, ...] = field(default_factory=tuple)
@property
def failed(self) -> tuple[VerificationCheck, ...]:
return tuple(check for check in self.checks if not check.passed)
def to_dict(self) -> dict[str, Any]:
log.info(f'DLSSVerify: level="{self.level}" python="{self.python}" platform="{self.platform}"')
log.info(f'DLSSVerify: paths={self.paths}')
log.info(f'DLSSVerify: gpu={self.gpu}')
log.debug(f'DLSSVerify: diagnostics={self.diagnostics}')
return {
"ok": self.ok,
"python": self.python,
"platform": self.platform,
"gpu": self.gpu,
"paths": self.paths,
"checks": [check.to_dict() for check in self.checks],
"diagnostics": self.diagnostics,
}
_RUNTIME_PATHS = {
"ffmpeg": FFMPEG,
"ffprobe": FFPROBE,
"worker": WORKER,
"host_dxgi": HOST_DXGI,
"dlss_addon": ADDON,
"dlss_superres": DLSS_SUPERRES,
"dlss_neural": NEURAL_RUNTIME,
}
class DLSSVerify:
"""Perform side-effect-free runtime preflight checks by default."""
def __init__(self) -> None:
self.last_report: VerificationReport | None = None
def __call__(self, gpu_uuid: str = "auto", options: VerifyOptions | None = None) -> VerificationReport:
options = options or VerifyOptions()
options.validate()
checks: list[VerificationCheck] = []
diagnostics: list[str] = []
selected_gpu: dict[str, Any] | None = None
checks.extend(self._check_files())
checks.append(self._check_import("numpy"))
checks.append(self._check_import("PIL"))
checks.append(self._check_import("cv2"))
checks.append(self._check_import("av"))
try:
gpus = detect_gpus()
selected_gpu = resolve_ai_gpu(gpus, gpu_uuid)
checks.append(VerificationCheck("gpu", True, self._gpu_detail(selected_gpu)))
except (OSError, RuntimeError, ValueError) as exc:
checks.append(VerificationCheck("gpu", False, str(exc)))
diagnostics.append(str(exc))
try:
validate_runtime_files()
checks.append(VerificationCheck("runtime", True, "Required runtime files are present."))
except (OSError, RuntimeError, ValueError) as exc:
checks.append(VerificationCheck("runtime", False, str(exc)))
diagnostics.append(str(exc))
if options.level == "deep":
checks.extend(self._deep_checks(options, gpu_uuid, selected_gpu))
else:
checks.append(VerificationCheck("deep_capabilities", True, "Deep capability checks were not requested."))
filtered = tuple(check for check in checks if self._feature_enabled(check.name, options))
report = VerificationReport(
ok=all(check.passed or check.status == "not_run" for check in filtered),
level=options.level,
python=platform.python_version(),
platform=sys.platform,
gpu=selected_gpu,
paths={name: str(path) for name, path in _RUNTIME_PATHS.items()},
checks=filtered,
diagnostics=tuple(diagnostics),
)
self.last_report = report
return report
@staticmethod
def _check_files() -> list[VerificationCheck]:
return [
VerificationCheck(
name=f"file:{name}",
passed=path.is_file(),
detail=str(path),
)
for name, path in _RUNTIME_PATHS.items()
]
@staticmethod
def _check_import(name: str) -> VerificationCheck:
available = importlib.util.find_spec(name) is not None
return VerificationCheck(
name=f"dependency:{name}",
passed=available,
detail="available" if available else "not installed",
)
@staticmethod
def _gpu_detail(gpu: dict[str, Any]) -> str:
return f"{gpu.get('name', 'NVIDIA GPU')} driver={gpu.get('driver', 'unknown')} uuid={gpu.get('uuid', 'unknown')}"
@staticmethod
def _feature_enabled(name: str, options: VerifyOptions) -> bool:
if name.startswith("neural:"):
return options.check_neural
if name.startswith("upscale:"):
return options.check_upscale
if name.startswith("interpolation:"):
return options.check_interpolation
return True
@staticmethod
def _deep_checks(options: VerifyOptions, gpu_uuid: str, selected_gpu: dict[str, Any] | None) -> list[VerificationCheck]:
del selected_gpu
checks: list[VerificationCheck] = []
if options.check_neural:
try:
from src.core.runtime import prepare_runtime
prepared = prepare_runtime()
checks.append(VerificationCheck("neural:runtime", True, f"Prepared {len(prepared.warmed_files)} runtime components."))
except (ImportError, OSError, RuntimeError, ValueError) as exc:
checks.append(VerificationCheck("neural:runtime", False, str(exc)))
if options.check_upscale:
try:
from src.upscale.video.native import probe_capabilities
capabilities = probe_capabilities(gpu_uuid)
checks.append(VerificationCheck("upscale:capability", bool(capabilities.vsr.get("available")), str(capabilities.vsr)))
except (ImportError, OSError, RuntimeError, ValueError) as exc:
checks.append(VerificationCheck("upscale:capability", False, str(exc)))
if options.check_interpolation:
try:
from src.frame_interpolation.capabilities import probe_frame_interpolation_capabilities
capabilities = probe_frame_interpolation_capabilities(gpu_uuid)
checks.append(VerificationCheck("interpolation:capability", capabilities.available, capabilities.detail or f"native_multiplier={capabilities.native_multiplier}"))
except (ImportError, OSError, RuntimeError, ValueError) as exc:
checks.append(VerificationCheck("interpolation:capability", False, str(exc)))
return checks
scripts/dlss_ext.py
+463
View File
@@ -0,0 +1,463 @@
import os
import time
import textwrap
import gradio as gr
from modules.logger import log
from modules import shared, devices, processing, timer, errors, scripts_manager, scripts_postprocessing
from scripts.dlss import controller_cli as c
registered = False
debug = os.environ.get('SD_DLSS_DEBUG', None) is not None
FPS_CHOICES = ['23.976', '25', '29.97', '30', '50', '59.94', '60', '90', '119.88', '120', '144', '165', '180', '240', '360', '480']
NR_STYLES = ['None', 'Default', 'Natural', 'Cinematic']
NR_MODELS = ['None','Default', 'J', 'K', 'L', 'M']
NR_PRESETS = ['Default', 'Preset #1', 'Preset #2', 'Preset #3']
def create_ui(parent):
with gr.Accordion('nVidia DLSS', open=False, elem_id=f'{parent}_dlss_accordion'):
with gr.Row():
btn_install = gr.Button(value="Install", elem_id='dlss_install')
btn_verify = gr.Button(value="Verify", elem_id='dlss_verify')
btn_status = gr.Button(value="Status", elem_id='dlss_status_btn')
btn_reset = gr.Button(value="Reset", elem_id='dlss_reset')
btn_shutdown = gr.Button(value="Shutdown", elem_id='dlss_shutdown')
with gr.Row():
install_note = gr.Markdown("", elem_id='dlss_install_note', visible=False)
with gr.Accordion('DLSS NeuralRender', open=False, elem_id='dlss_nn'):
with gr.Row():
nr_enabled = gr.Checkbox(label='NR enable', value=False, elem_id='dlss_nr_enabled')
nr_append = gr.Checkbox(label='NR append result', value=False, elem_id='dlss_nr_append')
with gr.Row():
nr_style = gr.Dropdown(label='NR style', choices=NR_STYLES, value='Default', elem_id='dlss_nr_style')
nr_preset = gr.Dropdown(label='NR preset', choices=NR_PRESETS, value='Default', elem_id='dlss_nr_preset')
nr_model_preset = gr.Dropdown(label='NR model', choices=NR_MODELS, value='Default', elem_id='dlss_nr_model_preset')
with gr.Row():
nr_intensity = gr.Slider(label='NR intensity', minimum=0.0, maximum=2.0, step=0.05, value=1.0, elem_id='dlss_nr_intensity')
nr_local_tone = gr.Slider(label='NR tone strength', minimum=0.0, maximum=2.0, step=0.05, value=1.0, elem_id='dlss_nr_local_tone')
with gr.Row():
nr_local_structure = gr.Slider(label='NR local structure', minimum=0.0, maximum=2.0, step=0.05, value=1.0, elem_id='dlss_nr_local_structure')
nr_skin_structure = gr.Slider(label='NR skin structure', minimum=-1.0, maximum=2.0, step=0.05, value=-1.0, elem_id='dlss_nr_skin_structure')
with gr.Row():
nr_upscaling_factor = gr.Dropdown(label='NR upscaling factor', choices=["1.0", "1.5", "1.724", "2.0", "3.0"], value="1.0", elem_id='dlss_nr_upscaling_factor')
nr_automatic_mask = gr.Checkbox(label='NR automatic mask', value=False, elem_id='dlss_nr_automatic_mask')
with gr.Accordion('DLSS SuperSample', open=False, elem_id='dlss_ss'):
with gr.Row():
ss_enabled = gr.Checkbox(label='SS enable', value=False, elem_id='dlss_ss_enabled')
ss_append = gr.Checkbox(label='SS append result', value=False, elem_id='dlss_ss_append')
with gr.Row():
ss_vsr_quality = gr.Dropdown(label='SS VSR quality', choices=["1: Low", "2: Medium", "3: High", "4: Ultra"], value="4: Ultra", type='value', elem_id='dlss_ss_vsr_quality')
with gr.Row():
ss_size_mode = gr.Dropdown(label='SS size mode', choices=['Scale factor', 'Target size'], value='Scale factor', elem_id='dlss_ss_size_mode')
with gr.Row():
ss_scale_factor = gr.Slider(label='SS scale factor', minimum=1.0, maximum=8.0, step=0.05, value=2.0, elem_id='dlss_ss_scale_factor')
with gr.Row():
ss_width = gr.Number(label='SS width', minimum=64, maximum=16384, step=8, value=3840, elem_id='dlss_ss_width')
ss_height = gr.Number(label='SS height', minimum=64, maximum=16384, step=8, value=2160, elem_id='dlss_ss_height')
with gr.Accordion('DLSS FrameGen', open=False, elem_id='dlss_fg'):
with gr.Row():
fg_enabled = gr.Checkbox(label='FG enable', value=False, elem_id='dlss_fg_enabled')
with gr.Row():
fg_source_fps = gr.Dropdown(label='Source FPS', choices=FPS_CHOICES, value='24', elem_id='dlss_fg_source_fps')
fg_target_fps = gr.Dropdown(label='Target FPS', choices=FPS_CHOICES, value='60', elem_id='dlss_fg_target_fps')
with gr.Row():
fg_engine = gr.Dropdown(label='Engine', choices=['Auto', 'Native DLSSG', 'Cascade'], value='Auto', elem_id='dlss_fg_engine')
with gr.Accordion('DLSS Status', open=True, elem_id='dlss_status'):
ss_status = gr.JSON({ 'Status': 'unknown' if len(shared.opts.dlss_pkg_path) < 4 else 'stored'})
with gr.Row():
pkg_path = gr.Textbox(label='DLSS Package path', value=shared.opts.dlss_pkg_path, placeholder='path to dlss 5 visual enhancer', elem_id='dlss_pkg_path')
btn_install.click(install, inputs=[], outputs=[install_note])
btn_verify.click(verify, inputs=[pkg_path], outputs=[ss_status])
btn_status.click(status, inputs=[pkg_path], outputs=[ss_status])
btn_reset.click(reset, inputs=[pkg_path], outputs=[ss_status])
btn_shutdown.click(shutdown, inputs=[pkg_path], outputs=[ss_status])
return [nr_enabled, nr_append, nr_style, nr_intensity, nr_local_tone, nr_local_structure, nr_skin_structure, nr_upscaling_factor, nr_preset, nr_automatic_mask, nr_model_preset, ss_enabled, ss_append, ss_vsr_quality, ss_size_mode, ss_scale_factor, ss_width, ss_height, fg_enabled, fg_source_fps, fg_target_fps, fg_engine]
def install():
note = textwrap.dedent("""\
### Install
1. Download and unpack: [DLSS 5 Visual Enhancer](https://github.com/Merserk/dlss5-visual-enhancer/releases/tag/v7.0)
2. Enter the path to the unpacked package
3. Press verify
### Notes
- Package info is stored for future use on successful verification
- DLSS controller process is started on first use
- Use status to check the current state of the DLSS controller
- Use reset to restore the DLSS controller to its default state
- Use shutdown to stop the DLSS controller process
""")
return gr.update(value=note, visible=True)
def verify(pkg_path):
log.info(f'DLSS verify: path="{pkg_path}"')
if not os.path.exists(pkg_path) or not os.path.isdir(pkg_path):
log.error(f'DLSS: path="{pkg_path}" not found')
return { 'error': 'package path not found' }
if not c.controller.get_python(pkg_path):
return { 'error': 'python not found in package path' }
response = c.controller.call(pkg_path, 'verify', { 'gpu_uuid': 'auto', 'options': { 'level': 'deep' } })
if response.get('status') != 'ok':
error = response.get('error') or {}
log.error(f'DLSS: {error.get("message")}')
return { 'error': error.get('message', 'unknown error') }
report = (response.get('result') or {}).get('report', {})
if debug:
log.trace(f'DLSS raw: {report}')
checks = { 'passed': 0, 'failed': 0 }
for check in report.get('checks', []):
if check.get('passed', False):
checks['passed'] += 1
else:
checks['failed'] += 1
log.error(f'DLSS : {check}')
shared.opts.dlss_pkg_path = pkg_path
shared.opts.save()
log.debug(f'DLSS: gpu={report.get("gpu", "unknown")} checks={checks}')
return report
def status(pkg_path):
log.info(f'DLSS status: path="{pkg_path}"')
response = c.controller.call(pkg_path, 'status', {})
if response.get('status') != 'ok':
error = response.get('error') or {}
log.error(f'DLSS: {error.get("message")}')
return { 'error': error.get('message', 'unknown error') }
return response.get('result', {})
def reset(pkg_path):
log.info(f'DLSS reset: path="{pkg_path}"')
response = c.controller.call(pkg_path, 'reset', {})
if response.get('status') != 'ok':
error = response.get('error') or {}
log.error(f'DLSS: {error.get("message")}')
return { 'error': error.get('message', 'unknown error') }
return response.get('result', {})
def shutdown(pkg_path):
log.info(f'DLSS shutdown: path="{pkg_path}"')
if not c.controller.is_alive():
return { 'shutdown': True, 'note': 'controller was not running' }
c.controller.stop()
return { 'shutdown': True }
def supersample(pkg_path, images, ss_vsr_quality, ss_size_mode, ss_scale_factor, ss_width, ss_height):
try:
options = {
'vsr_quality': int(ss_vsr_quality[0]),
'size_mode': ss_size_mode,
'scale_factor': float(ss_scale_factor),
'width': int(ss_width),
'height': int(ss_height),
'aspect_lock': False,
}
frames = c.images_to_nchw(images)
if debug:
log.trace(f'DLSS: method=SuperSample input={frames.shape} options={options}')
response = c.controller.call(
pkg_path,
'upscale',
{ 'images': frames, 'options': options },
timeout=300.0,
)
if response.get('status') != 'ok':
error = response.get('error') or {}
log.error(f'DLSS: {error.get("message")}')
return None
return c.nchw_to_images(response.get('result'))
except Exception as e:
log.error(f'DLSS: {e}')
errors.display(e, 'DLSS')
return None
def neuralrender(pkg_path, images, nr_style, nr_intensity, nr_local_tone, nr_local_structure, nr_skin_structure, nr_upscaling_factor, nr_preset, nr_automatic_mask, nr_model_preset):
try:
options = {
'nr_style': nr_style,
'nr_intensity': float(nr_intensity),
'local_tone_strength': float(nr_local_tone),
'local_structure_strength': float(nr_local_structure),
'skin_structure_strength': float(nr_skin_structure),
'upscaling_factor': float(nr_upscaling_factor),
'warmup_frames': 0,
'nr_preset': nr_preset,
'automatic_mask': bool(nr_automatic_mask),
'dlss_model_preset': nr_model_preset,
}
frames = c.images_to_nchw(images)
if debug:
log.trace(f'DLSS: method=NeuralRender input={frames.shape} options={options}')
response = c.controller.call(
pkg_path,
'render',
{ 'images': frames, 'options': options },
timeout=600.0,
)
if response.get('status') != 'ok':
error = response.get('error') or {}
log.error(f'DLSS: {error.get("message")}')
return None
return c.nchw_to_images(response.get('result'))
except Exception as e:
log.error(f'DLSS: {e}')
errors.display(e, 'DLSS')
return None
def framegen(pkg_path, images, fg_source_fps, fg_target_fps, fg_engine):
try:
if len(images) < 2:
log.warning('DLSS: FrameGen requires at least two frames, skipping')
return None
options = { 'ai_gpu_uuid': 'auto', 'engine': fg_engine }
frames = c.images_to_nchw(images)
if debug:
log.trace(f'DLSS: method=FrameGen input={frames.shape} options={options}')
response = c.controller.call(
pkg_path, 'framegen',
{ 'frames': frames, 'source_fps': fg_source_fps, 'target_fps': fg_target_fps, 'options': options },
timeout=300.0,
)
if response.get('status') != 'ok':
error = response.get('error') or {}
log.error(f'DLSS: {error.get("message")}')
return None
return c.nchw_to_images(response.get('result'))
except Exception as e:
log.error(f'DLSS: {e}')
errors.display(e, 'DLSS')
return None
def dlss(p: processing.StableDiffusionProcessing | None, pp: processing.Processed | scripts_postprocessing.PostprocessedImage,
nr_enabled, nr_append, nr_style, nr_intensity, nr_local_tone, nr_local_structure, nr_skin_structure, nr_upscaling_factor, nr_preset,nr_automatic_mask, nr_model_preset,
ss_enabled, ss_append, ss_vsr_quality, ss_size_mode, ss_scale_factor, ss_width, ss_height,
fg_enabled, fg_source_fps, fg_target_fps, fg_engine,
*args, **kwargs
):
if not (ss_enabled or nr_enabled or fg_enabled):
return None
pkg_path = shared.opts.dlss_pkg_path
if not pkg_path or not c.controller.get_python(pkg_path):
log.error('DLSS: package path not configured')
return None
if debug:
log.trace(f'DLSS: path="{pkg_path}" args={args} kwargs={kwargs}')
update = 'none'
if hasattr(pp, 'images') and pp.images is not None and len(pp.images) > 0:
update = 'images'
inputs = pp.images
elif hasattr(pp, 'image') and pp.image is not None:
update = 'image'
inputs = [pp.image]
else:
return None
# cast to appropriate types
nr_style = str(getattr(p, 'nr_style', nr_style))
nr_preset = str(getattr(p, 'nr_preset', nr_preset))
nr_model_preset = str(getattr(p, 'nr_model_preset', nr_model_preset))
nr_intensity = float(getattr(p, 'nr_intensity', nr_intensity))
nr_local_tone = float(getattr(p, 'nr_local_tone', nr_local_tone))
nr_local_structure = float(getattr(p, 'nr_local_structure', nr_local_structure))
nr_skin_structure = float(getattr(p, 'nr_skin_structure', nr_skin_structure))
nr_upscaling_factor = float(getattr(p, 'nr_upscaling_factor', nr_upscaling_factor))
ss_width = int(getattr(p, 'ss_width', ss_width))
ss_height = int(getattr(p, 'ss_height', ss_height))
ss_scale_factor = float(getattr(p, 'ss_scale_factor', ss_scale_factor))
fg_source_fps = str(getattr(p, 'fg_source_fps', fg_source_fps))
fg_target_fps = str(getattr(p, 'fg_target_fps', fg_target_fps))
if (p is not None) and ('video' in p.ops): # should not add video frames
nr_append = False
ss_append = False
images = []
originals = []
current_images = inputs
t = timer.Timer()
jobid = shared.state.begin('DLSS')
t_start = time.time()
if ss_enabled:
t0 = time.time()
if p:
p.extra_generation_params["DLSSSuperSample"] = True
log.debug(f'DLSS: method=SuperSample quality="{ss_vsr_quality}" mode="{ss_size_mode}" scale={ss_scale_factor} width={ss_width} height={ss_height}')
if ss_append:
originals.extend(current_images)
output = supersample(pkg_path, current_images, ss_vsr_quality, ss_size_mode, ss_scale_factor, ss_width, ss_height)
if debug:
log.trace(f'DLSS: method=SuperSample images={len(output) if output else 0} time={time.time() - t0:.3f}')
if output:
images.extend(output)
current_images = output
t.ts('supersample', t0)
if nr_style == 'None' or nr_model_preset == 'None':
nr_enabled = False
if nr_enabled:
t0 = time.time()
if p:
p.extra_generation_params["DLSSNeuralRender"] = True
log.debug(f'DLSS: method=NeuralRender style={nr_style} intensity={nr_intensity} tone={nr_local_tone} structure={nr_local_structure} skin={nr_skin_structure} scale={nr_upscaling_factor} preset={nr_preset} mask={nr_automatic_mask} model={nr_model_preset}')
if nr_append:
originals.extend(current_images)
output = neuralrender(pkg_path, current_images, nr_style, nr_intensity, nr_local_tone, nr_local_structure, nr_skin_structure, nr_upscaling_factor, nr_preset, nr_automatic_mask, nr_model_preset)
if debug:
log.trace(f'DLSS: method=NeuralRender images={len(output) if output else 0} time={time.time() - t0:.3f}')
if output:
images.extend(output)
current_images = output
t.ts('neuralrender', t0)
if fg_enabled:
t0 = time.time()
if p:
p.extra_generation_params["DLSSFrameGen"] = True
log.debug(f'DLSS: method=FrameGen source={fg_source_fps} target={fg_target_fps} engine={fg_engine}')
output = framegen(pkg_path, current_images, fg_source_fps, fg_target_fps, fg_engine)
if debug:
log.trace(f'DLSS: method=FrameGen images={len(output) if output else 0} time={time.time() - t0:.3f}')
if output:
images.extend(output)
current_images = output
t.ts('framegen', t0)
shared.state.end(jobid)
timer.process.ts('dlss', t_start)
log.debug(f'DLSS: frames={len(images)} {t.summary(min_time=0)}')
if update == 'images':
pp.images = images
elif update == 'image' and len(images) > 0:
pp.image = images[-1]
pp.originals = originals
return pp
class DLSSScript(scripts_manager.Script):
def __init__(self):
super().__init__()
self.video_capable = scripts_manager.AlwaysVisible
self.register()
def title(self):
return 'nVidia DLSS'
def show(self, _is_img2img):
if devices.backend != 'cuda':
return False
return scripts_manager.AlwaysVisible
def ui(self, _is_img2img):
return create_ui(self.parent)
def register(self): # register xyz grid elements
global registered # pylint: disable=global-statement
if registered:
return
registered = True
def apply_field(field):
def fun(p, x, xs): # pylint: disable=unused-argument
setattr(p, field, x)
self.run(p)
return fun
import sys
xyz_classes = [v for k, v in sys.modules.items() if 'xyz_grid_classes' in k]
if xyz_classes and len(xyz_classes) > 0:
xyz_classes = xyz_classes[0]
options = [
xyz_classes.AxisOption("[DLSS] NR style", str, apply_field("nr_style"), choices=lambda: NR_STYLES),
xyz_classes.AxisOption("[DLSS] NR preset", str, apply_field("nr_preset"), choices=lambda: NR_PRESETS),
xyz_classes.AxisOption("[DLSS] NR model", str, apply_field("nr_model_preset"), choices=lambda: NR_MODELS),
xyz_classes.AxisOption("[DLSS] NR intensity", float, apply_field("nr_intensity")),
xyz_classes.AxisOption("[DLSS] NR local tone", float, apply_field("nr_local_tone")),
xyz_classes.AxisOption("[DLSS] NR local structure", float, apply_field("nr_local_structure")),
xyz_classes.AxisOption("[DLSS] NR skin structure", float, apply_field("nr_skin_structure")),
]
for option in options:
if option not in xyz_classes.axis_options:
xyz_classes.axis_options.append(option)
def postprocess_image(self, p: processing.StableDiffusionProcessing, pp: scripts_manager.PostprocessImageArgs, *args, **kwargs):
if p.xyz:
pp = dlss(p, pp, *args, **kwargs)
def postprocess(self, p: processing.StableDiffusionProcessing, pp: processing.Processed, *args, **kwargs): # pylint: disable=arguments-differ,unused-argument
if p.xyz: # do not postprocessing when running in xyz mode
return pp
_pp = dlss(p, pp, *args, **kwargs)
# postprocess triggers after initial images have already been saved
if _pp is not None and hasattr(_pp, 'images') and _pp.images is not None:
pp = _pp
orig_infos = pp.infotexts if hasattr(pp, 'infotexts') else []
out_images, out_infos = processing.process_samples(p, pp.images)
pp.images = out_images
pp.infotexts = out_infos
if hasattr(pp, 'originals') and pp.originals is not None and len(pp.originals) > 0:
pp.infotexts = orig_infos + pp.infotexts
pp.images = pp.originals + pp.images
return pp
class DLSSPostprocessingScript(scripts_postprocessing.ScriptPostprocessing):
name = "nVidia DLSS"
order = 30000
def ui(self):
nr_enabled, nr_append, nr_style, nr_intensity, nr_local_tone, nr_local_structure, nr_skin_structure, nr_upscaling_factor, nr_preset, nr_automatic_mask, nr_model_preset, ss_enabled, ss_append, ss_vsr_quality, ss_size_mode, ss_scale_factor, ss_width, ss_height, fg_enabled, fg_source_fps, fg_target_fps, fg_engine = create_ui('postprocess')
return {
"nr_enabled": nr_enabled,
"nr_append": nr_append,
"nr_style": nr_style,
"nr_intensity": nr_intensity,
"nr_local_tone": nr_local_tone,
"nr_local_structure": nr_local_structure,
"nr_skin_structure": nr_skin_structure,
"nr_upscaling_factor": nr_upscaling_factor,
"nr_preset": nr_preset,
"nr_automatic_mask": nr_automatic_mask,
"nr_model_preset": nr_model_preset,
"ss_enabled": ss_enabled,
"ss_append": ss_append,
"ss_vsr_quality": ss_vsr_quality,
"ss_size_mode": ss_size_mode,
"ss_scale_factor": ss_scale_factor,
"ss_width": ss_width,
"ss_height": ss_height,
"fg_enabled": fg_enabled,
"fg_source_fps": fg_source_fps,
"fg_target_fps": fg_target_fps,
"fg_engine": fg_engine,
}
def process(self, pp: scripts_postprocessing.PostprocessedImage, *args, **kwargs):
nr_enabled = kwargs.get("nr_enabled", False)
ss_enabled = kwargs.get("ss_enabled", False)
fg_enabled = kwargs.get("fg_enabled", False)
if not (nr_enabled or ss_enabled or fg_enabled):
return
if pp.image is None:
return
result = dlss(None, pp, *args, **kwargs)
if result is None or not hasattr(result, "images") or len(result.images) == 0:
return
pp.image = result.images[0]
pp.info["DLSS"] = f'NR: {nr_enabled} SS: {ss_enabled} FG: {fg_enabled}'
scripts/prompt_enhance/options.py
+11 -3
View File
@@ -144,6 +144,12 @@ class Options:
Keep the result faithful to the original prompt and the intended image.
Do not add unrelated concepts, non-visual commentary, or fluff.
''')
t2i_noprompt: str = textwrap.dedent('''\
You are an expert AI image prompt engineer.
Create an original, imaginative and visually rich prompt for image generation.
Be Creative!
Use concrete, image-generation language and avoid unrelated fluff or commentary.
''')
i2i_prompt: str = textwrap.dedent('''\
You are an expert AI image prompt engineer.
You will receive an image and a user prompt for editing or refinement.
@@ -159,6 +165,7 @@ class Options:
Base the prompt on what is present in the image, its style, and its atmosphere.
Use concrete, image-generation language and avoid unrelated fluff or commentary.
''')
t2v_prompt: str = textwrap.dedent('''\
You are an expert AI video prompt engineer.
You will receive a user prompt for video generation.
@@ -185,6 +192,7 @@ class Options:
Specify subject and camera movement, scene transitions, and temporal dynamics.
Use concrete, video-generation language and avoid unrelated fluff or commentary.
''')
nsfw_ok: str = textwrap.dedent('''\
NSFW, nudity and explicit content and language are allowed.
''')
@@ -211,10 +219,10 @@ class Options:
max_delim_index: int = 60
min_tokens: int = 0
max_tokens: int = 256
max_tokens: int = 512
do_sample: bool = True
temperature: float = 0.6
repetition_penalty: float = 1.2
temperature: float = 0.75
repetition_penalty: float = 1.05
top_k: int = 0
top_p: float = 0.0
thinking_mode: bool = False
scripts/prompt_enhance/template.py
+4 -3
View File
@@ -85,15 +85,16 @@ def set_template(
has_image = image is not None and isinstance(image, Image.Image)
is_video = module == 'video'
debug_log(f'Prompt enhance template: module={module} prompt={has_prompt} image={has_image} video={is_video} model="{model}" nsfw={nsfw} processor={has_processor}')
debug_log(f'Prompt enhance template: module={module} prompt={has_prompt} image={has_image} video={is_video} model="{model}" nsfw={nsfw} processor={has_processor} cloud={is_cloud_model(model)}')
"""
if has_image:
if is_cloud_model(model):
pass
elif options.processor is None:
elif not has_processor:
log.error('Prompt enhance: image not supported by model')
return prompt if prompt is not None else '' # Return original text part if image cannot be processed
"""
if has_image:
chat_template = get_image_template(system, prompt, options, nsfw, has_prompt, has_processor, is_video, image)
else:
scripts/prompt_enhance_ext.py
+19 -13
View File
@@ -279,14 +279,16 @@ class PromptEnhanceScript(scripts_manager.Script):
def get_image(self, image):
current_image = None
try:
if image is not None and isinstance(image, gr.Image):
if (image is not None) and isinstance(image, list) and len(image) > 0:
current_image = image[0]
if (image is not None) and isinstance(image, gr.Image):
current_image = image.value
elif image is not None and isinstance(image, Image.Image): # if image is already a PIL image
elif (image is not None) and isinstance(image, Image.Image): # if image is already a PIL image
current_image = image
if current_image is not None and (current_image.width <= 64 or current_image.height <= 64):
if (current_image is not None) and (current_image.width <= 64 or current_image.height <= 64):
current_image = None
# Fallback to Kanvas/Control input if no image from Gradio component (e.g., when Kanvas is active)
if current_image is None and ui_control_helpers.input_source is not None:
if (current_image is None) and (ui_control_helpers.input_source is not None):
if isinstance(ui_control_helpers.input_source, list) and len(ui_control_helpers.input_source) > 0:
current_image = ui_control_helpers.input_source[0]
elif isinstance(ui_control_helpers.input_source, Image.Image):
@@ -325,11 +327,10 @@ class PromptEnhanceScript(scripts_manager.Script):
# Strip symbols from model name if present
model = get_model_repo_from_display(model) if model else self.options.default
prompt = prompt or (self.prompt.value if self.prompt else "") # Check if self.prompt is None
image = None
if use_vision and is_vision_model(model): # handle vision toggle
image = image or self.image
if image is None:
use_vision = False
else:
image = None
prefix = prefix or ''
suffix = suffix or ''
min_tokens = min_tokens or self.options.min_tokens
@@ -341,7 +342,7 @@ class PromptEnhanceScript(scripts_manager.Script):
thinking = thinking or self.options.thinking_mode
sample = sample if sample is not None else self.options.do_sample
nsfw = nsfw if nsfw is not None else True # Default nsfw to True if not provided
debug_log(f'Prompt enhance: model="{model}" model_class="{self.llm.__class__.__name__ if self.llm is not None else "not loaded"}" nsfw={nsfw} thinking={thinking} prefill="{prefill[:30] if prefill else ""}" use_vision={use_vision} image={image is not None}')
debug_log(f'Prompt enhance: model="{model}" model_class="{self.llm.__class__.__name__ if self.llm is not None else "not loaded"}" nsfw={nsfw} thinking={thinking} prefill="{prefill[:30] if prefill else ""}" vision={use_vision} image={image}')
while self.busy:
time.sleep(0.1)
@@ -363,7 +364,9 @@ class PromptEnhanceScript(scripts_manager.Script):
# Only process images if vision is enabled and model supports it
if use_vision and is_vision_model(model):
current_image = self.get_image(image)
debug_log(f'Prompt enhance: image={current_image}')
if current_image is None:
use_vision = False
debug_log(f'Prompt enhance: image={current_image} use_vision={use_vision}')
# Check if vision was requested but no image is available
if use_vision and is_vision_model(model) and current_image is None:
@@ -396,8 +399,10 @@ class PromptEnhanceScript(scripts_manager.Script):
self.busy = True
if is_cloud_model(model):
has_prompt = prompt_text is not None and len(prompt_text) > 4
has_prompt = (prompt_text is not None) and (len(prompt_text) > 4)
has_prefill = (prefill_text is not None) and (len(prefill_text) > 4)
system = get_system_prompt(system, self.options, nsfw, has_prompt=has_prompt, is_video=self.parent=='video', is_image=current_image is not None)
debug_log(f'Prompt enhance: prompt="{prompt_text}"')
if 'gemini' in model:
from modules.caption import gemini
kwargs = {
@@ -408,7 +413,7 @@ class PromptEnhanceScript(scripts_manager.Script):
model_name = model.replace('google/', '')
response = gemini.predict(prompt_text, current_image, model_name, system, prefill_text, thinking, kwargs)
t1 = time.time()
log.info(f'Prompt enhance: model="{model}" nsfw={nsfw} time={t1-t0:.2f} prefill="{prefill_text[:20] if prefill_text else None}" response={len(response)}')
log.info(f'Prompt enhance: model="{model}" nsfw={nsfw} time={t1-t0:.2f} prompt={has_prompt} prefill={has_prefill} image={current_image} thinking={thinking} response={len(response)}')
debug_log(f'Prompt enhance: response="{response}"')
self.busy = False
return response
@@ -420,7 +425,7 @@ class PromptEnhanceScript(scripts_manager.Script):
model_name = model.replace('xai/', '')
response = grok.predict(prompt_text, current_image, model_name, system, prefill_text, thinking, kwargs)
t1 = time.time()
log.info(f'Prompt enhance: model="{model}" nsfw={nsfw} time={t1-t0:.2f} prefill="{prefill_text[:20] if prefill_text else None}" response={len(response)}')
log.info(f'Prompt enhance: model="{model}" nsfw={nsfw} time={t1-t0:.2f} prompt={has_prompt} prefill={has_prefill} image={current_image} thinking={thinking} response={len(response)}')
debug_log(f'Prompt enhance: response="{response}"')
self.busy = False
return response
@@ -731,10 +736,11 @@ class PromptEnhanceScript(scripts_manager.Script):
jobid = shared.state.begin('LLM')
p.extra_generation_params['LLM'] = get_model_repo_from_display(llm_model)
p.extra_generation_params['Original'] = p.prompt
image = self_image or p.init_images
p.prompt = self.enhance(
prompt=p.prompt,
seed=p.seed,
image=self_image,
image=image,
prefix=prompt_prefix,
suffix=prompt_suffix,
model=llm_model,
scripts/rocm/rocm_log.py
+198
View File
@@ -0,0 +1,198 @@
"""Capture and report native MIOpen convolution selections."""
import atexit
import os
import re
import sys
import threading
import time
from modules.logger import log
_ALGORITHM_PATTERN = re.compile(r"FW Chosen Algorithm:\s*([^,\s]+)")
_CHOSEN_PATTERN = re.compile(r"FW Chosen Algorithm:\s*([^,\s]+)\s*,\s*[^,]*,\s*([0-9.eE+-]+)")
_MIOPEN_PREFIX = "MIOpen(HIP):"
_logging_capture = None
def _forward_stderr(fd, line):
data = line if line.endswith(b"\n") else line + b"\n"
while data:
written = os.write(fd, data)
data = data[written:]
def _process_line(line, saved_stderr):
text = line.decode(errors="replace").rstrip("\r\n")
if not text.lstrip().startswith(_MIOPEN_PREFIX):
_forward_stderr(saved_stderr, line)
return
match = _CHOSEN_PATTERN.search(text)
if match:
log.info(f'MIOpen: algorithm={match.group(1)} time={float(match.group(2)):.3f}')
class MIOpenLogRedirect:
"""Redirect native MIOpen diagnostics into structured informational logs."""
def __init__(self):
self.read_fd = -1
self.saved_stderr = -1
self.saved_python_stderr = sys.stderr
self.safe_stderr = None
self.reader = None
def __enter__(self):
write_fd = -1
try:
self.read_fd, write_fd = os.pipe()
self.saved_stderr = os.dup(2)
self.saved_python_stderr = sys.stderr
self.safe_stderr = os.fdopen(os.dup(self.saved_stderr), "w", encoding=getattr(sys.stderr, "encoding", None) or "utf-8", buffering=1)
os.dup2(write_fd, 2)
os.close(write_fd)
write_fd = -1
sys.stderr = self.safe_stderr
def read_output():
pending = b""
while True:
chunk = os.read(self.read_fd, 4096)
if not chunk:
break
pending += chunk
while b"\n" in pending:
line, pending = pending.split(b"\n", 1)
_process_line(line + b"\n", self.saved_stderr)
if pending:
_process_line(pending, self.saved_stderr)
self.reader = threading.Thread(target=read_output, daemon=True)
self.reader.start()
return self
except Exception:
if write_fd >= 0:
os.close(write_fd)
if self.saved_stderr >= 0:
os.dup2(self.saved_stderr, 2)
sys.stderr = self.saved_python_stderr
if self.safe_stderr is not None:
self.safe_stderr.close()
if self.reader is not None:
self.reader.join()
if self.saved_stderr >= 0:
os.close(self.saved_stderr)
if self.read_fd >= 0:
os.close(self.read_fd)
raise
def __exit__(self, _exc_type, _exc_value, _traceback):
os.dup2(self.saved_stderr, 2)
sys.stderr = self.saved_python_stderr
self.safe_stderr.close()
self.reader.join()
os.close(self.saved_stderr)
os.close(self.read_fd)
return False
def start_miopen_logging():
"""Start filtering native MIOpen diagnostics without changing the environment."""
global _logging_capture # pylint: disable=global-statement
if _logging_capture is None:
try:
_logging_capture = MIOpenLogRedirect()
_logging_capture.__enter__()
except Exception as err:
log.warning(f'MIOpen logging: failed to start: {err}')
_logging_capture = None
def stop_miopen_logging():
"""Stop filtering native MIOpen diagnostics and restore stderr."""
global _logging_capture # pylint: disable=global-statement
if _logging_capture is not None:
_logging_capture.__exit__(None, None, None)
_logging_capture = None
atexit.register(stop_miopen_logging)
class MIOpenLogCapture:
"""Capture one native MIOpen operation and log its selected algorithm and time."""
def __init__(self, operation: str = "convolution", repeats: int = 1):
self.operation = operation
self.repeats = max(1, repeats)
self.lines: list[str] = []
self.elapsed_ms = 0.0
self.algorithms: list[str] = []
self.read_fd = -1
self.saved_stderr = -1
self.saved_python_stderr = sys.stderr
self.safe_stderr = None
self.start = 0.0
self.reader = None
def __enter__(self):
write_fd = -1
try:
self.read_fd, write_fd = os.pipe()
self.saved_stderr = os.dup(2)
self.saved_python_stderr = sys.stderr
self.safe_stderr = os.fdopen(
os.dup(self.saved_stderr),
"w",
encoding=getattr(sys.stderr, "encoding", None) or "utf-8",
buffering=1,
)
os.dup2(write_fd, 2)
os.close(write_fd)
write_fd = -1
sys.stderr = self.safe_stderr
self.lines = []
self.start = time.perf_counter()
def read_output():
chunks = []
while True:
chunk = os.read(self.read_fd, 4096)
if not chunk:
break
chunks.append(chunk)
self.lines.extend(b"".join(chunks).decode(errors="replace").splitlines())
self.reader = threading.Thread(target=read_output, daemon=True)
self.reader.start()
return self
except Exception:
if write_fd >= 0:
os.close(write_fd)
if self.saved_stderr >= 0:
os.dup2(self.saved_stderr, 2)
sys.stderr = self.saved_python_stderr
if self.safe_stderr is not None:
self.safe_stderr.close()
if self.reader is not None:
self.reader.join()
if self.saved_stderr >= 0:
os.close(self.saved_stderr)
if self.read_fd >= 0:
os.close(self.read_fd)
raise
def __exit__(self, _exc_type, _exc_value, _traceback):
os.dup2(self.saved_stderr, 2)
sys.stderr = self.saved_python_stderr
self.safe_stderr.close()
os.close(self.saved_stderr)
self.reader.join()
os.close(self.read_fd)
return False
def capture_miopen(operation: str = "convolution", repeats: int = 1):
"""Return a side-effect-free context manager for one MIOpen operation."""
return MIOpenLogCapture(operation=operation, repeats=repeats)
scripts/rocm/rocm_mgr.py
+41 -7
View File
@@ -11,6 +11,7 @@ from modules.shared import opts
from scripts.rocm.rocm_vars import ROCM_ENV_VARS # pylint: disable=no-name-in-module
from scripts.rocm import rocm_profiles # pylint: disable=no-name-in-module
from scripts.rocm import rocm_log # pylint: disable=no-name-in-module
CONFIG = Path(os.path.abspath(os.path.join('data', 'rocm.json')))
@@ -275,6 +276,16 @@ def apply_env(config: Optional[Dict[str, str]] = None) -> None:
os.environ[var] = "0"
def start_miopen_logging() -> None:
"""Start explicit MIOpen diagnostic capture for a scoped operation."""
rocm_log.start_miopen_logging()
def stop_miopen_logging() -> None:
"""Stop explicit MIOpen diagnostic capture and restore stderr."""
rocm_log.stop_miopen_logging()
def apply_all(names: list, values: list) -> None:
config = load_config().copy()
arch = config.get(_ARCH_KEY, "")
@@ -333,6 +344,17 @@ def clear_env() -> None:
log.info(f'ROCm clear_env: cleared={cleared}')
def _miopen_user_db_path() -> Path:
"""Resolve the MIOpen user DB path for the current platform."""
configured = os.environ.get("MIOPEN_USER_DB_PATH", "")
if configured:
return Path(os.path.expandvars(os.path.expanduser(configured)))
if sys.platform == "win32":
return Path.home() / ".miopen" / "db"
cache_home = os.environ.get("XDG_CACHE_HOME", str(Path.home() / ".cache"))
return Path(cache_home) / "miopen"
def delete_config() -> None:
"""Delete the saved config file, clear all vars, and wipe the MIOpen user DB cache."""
import shutil # pylint: disable=import-outside-toplevel
@@ -342,8 +364,8 @@ def delete_config() -> None:
CONFIG.unlink()
log.info(f'ROCm delete_config: deleted {CONFIG}')
_cache = None
# Delete the MIOpen user DB (~/.miopen/db) - stale entries can cause solver mismatches
miopen_db = Path(os.path.expanduser('~')) / '.miopen' / 'db'
# Delete the MIOpen user DB - stale entries can cause solver mismatches.
miopen_db = _miopen_user_db_path()
if miopen_db.exists():
shutil.rmtree(miopen_db, ignore_errors=True)
log.info(f'ROCm delete_config: wiped MIOpen user DB at {miopen_db}')
@@ -492,8 +514,8 @@ def info() -> dict:
else:
sdb["exists"] = False
# --- User DB (~/.miopen/db) ---
user_db_path = Path.home() / ".miopen" / "db"
# --- User DB ---
user_db_path = _miopen_user_db_path()
udb = {"path": str(user_db_path), "exists": user_db_path.exists()}
if user_db_path.exists():
ufiles = _user_db_summary(user_db_path)
@@ -520,9 +542,21 @@ def info() -> dict:
}
# Apply saved config to os.environ at import time (only when ROCm is present)
if installer.torch_info.get('type', None) == 'rocm' and CONFIG.exists():
def _is_rocm_runtime() -> bool:
if installer.torch_info.get('type', None) == 'rocm':
return True
try:
apply_env()
import torch # pylint: disable=import-outside-toplevel
return bool(getattr(torch.version, 'hip', None))
except Exception:
return False
# Apply saved config to os.environ at import time (only when ROCm is present).
if _is_rocm_runtime():
try:
if CONFIG.exists():
apply_env()
rocm_log.start_miopen_logging()
except Exception as _e:
log.debug(f"[rocm_mgr] Warning: failed to apply env at import: {_e}")
scripts/rocm_ext.py
+1 -25
View File
@@ -47,8 +47,8 @@ class ROCmScript(scripts_manager.Script):
section("ROCm / HIP")
for k, v in d.get("rocm", {}).items():
row(k, v)
section("User DB (~/.miopen/db)")
udb = d.get("user_db", {})
section("User DB")
row("path", udb.get("path", ""))
for fname, finfo in udb.get("files", {}).items():
row(fname, finfo)
@@ -72,10 +72,6 @@ class ROCmScript(scripts_manager.Script):
btn_reset = gr.Button("Defaults", elem_id="rocm_btn_reset", size="sm")
btn_clear = gr.Button("Clear Run Vars", elem_id="rocm_btn_clear", size="sm")
btn_delete = gr.Button("Delete UserDb", variant="stop", elem_id="rocm_btn_delete", size="sm")
with gr.Row():
btn_rdna2 = gr.Button("RDNA2 (RX 6000)", elem_id="rocm_btn_rdna2")
btn_rdna3 = gr.Button("RDNA3 (RX 7000)", elem_id="rocm_btn_rdna3")
btn_rdna4 = gr.Button("RDNA4 (RX 9000)", elem_id="rocm_btn_rdna4")
_init_gemm = config.get("MIOPEN_GEMM_ENFORCE_BACKEND", "1")
_init_arch = config.get(rocm_mgr._ARCH_KEY, "")
_init_unavailable = rocm_profiles.UNAVAILABLE.get(_init_arch, set()) if _init_arch else set()
@@ -209,30 +205,10 @@ class ROCmScript(scripts_manager.Script):
result.append(gr.update(value=""))
return result
def profile_fn(arch):
rocm_mgr.apply_profile(arch)
updated = rocm_mgr.load_config()
unavailable = rocm_profiles.UNAVAILABLE.get(arch, set())
gemm_val = updated.get("MIOPEN_GEMM_ENFORCE_BACKEND", "1")
result = [gr.update(value=_build_style(unavailable, gemm_val == "1"))]
for pname in var_names:
meta = rocm_vars.ROCM_ENV_VARS[pname]
val = updated.get(pname, meta["default"])
if meta["widget"] == "checkbox":
result.append(gr.update(value=val == "1"))
elif meta["widget"] == "dropdown":
result.append(gr.update(value=rocm_mgr._dropdown_display(val, meta["options"])))
else:
result.append(gr.update(value=rocm_mgr._expand_venv(val)))
return result
btn_info.click(fn=_info_html, inputs=[], outputs=[info_out], show_progress='hidden')
btn_apply.click(fn=apply_fn, inputs=components, outputs=[style_out] + components, show_progress='hidden')
btn_reset.click(fn=reset_fn, inputs=[], outputs=[style_out] + components, show_progress='hidden')
btn_clear.click(fn=clear_fn, inputs=[], outputs=[style_out] + components, show_progress='hidden')
btn_delete.click(fn=delete_fn, inputs=[], outputs=[style_out] + components, show_progress='hidden')
btn_rdna2.click(fn=lambda: profile_fn("RDNA2"), inputs=[], outputs=[style_out] + components, show_progress='hidden')
btn_rdna3.click(fn=lambda: profile_fn("RDNA3"), inputs=[], outputs=[style_out] + components, show_progress='hidden')
btn_rdna4.click(fn=lambda: profile_fn("RDNA4"), inputs=[], outputs=[style_out] + components, show_progress='hidden')
return components
scripts/softfill.py
+1 -1
View File
@@ -964,7 +964,7 @@ class StableDiffusionXLSoftFillPipeline(
aesthetic_score: float = 6.0,
negative_aesthetic_score: float = 2.5,
clip_skip: Optional[int] = None,
callback_on_step_end: Optional[Callable[[int, int, Dict], None]] = None,
callback_on_step_end: Optional[Callable[..., Dict]] = None, # invoked as (self, step_index, timestep, callback_kwargs) -> dict
callback_on_step_end_tensor_inputs: List[str] = ["latents"],
**kwargs,
):
scripts/xyz/xyz_grid_draw.py
+2
View File
@@ -117,6 +117,7 @@ def draw_xyz_grid(p, xs, ys, zs, x_labels, y_labels, z_labels, cell, draw_legend
continue
if (not no_grid or include_sub_grids) and images.check_grid_size(to_process):
grid = images.image_grid(to_process, rows=len(ys))
p.is_grid = True
if draw_legend:
grid = images.draw_grid_annotations(grid, w, h, x_texts, y_texts, margin_size, title=z_texts[i])
processed_result.images.insert(i, grid)
@@ -124,6 +125,7 @@ def draw_xyz_grid(p, xs, ys, zs, x_labels, y_labels, z_labels, cell, draw_legend
processed_result.all_seeds.insert(i, processed_result.all_seeds[idx0])
processed_result.infotexts.insert(i, processed_result.infotexts[idx0])
if len(zs) > 1 and not no_grid and images.check_grid_size(processed_result.images[:len(zs)]): # create grid-of-grids
p.is_grid = True
grid = images.image_grid(processed_result.images[:len(zs)], rows=1)
processed_result.images.insert(0, grid)
processed_result.all_prompts.insert(0, processed_result.all_prompts[0])
scripts/xyz_grid_on.py
+2
View File
@@ -387,6 +387,7 @@ class XYZGridScript(scripts_manager.Script):
pc.extra_generation_params["Fixed Y Values"] = ", ".join([str(y) for y in ys])
info = processing.create_infotext(pc, pc.all_prompts, pc.all_seeds, pc.all_subseeds, grid=f'{len(xs)}x{len(ys)}')
grid_infotext.append(info)
if ix == 0 and iy == 0 and iz == 0 and len(zs) > 1: # create main grid info text
pc.extra_generation_params = copy(pc.extra_generation_params)
if z_opt.label != 'Nothing':
@@ -396,6 +397,7 @@ class XYZGridScript(scripts_manager.Script):
pc.extra_generation_params["Fixed Z Values"] = ", ".join([str(z) for z in zs])
info = processing.create_infotext(pc, pc.all_prompts, pc.all_seeds, pc.all_subseeds, grid=f'{len(zs)}x{len(xs)}x{len(ys)}')
grid_infotext.insert(0, info)
t1 = time.time()
return processed, t1-t0
test/test-offload-roles.py
+76 -53
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env python
"""
Offline unit tests for group offload placement in modules.sd_offload.
Offline unit tests for group offload placement in modules.sd_offload_group.
Every component takes exactly one role, derived from the component itself:
@@ -59,7 +59,7 @@ modules.cmd_args.parsed, _ = modules.cmd_args.parser.parse_known_args([])
from diffusers.utils.accelerate_utils import apply_forward_hook # pylint: disable=wrong-import-position
from modules.errors import log # pylint: disable=wrong-import-position
from modules import shared # pylint: disable=wrong-import-position,unused-import
from modules import sd_offload # pylint: disable=wrong-import-position
from modules import sd_offload_group, sd_offload_state, sd_offload_utils # pylint: disable=wrong-import-position
# ============================================================
@@ -246,28 +246,28 @@ ROLE_CASES = [
def test_role_table():
wrong = []
for module_name, cls, expected in ROLE_CASES:
role = sd_offload.group_offload_role(module_name, cls())
role = sd_offload_group.group_offload_role(module_name, cls())
if role != expected:
wrong.append(f'{module_name}/{cls.__name__}: {role} != {expected}')
assert not wrong, '; '.join(wrong)
def test_role_bridge_overrides_denoiser_slot_name():
assert sd_offload.group_offload_role('decoder', BridgeModule()) == 'ondemand'
assert sd_offload_group.group_offload_role('decoder', BridgeModule()) == 'ondemand'
def test_role_upstream_optout_overrides_denoiser_slot_name():
assert sd_offload.group_offload_role('transformer', UnsupportedModule()) == 'ondemand'
assert sd_offload_group.group_offload_role('transformer', UnsupportedModule()) == 'ondemand'
def test_role_undecorated_entry_points_stay_resident():
# neither group hooks nor the on-demand hook fire for a plain method call, so residency is the only safe placement
assert sd_offload.group_offload_role('vae', NoBridgeModule()) == 'resident'
assert sd_offload_group.group_offload_role('vae', NoBridgeModule()) == 'resident'
def test_role_unknown_component_is_aux():
# aux is the direction that stays correct when the guess is wrong
assert sd_offload.group_offload_role('some_future_head', PlainModule()) == 'aux'
assert sd_offload_group.group_offload_role('some_future_head', PlainModule()) == 'aux'
def role_with_opts(module_name, module, **opts):
@@ -275,7 +275,7 @@ def role_with_opts(module_name, module, **opts):
for key, value in opts.items():
setattr(shared.opts, key, value)
try:
return sd_offload.group_offload_role(module_name, module)
return sd_offload_group.group_offload_role(module_name, module)
finally:
for key, value in saved.items():
setattr(shared.opts, key, value)
@@ -303,7 +303,7 @@ def test_role_empty_exclusions_match_nothing():
def test_role_main_list_has_no_encoder_names():
encoders = [n for n in sd_offload.group_offload_main if 'encoder' in n or 'vae' in n]
encoders = [n for n in sd_offload_state.group_offload_main if 'encoder' in n or 'vae' in n]
assert not encoders, f'encoder-shaped names in the per-step list: {encoders}'
@@ -317,28 +317,28 @@ def dispatch_calls(pipe):
seen: dict[str, object] = {}
def name_of(module):
return next((n for n in sd_offload.get_module_names(pipe) if getattr(pipe, n, None) is module), module.__class__.__name__)
return next((n for n in sd_offload_utils.get_module_names(pipe) if getattr(pipe, n, None) is module), module.__class__.__name__)
def record(name, role, module):
calls.setdefault(name, []).append(role)
seen[name] = module
return True
orig_component = sd_offload.apply_group_offload_component
orig_ondemand = sd_offload.apply_group_offload_ondemand
orig_resident = sd_offload.set_group_resident
orig_stats = sd_offload.report_group_stats
sd_offload.apply_group_offload_component = lambda module, module_name, main: record(module_name, 'main' if main else 'aux', module)
sd_offload.apply_group_offload_ondemand = lambda module: record(name_of(module), 'ondemand', module)
sd_offload.set_group_resident = lambda module: record(name_of(module), 'resident', module)
sd_offload.report_group_stats = lambda sd_model, module_names: None
orig_component = sd_offload_group.apply_group_offload_component
orig_ondemand = sd_offload_group.apply_group_offload_ondemand
orig_resident = sd_offload_group.set_group_resident
orig_stats = sd_offload_group.report_group_stats
sd_offload_group.apply_group_offload_component = lambda module, module_name, main: record(module_name, 'main' if main else 'aux', module)
sd_offload_group.apply_group_offload_ondemand = lambda module: record(name_of(module), 'ondemand', module)
sd_offload_group.set_group_resident = lambda module: record(name_of(module), 'resident', module)
sd_offload_group.report_group_stats = lambda sd_model, module_names: None
try:
sd_offload.apply_group_offload(pipe)
sd_offload_group.apply_group_offload(pipe)
finally:
sd_offload.apply_group_offload_component = orig_component
sd_offload.apply_group_offload_ondemand = orig_ondemand
sd_offload.set_group_resident = orig_resident
sd_offload.report_group_stats = orig_stats
sd_offload_group.apply_group_offload_component = orig_component
sd_offload_group.apply_group_offload_ondemand = orig_ondemand
sd_offload_group.set_group_resident = orig_resident
sd_offload_group.report_group_stats = orig_stats
return calls, seen
@@ -364,12 +364,34 @@ def test_dispatch_skips_non_modules():
assert list(calls) == ['transformer'], f'dispatched {list(calls)}'
def test_stats_report_once_per_component():
seen = []
orig_stats = sd_offload_group.report_model_stats
sd_offload_group.report_model_stats = lambda module_name, module: seen.append(module_name)
try:
transformer, vae = PlainModule(), BridgeModule()
pipe = FakePipe({'transformer': transformer, 'vae': vae})
names = sd_offload_utils.get_module_names(pipe)
sd_offload_group.report_group_stats(pipe, names)
assert sorted(seen) == ['transformer', 'vae'], f'first report covered {seen}'
sd_offload_group.report_group_stats(pipe, names)
assert len(seen) == 2, f'a reapply reported again: {seen}'
switched = FakePipe({'transformer': transformer, 'vae': vae}) # a task switch rebuilds the pipe around the same components
sd_offload_group.report_group_stats(switched, names)
assert len(seen) == 2, f'a task switch reported again: {seen}'
reloaded = FakePipe({'transformer': PlainModule(), 'vae': vae}) # a reload or a component swap brings a new module
sd_offload_group.report_group_stats(reloaded, names)
assert seen[2:] == ['transformer'], f'a new component was not reported on its own: {seen}'
finally:
sd_offload_group.report_model_stats = orig_stats
def test_force_sweep_moves_only_stamped_components():
stamped = SweepModule()
stamped.sdnext_ondemand = True
unstamped = SweepModule()
pipe = FakePipe({'vae': stamped, 'transformer': unstamped})
sd_offload.offload_ondemand(pipe, reason='test', force=True)
sd_offload_group.offload_ondemand(pipe, reason='test', force=True)
assert stamped.moved, 'the stamped component must be swept to cpu'
assert not unstamped.moved, 'a component with no onload path must not be swept'
@@ -378,34 +400,34 @@ def test_reapply_after_clearing_the_never_list_restores_hooks():
module = PlainModule()
pipe = FakePipe({'text_encoder': module})
saved_never = shared.opts.diffusers_offload_never
orig_device = sd_offload.devices.device
orig_stats = sd_offload.report_group_stats
sd_offload.devices.device = torch.device('cpu') # residency moves to the accelerator, so pin the target to cpu
sd_offload.report_group_stats = lambda sd_model, module_names: None
orig_device = sd_offload_group.devices.device
orig_stats = sd_offload_group.report_group_stats
sd_offload_group.devices.device = torch.device('cpu') # residency moves to the accelerator, so pin the target to cpu
sd_offload_group.report_group_stats = lambda sd_model, module_names: None
try:
shared.opts.diffusers_offload_never = 'text_encoder'
sd_offload.apply_group_offload(pipe)
sd_offload_group.apply_group_offload(pipe)
assert getattr(module, 'sdnext_group_offload_sig', None) is None, 'a resident component must carry no group signature'
shared.opts.diffusers_offload_never = ''
sd_offload.apply_group_offload(pipe)
sd_offload_group.apply_group_offload(pipe)
assert getattr(module, 'sdnext_group_offload_sig', None) not in (None, 'partial'), 'clearing the exclusion must re-place the component'
finally:
shared.opts.diffusers_offload_never = saved_never
sd_offload.devices.device = orig_device
sd_offload.report_group_stats = orig_stats
sd_offload_group.devices.device = orig_device
sd_offload_group.report_group_stats = orig_stats
def test_ondemand_list_tracks_the_stamps():
pipe = FakePipe({'transformer': PlainModule(), 'vae': BridgeModule()})
orig_component = sd_offload.apply_group_offload_component
orig_stats = sd_offload.report_group_stats
sd_offload.apply_group_offload_component = lambda module, module_name, main: True
sd_offload.report_group_stats = lambda sd_model, module_names: None
orig_component = sd_offload_group.apply_group_offload_component
orig_stats = sd_offload_group.report_group_stats
sd_offload_group.apply_group_offload_component = lambda module, module_name, main: True
sd_offload_group.report_group_stats = lambda sd_model, module_names: None
try:
sd_offload.apply_group_offload(pipe)
sd_offload_group.apply_group_offload(pipe)
finally:
sd_offload.apply_group_offload_component = orig_component
sd_offload.report_group_stats = orig_stats
sd_offload_group.apply_group_offload_component = orig_component
sd_offload_group.report_group_stats = orig_stats
assert pipe.sdnext_ondemand_modules == ['vae'], f'on-demand list is {pipe.sdnext_ondemand_modules}'
assert getattr(pipe.vae, 'sdnext_ondemand', False), 'the vae must carry the on-demand stamp'
@@ -416,8 +438,8 @@ def test_ondemand_list_tracks_the_stamps():
def test_ondemand_apply_returns_bool_and_is_idempotent():
module = BridgeModule()
first = sd_offload.apply_group_offload_ondemand(module)
second = sd_offload.apply_group_offload_ondemand(module)
first = sd_offload_group.apply_group_offload_ondemand(module)
second = sd_offload_group.apply_group_offload_ondemand(module)
assert isinstance(first, bool) and isinstance(second, bool), 'placement must report a bool'
assert first is True, 'the first placement changes the component'
assert second is False, 'an unchanged component must report no change'
@@ -426,19 +448,19 @@ def test_ondemand_apply_returns_bool_and_is_idempotent():
def test_ondemand_apply_leaves_weights_on_cpu():
module = BridgeModule()
sd_offload.apply_group_offload_ondemand(module)
sd_offload_group.apply_group_offload_ondemand(module)
assert next(module.parameters()).device.type == 'cpu', 'on-demand components rest on cpu'
def test_resident_placement_clears_the_ondemand_stamp():
module = BridgeModule()
sd_offload.apply_group_offload_ondemand(module)
orig_device = sd_offload.devices.device
sd_offload.devices.device = torch.device('cpu') # residency moves to the accelerator, so pin the target to cpu
sd_offload_group.apply_group_offload_ondemand(module)
orig_device = sd_offload_group.devices.device
sd_offload_group.devices.device = torch.device('cpu') # residency moves to the accelerator, so pin the target to cpu
try:
changed = sd_offload.set_group_resident(module)
changed = sd_offload_group.set_group_resident(module)
finally:
sd_offload.devices.device = orig_device
sd_offload_group.devices.device = orig_device
assert isinstance(changed, bool) and changed is True, 'moving off the on-demand hook is a change'
assert not getattr(module, 'sdnext_ondemand', False), 'the on-demand stamp must not survive'
assert not hasattr(module, '_hf_hook'), 'the on-demand hook must be removed'
@@ -451,7 +473,7 @@ def test_resident_placement_clears_the_ondemand_stamp():
def test_module_names_reads_specs_on_modular_pipelines():
# transformer_2 exists only in the specs, so only the specs branch can find it
pipe = FakeModularPipe({'transformer': PlainModule(), 'vae': BridgeModule(), 'scheduler': object()}, spec_only={'transformer_2': PlainModule()})
names = sd_offload.get_module_names(pipe)
names = sd_offload_utils.get_module_names(pipe)
assert names == ['transformer', 'transformer_2', 'vae'], f'got {names}'
assert 'canvas_short_edge' not in names, 'config scalars must not be enumerated'
@@ -464,7 +486,7 @@ def test_module_names_ignores_the_component_registry_on_classic_pipelines():
raise ValueError('config and signature disagree')
pipe = RaisingPipe({'transformer': PlainModule(), 'vae': BridgeModule()})
names = sd_offload.get_module_names(pipe)
names = sd_offload_utils.get_module_names(pipe)
assert names == ['transformer', 'vae'], f'got {names}'
@@ -474,13 +496,13 @@ def test_module_names_ignores_the_component_registry_on_classic_pipelines():
def test_autoencoders_carry_the_entry_bridge():
from diffusers import AutoencoderKL, VQModel
missing = [cls.__name__ for cls in (AutoencoderKL, VQModel) if not sd_offload.has_entry_bridge(cls)]
missing = [cls.__name__ for cls in (AutoencoderKL, VQModel) if not sd_offload_group.has_entry_bridge(cls)]
assert not missing, f'no entry bridge detected on {missing}'
def test_denoisers_do_not_carry_the_entry_bridge():
from diffusers import SD3Transformer2DModel, UNet2DConditionModel
bridged = [cls.__name__ for cls in (UNet2DConditionModel, SD3Transformer2DModel) if sd_offload.has_entry_bridge(cls)]
bridged = [cls.__name__ for cls in (UNet2DConditionModel, SD3Transformer2DModel) if sd_offload_group.has_entry_bridge(cls)]
assert not bridged, f'entry bridge detected on denoisers {bridged}'
@@ -492,7 +514,7 @@ def test_upstream_still_opts_hunyuandit_out_of_group_offload():
def test_mageflow_vae_carries_the_entry_bridge():
from pipelines.mageflow.autoencoder_mage_vae import AutoencoderMageVAE
assert sd_offload.has_entry_bridge(AutoencoderMageVAE), 'the mageflow vae lost its entry decorators'
assert sd_offload_group.has_entry_bridge(AutoencoderMageVAE), 'the mageflow vae lost its entry decorators'
# ============================================================
@@ -597,7 +619,7 @@ def inventory_roles(never=''):
shared.opts.diffusers_offload_never = never
shared.opts.models_not_to_offload = ''
try:
return {(pipe, slot, comp.__name__): sd_offload.group_offload_role(slot, weightless(comp)) for pipe, slot, comp in rows}
return {(pipe, slot, comp.__name__): sd_offload_group.group_offload_role(slot, weightless(comp)) for pipe, slot, comp in rows}
finally:
shared.opts.diffusers_offload_never, shared.opts.models_not_to_offload = saved
@@ -671,6 +693,7 @@ def run_all():
for fn in [
test_dispatch_is_one_arm_per_component,
test_dispatch_skips_non_modules,
test_stats_report_once_per_component,
test_ondemand_list_tracks_the_stamps,
test_force_sweep_moves_only_stamped_components,
test_reapply_after_clearing_the_never_list_restores_hooks,
ui/autocomplete_xn.ts
+19 -11
View File
@@ -108,19 +108,27 @@ export const xnEngine: XnEngine = {
this.lora = new XnIndex(items);
}
// Embeddings: {loaded: [...], skipped: [...]}
const embData = await this.fetchJson('/embeddings') as Record<string, unknown> | null;
if (embData && typeof embData === 'object') {
const loaded = Array.isArray(embData.loaded) ? embData.loaded : [];
this.embed = new XnIndex(loaded.map((name) => ({ name: String(name) })));
if (window.opts.diffusers_enable_embed) {
const embData = await this.fetchJson('/embeddings') as Record<string, unknown> | null;
if (embData && typeof embData === 'object') {
const loaded = Array.isArray(embData.loaded) ? embData.loaded : [];
this.embed = new XnIndex(loaded.map((name) => ({ name: String(name) })));
}
} else {
this.embed = new XnIndex([]);
}
// Wildcards: [{name}, ...]
const wcData = await this.fetchJson('/wildcards');
if (Array.isArray(wcData)) {
this.wildcard = new XnIndex(
wcData
.filter((w) => typeof w === 'object' && w && 'name' in w && typeof w.name === 'string')
.map((w) => ({ name: w.name })),
);
if (window.opts.wildcards_enabled) {
const wcData = await this.fetchJson('/wildcards');
if (Array.isArray(wcData)) {
this.wildcard = new XnIndex(
wcData
.filter((w) => typeof w === 'object' && w && 'name' in w && typeof w.name === 'string')
.map((w) => ({ name: w.name })),
);
}
} else {
this.wildcard = new XnIndex([]);
}
log('autoComplete', {
xnLoaded: true,
ui/css/base.css
+2 -1
View File
@@ -1,4 +1,5 @@
@font-face { font-family: 'NotoSans'; font-display: swap; font-style: normal; font-weight: 100; src: local('NotoSansNerd'), url('fonts/notosans-nerdfont-regular.ttf') }
@font-face { font-family: 'Ubuntu'; font-display: swap; font-style: normal; font-weight: 100; src: local('Ubuntu'), url('fonts/ubuntu-nerdfont.ttf') }
/* toolbutton */
.gradio-button.tool { max-width: min-content; min-width: min-content !important; align-self: end; font-size: 1.4em; color: var(--body-text-color) !important; }
@@ -32,7 +33,7 @@
/* fullpage image viewer */
#lightboxModal { display: none; position: fixed; z-index: 1001; left: 0; top: 0; width: 100%; height: 100%; overflow: auto; background-color: rgba(20, 20, 20, 0.75); backdrop-filter: blur(6px);
user-select: none; -webkit-user-select: none; flex-direction: row; font-family: 'NotoSans'; }
user-select: none; -webkit-user-select: none; flex-direction: row; font-family: 'Ubuntu'; }
.modalControls { display: flex; justify-content: space-evenly; background-color: transparent; position: absolute; width: 99%; z-index: 1; }
.modalControls:hover { background-color: #50505050; }
.modalControls span { color: white; font-size: 2em; font-weight: bold; cursor: pointer; filter: grayscale(100%); }
ui/css/sdnext.css
+10 -3
View File
@@ -5,6 +5,13 @@
font-weight: 100;
src: local('NotoSansNerd'), url('fonts/notosans-nerdfont-regular.ttf');
}
@font-face {
font-display: swap;
font-family: 'Ubuntu';
font-style: normal;
font-weight: 100;
src: local('Ubuntu'), url('fonts/ubuntu-nerdfont.ttf');
}
:root {
--card-size: 160px;
@@ -72,7 +79,7 @@
--sd-tooltip-text-color: var(--sd-input-background-color);
--sd-body-font: 'IBM Plex Mono', monospace;
--sd-text-font: 'IBM Plex Mono', monospace;
--sd-button-font: 'NotoSans', sans-serif;
--sd-button-font: 'Ubuntu', sans-serif;
--sd-image-fit: scale-down;
--sd-panel-min-width: 256px;
--sd-grid-image-size: 150px;
@@ -289,12 +296,12 @@ input::-webkit-outer-spin-button, input::-webkit-inner-spin-button {
.gradio-dropdown .token {
overflow-x: hidden;
padding: var(--spacing-xs) !important;
font-family: 'NotoSans', var(--font);
font-family: 'Ubuntu', var(--font);
}
.gradio-dropdown .wrap input,
.gradio-dropdown input {
font-family: 'NotoSans', var(--font);
font-family: 'Ubuntu', var(--font);
}
.gradio-html {
ui/dist/sdnext.mjs
Vendored
+49 -20
View File
@@ -10051,6 +10051,7 @@ var ignoreElements = ["logMonitorData", "logWarnings", "logErrors", "tooltip-con
var ignoreElementsSet = new Set(ignoreElements);
var ignoreClasses = ["wrap"];
var mutationTimer;
var mutationTS;
var validMutations = [];
async function mutationCallback(mutations) {
if (mutations.length <= 0) return;
@@ -10064,7 +10065,10 @@ async function mutationCallback(mutations) {
if (validMutations.length < 1) return;
if (mutationTimer) clearTimeout(mutationTimer);
mutationTimer = setTimeout(async () => {
const ts = Date.now() - mutationTS;
if (!executedOnLoaded && ts > 1e3) log("onUiLoaded delayed", { ts, prompts: anyPromptExists() });
if (!executedOnLoaded && anyPromptExists()) {
log("onUiLoaded", ts);
executedOnLoaded = true;
executeCallbacks(uiLoadedCallbacks);
}
@@ -10083,6 +10087,7 @@ async function mutationCallback(mutations) {
}
document.addEventListener("DOMContentLoaded", () => {
log("DOMContentLoaded");
mutationTS = Date.now();
gradioObserver = new MutationObserver(mutationCallback);
gradioObserver.observe(gradioApp(), { childList: true, subtree: true, attributes: false });
});
@@ -10403,7 +10408,7 @@ function readCardTags(el2, tags) {
textarea.value = new_prompt;
updateInput(textarea);
};
if (tags.length === 0) return;
if (!tags || tags.length === 0) return;
const cardTags = tags.split("|");
if (!cardTags || cardTags.length === 0) return;
const tagsEl = el2.getElementsByClassName("tags")[0];
@@ -12556,7 +12561,7 @@ var ConnectionMonitorState = class _ConnectionMonitorState {
else return;
}
this.element.dataset.hint = this.toHTML();
this.element.style.backgroundColor = this.online ? "var(--sd-main-accent-color)" : "var(--color-error)";
this.element.style.background = this.online ? "var(--sd-main-accent-color)" : "var(--color-error)";
}
};
async function updateIndicator(online, data = {}, msg) {
@@ -13569,7 +13574,7 @@ Resolution: ${this.width} x ${this.height}`;
}
}
};
let ok2 = true;
let ok = true;
if (cachedData?.img) {
img.src = cachedData.img;
this.exif = cachedData.exif;
@@ -13582,7 +13587,7 @@ Resolution: ${this.width} x ${this.height}`;
try {
const json = await delayFetchThumb(this.src, this.#signal);
if (!json) {
ok2 = false;
ok = false;
pb.stats.failed = (pb.stats.failed || 0) + 1;
} else {
img.src = json.data;
@@ -13617,7 +13622,7 @@ Resolution: ${this.width} x ${this.height}`;
pb.stats.callback = (pb.stats.callback || 0) + Math.round(performance.now() - t0);
if (this.#signal.aborted) return;
galleryHashes.add(this.hash);
if (!ok2) return;
if (!ok) return;
img.onclick = () => {
setGallerySelectionByElement(this, { send: true });
};
@@ -14754,16 +14759,24 @@ var xnEngine = {
}
this.lora = new XnIndex(items);
}
const embData = await this.fetchJson("/embeddings");
if (embData && typeof embData === "object") {
const loaded = Array.isArray(embData.loaded) ? embData.loaded : [];
this.embed = new XnIndex(loaded.map((name) => ({ name: String(name) })));
if (window.opts.diffusers_enable_embed) {
const embData = await this.fetchJson("/embeddings");
if (embData && typeof embData === "object") {
const loaded = Array.isArray(embData.loaded) ? embData.loaded : [];
this.embed = new XnIndex(loaded.map((name) => ({ name: String(name) })));
}
} else {
this.embed = new XnIndex([]);
}
const wcData = await this.fetchJson("/wildcards");
if (Array.isArray(wcData)) {
this.wildcard = new XnIndex(
wcData.filter((w) => typeof w === "object" && w && "name" in w && typeof w.name === "string").map((w) => ({ name: w.name }))
);
if (window.opts.wildcards_enabled) {
const wcData = await this.fetchJson("/wildcards");
if (Array.isArray(wcData)) {
this.wildcard = new XnIndex(
wcData.filter((w) => typeof w === "object" && w && "name" in w && typeof w.name === "string").map((w) => ({ name: w.name }))
);
}
} else {
this.wildcard = new XnIndex([]);
}
log("autoComplete", {
xnLoaded: true,
@@ -16086,8 +16099,8 @@ async function createSplash() {
<div id="splashLog" class="splash-log" style="position: fixed; bottom: 0; text-align: left; padding: 8vh 8px 8px 8px; font-size: 12px; width: 100%; background: linear-gradient(0deg, darkslategray, transparent); opacity: 50%;"></div>
</div>`;
document.body.insertAdjacentHTML("beforeend", splash);
const ok2 = await preloadImages();
if (!ok2) {
const ok = await preloadImages();
if (!ok) {
removeSplash();
return;
}
@@ -16106,6 +16119,15 @@ async function createSplash() {
if (motdEl) motdEl.innerHTML = clean;
}).catch((err) => error(`getMOTD: ${err}`));
log("loadGradioUi");
const splashMonitor = setInterval(() => {
const splashVisible = !!document.getElementById("splash");
if (splashVisible) {
log("splashVisible", { visible: true, elapsed: Math.round(performance.now() - appStartTime) });
} else {
log("splashVisible", { visible: false, elapsed: Math.round(performance.now() - appStartTime) });
clearInterval(splashMonitor);
}
}, 5e3);
}
window.onload = createSplash;
@@ -16124,23 +16146,27 @@ function addLegacyNotice() {
window.api = "/sdapi/v1";
window.subpath = "";
var startupPromises = [];
var ok = false;
var optsReady = false;
var initialized = false;
async function waitForOpts() {
const t0 = performance.now();
let t1 = performance.now();
while (true) {
if (t1 - t0 > 12e4) {
if (t1 - t0 > 6e4) {
log("waitForOpts timeout");
break;
}
if (window.opts && Object.keys(window.opts).length > 0) {
ok = window.opts.theme_type === "Modern" ? "uiux_separator_appearance" in window.opts : true;
if (ok) {
optsReady = window.opts.theme_type === "Modern" ? "uiux_separator_appearance" in window.opts : true;
if (optsReady) {
log("waitForOpts", Math.round(t1 - t0));
timer("waitForOpts", t1 - t0);
break;
}
}
if (t1 - t0 > 15e3) {
log("waitForOpts delayed", Math.round(t1 - t0));
}
await sleep(100);
t1 = performance.now();
}
@@ -16159,6 +16185,8 @@ async function updateSubpath() {
log("API", { url: window.api });
}
async function initStartup() {
if (initialized) return;
initialized = true;
const t0 = performance.now();
log("initGradio", Math.round(t0 - appStartTime));
timer("initGradio", t0 - appStartTime);
@@ -16202,6 +16230,7 @@ async function initStartup() {
}
onUiLoaded(initStartup);
onUiReady(() => log("uiReady"));
window.initStartup = initStartup;
// ui/extensions.ts
function extensions_apply(_extensionsDisabledList, _extensionsUpdateList, disableAll) {
ui/dist/sdnext.mjs.map
Vendored
+3 -3
View File
File diff suppressed because one or more lines are too long
ui/extraNetworks.ts
+1 -1
View File
@@ -103,7 +103,7 @@ function readCardTags(el, tags) {
updateInput(textarea);
};
if (tags.length === 0) return;
if (!tags || tags.length === 0) return;
const cardTags = tags.split('|');
if (!cardTags || cardTags.length === 0) return;
const tagsEl = el.getElementsByClassName('tags')[0];
ui/gallery.ts
+1
View File
@@ -325,6 +325,7 @@ class SimpleProgressBar {
}
update(loaded, max) {
// log('galleryUpdate', { loaded, max });
this.#progress.style.width = `${Math.floor((loaded / max) * 100)}%`;
this.#text.textContent = `${loaded}/${max}`;
if (!this.#visible) {
ui/globals.d.ts
Vendored
+1
View File
@@ -23,6 +23,7 @@ declare global {
// state objects
api: string; // ui/startup.ts
subpath: string; // ui/startup.ts
initStartup?: () => void; // ui/startup.ts
opts: Record<string, any>; // ui/ui.ts
localization?: Record<string, any>; // ui/ui.ts
titles?: Record<string, any>; // ui/ui.ts
ui/js/login.js
+3
View File
@@ -34,6 +34,9 @@ function forceLogin() {
document.body.appendChild(form);
const status = form.querySelector('#loginStatus');
if (!status) {
console.error('forceLogin', 'loginStatus element not found');
}
form.addEventListener('submit', (event) => {
event.preventDefault();
ui/loader.ts
+10
View File
@@ -112,6 +112,16 @@ async function createSplash() {
.catch((err) => error(`getMOTD: ${err}`));
log('loadGradioUi');
const splashMonitor = setInterval(() => {
const splashVisible = !!document.getElementById('splash');
if (splashVisible) {
log('splashVisible', { visible: true, elapsed: Math.round(performance.now() - appStartTime) });
} else {
log('splashVisible', { visible: false, elapsed: Math.round(performance.now() - appStartTime) });
clearInterval(splashMonitor);
}
}, 5000);
}
window.onload = createSplash;
ui/monitor.ts
+1 -1
View File
@@ -65,7 +65,7 @@ export class ConnectionMonitorState {
else return;
}
this.element.dataset.hint = this.toHTML();
this.element.style.backgroundColor = this.online ? 'var(--sd-main-accent-color)' : 'var(--color-error)';
this.element.style.background = this.online ? 'var(--sd-main-accent-color)' : 'var(--color-error)';
}
}
ui/script.ts
+6 -1
View File
@@ -139,6 +139,7 @@ const ignoreElementsSet = new Set(ignoreElements);
const ignoreClasses = ['wrap'];
let mutationTimer: ReturnType<typeof setTimeout> | undefined;
let mutationTS: number | undefined;
let validMutations = [];
async function mutationCallback(mutations) {
@@ -152,9 +153,12 @@ async function mutationCallback(mutations) {
}
if (validMutations.length < 1) return;
if (mutationTimer) clearTimeout(mutationTimer);
if (mutationTimer) clearTimeout(mutationTimer); // bounce
mutationTimer = setTimeout(async () => {
const ts = Date.now() - mutationTS;
if (!executedOnLoaded && (ts > 1000)) log('onUiLoaded delayed', { ts, prompts: anyPromptExists() });
if (!executedOnLoaded && anyPromptExists()) { // execute once
log('onUiLoaded', ts);
executedOnLoaded = true;
executeCallbacks(uiLoadedCallbacks);
}
@@ -174,6 +178,7 @@ async function mutationCallback(mutations) {
document.addEventListener('DOMContentLoaded', () => {
log('DOMContentLoaded');
mutationTS = Date.now();
gradioObserver = new MutationObserver(mutationCallback);
gradioObserver.observe(gradioApp(), { childList: true, subtree: true, attributes: false });
});
ui/startup.ts
+13 -5
View File
@@ -27,25 +27,29 @@ window.api = '/sdapi/v1';
window.subpath = '';
const startupPromises: Promise<unknown>[] = [];
let ok = false;
let optsReady = false;
let initialized = false;
async function waitForOpts() {
// make sure all of the ui is ready and options are loaded
const t0 = performance.now();
let t1 = performance.now();
while (true) {
if (t1 - t0 > 120000) {
if (t1 - t0 > 60000) {
log('waitForOpts timeout');
break;
}
if (window.opts && Object.keys(window.opts).length > 0) {
ok = window.opts.theme_type === 'Modern' ? 'uiux_separator_appearance' in window.opts : true;
if (ok) {
optsReady = (window.opts.theme_type === 'Modern') ? 'uiux_separator_appearance' in window.opts : true;
if (optsReady) {
log('waitForOpts', Math.round(t1 - t0));
timer('waitForOpts', t1 - t0);
break;
}
}
if (t1 - t0 > 15000) {
log('waitForOpts delayed', Math.round(t1 - t0));
}
await sleep(100);
t1 = performance.now();
}
@@ -66,7 +70,9 @@ async function updateSubpath() {
log('API', { url: window.api });
}
async function initStartup() {
export async function initStartup() {
if (initialized) return;
initialized = true;
const t0 = performance.now();
log('initGradio', Math.round(t0 - appStartTime));
timer('initGradio', t0 - appStartTime);
@@ -126,6 +132,8 @@ async function initStartup() {
onUiLoaded(initStartup);
onUiReady(() => log('uiReady'));
window.initStartup = initStartup;
// onAfterUiUpdate(() => log('evt onAfterUiUpdate'));
// onUiLoaded(() => log('evt onUiLoaded'));
// onOptionsChanged(() => log('evt onOptionsChanged'));
webui.py
+3 -2
View File
@@ -249,7 +249,7 @@ def async_policy():
if shared.cmd_opts.profile:
loop.slow_callback_duration = 0.001
loop.set_debug(shared.cmd_opts.profile)
log.debug(f'AsyncIO: loop={loop}')
log.debug(f'AsyncIO: loop={loop.__class__.__name__}')
loop.set_task_factory(verbose_task_factory)
loop.set_exception_handler(self.handle_exception)
return loop
@@ -429,7 +429,8 @@ def start_ui():
uc = shared.demo.server.config
get_name = lambda c: getattr(c, '__name__', c) # pylint: disable=unnecessary-lambda-assignment
log.debug(f'Server config: loop={shared.demo.server.loop} http={get_name(uc.http_protocol_class)} ws={get_name(uc.ws_protocol_class)} interface={uc.interface} workers={uc.workers} backlog={uc.backlog} timeout_keep_alive={uc.timeout_keep_alive} timeout_notify={uc.timeout_notify} ws_max_size={uc.ws_max_size} ws_max_queue={uc.ws_max_queue} ws_ping_interval={uc.ws_ping_interval} ws_ping_timeout={uc.ws_ping_timeout}')
log.debug(f'Server config: loop={shared.demo.server.loop} http={get_name(uc.http_protocol_class)} ws={get_name(uc.ws_protocol_class)} interface={uc.interface} workers={uc.workers} backlog={uc.backlog} timeout_keep_alive={uc.timeout_keep_alive}')
# log.debug(f'Server config: loop={shared.demo.server.loop} http={get_name(uc.http_protocol_class)} ws={get_name(uc.ws_protocol_class)} interface={uc.interface} workers={uc.workers} backlog={uc.backlog} timeout_keep_alive={uc.timeout_keep_alive} timeout_notify={uc.timeout_notify} ws_max_size={uc.ws_max_size} ws_max_queue={uc.ws_max_queue} ws_ping_interval={uc.ws_ping_interval} ws_ping_timeout={uc.ws_ping_timeout}')
if shared.cmd_opts.data_dir is not None:
modules.gr_tempdir.register_tmp_file(shared.demo, os.path.join(shared.cmd_opts.data_dir, 'x'))
wiki
+1 -1
Submodule wiki updated: 3912bb1255...5e3c670429