17 KiB
CLI
TheNoise's command-line interface. The model type is auto-detected from the DiT checkpoint.
Commands assume a dev checkout (
.venv/bin/python,./thenoise.sh). On a portable bundle use./bin/thenoiseand./bin/python3instead.
Operation modes
| Command | What it does | Model needed |
|---|---|---|
generate |
Generate one image from a text prompt | yes |
edit |
Edit an image from an instruction (image + prompt → edited image) | model that takes a reference |
video |
Generate one clip (text, or an image to start from) | video model |
upscale |
Pixel-upscale an existing image | no (upscaler only) |
controlmap |
Extract a control map (canny / depth / pose) from an image | no |
serve |
Serve a model over HTTP: JSON API + web UI at http://localhost:8000/ | yes (lazily) |
Shared flags (serve, generate, edit, video)
| Flag | Required | Default | Description |
|---|---|---|---|
--dit |
generate / edit / video |
— | Path to the DiT checkpoint (.safetensors) |
--vae |
generate / edit / video |
— | Path to the VAE checkpoint (.safetensors) |
--text-encoder |
generate / edit / video |
— | Path to the text encoder checkpoint (.safetensors) |
--lora-dir |
no | — | Directory containing LoRA .safetensors files |
--device |
no | cuda |
Inference device (ROCm aliases cuda → hip) |
serve requires only --dit. Any further checkpoint is determined by the loaded
model, and named by flag if it is missing.
Serve a model over HTTP
./thenoise.sh serve \
--dit ./models/krea2/diffusion_models/krea2_turbo_bf16.safetensors \
--vae ./models/krea2/vae/qwen_image_vae.safetensors \
--text-encoder ./models/krea2/text_encoders/qwen3vl_4b_bf16.safetensors \
--host 127.0.0.1 --port 8000
Then open http://localhost:8000/ for the web UI. Full endpoint reference: HTTP API.
| Flag | Default | Description |
|---|---|---|
--dit-low |
— | Low-noise DiT of a two-expert model, e.g. Wan 2.2 A14B; --dit then selects the high-noise expert |
--audio-vae |
— | Audio VAE and vocoder checkpoint, for a release that ships them apart from the transformer |
--spatial-upscaler |
— | Spatial latent upscaler checkpoint, for a release that ships it apart |
--text-projection |
— | Text projection checkpoint, for a release that packs it apart from the transformer |
--host |
127.0.0.1 |
Bind host |
--port |
8000 |
Bind port |
--upscaler-dir |
— | Directory containing pixel upscaler .safetensors files (e.g. Real-ESRGAN); selected per-request via pixel_upscaler |
--controlmap-dir |
— | Directory of control-map checkpoints; each subdirectory's config.json declares the kind it implements |
--cache-dir |
~/.cache/thenoise |
Where generated files are kept; queued jobs' images and clips land in <cache-dir>/jobs |
Generate a single image
./thenoise.sh generate \
--dit ./models/anima/split_files/diffusion_models/anima-turbo-v1.0.safetensors \
--vae ./models/anima/split_files/vae/qwen_image_vae.safetensors \
--text-encoder ./models/anima/split_files/text_encoders/qwen_3_06b_base.safetensors \
--prompt "a fox walking in the snow" --steps 8 --guidance-scale 1 \
--out /tmp/fox.png
| Flag | Required | Default | Description |
|---|---|---|---|
--prompt |
yes | — | Text prompt |
--negative-prompt |
no | "" |
Negative prompt |
--width |
no | model default | Output width (omit for auto) |
--height |
no | model default | Output height (omit for auto) |
--steps |
no | model default | Denoising steps |
--guidance-scale |
no | model default | CFG scale (≤ 1.0 disables CFG) |
--seed |
no | random | Random seed |
--out |
no | out.png |
Output file path |
--lora |
no | — | LoRA to apply (repeatable, format: file:weight) |
--pixel-upscaler |
no | — | Full path to the pixel upscaler model (one-shot; e.g. a Real-ESRGAN .safetensors) |
--upscale-type |
no | refined |
refined or no-refiner |
--upscale |
no | off | 2× latent upscale with refine denoise (legacy alias for --upscale-type refined --upscale-factor 2) |
--upscale-factor |
no | 1.0 |
Upscale factor (> 0.0; max depends on the pixel upscaler scale, see Upscaling) |
--sampler |
no | model default | Solver: er_sde, euler or unipc |
--qwen-vae-enhance |
no | off | Nyquist notch post-filter (removes 2px grid artifacts) |
--saturation |
no | 0.0 |
Colour saturation around neutral: -1.0 greyscales, 1.0 doubles (-1.0–1.0) |
--vibrance |
no | 0.0 |
Saturation that lifts washed-out pixels and leaves vivid ones alone (-1.0–1.0) |
--temperature |
no | 0.0 |
White balance: -1.0 cools to blue, 1.0 warms to amber (-1.0–1.0) |
--tint |
no | 0.0 |
White balance across the green–magenta axis: -1.0 greens, 1.0 magentas (-1.0–1.0) |
--sharpening |
no | 0.0 |
RCAS sharpening strength (0.0–1.0) |
--film-grain |
no | 0.0 |
Film grain strength (0.0–10.0) |
--kv-cache / --no-kv-cache |
no | auto | Reuse step-invariant K/V across denoising steps. T2I defaults off (opt-in); compatible edits auto-enable for index_timestep_zero |
--ref-method |
no | auto | How a reference is used for editing: index, index_timestep_zero (reference tokens conditioned at timestep zero, which is what makes the KV cache valid) or vision (the text encoder reads the picture instead of the VAE, so nothing is repainted). Auto = detected from the checkpoint, else the model's first listed method. The cache default follows it |
Note: T2I caching is supported by models with a cacheable text prefix; other cache-capable models may accept the flag without a T2I speedup.
Edit an image
Edits an existing image from an instruction (image + prompt → edited image).
Requires an editing-capable model and shares all generation flags with generate.
--image is repeatable. The first image sets the output size when
--width/--height are omitted: its own aspect ratio at about a megapixel.
A model whose only way with a picture is the text encoder — MOSS-Image — lists
vision and nothing else, so edit is how you show it one.
./thenoise.sh edit \
--dit ./models/klein/split_files/diffusion_models/flux-2-klein-4b.safetensors \
--vae ./models/klein/split_files/vae/flux2-vae.safetensors \
--text-encoder ./models/klein/split_files/text_encoders/qwen_3_4b.safetensors \
--image /tmp/fox.png \
--prompt "a fox wearing a red scarf" --steps 4 --sampler euler \
--out /tmp/fox_edited.png
| Flag | Required | Default | Description |
|---|---|---|---|
--image |
yes | — | Input image(s) to edit; repeatable for multiple reference images (first sets the output size) |
--out |
no | out_edit.png |
Output file path |
Generate a single clip
Renders one clip and writes it to --out. Needs a video model — the ones
supported today are in the video models list — and a
two-expert variant takes --dit-low beside --dit. A model that renders sound
takes the component flags above wherever its release ships those parts as their
own files.
./thenoise.sh video \
--dit ./models/wan22/split_files/diffusion_models/fastwan2.2_ti2v-5b_bf16.safetensors \
--vae ./models/wan22/split_files/vae/wan2.2_vae.safetensors \
--text-encoder ./models/wan22/split_files/text_encoders/umt5_xxl_fp16.safetensors \
--prompt "a fox walking through snow" --frames 33 --steps 4 --guidance-scale 1 \
--out /tmp/fox.mp4
Unlike generate, no flag of this one has a default: what each release ships is
the variant's to say, so a flag you leave out stays unset. The run logs the size,
frame count, frame rate, solver and seed it settled on before the first step, and
bars the stages as it goes.
| Flag | Required | Default | Description |
|---|---|---|---|
--prompt |
yes | — | Text prompt |
--negative-prompt |
no | model default | Negative prompt (not sent at all at CFG ≤ 1.0) |
--width / --height |
no | model default | Clip size, as the model rounds it; with --image and neither given, the image's aspect ratio decides |
--frames |
no | model default | Frames, rounded onto the model's temporal grid |
--duration |
no | — | Clip length in seconds, an alternative to --frames |
--fps |
no | model default | Frame rate, both of the clip and of the container |
--steps |
no | model default | Denoising steps |
--guidance-scale |
no | model default | CFG; on a two-expert model, of the high-noise half |
--flow-shift |
no | model default | Schedule shift; a distilled LoRA wants the one it ships with |
--sampler |
no | model default | Denoising solver: er_sde, euler or unipc |
--seed |
no | random | Random seed |
--image |
no | — | image the clip starts on |
--end-image |
no | — | image the clip ends on; needs --image, and a model that can be told where a clip ends (FLF2V) |
--lora |
no | — | LoRA for the primary transformer; repeatable, file:weight |
--lora-low |
no | — | LoRA for the low-noise expert of a --dit-low model; a LoRA meant for the whole clip goes to both flags |
--output-format |
no | model default | mp4 or webm; --out's extension answers when this is left out |
--out |
no | out.mp4 |
Output clip path; with no extension, the container the clip was written in |
Waiting for a clip is the difference from the job API: queue the same render with
POST /jobs/video instead — see HTTP API.
Upscale a single image
Pixel-upscales an existing image. Model-free - no --dit/--vae/
--text-encoder needed:
./thenoise.sh upscale \
--pixel-upscaler ./models/esrgan/RealESRGAN_x4plus.safetensors \
--input /tmp/fox.png --upscale-factor 4 \
--out /tmp/fox_4x.png
| Flag | Required | Default | Description |
|---|---|---|---|
--pixel-upscaler |
yes | — | Full path to the pixel upscaler model (e.g. a Real-ESRGAN .safetensors) |
--input |
yes | — | Input image to upscale |
--upscale-factor |
no | 0.0 |
Upscale factor (0.0 = the model's detected scale; must be in [1, that scale]; larger values are rejected) |
--out |
no | out_upscaled.png |
Output image path |
Control-map a control map
Extracts a conditioning map - Canny edges, a Depth-Anything-V2 depth map or an openpose-style pose skeleton - from an existing image. Model-free.
./thenoise.sh controlmap \
--input /tmp/photo.jpg --type canny --out /tmp/photo_canny.png
./thenoise.sh controlmap \
--input /tmp/photo.jpg --type depth --out /tmp/photo_depth.png \
--controlmap-dir ./models/controlmaps
./thenoise.sh controlmap \
--input /tmp/photo.jpg --type pose --out /tmp/photo_pose.png \
--controlmap-dir ./models/controlmaps
Depth needs its checkpoint and pose needs two (a keypoint model and a person detector): see Control-map extraction.
| Flag | Required | Default | Description |
|---|---|---|---|
--input |
yes* | — | Input image to extract from (*unless --list) |
--type |
yes* | — | Control map to extract; repeatable, each extra type writes its own <out>_<type>.png |
--model |
no | type default | Implementation to use for that type |
--controlmap-dir |
no | — | Directory of control-map checkpoints (e.g. ./models/controlmaps); each subdirectory's config.json declares the kind it implements |
--list |
no | — | List the available control-map types, and what a half-installed kind still needs, then exit |
--canny-low |
no | 100 |
Canny low threshold, in Sobel magnitude units |
--canny-high |
no | 200 |
Canny high threshold; weak edges connected to a strong one are kept |
--canny-thickness |
no | 1 |
Rendered edge thickness in pixels |
--lineart-threshold |
no | 0.5 |
Line-art: line intensity threshold, 0-1 |
--lineart-thickness |
no | 1 |
Line-art: line thickness in pixels |
--softedge-threshold |
no | 0.2 |
Soft-edge: where the gradient starts to count, 0-1 |
--softedge-blur |
no | 1.5 |
Soft-edge: smoothing sigma in pixels, 0 = none |
--depth-invert / --no-depth-invert |
no | false |
Depth: swap near/far (near is bright by default) |
--depth-colormap |
no | gray |
Depth: gray or turbo (a vivid rainbow) |
--pose-detector |
no | first found | Pose: which RT-DETR checkpoint to crop people with |
--pose-threshold |
no | 0.5 |
Pose: minimum detector score for a person to be in the map at all |
--pose-kp-threshold |
no | 0.3 |
Pose: minimum confidence for a joint to be drawn |
--pose-line-width |
no | 4 |
Pose: skeleton bone thickness in pixels |
--pose-radius |
no | 4 |
Pose: radius of the joint dots |
--pose-background |
no | black |
Pose: black or green (the canvas SD's original openpose map used) |
--out |
no | out_control.png |
Output image path |
The --<type>-<option> flags, the /controlmap request fields and the web UI form all
come from the same declaration.
The map is white edges on black (--type canny), dark lines on white
(--type lineart), soft dark edges on white (--type softedge), a grey or turbo
rainbow depth field (--type depth) or a coloured skeleton (--type pose), at the
size of the image it was extracted from, so it can be paired with that image later. It
is written as a PNG with a controlmap_data text chunk recording the type, the model
and every option that differed from its default, alongside whatever metadata the input
already carried.
Upscaling
TheNoise supports up to 8× upscaling through two complementary mechanisms. Both are optional and can be combined.
Latent upscale + refiner (refined)
Every model ships a built-in latent upscaler that runs in latent
space before the VAE decode: it upscales the latent 2× and then runs a short,
low-strength refine denoise at the upscaled size. This is the default
upscale_type and needs no extra model files - a 2× upscale works out of the
box on any supported model.
./thenoise.sh generate \
--dit ... --vae ... --text-encoder ... \
--prompt "a fox walking in the snow" \
--upscale-type refined --upscale-factor 2 \
--out fox_2x.png
Pixel-domain upscaler (no-refiner, and beyond 2×)
Pixel upscaling operates purely in pixel space (after decode) and uses a
dedicated upscaler model, Real-ESRGAN. It is not a model concern:
the upscaler directory is server configuration (--upscaler-dir), and the named
model is selected per-request via pixel_upscaler. Only the last-used upscaler
is kept loaded (switched on change).
# download the optional Real-ESRGAN x4 pixel upscaler (portable bundle:
# ./bin/python3 scripts/download.py --model esrgan)
.venv/bin/python scripts/download.py --model esrgan
A pixel upscaler is required for no-refiner mode (pixel upscaler only, no
latent 2×), and for refined factors above the latent 2×. Without one, only
refined factors up to 2× are available.
The max factor follows the detected upscaler scale. For a 4× Real-ESRGAN model:
no-refiner is limited to 4×, and refined to 2× (latent) × 4× (pixel) = 8×.
# generate with an 8x total upscale (2x latent refine + 4x pixel)
./thenoise.sh generate \
--dit ... --vae ... --text-encoder ... \
--prompt "a fox walking in the snow" \
--upscale-type refined --upscale-factor 8 \
--pixel-upscaler ./models/esrgan/RealESRGAN_x4plus.safetensors \
--out fox_8x.png
LoRAs
Place .safetensors LoRA files in a directory and point --lora-dir at it
(both serve and generate). Then apply LoRAs per-request:
./thenoise.sh generate \
--dit ... --vae ... --text-encoder ... \
--lora-dir ./models/loras \
--prompt "a cyberpunk cityscape" \
--lora "style-cyberpunk:0.8" \
--lora "sub/detail-booster:0.5" \
--out /tmp/city.png
LoRA format is filename:weight - the .safetensors extension is appended
automatically. Omit :weight to use the default of 1.0. LoRAs are switched
in-memory without reloading the base model.
On serve, available LoRAs are listed at GET /lora and requested per-call
via lora_specs (see HTTP API).
First run
The DiT model is compiled with torch.compile on load. The first generation
will be noticeably slower while the inductor traces and compiles kernels, and
you will see some warnings on the console — these are normal. All subsequent
generations use the cached compiled code and run at full speed. Compilation is
transparent: no configuration needed.