h3-latent-upscaler

KO | EN

gitlite — search

#c #deep-learning #devops #open-source #mcp #ios #kubernetes #chatbot #monitoring #php

h3-latent-upscaler

(★ 9)

ComfyUI custom nodes for spatially upscaling a MiniMax H3 video latent between two sampling passes (low-res -> upscale -> high-res)

File Explorer

ZIP Download

- minimax_h3_fl2va_two_pass_upscale.json
- minimax_h3_ref2va_two_pass_upscale.json
- conftest.py
- test_conditioning.py
- test_nodes.py
- test_upscale.py
.gitignore
__init__.py
LICENSE
nodes.py
pyproject.toml
README.md
upscale.py

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

ComfyUI custom nodes for spatially upscaling a MiniMax H3 video latent between
two sampling passes (low-res -> upscale -> high-res).

## Nodes

**MiniMax H3 Latent Upscale** (`model/latent/minimax`)
Inputs: `samples` (`LATENT`, plain video tensor — use after `LTXVSeparateAVLatent`),
`scale_by` (`FLOAT`), `upscale_method` (`nearest-exact` / `bilinear` / `area` /
`bicubic` / `bislerp`).
Output: `LATENT`.
Upscales H/W by `scale_by`, snapped up to a multiple of 2 in latent space (the
DiT's patch size is `(1, 2, 2)` and unpadded odd sizes will crash).

**MiniMax H3 Conditioning Upscale** (`model/conditioning/minimax`)
Inputs: `conditioning` (`CONDITIONING`), `scale_by`, `upscale_method` — use the
same values as the paired Latent Upscale node.
Output: `CONDITIONING`.
Rescales `minimax_refs` / `minimax_keyframes` visual latents (from
`MiniMaxH3ReferenceToVideo` / `MiniMaxH3ImageToVideo`) so reference/keyframe
tokens stay at the right relative scale and RoPE row layout after the video
canvas grows. Safe no-op if neither is present — leave it wired even for plain
t2va graphs.

**MiniMax H3 Add Noise (resample)** (`model/sampling/noise/minimax`)
Inputs: `model`, `noise` (`NOISE`), `sigmas`, `latent_image` — same inputs as
stock `AddNoise`.
Output: `LATENT`.
Drop-in replacement for stock `AddNoise` when continuing a MiniMax H3 (or any
other CONST/flow-parameterized) latent into a second `SamplerCustomAdvanced`
pass via `DisableNoise`. See "Wiring" below for why stock `AddNoise` produces
corrupted output here.

**MiniMax H3 Shift Sigmas (video -> audio)** (`model/sampling/sigmas/minimax`)
Inputs: `sigmas` (`SIGMAS`), `shift_video` (`FLOAT`, default `12.0`),
`shift_audio` (`FLOAT`, default `3.0`).
Output: `SIGMAS`.
Remaps a video-schedule `sigmas` array onto MiniMax H3's audio schedule.
Insert between the pass-2 `BasicScheduler` and the **audio** branch's
`MiniMaxH3AddNoise` only — see "Wiring" below for why the video and audio
branches need different sigmas.

## Wiring

`LTXVSeparateAVLatent` / `LTXVConcatAVLatent` (split/join the NestedTensor AV
latent) are stock ComfyUI core nodes that work correctly on MiniMax H3's AV
latent as-is. Stock `AddNoise`, however, does **not**, for two independent
reasons:

1. Its implementation calls `torch.count_nonzero(...)` and
   `torch.nan_to_num(...)` directly on the latent tensor
   (`comfy_extras/nodes_custom_sampler.py`), and neither is one of the
   operators `NestedTensor` overloads, so feeding it the joint AV latent
   crashes with `count_nonzero(): argument 'input' ... must be Tensor, not
   NestedTensor`. Fixed by running noise injection on the separated video and
   audio tensors individually (each a plain `Tensor`) and only concatenating
   them back into a NestedTensor afterward, right before the pass-2 sampler.
2. Even on a plain tensor, stock `AddNoise`'s output is wrong for a
   CONST-parameterized flow model like MiniMax H3. `SamplerCustomAdvanced` +
   `DisableNoise` internally calls
   `model_sampling.noise_scaling(sigma, zero_noise, latent_image)` again as
   part of its own setup — for `CONST` that's `(1-sigma)*latent_image`,
   silently scaling stock `AddNoise`'s already-correctly-noised output down a
   *second* time. Verified against a real run: decoding stock `AddNoise`'s
   output directly (before the second `SamplerCustomAdvanced` even runs)
   showed pure noise at `denoise=0.4`, and the final video was a corrupted
   checkerboard/waffle pattern with zero resemblance to the source. Fixed by
   `MiniMaxH3AddNoise` — identical to stock `AddNoise` plus one extra
   `inverse_noise_scaling` call that pre-cancels the sampler's redundant
   rescale. With it, the same `denoise=0.4` run produces a correctly upscaled,
   coherent result.

There is a third, separate issue specific to the **audio** branch:
`comfy/ldm/minimax/model.py` (core) documents that the DiT receives a single
*video*-schedule sigma from the sampler and internally derives the audio
stream's own timestep via a closed-form shift
(`shift_video=12.0` / `shift_audio=3.0` by default — matches the checkpoint's
`sampling_settings`), then rescales the audio velocity it returns to
compensate. That only works if the audio latent handed to the model was
actually noised to the *audio*-schedule sigma, not the video one. Feeding the
audio branch's `MiniMaxH3AddNoise` the same `sigmas_pass2` as video (as if
audio needed "the same noise level") noises it far more than the model
expects at that point in the schedule — e.g. at `denoise=0.4`
(`sigma_v=0.4`), the model's own math expects the audio latent noised to
`sigma_a≈0.14`, not `0.4`. This reads as audio distortion after pass 2.
Fixed by `MiniMaxH3ShiftSigmas`, inserted between the pass-2 `BasicScheduler`
and the audio branch's `MiniMaxH3AddNoise` only (the video branch keeps the
raw `sigmas_pass2`).

```
MiniMaxH3ImageToVideo / MiniMaxH3ReferenceToVideo
  -> conditioning, latent
  -> RandomNoise + BasicGuider + SamplerCustomAdvanced   (pass 1)
  -> denoised latent

-> LTXVSeparateAVLatent(denoised latent) -> video, audio
  -> MiniMax H3 Latent Upscale(video, scale_by, method) -> video_up

-> MiniMax H3 Add Noise (resample)(model, RandomNoise2, sigmas_pass2, video_up) -> noised_video
  -> MiniMax H3 Shift Sigmas(sigmas_pass2, shift_video=12.0, shift_audio=3.0) -> sigmas_pass2_audio
  -> MiniMax H3 Add Noise (resample)(model, RandomNoise2, sigmas_pass2_audio, audio) -> noised_audio
  -> LTXVConcatAVLatent(noised_video, noised_audio) -> noised_latent_up

-> MiniMax H3 Conditioning Upscale(conditioning, scale_by, method) -> conditioning_up
  -> rebuild Guider from conditioning_up

-> SamplerCustomAdvanced (pass 2, DisableNoise + sigmas_pass2 + guider_up + noised_latent_up)
  -> final denoised latent
```

## Example Workflows

`example_workflows/` has the two-pass pipeline above already wired into both
MiniMax H3 task workflows:

- `minimax_h3_ref2va_two_pass_upscale.json` — ref2va (`MiniMaxH3ReferenceToVideo`)
- `minimax_h3_fl2va_two_pass_upscale.json` — fl2va (`MiniMaxH3ImageToVideo`, wired
  inside the "Image to Video (MiniMax H3)" subgraph)

`scale_by=1.5`, `bilinear`, and the pass-2 `BasicScheduler` (`simple`, 15 steps,
`denoise=0.4`) are starting points, not tuned values — adjust by eye once you
see a result.

## Tests

Run from inside this directory, using ComfyUI's own venv (needed for
`comfy_api.latest` and friends):

```bash
cd custom_nodes/h3-latent-upscaler && uv run --project /path/to/ComfyUI pytest -v
```

`pyproject.toml`'s `[tool.pytest.ini_options]` sets `--import-mode=importlib`
(pytest's default "prepend" import mode gets confused walking up through this
hyphenated package's `__init__.py`) and `-p no:torchtyping` (a broken
`torchtyping` pytest plugin registered in the ComfyUI venv otherwise crashes
pytest at startup).

h3-latent-upscaler

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

h3-latent-upscaler

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

Download README