AI on the Lot 2026_LTX-2.3_IC LoRA (Union Control)
The second bonus workflow for AI on the Lot 2026, made available for ComfyUI 101 workshop attendees to open and follow along if time allows. A video editing workflow using LTX 2.3 with the Union Control IC LoRA and canny control. Provide a modified frame from your video and the workflow propagates that change across the clip. For example, take a frame from a video of a shirtless boxer, edit a red shirt onto him, and the output video will show him wearing the red shirt throughout.
概要
This workflow performs reference-based video-to-video editing with LTX 2.3, steered by the Union Control IC LoRA and canny edge guidance. You supply a single edited frame from your clip (for example, painting a red shirt onto a boxer), and the pipeline propagates that visual change to every frame in the video while preserving motion and structure. The core processing happens in the custom processing block (UUID: 9278a6c3-b46a-4e21-8d09-5103f85ffec6), which runs LTX 2.3 inference with the Union Control IC LoRA and integrates canny control to keep edges and silhouettes consistent across time.
Technically, LoadVideo ingests your source clip and feeds frames into the inference block, where canny edge maps are used as a structural prior so the subject’s pose and scene layout remain stable. LoadImage brings in your manually edited reference frame, which the IC LoRA uses as an appearance guide to replicate the change throughout the sequence.
After processing, VHS_VideoCombine assembles the generated frames back into a video, and SaveVideo writes the final output to disk. This approach is ideal when you want targeted, consistent replacements—like color swaps, wardrobe fixes, or logo/graphic updates—without re-shoots or heavy roto work.
入力するもの
- ワークフローを読み込み中
ComfyUI で実行できます。 このワークフローはこのページでは実行できません。すべてが読み込まれた状態でComfy Cloudで開くか、ご自身のComfyUI用にダウンロードしてください。
Cloudで試す例を試す
ワークフローの内部
ノードグラフの読み取り専用プレビューです。ノードを編集するには、Comfy Cloudで開くかダウンロードしてください。
Loading the graph
Open full-size workflow previewカスタムノードが必要です
- ワークフローを読み込み中
コードからこのワークフローを使用する
可能な場合はワークフローグラフと入力から生成されます。それ以外の場合は保存された例が表示されます。料金が発生する場合があるため、コードを実行する前に認証情報とエンドポイントを確認してください。
What you need
- Runs on
- Comfy Cloud
- キー
- COMFY_API_KEY
有料の Cloud プランとクレジットが必要です。
# Python 3.10+: pip install comfy-sdk==0.4.0
import os
from comfy_sdk import Comfy
api_key = os.environ.get("COMFY_API_KEY", "").strip()
if not api_key: raise RuntimeError("Set COMFY_API_KEY")
os.environ.setdefault("COMFY_BASE_URL", "https://cloud.comfy.org")
client = Comfy(api_key=api_key)
workflow = client.workflows.from_json({
"5074": {
"class_type": "SaveVideo",
"inputs": {
"filename_prefix": "output",
"format": "auto",
"format.codec": "auto",
"video": [
"5073:4849",
0
]
},
"_meta": {
"title": "Save Video"
}
},
"5075": {
"class_type": "VHS_VideoCombine",
"inputs": {
"crf": 14,
"filename_prefix": "AnimateDiff",
"format": "video/h264-mp4",
"frame_rate": 8,
"images": [
"5073:4991",
0
],
"loop_count": 0,
"pingpong": False,
"pix_fmt": "yuv420p",
"save_metadata": True,
"save_output": False,
"trim_to_audio": False
},
"_meta": {
"title": "Video Combine 🎥🅥🅗🅢"
}
},
"5076": {
"class_type": "LoadImage",
"inputs": {
"image": "ad51bba819a91ed64386b035c44b36da9d29b2cef9dc02389b96738dbfa974b8.png"
},
"_meta": {
"title": "Load Image"
}
},
"5077": {
"class_type": "LoadVideo",
"inputs": {
"file": "2660a79624272b901d33725cc770e2b16d47528608c5487c553444c6502d0358.mp4"
},
"_meta": {
"title": "Load Video"
}
},
"5073:1241": {
"class_type": "LTXVConditioning",
"inputs": {
"frame_rate": [
"5073:5000",
2
],
"negative": [
"5073:2612",
0
],
"positive": [
"5073:2483",
0
]
},
"_meta": {
"title": "LTXVConditioning"
}
},
"5073:2483": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"5073:5023",
0
],
"text": "Boxer breathing hard"
},
"_meta": {
"title": "CLIP Text Encode (Positive Prompt)"
}
},
"5073:2612": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"5073:5023",
0
],
"text": "pc game, console game, video game, cartoon, childish, ugly"
},
"_meta": {
"title": "CLIP Text Encode (Negative Prompt)"
}
},
"5073:3059": {
"class_type": "EmptyLTXVLatentVideo",
"inputs": {
"batch_size": 1,
"height": [
"5073:5029",
1
],
"length": [
"5073:5029",
2
],
"width": [
"5073:5029",
0
]
},
"_meta": {
"title": "Empty LTXV Latent Video"
}
},
"5073:3159": {
"class_type": "LTXVImgToVideoConditionOnly",
"inputs": {
"bypass": [
"5073:5019",
0
],
"image": [
"5073:5035",
0
],
"latent": [
"5073:3059",
0
],
"strength": 1,
"vae": [
"5073:3940",
2
]
},
"_meta": {
"title": "🅛🅣🅧 LTXV Img To Video Condition Only"
}
},
"5073:3940": {
"class_type": "CheckpointLoaderSimple",
"inputs": {
"ckpt_name": "ltx-2.3-22b-distilled-fp8.safetensors"
},
"_meta": {
"title": "Load Checkpoint"
}
},
"5073:3980": {
"class_type": "LTXVEmptyLatentAudio",
"inputs": {
"audio_vae": [
"5073:4010",
0
],
"batch_size": 1,
"frame_rate": [
"5073:5066",
0
],
"frames_number": [
"5073:5029",
2
]
},
"_meta": {
"title": "Empty LTXV Latent Audio"
}
},
"5073:4010": {
"class_type": "LTXVAudioVAELoader",
"inputs": {
"ckpt_name": "ltx-2.3-22b-distilled-fp8.safetensors"
},
"_meta": {
"title": "Load LTXV Audio VAE"
}
},
"5073:4528": {
"class_type": "LTXVConcatAVLatent",
"inputs": {
"audio_latent": [
"5073:3980",
0
],
"video_latent": [
"5073:5072",
2
]
},
"_meta": {
"title": "Concat AV Latent"
}
},
"5073:4828": {
"class_type": "CFGGuider",
"inputs": {
"cfg": 1,
"model": [
"5073:5069",
0
],
"negative": [
"5073:5072",
1
],
"positive": [
"5073:5072",
0
]
},
"_meta": {
"title": "CFG Guider"
}
},
"5073:4829": {
"class_type": "SamplerCustomAdvanced",
"inputs": {
"guider": [
"5073:4828",
0
],
"latent_image": [
"5073:4528",
0
],
"noise": [
"5073:4832",
0
],
"sampler": [
"5073:4831",
0
],
"sigmas": [
"5073:5025",
0
]
},
"_meta": {
"title": "SamplerCustomAdvanced"
}
},
"5073:4831": {
"class_type": "KSamplerSelect",
"inputs": {
"sampler_name": "euler_ancestral_cfg_pp"
},
"_meta": {
"title": "KSamplerSelect"
}
},
"5073:4832": {
"class_type": "RandomNoise",
"inputs": {
"noise_seed": 42
},
"_meta": {
"title": "RandomNoise"
}
},
"5073:4845": {
"class_type": "LTXVSeparateAVLatent",
"inputs": {
"av_latent": [
"5073:4829",
0
]
},
"_meta": {
"title": "Separate AV Latent"
}
},
"5073:4848": {
"class_type": "LTXVAudioVAEDecode",
"inputs": {
"audio_vae": [
"5073:4010",
0
],
"samples": [
"5073:4845",
1
]
},
"_meta": {
"title": "LTXV Audio VAE Decode"
}
},
"5073:4849": {
"class_type": "CreateVideo",
"inputs": {
"audio": [
"5073:4848",
0
],
"bit_depth": "auto",
"codec": "none",
"color_space": "sRGB",
"fps": [
"5073:5000",
2
],
"images": [
"5073:5065",
0
]
},
"_meta": {
"title": "Create Video"
}
},
"5073:4922": {
"class_type": "LoraLoaderModelOnly",
"inputs": {
"lora_name": "ltx-2.3-22b-distilled-lora-384-1.1.safetensors",
"model": [
"5073:3940",
0
],
"strength_model": 0.5
},
"_meta": {
"title": "Load LoRA"
}
},
"5073:4991": {
"class_type": "CannyEdgePreprocessor",
"inputs": {
"high_threshold": 200,
"image": [
"5073:5026",
0
],
"low_threshold": 92,
"resolution": 1024
},
"_meta": {
"title": "Canny Edge"
}
},
"5073:5000": {
"class_type": "GetVideoComponents",
"inputs": {
"video": [
"5077",
0
]
},
"_meta": {
"title": "Get Video Components"
}
},
"5073:5011": {
"class_type": "LTXICLoRALoaderModelOnly",
"inputs": {
"lora_name": "ltx-2-19b-ic-lora-canny-control.safetensors",
"model": [
"5073:5069",
0
],
"strength_model": 1
},
"_meta": {
"title": "🅛🅣🅧 IC-LoRA Loader Model Only"
}
},
"5073:5013": {
"class_type": "LTXVCropGuides",
"inputs": {
"latent": [
"5073:4845",
0
],
"negative": [
"5073:5072",
1
],
"positive": [
"5073:5072",
0
]
},
"_meta": {
"title": "LTXVCropGuides"
}
},
"5073:5019": {
"class_type": "PrimitiveBoolean",
"inputs": {
"value": False
},
"_meta": {
"title": "bypass_i2v"
}
},
"5073:5023": {
"class_type": "LTXAVTextEncoderLoader",
"inputs": {
"ckpt_name": "ltx-2.3-22b-distilled-fp8.safetensors",
"device": "default",
"text_encoder": "gemma_3_12B_it_fp4_mixed.safetensors"
},
"_meta": {
"title": "Load LTXV Audio Text Encoder"
}
},
"5073:5025": {
"class_type": "ManualSigmas",
"inputs": {
"sigmas": "1.0, 0.99375, 0.9875, 0.98125, 0.975, 0.909375, 0.725, 0.421875, 0.0"
},
"_meta": {
"title": "ManualSigmas"
}
},
"5073:5026": {
"class_type": "ResizeImageMaskNode",
"inputs": {
"input": [
"5073:5000",
0
],
"resize_type": "scale shorter dimension",
"resize_type.shorter_size": 544,
"scale_method": "lanczos"
},
"_meta": {
"title": "Resize Image/Mask"
}
},
"5073:5028": {
"class_type": "ResizeImageMaskNode",
"inputs": {
"input": [
"5073:4991",
0
],
"resize_type": "scale to multiple",
"resize_type.multiple": [
"5073:5034",
0
],
"scale_method": "lanczos"
},
"_meta": {
"title": "Resize Image/Mask"
}
},
"5073:5029": {
"class_type": "GetImageSize",
"inputs": {
"image": [
"5073:5028",
0
]
},
"_meta": {
"title": "Get Image Size"
}
},
"5073:5034": {
"class_type": "SimpleMath+",
"inputs": {
"a": [
"5073:5011",
1
],
"b": 0,
"value": "a*32"
},
"_meta": {
"title": "🔧 Simple Math"
}
},
"5073:5035": {
"class_type": "ResizeImageMaskNode",
"inputs": {
"input": [
"5076",
0
],
"resize_type": "scale longer dimension",
"resize_type.longer_size": 1536,
"scale_method": "lanczos"
},
"_meta": {
"title": "Resize Image/Mask"
}
},
"5073:5065": {
"class_type": "LTXVTiledVAEDecode",
"inputs": {
"horizontal_tiles": 2,
"last_frame_fix": False,
"latents": [
"5073:5013",
2
],
"overlap": 6,
"vae": [
"5073:3940",
2
],
"vertical_tiles": 2,
"working_device": "auto",
"working_dtype": "auto"
},
"_meta": {
"title": "🅛🅣🅧 LTXV Tiled VAE Decode"
}
},
"5073:5066": {
"class_type": "LTXFloatToInt",
"inputs": {
"a": [
"5073:5000",
2
]
},
"_meta": {
"title": "🅛🅣🅧 Float To Int"
}
},
"5073:5069": {
"class_type": "LoraLoaderModelOnly",
"inputs": {
"lora_name": "ltx-2.3-22b-ic-lora-union-control-ref0.5.safetensors",
"model": [
"5073:4922",
0
],
"strength_model": 1
},
"_meta": {
"title": "Load LoRA"
}
},
"5073:5070": {
"class_type": "GetICLoRAParameters",
"inputs": {
"iclora_model": [
"5073:5069",
0
]
},
"_meta": {
"title": "Get IC-LoRA Parameters"
}
},
"5073:5072": {
"class_type": "LTXVAddGuide",
"inputs": {
"frame_idx": 0,
"iclora_parameters": [
"5073:5070",
0
],
"image": [
"5073:5028",
0
],
"latent": [
"5073:3159",
0
],
"negative": [
"5073:1241",
1
],
"positive": [
"5073:1241",
0
],
"strength": 1,
"vae": [
"5073:3940",
2
]
},
"_meta": {
"title": "LTXVAddGuide"
}
}
})
# run() also passes api_key to the partner nodes in this workflow.
job = client.run(workflow, api_key=api_key)よくある質問
No. The workflow uses LTX 2.3 plus the Union Control IC LoRA as a general-purpose conditioning method. You provide one edited frame as a reference, and the model propagates that change across the clip.
Pick a clear frame where the target area is fully visible and not heavily motion‑blurred or occluded. The more unambiguous your edit, the easier it is for the model to generalize it across the sequence.
Canny thresholds control edge sensitivity. Lower thresholds capture more fine detail (pushing stronger edits but risking overreach), while higher thresholds preserve broader structure (maintaining pose and layout but reducing how much the edit can reshape details). Adjust alongside control strength to balance fidelity and change.
That depends on your GPU memory and patience. Start with short segments at moderate resolution. If needed, split long videos into sections, process each, and reassemble downstream in your editor.