Wan2.2 Animate, Character Animation and Replacement
Unified character animation and replacement framework with precise motion and expression replication.
About
This ComfyUI workflow demonstrates a unified approach to character animation and replacement using the Wan 2.2 Animate video diffusion model. It extracts motion from an input video, segments the target subject, and synthesizes a new character (or restyles the original) that follows the same motion. Core components include UNETLoader loading the Wan2_2-Animate-14B_fp8_e4m3fn_scaled_KJ.safetensors checkpoint, VAELoader for decoding frames, CLIPLoader and CLIPTextEncode for positive/negative prompts, and SaveVideo for final output.
Motion cues are produced with comfyui_controlnet_aux’s DWPreprocessor (human pose/whole-body keypoints), while subject isolation comes from ComfyUI-segment-anything-2 via DownloadAndLoadSAM2Model and Sam2Segmentation. If you want to lock identity or style from an image, CLIPVisionLoader + CLIPVisionEncode provide additional reference conditioning, and LoraLoaderModelOnly lets you inject a character LoRA.
Technically, the workflow wires LoadVideo through GetVideoComponents, then preprocesses the background and subject: Sam2Segmentation builds a character mask (guided by user “green points”), and DWPreprocessor extracts pose from each frame. PixelPerfectResolution ensures the working size fits your GPU and stays compliant with the WanAnimateToVideo constraint (width and height must be multiples of 16). ModelSamplingSD3 controls your sampler and schedule for the Wan UNet, and comfyui-kjnodes utilities help with the optimized FP8-scaled model.
The result is a flexible pipeline for motion-consistent swaps or restyling—use a single reference image, a LoRA, and prompts to steer identity and appearance while preserving the original motion.
What you provide
- Reading the workflow
Runs in ComfyUI. This workflow can't run on this page. Open it in Comfy Cloud with everything loaded, or download it for your own ComfyUI.
Try in CloudInside the workflow
A read-only preview of the node graph. Open it in Comfy Cloud or download it to edit the nodes.
Loading the graph
Open full-size workflow preview- Author
- ComfyUI
Requires custom nodes
- Reading the workflow
Use this workflow from code
Generated from the workflow graph and inputs when possible; stored examples are shown otherwise. Review credentials and endpoints before running code, which may incur charges.
What you need
- Runs on
- Comfy Cloud
- Key
- COMFY_API_KEY
Needs a paid Cloud plan and credits.
# Python 3.10+: pip install comfy-sdk==0.4.0
import os
from comfy_sdk import Comfy
api_key = os.environ.get("COMFY_API_KEY", "").strip()
if not api_key: raise RuntimeError("Set COMFY_API_KEY")
os.environ.setdefault("COMFY_BASE_URL", "https://cloud.comfy.org")
client = Comfy(api_key=api_key)
workflow = client.workflows.from_json({
"1": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"2",
0
],
"text": "色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走"
},
"_meta": {
"title": "CLIP Text Encode (Negative Prompt)"
}
},
"2": {
"class_type": "CLIPLoader",
"inputs": {
"clip_name": "umt5_xxl_fp8_e4m3fn_scaled.safetensors",
"device": "default",
"type": "wan"
},
"_meta": {
"title": "Load CLIP"
}
},
"3": {
"class_type": "VAELoader",
"inputs": {
"vae_name": "wan_2.1_vae.safetensors"
},
"_meta": {
"title": "Load VAE"
}
},
"4": {
"class_type": "CLIPVisionLoader",
"inputs": {
"clip_name": "clip_vision_h.safetensors"
},
"_meta": {
"title": "Load CLIP Vision"
}
},
"9": {
"class_type": "CLIPVisionEncode",
"inputs": {
"clip_vision": [
"4",
0
],
"crop": "none",
"image": [
"10",
0
]
},
"_meta": {
"title": "CLIP Vision Encode"
}
},
"10": {
"class_type": "LoadImage",
"inputs": {
"image": "video_wan2_2_14B_animate_reference_image.png"
},
"_meta": {
"title": "Load Image"
}
},
"18": {
"class_type": "LoraLoaderModelOnly",
"inputs": {
"lora_name": "lightx2v_I2V_14B_480p_cfg_step_distill_rank64_bf16.safetensors",
"model": [
"20",
0
],
"strength_model": 1
},
"_meta": {
"title": "Load LoRA"
}
},
"19": {
"class_type": "SaveVideo",
"inputs": {
"filename_prefix": "video/ComfyUI",
"format": "auto",
"format.codec": "auto",
"video": [
"232:15",
0
]
},
"_meta": {
"title": "Save Video"
}
},
"20": {
"class_type": "UNETLoader",
"inputs": {
"unet_name": "Wan2_2-Animate-14B_fp8_e4m3fn_scaled_KJ.safetensors",
"weight_dtype": "default"
},
"_meta": {
"title": "Load Diffusion Model"
}
},
"21": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"2",
0
],
"text": "The character is dancing in the room"
},
"_meta": {
"title": "CLIP Text Encode (Positive Prompt)"
}
},
"23": {
"class_type": "GetVideoComponents",
"inputs": {
"video": [
"145",
0
]
},
"_meta": {
"title": "Get Video Components"
}
},
"60": {
"class_type": "ModelSamplingSD3",
"inputs": {
"model": [
"99",
0
],
"shift": 8
},
"_meta": {
"title": "ModelSamplingSD3"
}
},
"99": {
"class_type": "LoraLoaderModelOnly",
"inputs": {
"lora_name": "WanAnimate_relight_lora_fp16.safetensors",
"model": [
"18",
0
],
"strength_model": 1
},
"_meta": {
"title": "Load LoRA"
}
},
"100": {
"class_type": "DWPreprocessor",
"inputs": {
"bbox_detector": "yolox_l.onnx",
"detect_body": "disable",
"detect_face": "enable",
"detect_hand": "disable",
"image": [
"212",
0
],
"pose_estimator": "dw-ll_ucoco_384_bs5.torchscript.pt",
"resolution": [
"158",
0
],
"scale_stick_for_xinsr_cn": "disable"
},
"_meta": {
"title": "DWPose Estimator"
}
},
"101": {
"class_type": "DWPreprocessor",
"inputs": {
"bbox_detector": "yolox_l.onnx",
"detect_body": "enable",
"detect_face": "disable",
"detect_hand": "enable",
"image": [
"212",
0
],
"pose_estimator": "dw-ll_ucoco_384_bs5.torchscript.pt",
"resolution": [
"158",
0
],
"scale_stick_for_xinsr_cn": "disable"
},
"_meta": {
"title": "DWPose Estimator"
}
},
"107": {
"class_type": "Sam2Segmentation",
"inputs": {
"coordinates_positive": [
"229",
0
],
"image": [
"212",
0
],
"individual_objects": False,
"keep_model_loaded": True,
"sam2_model": [
"108",
0
]
},
"_meta": {
"title": "Sam2Segmentation"
}
},
"108": {
"class_type": "DownloadAndLoadSAM2Model",
"inputs": {
"device": "cuda",
"model": "sam2_hiera_base_plus.safetensors",
"precision": "fp16",
"segmentor": "video"
},
"_meta": {
"title": "(Down)Load SAM2Model"
}
},
"145": {
"class_type": "LoadVideo",
"inputs": {
"file": "video_wan2_2_14B_animate_original_video.mp4"
},
"_meta": {
"title": "Load Video"
}
},
"158": {
"class_type": "PixelPerfectResolution",
"inputs": {
"image_gen_height": [
"160",
0
],
"image_gen_width": [
"159",
0
],
"original_image": [
"23",
0
],
"resize_mode": "Just Resize"
},
"_meta": {
"title": "Pixel Perfect Resolution"
}
},
"159": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 640
},
"_meta": {
"title": "Width"
}
},
"160": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 640
},
"_meta": {
"title": "Height"
}
},
"212": {
"class_type": "ImageScale",
"inputs": {
"crop": "center",
"height": [
"160",
0
],
"image": [
"23",
0
],
"upscale_method": "lanczos",
"width": [
"159",
0
]
},
"_meta": {
"title": "Upscale Image"
}
},
"229": {
"class_type": "PointsEditor",
"inputs": {
"bbox_format": "xyxy",
"bbox_store": "[{}]",
"bboxes": "[{}]",
"bg_image": [
"212",
0
],
"coordinates": "[{\"x\":256,\"y\":256}]",
"height": 640,
"neg_coordinates": "[{\"x\":0,\"y\":0}]",
"normalize": False,
"points_store": "{\"positive\":[{\"x\":256,\"y\":256}],\"negative\":[{\"x\":0,\"y\":0}]}",
"width": 640
},
"_meta": {
"title": "Points Editor"
}
},
"243": {
"class_type": "SaveVideo",
"inputs": {
"filename_prefix": "video/ComfyUI",
"format": "auto",
"format.codec": "auto",
"video": [
"242:88",
0
]
},
"_meta": {
"title": "Save Video"
}
},
"274": {
"class_type": "GrowMask",
"inputs": {
"expand": 10,
"mask": [
"107",
0
],
"tapered_corners": True
},
"_meta": {
"title": "Grow Mask"
}
},
"275": {
"class_type": "DrawMaskOnImage",
"inputs": {
"color": "0, 0, 0",
"device": "cpu",
"image": [
"212",
0
],
"mask": [
"276",
0
]
},
"_meta": {
"title": "Draw Mask On Image"
}
},
"276": {
"class_type": "BlockifyMask",
"inputs": {
"block_size": 32,
"device": "cpu",
"masks": [
"274",
0
]
},
"_meta": {
"title": "Blockify Mask"
}
},
"232:15": {
"class_type": "CreateVideo",
"inputs": {
"audio": [
"23",
1
],
"bit_depth": "auto",
"codec": "none",
"color_space": "sRGB",
"fps": 16,
"images": [
"232:230",
0
]
},
"_meta": {
"title": "Create Video"
}
},
"232:230": {
"class_type": "ImageFromBatch",
"inputs": {
"batch_index": [
"232:62",
4
],
"image": [
"232:58",
0
],
"length": 4096
},
"_meta": {
"title": "Get Image from Batch"
}
},
"232:57": {
"class_type": "TrimVideoLatent",
"inputs": {
"samples": [
"232:63",
0
],
"trim_amount": [
"232:62",
3
]
},
"_meta": {
"title": "Trim Video Latent"
}
},
"232:58": {
"class_type": "VAEDecode",
"inputs": {
"samples": [
"232:57",
0
],
"vae": [
"3",
0
]
},
"_meta": {
"title": "VAE Decode"
}
},
"232:62": {
"class_type": "WanAnimateToVideo",
"inputs": {
"background_video": [
"275",
0
],
"batch_size": 1,
"character_mask": [
"276",
0
],
"clip_vision_output": [
"9",
0
],
"continue_motion_max_frames": 5,
"face_video": [
"100",
0
],
"height": [
"160",
0
],
"length": 77,
"negative": [
"1",
0
],
"pose_video": [
"101",
0
],
"positive": [
"21",
0
],
"reference_image": [
"10",
0
],
"vae": [
"3",
0
],
"video_frame_offset": 0,
"width": [
"159",
0
]
},
"_meta": {
"title": "WanAnimateToVideo"
}
},
"232:63": {
"class_type": "KSampler",
"inputs": {
"cfg": 1,
"denoise": 1,
"latent_image": [
"232:62",
2
],
"model": [
"60",
0
],
"negative": [
"232:62",
1
],
"positive": [
"232:62",
0
],
"sampler_name": "euler",
"scheduler": "simple",
"seed": 1106558644923357,
"steps": 6
},
"_meta": {
"title": "KSampler"
}
},
"242:85": {
"class_type": "ImageFromBatch",
"inputs": {
"batch_index": [
"242:90",
4
],
"image": [
"242:87",
0
],
"length": 4096
},
"_meta": {
"title": "Get Image from Batch"
}
},
"242:86": {
"class_type": "ImageBatch",
"inputs": {
"image1": [
"232:58",
0
],
"image2": [
"242:85",
0
]
},
"_meta": {
"title": "Batch Images (DEPRECATED)"
}
},
"242:87": {
"class_type": "VAEDecode",
"inputs": {
"samples": [
"242:92",
0
],
"vae": [
"3",
0
]
},
"_meta": {
"title": "VAE Decode"
}
},
"242:88": {
"class_type": "CreateVideo",
"inputs": {
"audio": [
"23",
1
],
"bit_depth": "auto",
"codec": "none",
"color_space": "sRGB",
"fps": 16,
"images": [
"242:86",
0
]
},
"_meta": {
"title": "Create Video"
}
},
"242:90": {
"class_type": "WanAnimateToVideo",
"inputs": {
"background_video": [
"275",
0
],
"batch_size": 1,
"character_mask": [
"276",
0
],
"clip_vision_output": [
"9",
0
],
"continue_motion": [
"232:58",
0
],
"continue_motion_max_frames": 5,
"face_video": [
"100",
0
],
"height": [
"159",
0
],
"length": 77,
"negative": [
"1",
0
],
"pose_video": [
"101",
0
],
"positive": [
"21",
0
],
"reference_image": [
"10",
0
],
"vae": [
"3",
0
],
"video_frame_offset": [
"232:62",
5
],
"width": [
"159",
0
]
},
"_meta": {
"title": "WanAnimateToVideo"
}
},
"242:91": {
"class_type": "KSampler",
"inputs": {
"cfg": 1,
"denoise": 1,
"latent_image": [
"242:90",
2
],
"model": [
"60",
0
],
"negative": [
"242:90",
1
],
"positive": [
"242:90",
0
],
"sampler_name": "euler",
"scheduler": "simple",
"seed": 387378783691198,
"steps": 6
},
"_meta": {
"title": "KSampler"
}
},
"242:92": {
"class_type": "TrimVideoLatent",
"inputs": {
"samples": [
"242:91",
0
],
"trim_amount": [
"242:90",
3
]
},
"_meta": {
"title": "Trim Video Latent"
}
}
})
# run() also passes api_key to the partner nodes in this workflow.
job = client.run(workflow, api_key=api_key)Questions & answers
Required: Wan2_2-Animate-14B_fp8_e4m3fn_scaled_KJ.safetensors (UNETLoader), VAELoader, CLIPLoader, CLIPTextEncode, SaveVideo. For guidance and masking: comfyui_controlnet_aux (DWPreprocessor), ComfyUI-segment-anything-2 (DownloadAndLoadSAM2Model, Sam2Segmentation). Optional: CLIPVisionLoader/CLIPVisionEncode for reference image conditioning, LoraLoaderModelOnly for character LoRA, and comfyui-kjnodes utilities.
Use PixelPerfectResolution (or Image Scale, if present) to set a manageable size and ensure width/height are multiples of 16 (WanAnimateToVideo requirement). Start smaller if VRAM is limited, then scale up. Longer videos and higher resolutions increase preprocessing and sampling time.
Load the SAM2 model via DownloadAndLoadSAM2Model, then in Sam2Segmentation add green points (hold Shift and click) on the subject you want. Add a few background (red) points if your node exposes them, and preview with MaskPreview. Iterate: more targeted points near boundaries improve hair and limb separation.
Yes. Feed a reference image through CLIPVisionEncode (with CLIPVisionLoader) and/or load a matching LoRA via LoraLoaderModelOnly. If identity flickers, reduce resolution slightly, increase steps, fix the seed, strengthen the mask around the face, and use a concise, consistent prompt with a strong negative prompt.