Wan2.2 Animate, Character Animation and Replacement
Unified character animation and replacement framework with precise motion and expression replication.
概要
This ComfyUI workflow demonstrates a unified approach to character animation and replacement using the Wan 2.2 Animate video diffusion model. It extracts motion from an input video, segments the target subject, and synthesizes a new character (or restyles the original) that follows the same motion. Core components include UNETLoader loading the Wan2_2-Animate-14B_fp8_e4m3fn_scaled_KJ.safetensors checkpoint, VAELoader for decoding frames, CLIPLoader and CLIPTextEncode for positive/negative prompts, and SaveVideo for final output.
Motion cues are produced with comfyui_controlnet_aux’s DWPreprocessor (human pose/whole-body keypoints), while subject isolation comes from ComfyUI-segment-anything-2 via DownloadAndLoadSAM2Model and Sam2Segmentation. If you want to lock identity or style from an image, CLIPVisionLoader + CLIPVisionEncode provide additional reference conditioning, and LoraLoaderModelOnly lets you inject a character LoRA.
Technically, the workflow wires LoadVideo through GetVideoComponents, then preprocesses the background and subject: Sam2Segmentation builds a character mask (guided by user “green points”), and DWPreprocessor extracts pose from each frame. PixelPerfectResolution ensures the working size fits your GPU and stays compliant with the WanAnimateToVideo constraint (width and height must be multiples of 16). ModelSamplingSD3 controls your sampler and schedule for the Wan UNet, and comfyui-kjnodes utilities help with the optimized FP8-scaled model.
The result is a flexible pipeline for motion-consistent swaps or restyling—use a single reference image, a LoRA, and prompts to steer identity and appearance while preserving the original motion.
入力するもの
- ワークフローを読み込み中
ComfyUI で実行できます。 このワークフローはこのページでは実行できません。すべてが読み込まれた状態でComfy Cloudで開くか、ご自身のComfyUI用にダウンロードしてください。
Cloudで試すワークフローの内部
ノードグラフの読み取り専用プレビューです。ノードを編集するには、Comfy Cloudで開くかダウンロードしてください。
Loading the graph
Open full-size workflow preview- 作成者
- ComfyUI
カスタムノードが必要です
- ワークフローを読み込み中
コードからこのワークフローを使用する
可能な場合はワークフローグラフと入力から生成されます。それ以外の場合は保存された例が表示されます。料金が発生する場合があるため、コードを実行する前に認証情報とエンドポイントを確認してください。
What you need
- Runs on
- Comfy Cloud
- キー
- COMFY_API_KEY
有料の Cloud プランとクレジットが必要です。
# Python 3.10+: pip install comfy-sdk==0.4.0
import os
from comfy_sdk import Comfy
api_key = os.environ.get("COMFY_API_KEY", "").strip()
if not api_key: raise RuntimeError("Set COMFY_API_KEY")
os.environ.setdefault("COMFY_BASE_URL", "https://cloud.comfy.org")
client = Comfy(api_key=api_key)
workflow = client.workflows.from_json({
"1": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"2",
0
],
"text": "色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走"
},
"_meta": {
"title": "CLIP Text Encode (Negative Prompt)"
}
},
"2": {
"class_type": "CLIPLoader",
"inputs": {
"clip_name": "umt5_xxl_fp8_e4m3fn_scaled.safetensors",
"device": "default",
"type": "wan"
},
"_meta": {
"title": "Load CLIP"
}
},
"3": {
"class_type": "VAELoader",
"inputs": {
"vae_name": "wan_2.1_vae.safetensors"
},
"_meta": {
"title": "Load VAE"
}
},
"4": {
"class_type": "CLIPVisionLoader",
"inputs": {
"clip_name": "clip_vision_h.safetensors"
},
"_meta": {
"title": "Load CLIP Vision"
}
},
"9": {
"class_type": "CLIPVisionEncode",
"inputs": {
"clip_vision": [
"4",
0
],
"crop": "none",
"image": [
"10",
0
]
},
"_meta": {
"title": "CLIP Vision Encode"
}
},
"10": {
"class_type": "LoadImage",
"inputs": {
"image": "video_wan2_2_14B_animate_reference_image.png"
},
"_meta": {
"title": "Load Image"
}
},
"18": {
"class_type": "LoraLoaderModelOnly",
"inputs": {
"lora_name": "lightx2v_I2V_14B_480p_cfg_step_distill_rank64_bf16.safetensors",
"model": [
"20",
0
],
"strength_model": 1
},
"_meta": {
"title": "Load LoRA"
}
},
"19": {
"class_type": "SaveVideo",
"inputs": {
"filename_prefix": "video/ComfyUI",
"format": "auto",
"format.codec": "auto",
"video": [
"232:15",
0
]
},
"_meta": {
"title": "Save Video"
}
},
"20": {
"class_type": "UNETLoader",
"inputs": {
"unet_name": "Wan2_2-Animate-14B_fp8_e4m3fn_scaled_KJ.safetensors",
"weight_dtype": "default"
},
"_meta": {
"title": "Load Diffusion Model"
}
},
"21": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"2",
0
],
"text": "The character is dancing in the room"
},
"_meta": {
"title": "CLIP Text Encode (Positive Prompt)"
}
},
"23": {
"class_type": "GetVideoComponents",
"inputs": {
"video": [
"145",
0
]
},
"_meta": {
"title": "Get Video Components"
}
},
"60": {
"class_type": "ModelSamplingSD3",
"inputs": {
"model": [
"99",
0
],
"shift": 8
},
"_meta": {
"title": "ModelSamplingSD3"
}
},
"99": {
"class_type": "LoraLoaderModelOnly",
"inputs": {
"lora_name": "WanAnimate_relight_lora_fp16.safetensors",
"model": [
"18",
0
],
"strength_model": 1
},
"_meta": {
"title": "Load LoRA"
}
},
"100": {
"class_type": "DWPreprocessor",
"inputs": {
"bbox_detector": "yolox_l.onnx",
"detect_body": "disable",
"detect_face": "enable",
"detect_hand": "disable",
"image": [
"212",
0
],
"pose_estimator": "dw-ll_ucoco_384_bs5.torchscript.pt",
"resolution": [
"158",
0
],
"scale_stick_for_xinsr_cn": "disable"
},
"_meta": {
"title": "DWPose Estimator"
}
},
"101": {
"class_type": "DWPreprocessor",
"inputs": {
"bbox_detector": "yolox_l.onnx",
"detect_body": "enable",
"detect_face": "disable",
"detect_hand": "enable",
"image": [
"212",
0
],
"pose_estimator": "dw-ll_ucoco_384_bs5.torchscript.pt",
"resolution": [
"158",
0
],
"scale_stick_for_xinsr_cn": "disable"
},
"_meta": {
"title": "DWPose Estimator"
}
},
"107": {
"class_type": "Sam2Segmentation",
"inputs": {
"coordinates_positive": [
"229",
0
],
"image": [
"212",
0
],
"individual_objects": False,
"keep_model_loaded": True,
"sam2_model": [
"108",
0
]
},
"_meta": {
"title": "Sam2Segmentation"
}
},
"108": {
"class_type": "DownloadAndLoadSAM2Model",
"inputs": {
"device": "cuda",
"model": "sam2_hiera_base_plus.safetensors",
"precision": "fp16",
"segmentor": "video"
},
"_meta": {
"title": "(Down)Load SAM2Model"
}
},
"145": {
"class_type": "LoadVideo",
"inputs": {
"file": "video_wan2_2_14B_animate_original_video.mp4"
},
"_meta": {
"title": "Load Video"
}
},
"158": {
"class_type": "PixelPerfectResolution",
"inputs": {
"image_gen_height": [
"160",
0
],
"image_gen_width": [
"159",
0
],
"original_image": [
"23",
0
],
"resize_mode": "Just Resize"
},
"_meta": {
"title": "Pixel Perfect Resolution"
}
},
"159": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 640
},
"_meta": {
"title": "Width"
}
},
"160": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 640
},
"_meta": {
"title": "Height"
}
},
"212": {
"class_type": "ImageScale",
"inputs": {
"crop": "center",
"height": [
"160",
0
],
"image": [
"23",
0
],
"upscale_method": "lanczos",
"width": [
"159",
0
]
},
"_meta": {
"title": "Upscale Image"
}
},
"229": {
"class_type": "PointsEditor",
"inputs": {
"bbox_format": "xyxy",
"bbox_store": "[{}]",
"bboxes": "[{}]",
"bg_image": [
"212",
0
],
"coordinates": "[{\"x\":256,\"y\":256}]",
"height": 640,
"neg_coordinates": "[{\"x\":0,\"y\":0}]",
"normalize": False,
"points_store": "{\"positive\":[{\"x\":256,\"y\":256}],\"negative\":[{\"x\":0,\"y\":0}]}",
"width": 640
},
"_meta": {
"title": "Points Editor"
}
},
"243": {
"class_type": "SaveVideo",
"inputs": {
"filename_prefix": "video/ComfyUI",
"format": "auto",
"format.codec": "auto",
"video": [
"242:88",
0
]
},
"_meta": {
"title": "Save Video"
}
},
"274": {
"class_type": "GrowMask",
"inputs": {
"expand": 10,
"mask": [
"107",
0
],
"tapered_corners": True
},
"_meta": {
"title": "Grow Mask"
}
},
"275": {
"class_type": "DrawMaskOnImage",
"inputs": {
"color": "0, 0, 0",
"device": "cpu",
"image": [
"212",
0
],
"mask": [
"276",
0
]
},
"_meta": {
"title": "Draw Mask On Image"
}
},
"276": {
"class_type": "BlockifyMask",
"inputs": {
"block_size": 32,
"device": "cpu",
"masks": [
"274",
0
]
},
"_meta": {
"title": "Blockify Mask"
}
},
"232:15": {
"class_type": "CreateVideo",
"inputs": {
"audio": [
"23",
1
],
"bit_depth": "auto",
"codec": "none",
"color_space": "sRGB",
"fps": 16,
"images": [
"232:230",
0
]
},
"_meta": {
"title": "Create Video"
}
},
"232:230": {
"class_type": "ImageFromBatch",
"inputs": {
"batch_index": [
"232:62",
4
],
"image": [
"232:58",
0
],
"length": 4096
},
"_meta": {
"title": "Get Image from Batch"
}
},
"232:57": {
"class_type": "TrimVideoLatent",
"inputs": {
"samples": [
"232:63",
0
],
"trim_amount": [
"232:62",
3
]
},
"_meta": {
"title": "Trim Video Latent"
}
},
"232:58": {
"class_type": "VAEDecode",
"inputs": {
"samples": [
"232:57",
0
],
"vae": [
"3",
0
]
},
"_meta": {
"title": "VAE Decode"
}
},
"232:62": {
"class_type": "WanAnimateToVideo",
"inputs": {
"background_video": [
"275",
0
],
"batch_size": 1,
"character_mask": [
"276",
0
],
"clip_vision_output": [
"9",
0
],
"continue_motion_max_frames": 5,
"face_video": [
"100",
0
],
"height": [
"160",
0
],
"length": 77,
"negative": [
"1",
0
],
"pose_video": [
"101",
0
],
"positive": [
"21",
0
],
"reference_image": [
"10",
0
],
"vae": [
"3",
0
],
"video_frame_offset": 0,
"width": [
"159",
0
]
},
"_meta": {
"title": "WanAnimateToVideo"
}
},
"232:63": {
"class_type": "KSampler",
"inputs": {
"cfg": 1,
"denoise": 1,
"latent_image": [
"232:62",
2
],
"model": [
"60",
0
],
"negative": [
"232:62",
1
],
"positive": [
"232:62",
0
],
"sampler_name": "euler",
"scheduler": "simple",
"seed": 1106558644923357,
"steps": 6
},
"_meta": {
"title": "KSampler"
}
},
"242:85": {
"class_type": "ImageFromBatch",
"inputs": {
"batch_index": [
"242:90",
4
],
"image": [
"242:87",
0
],
"length": 4096
},
"_meta": {
"title": "Get Image from Batch"
}
},
"242:86": {
"class_type": "ImageBatch",
"inputs": {
"image1": [
"232:58",
0
],
"image2": [
"242:85",
0
]
},
"_meta": {
"title": "Batch Images (DEPRECATED)"
}
},
"242:87": {
"class_type": "VAEDecode",
"inputs": {
"samples": [
"242:92",
0
],
"vae": [
"3",
0
]
},
"_meta": {
"title": "VAE Decode"
}
},
"242:88": {
"class_type": "CreateVideo",
"inputs": {
"audio": [
"23",
1
],
"bit_depth": "auto",
"codec": "none",
"color_space": "sRGB",
"fps": 16,
"images": [
"242:86",
0
]
},
"_meta": {
"title": "Create Video"
}
},
"242:90": {
"class_type": "WanAnimateToVideo",
"inputs": {
"background_video": [
"275",
0
],
"batch_size": 1,
"character_mask": [
"276",
0
],
"clip_vision_output": [
"9",
0
],
"continue_motion": [
"232:58",
0
],
"continue_motion_max_frames": 5,
"face_video": [
"100",
0
],
"height": [
"159",
0
],
"length": 77,
"negative": [
"1",
0
],
"pose_video": [
"101",
0
],
"positive": [
"21",
0
],
"reference_image": [
"10",
0
],
"vae": [
"3",
0
],
"video_frame_offset": [
"232:62",
5
],
"width": [
"159",
0
]
},
"_meta": {
"title": "WanAnimateToVideo"
}
},
"242:91": {
"class_type": "KSampler",
"inputs": {
"cfg": 1,
"denoise": 1,
"latent_image": [
"242:90",
2
],
"model": [
"60",
0
],
"negative": [
"242:90",
1
],
"positive": [
"242:90",
0
],
"sampler_name": "euler",
"scheduler": "simple",
"seed": 387378783691198,
"steps": 6
},
"_meta": {
"title": "KSampler"
}
},
"242:92": {
"class_type": "TrimVideoLatent",
"inputs": {
"samples": [
"242:91",
0
],
"trim_amount": [
"242:90",
3
]
},
"_meta": {
"title": "Trim Video Latent"
}
}
})
# run() also passes api_key to the partner nodes in this workflow.
job = client.run(workflow, api_key=api_key)よくある質問
Required: Wan2_2-Animate-14B_fp8_e4m3fn_scaled_KJ.safetensors (UNETLoader), VAELoader, CLIPLoader, CLIPTextEncode, SaveVideo. For guidance and masking: comfyui_controlnet_aux (DWPreprocessor), ComfyUI-segment-anything-2 (DownloadAndLoadSAM2Model, Sam2Segmentation). Optional: CLIPVisionLoader/CLIPVisionEncode for reference image conditioning, LoraLoaderModelOnly for character LoRA, and comfyui-kjnodes utilities.
Use PixelPerfectResolution (or Image Scale, if present) to set a manageable size and ensure width/height are multiples of 16 (WanAnimateToVideo requirement). Start smaller if VRAM is limited, then scale up. Longer videos and higher resolutions increase preprocessing and sampling time.
Load the SAM2 model via DownloadAndLoadSAM2Model, then in Sam2Segmentation add green points (hold Shift and click) on the subject you want. Add a few background (red) points if your node exposes them, and preview with MaskPreview. Iterate: more targeted points near boundaries improve hair and limb separation.
Yes. Feed a reference image through CLIPVisionEncode (with CLIPVisionLoader) and/or load a matching LoRA via LoraLoaderModelOnly. If identity flickers, reduce resolution slightly, increase steps, fix the seed, strengthen the mask around the face, and use a concise, consistent prompt with a strong negative prompt.