Wan VACE控制影片
使用Wan VACE透過控制輸入影片和參考圖像產生影片。
工作流程內部
來源範本的唯讀預覽。請在 Cloud 中開啟,以探索節點並進行變更。
從您的程式碼呼叫此模型
以下請求使用與 Playground 相同的已準備節點圖和設定。每次提交都可能啟動付費執行;若提交結果不明,請勿自動重試提交。
What you need
- Runs on
- Comfy Cloud
- 金鑰
- COMFY_API_KEY
- 您的檔案
- 由程式碼透過其 URL 上傳
需要付費 Cloud 方案與點數。
# Python 3.10+: pip install comfy-sdk==0.4.0
import os
from comfy_sdk import Comfy
api_key = os.environ.get("COMFY_API_KEY", "").strip()
if not api_key: raise RuntimeError("Set COMFY_API_KEY")
os.environ.setdefault("COMFY_BASE_URL", "https://cloud.comfy.org")
client = Comfy(api_key=api_key)
workflow = client.workflows.from_json({
"3": {
"class_type": "KSampler",
"inputs": {
"cfg": 1,
"denoise": 1,
"latent_image": [
"49",
2
],
"model": [
"48",
0
],
"negative": [
"49",
1
],
"positive": [
"49",
0
],
"sampler_name": "uni_pc",
"scheduler": "simple",
"seed": 654654950714624,
"steps": 4
},
"_meta": {
"title": "KSampler"
}
},
"6": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"107",
1
],
"text": "The pearlescent white mecha warrior from the reference image performs a sharp street-dance locking routine in a vast misty plaza of polished slate flagstones at dawn. It drops low with knees deeply flexed, snaps its arms into rigid right angles, points one hand straight out, then spins and freezes with mechanical precision, cyan coolant glowing through its translucent conduits on every hit. Its champagne titanium joints and faceted violet-gold visor catch the pale morning light, and fine mist curls around its broad stabilizer feet. Fog-shrouded brutalist concrete colonnades stand in the background and the wet stone reflects the sky. Smooth fluid motion, the whole figure stays in frame, cinematic lighting, highly detailed, sharp focus."
},
"_meta": {
"title": "CLIP Text Encode (Positive Prompt)"
}
},
"7": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"107",
1
],
"text": "过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走,"
},
"_meta": {
"title": "CLIP Text Encode (Negative Prompt)"
}
},
"8": {
"class_type": "VAEDecode",
"inputs": {
"samples": [
"58",
0
],
"vae": [
"105",
0
]
},
"_meta": {
"title": "VAE Decode"
}
},
"48": {
"class_type": "ModelSamplingSD3",
"inputs": {
"model": [
"107",
0
],
"shift": 8.000000000000002
},
"_meta": {
"title": "ModelSamplingSD3"
}
},
"49": {
"class_type": "WanVaceToVideo",
"inputs": {
"batch_size": 1,
"control_video": [
"144",
0
],
"height": 720,
"length": 81,
"negative": [
"7",
0
],
"positive": [
"6",
0
],
"reference_image": [
"134",
0
],
"strength": 1,
"vae": [
"105",
0
],
"width": 1280
},
"_meta": {
"title": "WanVaceToVideo"
}
},
"58": {
"class_type": "TrimVideoLatent",
"inputs": {
"samples": [
"3",
0
],
"trim_amount": [
"49",
3
]
},
"_meta": {
"title": "Trim Video Latent"
}
},
"68": {
"class_type": "CreateVideo",
"inputs": {
"bit_depth": "auto",
"codec": "none",
"color_space": "sRGB",
"fps": 16,
"images": [
"8",
0
]
},
"_meta": {
"title": "Create Video"
}
},
"105": {
"class_type": "VAELoader",
"inputs": {
"vae_name": "wan_2.1_vae.safetensors"
},
"_meta": {
"title": "Load VAE"
}
},
"106": {
"class_type": "UNETLoader",
"inputs": {
"unet_name": "wan2.1_vace_14B_fp16.safetensors",
"weight_dtype": "default"
},
"_meta": {
"title": "Load Diffusion Model"
}
},
"107": {
"class_type": "LoraLoader",
"inputs": {
"clip": [
"110",
0
],
"lora_name": "Wan21_CausVid_14B_T2V_lora_rank32.safetensors",
"model": [
"106",
0
],
"strength_clip": 1,
"strength_model": 0.30000000000000004
},
"_meta": {
"title": "Load LoRA (Model and CLIP)"
}
},
"110": {
"class_type": "CLIPLoader",
"inputs": {
"clip_name": "umt5_xxl_fp16.safetensors",
"device": "default",
"type": "wan"
},
"_meta": {
"title": "Load CLIP"
}
},
"114": {
"class_type": "SaveVideo",
"inputs": {
"filename_prefix": "video/ComfyUI",
"format": "auto",
"format.codec": "auto",
"video": [
"68",
0
]
},
"_meta": {
"title": "Save Video"
}
},
"134": {
"class_type": "LoadImage",
"inputs": {
"image": "UPLOADED_reference_image_FILENAME"
},
"_meta": {
"title": "Load Image"
}
},
"144": {
"class_type": "GetVideoComponents",
"inputs": {
"video": [
"145",
0
]
},
"_meta": {
"title": "Get Video Components"
}
},
"145": {
"class_type": "LoadVideo",
"inputs": {
"file": "UPLOADED_control_video_FILENAME"
},
"_meta": {
"title": "Load Video"
}
}
})
workflow.set_input("145", "file", client.assets.from_url("https://comfy.org/catalog-media/947491b3-7015-4bff-ba49-5409d49de262/d6f1cdc1-c3e1-4f53-9824-980f4131014f"))
workflow.set_input("134", "image", client.assets.from_url("https://comfy.org/catalog-media/5e646926-45c4-483b-9fac-5c2dd37660b0/e888991b-6876-4715-b11d-2a2f1132c9c8"))
# run() also passes api_key to the partner nodes in this workflow.
job = client.run(workflow, api_key=api_key)
for output in job.get_outputs("114"):
output.to_file(output.name)試試範例
關於此工作流程
Wan2.1 VACE Control Video 是一個引導式影片轉影片工作流程:在保留來源影片的動作以及時間節奏的同時,透過文字提示與參考圖片進行重新風格化或重新角色化。它會透過 UNETLoader 與 VAELoader 載入 Wan2.1 VACE checkpoints(1.3B 或 14B),用 CLIPTextEncode 編碼文字,並以簡單的影像前處理(預設為 Canny)產生的控制訊號來做條件設定。GetVideoComponents 會從輸入片段讀取畫格以及 FPS,TrimVideoLatent 會在取樣前依你選擇的畫格範圍與解析度,保持潛空間序列對齊。
在內部流程中,ModelSamplingSD3 會為 Wan2.1 VACE 設定預期的取樣器排程,並由 KSampler 對 WanVaceToVideo 組裝出的影片潛空間序列進行去雜訊;該序列會使用你的控制影片、提示以及參考圖片。VAEDecode 會把最終潛空間轉回畫格,CreateVideo 依指定 FPS 組合成影片,你可以匯出 MP4(SaveVideo)或動態 WebP(SaveAnimatedWEBP)。可選的 CausVid LoRA(LoraLoader)可在極低步數下加速或穩定動作,但可能需要調整強度。14B 模型能提供更高保真度(並支援 720p),但更慢且更吃 VRAM;1.3B 版本更快,適合 480p 輸出。
問答
若要更快迭代與 480p 輸出,使用 1.3B。若需要更高保真度或 720p,選 14B。14B 較慢且需要更多 VRAM,但通常能提供更乾淨的細節與更好的時間穩定性。
把 LoRA strength 降到 0.3–0.5、略增 steps(例如從 2 提到 3–4),並在 LoRA 預設下讓 CFG 維持接近 1.0。若仍有瑕疵,請直接略過 LoRA,或放寬 Canny thresholds 讓邊緣限制不要太強。
TrimVideoLatent 可用來限制或對齊畫格範圍。GetVideoComponents 會讀取來源 FPS;你可以保留,或在 CreateVideo 時覆寫。縮短片段並使用適中的 FPS 可降低 VRAM 並加快取樣。
可以。將 Canny 節點替換為其他 comfyui_controlnet_aux 前處理器(例如 Lineart 或 Depth),並把輸出路由到 WanVaceToVideo 使用的同一個 control 輸入。不同前處理器會改變結構保留的嚴格程度。