Wan2.2-S2V 音訊驅動影片產生
將靜態圖像與音訊轉為動態影片,音畫完美同步,分鐘級即可產生。
工作流程內部
來源範本的唯讀預覽。請在 Cloud 中開啟,以探索節點並進行變更。
從您的程式碼呼叫此模型
以下請求使用與 Playground 相同的已準備節點圖和設定。每次提交都可能啟動付費執行;若提交結果不明,請勿自動重試提交。
What you need
- Runs on
- Comfy Cloud
- 金鑰
- COMFY_API_KEY
- 您的檔案
- 由程式碼透過其 URL 上傳
需要付費 Cloud 方案與點數。
# Python 3.10+: pip install comfy-sdk==0.4.0
import os
from comfy_sdk import Comfy
api_key = os.environ.get("COMFY_API_KEY", "").strip()
if not api_key: raise RuntimeError("Set COMFY_API_KEY")
os.environ.setdefault("COMFY_BASE_URL", "https://cloud.comfy.org")
client = Comfy(api_key=api_key)
workflow = client.workflows.from_json({
"3": {
"class_type": "KSampler",
"inputs": {
"cfg": [
"105",
0
],
"denoise": 1,
"latent_image": [
"93",
2
],
"model": [
"54",
0
],
"negative": [
"93",
1
],
"positive": [
"93",
0
],
"sampler_name": "uni_pc",
"scheduler": "simple",
"seed": 20,
"steps": [
"103",
0
]
},
"_meta": {
"title": "KSampler"
}
},
"6": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"38",
0
],
"text": "A woman in a dark charcoal high-collar jacket sits in a softly lit broadcast studio and speaks warmly into the black condenser microphone in front of her, like a late-night radio host greeting her listeners. She talks naturally and calmly, her lips moving clearly with every word, with small nods, gentle eyebrow raises and a faint smile on the friendly lines. Her head tilts slightly as she leans a little closer to the microphone, then settles back. Her eyes stay on the camera, blinking naturally. Her hands rest out of frame below the microphone, and nothing passes in front of her face. The camera is locked off at chest height in a single continuous shot. Behind her, cyan and amber edge lights glow on the concrete and the hexagonal acoustic panels, and small indicator lights blink on the equipment racks."
},
"_meta": {
"title": "CLIP Text Encode (Positive Prompt)"
}
},
"7": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"38",
0
],
"text": "色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走"
},
"_meta": {
"title": "CLIP Text Encode (Negative Prompt)"
}
},
"37": {
"class_type": "UNETLoader",
"inputs": {
"unet_name": "wan2.2_s2v_14B_fp8_scaled.safetensors",
"weight_dtype": "default"
},
"_meta": {
"title": "Load Diffusion Model"
}
},
"38": {
"class_type": "CLIPLoader",
"inputs": {
"clip_name": "umt5_xxl_fp8_e4m3fn_scaled.safetensors",
"device": "default",
"type": "wan"
},
"_meta": {
"title": "Load CLIP"
}
},
"39": {
"class_type": "VAELoader",
"inputs": {
"vae_name": "wan_2.1_vae.safetensors"
},
"_meta": {
"title": "Load VAE"
}
},
"52": {
"class_type": "LoadImage",
"inputs": {
"image": "UPLOADED_image_FILENAME"
},
"_meta": {
"title": "Load Image"
}
},
"54": {
"class_type": "ModelSamplingSD3",
"inputs": {
"model": [
"107",
0
],
"shift": 8
},
"_meta": {
"title": "ModelSamplingSD3"
}
},
"56": {
"class_type": "AudioEncoderEncode",
"inputs": {
"audio": [
"58",
0
],
"audio_encoder": [
"57",
0
]
},
"_meta": {
"title": "AudioEncoderEncode"
}
},
"57": {
"class_type": "AudioEncoderLoader",
"inputs": {
"audio_encoder_name": "wav2vec2_large_english_fp16.safetensors"
},
"_meta": {
"title": "Load Audio Encoder"
}
},
"58": {
"class_type": "LoadAudio",
"inputs": {
"audio": "UPLOADED_audio_FILENAME"
},
"_meta": {
"title": "Load Audio"
}
},
"80": {
"class_type": "VAEDecode",
"inputs": {
"samples": [
"95",
0
],
"vae": [
"39",
0
]
},
"_meta": {
"title": "VAE Decode"
}
},
"82": {
"class_type": "CreateVideo",
"inputs": {
"audio": [
"58",
0
],
"bit_depth": "auto",
"codec": "none",
"color_space": "sRGB",
"fps": 16,
"images": [
"96",
0
]
},
"_meta": {
"title": "Create Video"
}
},
"93": {
"class_type": "WanSoundImageToVideo",
"inputs": {
"audio_encoder_output": [
"56",
0
],
"batch_size": 1,
"height": 480,
"length": [
"104",
0
],
"negative": [
"7",
0
],
"positive": [
"6",
0
],
"ref_image": [
"52",
0
],
"vae": [
"39",
0
],
"width": 832
},
"_meta": {
"title": "WanSoundImageToVideo"
}
},
"94": {
"class_type": "LatentCut",
"inputs": {
"amount": 1,
"dim": "t",
"index": 0,
"samples": [
"85:78",
0
]
},
"_meta": {
"title": "LatentCut"
}
},
"95": {
"class_type": "LatentConcat",
"inputs": {
"dim": "t",
"samples1": [
"94",
0
],
"samples2": [
"85:78",
0
]
},
"_meta": {
"title": "LatentConcat"
}
},
"96": {
"class_type": "ImageFromBatch",
"inputs": {
"batch_index": [
"100",
0
],
"image": [
"80",
0
],
"length": 4096
},
"_meta": {
"title": "Get Image from Batch"
}
},
"100": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 3
},
"_meta": {
"title": "Batch sizes"
}
},
"103": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 4
},
"_meta": {
"title": "Steps"
}
},
"104": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 77
},
"_meta": {
"title": "Chunk Length"
}
},
"105": {
"class_type": "PrimitiveFloat",
"inputs": {
"value": 1
},
"_meta": {
"title": "CFG"
}
},
"107": {
"class_type": "LoraLoaderModelOnly",
"inputs": {
"lora_name": "wan2.2_t2v_lightx2v_4steps_lora_v1.1_high_noise.safetensors",
"model": [
"37",
0
],
"strength_model": 1
},
"_meta": {
"title": "Load LoRA"
}
},
"113": {
"class_type": "SaveVideo",
"inputs": {
"filename_prefix": "video/ComfyUI",
"format": "auto",
"format.codec": "auto",
"video": [
"82",
0
]
},
"_meta": {
"title": "Save Video"
}
},
"79:76": {
"class_type": "WanSoundImageToVideoExtend",
"inputs": {
"audio_encoder_output": [
"56",
0
],
"length": [
"104",
0
],
"negative": [
"7",
0
],
"positive": [
"6",
0
],
"ref_image": [
"52",
0
],
"vae": [
"39",
0
],
"video_latent": [
"3",
0
]
},
"_meta": {
"title": "WanSoundImageToVideoExtend"
}
},
"79:77": {
"class_type": "KSampler",
"inputs": {
"cfg": [
"105",
0
],
"denoise": 1,
"latent_image": [
"79:76",
2
],
"model": [
"54",
0
],
"negative": [
"79:76",
1
],
"positive": [
"79:76",
0
],
"sampler_name": "uni_pc",
"scheduler": "simple",
"seed": 1,
"steps": [
"103",
0
]
},
"_meta": {
"title": "KSampler"
}
},
"79:78": {
"class_type": "LatentConcat",
"inputs": {
"dim": "t",
"samples1": [
"3",
0
],
"samples2": [
"79:77",
0
]
},
"_meta": {
"title": "LatentConcat"
}
},
"85:76": {
"class_type": "WanSoundImageToVideoExtend",
"inputs": {
"audio_encoder_output": [
"56",
0
],
"length": [
"104",
0
],
"negative": [
"7",
0
],
"positive": [
"6",
0
],
"ref_image": [
"52",
0
],
"vae": [
"39",
0
],
"video_latent": [
"79:78",
0
]
},
"_meta": {
"title": "WanSoundImageToVideoExtend"
}
},
"85:77": {
"class_type": "KSampler",
"inputs": {
"cfg": [
"105",
0
],
"denoise": 1,
"latent_image": [
"85:76",
2
],
"model": [
"54",
0
],
"negative": [
"85:76",
1
],
"positive": [
"85:76",
0
],
"sampler_name": "uni_pc",
"scheduler": "simple",
"seed": 250,
"steps": [
"103",
0
]
},
"_meta": {
"title": "KSampler"
}
},
"85:78": {
"class_type": "LatentConcat",
"inputs": {
"dim": "t",
"samples1": [
"79:78",
0
],
"samples2": [
"85:77",
0
]
},
"_meta": {
"title": "LatentConcat"
}
}
})
workflow.set_input("58", "audio", client.assets.from_url("https://comfy.org/catalog-media/3bb9ee36-53c7-459b-8344-8c127597fd3c/1b849bac-2e14-43ef-8edd-ea2859d20a09"))
workflow.set_input("52", "image", client.assets.from_url("https://comfy.org/catalog-media/6e2796d5-1dfd-42f2-b918-b7157f9c5067/2d1cb5de-3ee8-4dfc-97c5-bc7d189528cd"))
# run() also passes api_key to the partner nodes in this workflow.
job = client.run(workflow, api_key=api_key)
for output in job.get_outputs("113"):
output.to_file(output.name)試試範例
關於此工作流程
這個 ComfyUI 工作流程會使用 Wan 2.2 S2V(sound-to-video)模型,把單張參考圖片以及一段音訊轉成同步的影片。它透過 UNETLoader、CLIPLoader 以及 VAELoader 載入 Wan 主幹,並用 ModelSamplingSD3 設定正確的取樣排程。你的文字提示由 CLIPTextEncode 編碼,而音訊則由 AudioEncoderLoader 與 AudioEncoderEncode 處理。這些訊號再加上 LoadImage 載入的圖片,一起驅動 WanSoundImageToVideo 合成符合音訊節奏與語句停頓的動作。KSampler 控制步數、CFG 與 seed;VAEDecode 把潛空間轉成影格;CreateVideo 則組裝成最後的片段。
有兩個工程細節讓這個工作流程更適合量產:首幀穩定化與可延伸的時長。「Fix overbaked first frame」群組會用 LatentCut 與 LatentConcat 複製第一個潛空間影格再解碼,並在解碼後丟棄第一個影格,以避免常見 VAE 在第 0 幀出現的「過烤」外觀。要做長片段時,「Video S2V Extend」子圖可用 LatentCut/LatentConcat 以分段方式串接(預設每段 77 幀),讓你能製作到分鐘等級而不把 VRAM 撐爆。此工作流程也支援 lightning 路徑:搭配 Wan 2.2 lightning LoRA 可用 4 步取樣(建議 CFG 約 1.0)。若你更重視品質與穩定性,使用標準路徑約 20 步(CFG 約 6.0)。
問答
使用 Video S2V Extend 子圖。每個子圖會透過 LatentCut 與 LatentConcat 切片並拼接 latent,新增一段新的片段(預設 77 影格)。每需要多一段就複製一次子圖。為了轉場平順,請在各段之間保持 seed、解析度、FPS 與取樣器設定一致。
有些 VAE 會把第一張解碼影格「烤」得過頭。Fix overbaked first frame 群組會在解碼前複製第一個 latent,並在解碼後丟棄第一張解碼影格。建議保持啟用以避免此問題;只有在你的 VAE 沒有出現此瑕疵時才關閉。
Lightning LoRA 速度快很多(約 4 步、CFG ≈ 1.0),但可能降低細節或穩定性。標準路徑(約 20 步、CFG ≈ 6.0)較慢,但通常更乾淨。如果你看到品質下降或 LoRA key 不匹配,請切換到 20-step 工作流程。也可以先用短片對兩條路徑做 A/B 測試再決定。
通常 CreateVideo 會輸出無聲的 MP4,不會封裝音訊。音訊會透過 AudioEncoderEncode 用於動作條件,但不會內嵌。若要合併,可在後製中把音訊與結果 mux 起來(例如用 ffmpeg:ffmpeg -i video.mp4 -i audio.wav -c copy -shortest out.mp4)。
相關工作流程
- Wan2.2
1.2 入門範本-圖像轉影片
- Qwen-Image-Edit, Wan2.2
1個圖像輸入分割堆疊 - Qwen多角度 + Wan 2.2
- Wan2.2, Z-Image-Turbo
AIrt MAchIne
- Wan2.2, Wan
Wan 2.2 14B Fun Camera控制
- Wan2.2, Wan
Wan 2.2 14B Fun Control
- Wan2.2, Wan
Wan 2.2 14B 趣味區域性重繪
- Wan2.2, Wan
Wan 2.2 5B Fun Control
- Wan2.2, Wan
Wan 2.2 5B Fun Inpaint
- Wan2.2, Wan
Wan 2.2 5B影片產生
- Wan, Wan2.2
Wan萬相 2.2 14B 圖生影片
- Wan2.2, Wan
Wan萬相 2.2 14B 文生影片
- Wan2.2, Wan
Wan萬相 2.2 14B 首尾幀產生影片