This ComfyUI workflow is designed to create a lip-synced video by combining a static character image with an audio file. Utilizing the power of the Wan2.1 InfiniteTalk model, the workflow analyzes the uploaded audio to generate mouth movements that match the spoken words, creating a realistic lip-sync effect. The process begins with the 'LoadAudio' node, which ingests the audio file and prepares it for analysis. Simultaneously, the 'LoadImage' node allows users to upload the character image that will be animated. The workflow leverages custom nodes like 'ComfyUI-WanVideoWrapper' and 'audio-separation-nodes-comfyui' to process the audio and synchronize it with the visual elements. The final output is a video file, saved using the 'SaveVideo' node, where the character's mouth movements are in perfect harmony with the audio track.
Technically, this workflow excels by utilizing advanced audio analysis and video generation techniques. The 'audio-separation-nodes-comfyui' node is crucial for breaking down the audio into components that can be mapped to specific mouth shapes, while the 'comfyui-kjnodes' handle the intricate task of blending these movements into the character's image. This workflow is particularly useful for content creators looking to produce animated videos without the need for complex animation software, making it an accessible tool for enhancing storytelling and engagement through visual media.
API
Use this workflow from code
Every Comfy workflow is a JSON graph. The payload below is this workflow, exactly as ComfyUI runs it — fetch it from the URL, keep it in version control, or load it in ComfyUI and run it node by node.
Run it from TypeScript or Python with the Comfy SDK. The same code targets Comfy Cloud or a ComfyUI you host yourself — only the base URL changes.
// Install (beta)
npm i @comfyorg/sdk
// Run "Audio-Driven Character Lip Sync Video" (TypeScript)
import { Comfy } from "@comfyorg/sdk";
const client = new Comfy({ apiKey: "comfyui-..." });
// This workflow, exported in API format (see note below)
const wf = await client.workflows.fromFile("templates-wan2_1_infinitetalk_music_api.json");
const asset = client.assets.fromFile("input.png");
wf.setInput("284", "image", asset); // LoadImage
const job = await client.run(wf);
await job.getOutputs("325")[0].toFile("output.png"); // SaveVideoThe SDK takes a workflow in API format: open this workflow in ComfyUI and use File → Export Workflow (API).
Comfy Cloud API access requires a plan with an API key.
FAQ
