Audio-Driven Character Lip Sync Video

Upload a character image and an audio file. Generate a lip-sync video where the character's mouth movements match the provided audio track.

Input

Speech or singing the character lip-syncs to.

  • 0:00 / 0:00

One person, face clearly visible. Cropped to the width × height below.

Describe the performance: expression, head and body movement, camera.

Pixels, a multiple of 16.

Frames at 25 fps: set it to audio seconds × 25 (the graph can't read the audio length). The example's 17.6 s clip used 440.

Default: 440

Pixels, a multiple of 16.

Advanced settings

What to avoid.

Leave empty for a new result each run. Enter a number to repeat one.

Default: 7

Output
00:00
Example

例を試す

このワークフローについて

This ComfyUI workflow is designed to create a lip-synced video by combining a static character image with an audio file. Utilizing the power of the Wan2.1 InfiniteTalk model, the workflow analyzes the uploaded audio to generate mouth movements that match the spoken words, creating a realistic lip-sync effect.

The process begins with the 'LoadAudio' node, which ingests the audio file and prepares it for analysis. Simultaneously, the 'LoadImage' node allows users to upload the character image that will be animated. The workflow leverages custom nodes like 'ComfyUI-WanVideoWrapper' and 'audio-separation-nodes-comfyui' to process the audio and synchronize it with the visual elements. The final output is a video file, saved using the 'SaveVideo' node, where the character's mouth movements are in perfect harmony with the audio track.

Technically, this workflow excels by utilizing advanced audio analysis and video generation techniques. The 'audio-separation-nodes-comfyui' node is crucial for breaking down the audio into components that can be mapped to specific mouth shapes, while the 'comfyui-kjnodes' handle the intricate task of blending these movements into the character's image. This workflow is particularly useful for content creators looking to produce animated videos without the need for complex animation software, making it an accessible tool for enhancing storytelling and engagement through visual media.

よくある質問