The LTX-2: Audio to Video workflow is designed to transform an audio file and a starting image frame into a synchronized video. This process leverages the LTX-2 model, known for its ability to interpret audio cues and translate them into dynamic visual content. The workflow begins with the 'LoadImage' and 'LoadAudio' nodes, which allow users to input the initial image and audio file that will serve as the foundation for the video generation. These inputs are then processed through a series of custom nodes, including the 'comfyui-kjnodes', which are crucial for maintaining synchronization between the audio and visual elements.
Technically, the workflow utilizes a unique node identified by the ID '3458504c-2bd1-4267-8722-6f2478115b42', which plays a pivotal role in aligning the audio's rhythm and tempo with visual transitions. The 'SaveVideo' node is the final step, converting the processed data into a video file. This workflow is particularly useful for creators looking to produce engaging video content from audio tracks, such as music videos or audio-visual art pieces. Its experimental nature means that results can vary, offering a unique opportunity for creative exploration.
API
Use this workflow from code
Every Comfy workflow is a JSON graph. The payload below is this workflow, exactly as ComfyUI runs it — fetch it from the URL, keep it in version control, or load it in ComfyUI and run it node by node.
Run it from TypeScript or Python with the Comfy SDK. The same code targets Comfy Cloud or a ComfyUI you host yourself — only the base URL changes.
// Install (beta)
npm i @comfyorg/sdk
// Run "LTX-2: Audio to VIdeo" (TypeScript)
import { Comfy } from "@comfyorg/sdk";
const client = new Comfy({ apiKey: "comfyui-..." });
// This workflow, exported in API format (see note below)
const wf = await client.workflows.fromFile("video_ltx_2_audio_to_video_api.json");
const asset = client.assets.fromFile("input.png");
wf.setInput("444", "image", asset); // LoadImage
const job = await client.run(wf);
await job.getOutputs("984")[0].toFile("output.png"); // SaveVideoThe SDK takes a workflow in API format: open this workflow in ComfyUI and use File → Export Workflow (API).
Comfy Cloud API access requires a plan with an API key.
FAQ
