SVD Text to Image to Video

Generate videos by first creating images from text prompts.

Download Workflow Try in Cloud API documentation

Input

Prompt

Height

Length (frames)

Width

Advanced settings

Frame rate

Default: 10

Motion amount

Default: 127

Negative prompt

Seed

Run

Output

00:00

Example

Call this model from your code

The request below uses the same prepared graph and settings as the Playground. Each submission can start a paid run; do not automatically retry a submission whose outcome is unknown.

Get API key

What you need

Runs on: Comfy Cloud
Key: COMFY_API_KEY

Needs a paid Cloud plan and credits.

Download API JSON

# Python 3.10+: pip install comfy-sdk==0.4.0
import os

from comfy_sdk import Comfy

api_key = os.environ.get("COMFY_API_KEY", "").strip()
if not api_key: raise RuntimeError("Set COMFY_API_KEY")
os.environ.setdefault("COMFY_BASE_URL", "https://cloud.comfy.org")
client = Comfy(api_key=api_key)
workflow = client.workflows.from_json({
    "3": {
        "class_type": "KSampler",
        "inputs": {
            "cfg": 2.5,
            "denoise": 1,
            "latent_image": [
                "12",
                2
            ],
            "model": [
                "14",
                0
            ],
            "negative": [
                "12",
                1
            ],
            "positive": [
                "12",
                0
            ],
            "sampler_name": "euler",
            "scheduler": "karras",
            "seed": 237514639057514,
            "steps": 20
        },
        "_meta": {
            "title": "KSampler"
        }
    },
    "8": {
        "class_type": "VAEDecode",
        "inputs": {
            "samples": [
                "3",
                0
            ],
            "vae": [
                "15",
                2
            ]
        },
        "_meta": {
            "title": "VAE Decode"
        }
    },
    "12": {
        "class_type": "SVD_img2vid_Conditioning",
        "inputs": {
            "augmentation_level": 0,
            "clip_vision": [
                "15",
                1
            ],
            "fps": 6,
            "height": 576,
            "init_image": [
                "20",
                0
            ],
            "motion_bucket_id": 127,
            "vae": [
                "15",
                2
            ],
            "video_frames": 25,
            "width": 1024
        },
        "_meta": {
            "title": "SVD_img2vid_Conditioning"
        }
    },
    "14": {
        "class_type": "VideoLinearCFGGuidance",
        "inputs": {
            "min_cfg": 1,
            "model": [
                "15",
                0
            ]
        },
        "_meta": {
            "title": "Video Linear CFG Guidance"
        }
    },
    "15": {
        "class_type": "ImageOnlyCheckpointLoader",
        "inputs": {
            "ckpt_name": "svd_xt.safetensors"
        },
        "_meta": {
            "title": "Load Checkpoint Image Only (img2vid model)"
        }
    },
    "16": {
        "class_type": "CheckpointLoaderSimple",
        "inputs": {
            "ckpt_name": "sd_xl_base_1.0.safetensors"
        },
        "_meta": {
            "title": "Load Checkpoint"
        }
    },
    "17": {
        "class_type": "KSampler",
        "inputs": {
            "cfg": 8,
            "denoise": 1,
            "latent_image": [
                "22",
                0
            ],
            "model": [
                "16",
                0
            ],
            "negative": [
                "19",
                0
            ],
            "positive": [
                "18",
                0
            ],
            "sampler_name": "uni_pc_bh2",
            "scheduler": "normal",
            "seed": 144698910769133,
            "steps": 15
        },
        "_meta": {
            "title": "KSampler"
        }
    },
    "18": {
        "class_type": "CLIPTextEncode",
        "inputs": {
            "clip": [
                "16",
                1
            ],
            "text": "A dynamic wildlife photograph of a powerful roan stallion galloping through churning ocean surf along an expansive, desolate coastline at sunrise. Blinding golden-hour backlighting cuts through thick, swirling coastal mist, outlining the horse’s taut, gleaming chestnut coat and windblown mane in radiant amber rim light. Driven by gale-force winds, cresting turquoise waves crash against the shore, erupting into towering plumes of crystalline sea spray and glittering droplets frozen mid-air around powerful, surging hooves. Deep indigo shadows stretch across the wet, reflective sand, mirroring the vibrant rose and copper hues of the dawn sky. Dense sea foam churns vigorously in the foreground, while sweeping gusts whip the horse's tail and scatter ocean vapor toward the distant, hazy headlands."
        },
        "_meta": {
            "title": "CLIP Text Encode (Prompt)"
        }
    },
    "19": {
        "class_type": "CLIPTextEncode",
        "inputs": {
            "clip": [
                "16",
                1
            ],
            "text": "text, watermark"
        },
        "_meta": {
            "title": "CLIP Text Encode (Prompt)"
        }
    },
    "20": {
        "class_type": "VAEDecode",
        "inputs": {
            "samples": [
                "17",
                0
            ],
            "vae": [
                "16",
                2
            ]
        },
        "_meta": {
            "title": "VAE Decode"
        }
    },
    "21": {
        "class_type": "PreviewImage",
        "inputs": {
            "images": [
                "20",
                0
            ]
        },
        "_meta": {
            "title": "Preview Image"
        }
    },
    "22": {
        "class_type": "EmptyLatentImage",
        "inputs": {
            "batch_size": 1,
            "height": 576,
            "width": 1024
        },
        "_meta": {
            "title": "Empty Latent Image"
        }
    },
    "24": {
        "class_type": "SaveVideo",
        "inputs": {
            "filename_prefix": "video/ComfyUI",
            "format": "auto",
            "format.codec": "auto",
            "video": [
                "25",
                0
            ]
        },
        "_meta": {
            "title": "Save Video"
        }
    },
    "25": {
        "class_type": "CreateVideo",
        "inputs": {
            "bit_depth": "auto",
            "codec": "none",
            "color_space": "sRGB",
            "fps": 10,
            "images": [
                "8",
                0
            ]
        },
        "_meta": {
            "title": "Create Video"
        }
    }
})

# run() also passes api_key to the partner nodes in this workflow.
job = client.run(workflow, api_key=api_key)
for output in job.get_outputs("24"):
    output.to_file(output.name)

API documentation

Try an example

About this workflow

The 'SVD Text to Image to Video' workflow in ComfyUI is designed to transform textual descriptions into dynamic video content. This process begins with generating images from text prompts using a combination of nodes like CLIPTextEncode, which encodes the text into a format that the image generation models can interpret.

The workflow leverages powerful models such as SVD and Stability to ensure high-quality image outputs. Once the images are created, the workflow transitions into video generation using nodes like SVD_img2vid_Conditioning and VideoLinearCFGGuidance, which guide the transformation of static images into fluid video sequences.

This workflow is particularly useful for creators looking to produce engaging video content from simple text inputs without needing extensive video editing skills.

Technically, the workflow is structured to maximize the quality and coherence of both images and videos. The KSampler node plays a crucial role in sampling latent spaces to produce varied and creative outputs, while the VAEDecode node decodes these latent representations into visible images. The CreateVideo node then compiles these images into a seamless video, which can be previewed and saved using PreviewImage and SaveVideo nodes, respectively. This approach is not only efficient but also allows for a high degree of customization and creativity, making it an invaluable tool for artists and marketers alike.

SVD Text to Image to Video

Input

Inside the workflow

Call this model from your code

What you need

Try an example

About this workflow

Questions & answers

Input

Inside the workflow

Call this model from your code

What you need

Try an example

About this workflow

Questions & answers

What type of text prompts work best with this workflow?

Can I adjust the style of the generated video?

How long does it take to generate a video?

Is it possible to use custom models with this workflow?