Capybara: Image to Video
Upload an image and provide a text instruction. Generate a new video where the input is edited according to your instruction, preserving structure and temporal coherence.
概要
The 'Capybara: Image to Video' workflow is designed to transform a static image into a dynamic video based on user-provided text instructions. This workflow leverages the Capybara model, which excels in maintaining the structure and temporal coherence of the original image while introducing the specified changes.
Technically, the process begins with the 'LoadImage' node, which imports the initial image. The core transformation is handled by the Capybara model node (4f4ddf39-1508-4d34-a35c-ff10e6ce995b), which interprets the text instructions to edit the image progressively across frames, creating a seamless video output.
Finally, the 'SaveVideo' node compiles these frames into a coherent video file.
This workflow is particularly useful for content creators and digital artists who wish to animate static images with specific thematic or stylistic changes. By preserving the original image's structure, the workflow ensures that the resulting video remains true to the source material, while the temporal coherence feature prevents jarring transitions between frames. The Capybara model's ability to handle various resolutions and aspect ratios, as detailed in the resolution configuration, adds versatility to this tool, making it suitable for different platforms and display requirements.
入力するもの
- ワークフローを読み込み中
ComfyUI で実行できます。 このワークフローはこのページでは実行できません。すべてが読み込まれた状態でComfy Cloudで開くか、ご自身のComfyUI用にダウンロードしてください。
Cloudで試すワークフローの内部
ノードグラフの読み取り専用プレビューです。ノードを編集するには、Comfy Cloudで開くかダウンロードしてください。
Loading the graph
Open full-size workflow preview- 作成者
- ComfyUI
カスタムノードが必要です
- ワークフローを読み込み中
コードからこのワークフローを使用する
可能な場合はワークフローグラフと入力から生成されます。それ以外の場合は保存された例が表示されます。料金が発生する場合があるため、コードを実行する前に認証情報とエンドポイントを確認してください。
What you need
- Runs on
- Comfy Cloud
- キー
- COMFY_API_KEY
有料の Cloud プランとクレジットが必要です。
# Python 3.10+: pip install comfy-sdk==0.4.0
import os
from comfy_sdk import Comfy
api_key = os.environ.get("COMFY_API_KEY", "").strip()
if not api_key: raise RuntimeError("Set COMFY_API_KEY")
os.environ.setdefault("COMFY_BASE_URL", "https://cloud.comfy.org")
client = Comfy(api_key=api_key)
workflow = client.workflows.from_json({
"114": {
"class_type": "SaveVideo",
"inputs": {
"filename_prefix": "video/ComfyUI",
"format": "auto",
"format.codec": "auto",
"video": [
"103:113",
0
]
},
"_meta": {
"title": "Save Video"
}
},
"126": {
"class_type": "LoadImage",
"inputs": {
"image": "joy_snap.png"
},
"_meta": {
"title": "Load Image"
}
},
"103:10": {
"class_type": "VAELoader",
"inputs": {
"vae_name": "hunyuanvideo15_vae_fp16.safetensors"
},
"_meta": {
"title": "Load VAE"
}
},
"103:11": {
"class_type": "DualCLIPLoader",
"inputs": {
"clip_name1": "qwen_2.5_vl_7b.safetensors",
"clip_name2": "byt5_small_glyphxl_fp16.safetensors",
"device": "default",
"type": "hunyuan_video_15"
},
"_meta": {
"title": "Load CLIP (Dual)"
}
},
"103:113": {
"class_type": "CreateVideo",
"inputs": {
"bit_depth": "auto",
"codec": "none",
"color_space": "sRGB",
"fps": 24,
"images": [
"103:8",
0
]
},
"_meta": {
"title": "Create Video"
}
},
"103:115": {
"class_type": "HunyuanVideo15ImageToVideo",
"inputs": {
"batch_size": 1,
"clip_vision_output": [
"103:119",
0
],
"height": [
"103:122",
0
],
"length": 81,
"negative": [
"103:93",
0
],
"positive": [
"103:44",
0
],
"start_image": [
"103:118",
0
],
"vae": [
"103:10",
0
],
"width": [
"103:121",
0
]
},
"_meta": {
"title": "HunyuanVideo15ImageToVideo"
}
},
"103:117": {
"class_type": "CLIPVisionLoader",
"inputs": {
"clip_name": "sigclip_vision_patch14_384.safetensors"
},
"_meta": {
"title": "Load CLIP Vision"
}
},
"103:118": {
"class_type": "ResizeAndPadImage",
"inputs": {
"image": [
"126",
0
],
"interpolation": "area",
"padding_color": "white",
"target_height": [
"103:122",
0
],
"target_width": [
"103:121",
0
]
},
"_meta": {
"title": "Resize And Pad Image"
}
},
"103:119": {
"class_type": "CLIPVisionEncode",
"inputs": {
"clip_vision": [
"103:117",
0
],
"crop": "center",
"image": [
"126",
0
]
},
"_meta": {
"title": "CLIP Vision Encode"
}
},
"103:12": {
"class_type": "UNETLoader",
"inputs": {
"unet_name": "capybara_v0.1.safetensors",
"weight_dtype": "default"
},
"_meta": {
"title": "Load Diffusion Model"
}
},
"103:121": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 640
},
"_meta": {
"title": "Width"
}
},
"103:122": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 640
},
"_meta": {
"title": "Height"
}
},
"103:13": {
"class_type": "SamplerCustomAdvanced",
"inputs": {
"guider": [
"103:91",
0
],
"latent_image": [
"103:115",
2
],
"noise": [
"103:25",
0
],
"sampler": [
"103:16",
0
],
"sigmas": [
"103:17",
0
]
},
"_meta": {
"title": "SamplerCustomAdvanced"
}
},
"103:16": {
"class_type": "KSamplerSelect",
"inputs": {
"sampler_name": "euler"
},
"_meta": {
"title": "KSamplerSelect"
}
},
"103:17": {
"class_type": "BasicScheduler",
"inputs": {
"denoise": 1,
"model": [
"103:12",
0
],
"scheduler": "simple",
"steps": 20
},
"_meta": {
"title": "BasicScheduler"
}
},
"103:25": {
"class_type": "RandomNoise",
"inputs": {
"noise_seed": 663555858763862
},
"_meta": {
"title": "RandomNoise"
}
},
"103:44": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"103:11",
0
],
"text": "A vibrant, sunny video scene: a young man and woman sitting on a green bench on a tennis court, taking a selfie with a white smartphone. The camera starts at a low angle and slowly tilts upward, revealing the clear blue sky. The man wears a light green jacket and shorts, the woman wears a light green blazer and trousers with a red and white striped turtleneck. The woman raises one arm playfully, and they hold hands, radiating youthful energy. The overall style is bright, cinematic, with smooth, slow camera movement and warm, vivid color grading."
},
"_meta": {
"title": "CLIP Text Encode (Positive Prompt)"
}
},
"103:67": {
"class_type": "ModelSamplingSD3",
"inputs": {
"model": [
"103:12",
0
],
"shift": 7
},
"_meta": {
"title": "ModelSamplingSD3"
}
},
"103:8": {
"class_type": "VAEDecode",
"inputs": {
"samples": [
"103:13",
0
],
"vae": [
"103:10",
0
]
},
"_meta": {
"title": "VAE Decode"
}
},
"103:91": {
"class_type": "CFGGuider",
"inputs": {
"cfg": 6,
"model": [
"103:67",
0
],
"negative": [
"103:115",
1
],
"positive": [
"103:115",
0
]
},
"_meta": {
"title": "CFG Guider"
}
},
"103:93": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"103:11",
0
],
"text": "blurry, low quality, distorted, ugly, watermark, text"
},
"_meta": {
"title": "CLIP Text Encode (Negative Prompt)"
}
}
})
# run() also passes api_key to the partner nodes in this workflow.
job = client.run(workflow, api_key=api_key)よくある質問
Images with clear structure and elements that can be logically animated based on the text instructions work best. High-resolution images are recommended for better video quality.
The Capybara model uses advanced algorithms to ensure smooth transitions between frames, maintaining the visual consistency of the original image while applying the specified changes.
It's best to set your desired resolution before running the workflow. However, you can use video editing software to adjust the resolution post-processing if needed.
Try refining your text instructions for clarity and specificity, and consider increasing the number of steps to 50 for improved detail and quality.