Capybara: Video Edit
Upload an image or video and a text instruction to perform generation or editing tasks. Generate a new or edited image or video based on the provided instruction.
概要
The Capybara: Video Edit workflow is a powerful tool designed for users looking to perform video editing and generation tasks using AI. At its core, this workflow leverages the Capybara model, which is adept at interpreting text instructions to modify or create video content. The workflow begins with the LoadVideo node, which allows users to upload their desired video or image file.
Once the media is loaded, users can input specific text instructions that dictate the editing or generation task they wish to perform. The Capybara model processes these instructions to produce the desired output, which can range from simple edits to entirely new video sequences.
Technically, the workflow is structured to facilitate seamless integration between user inputs and AI-driven video processing. The MarkdownNote node serves as a guide, providing users with essential information and tips for optimal usage. After processing, the SaveVideo node ensures that the output is stored efficiently, ready for review or further use. This workflow is particularly useful for content creators, marketers, and educators who need to quickly generate or edit video content based on specific requirements, without the need for extensive manual editing.
入力するもの
- ワークフローを読み込み中
ComfyUI で実行できます。 このワークフローはこのページでは実行できません。すべてが読み込まれた状態でComfy Cloudで開くか、ご自身のComfyUI用にダウンロードしてください。
Cloudで試すワークフローの内部
ノードグラフの読み取り専用プレビューです。ノードを編集するには、Comfy Cloudで開くかダウンロードしてください。
Loading the graph
Open full-size workflow preview- 作成者
- ComfyUI
カスタムノードが必要です
- ワークフローを読み込み中
コードからこのワークフローを使用する
可能な場合はワークフローグラフと入力から生成されます。それ以外の場合は保存された例が表示されます。料金が発生する場合があるため、コードを実行する前に認証情報とエンドポイントを確認してください。
What you need
- Runs on
- Comfy Cloud
- キー
- COMFY_API_KEY
有料の Cloud プランとクレジットが必要です。
# Python 3.10+: pip install comfy-sdk==0.4.0
import os
from comfy_sdk import Comfy
api_key = os.environ.get("COMFY_API_KEY", "").strip()
if not api_key: raise RuntimeError("Set COMFY_API_KEY")
os.environ.setdefault("COMFY_BASE_URL", "https://cloud.comfy.org")
client = Comfy(api_key=api_key)
workflow = client.workflows.from_json({
"114": {
"class_type": "SaveVideo",
"inputs": {
"filename_prefix": "video/ComfyUI",
"format": "auto",
"format.codec": "auto",
"video": [
"103:113",
0
]
},
"_meta": {
"title": "Save Video"
}
},
"127": {
"class_type": "LoadVideo",
"inputs": {
"file": "horse_running.mp4"
},
"_meta": {
"title": "Load Video"
}
},
"103:10": {
"class_type": "VAELoader",
"inputs": {
"vae_name": "hunyuanvideo15_vae_fp16.safetensors"
},
"_meta": {
"title": "Load VAE"
}
},
"103:11": {
"class_type": "DualCLIPLoader",
"inputs": {
"clip_name1": "qwen_2.5_vl_7b.safetensors",
"clip_name2": "byt5_small_glyphxl_fp16.safetensors",
"device": "default",
"type": "hunyuan_video_15"
},
"_meta": {
"title": "Load CLIP (Dual)"
}
},
"103:113": {
"class_type": "CreateVideo",
"inputs": {
"audio": [
"103:128",
1
],
"bit_depth": "auto",
"codec": "none",
"color_space": "sRGB",
"fps": 24,
"images": [
"103:8",
0
]
},
"_meta": {
"title": "Create Video"
}
},
"103:115": {
"class_type": "HunyuanVideo15ImageToVideo",
"inputs": {
"batch_size": 1,
"clip_vision_output": [
"103:119",
0
],
"height": [
"103:122",
0
],
"length": 121,
"negative": [
"103:93",
0
],
"positive": [
"103:44",
0
],
"start_image": [
"103:131",
0
],
"vae": [
"103:10",
0
],
"width": [
"103:121",
0
]
},
"_meta": {
"title": "HunyuanVideo15ImageToVideo"
}
},
"103:117": {
"class_type": "CLIPVisionLoader",
"inputs": {
"clip_name": "sigclip_vision_patch14_384.safetensors"
},
"_meta": {
"title": "Load CLIP Vision"
}
},
"103:119": {
"class_type": "CLIPVisionEncode",
"inputs": {
"clip_vision": [
"103:117",
0
],
"crop": "center",
"image": [
"103:131",
0
]
},
"_meta": {
"title": "CLIP Vision Encode"
}
},
"103:12": {
"class_type": "UNETLoader",
"inputs": {
"unet_name": "capybara_v0.1.safetensors",
"weight_dtype": "default"
},
"_meta": {
"title": "Load Diffusion Model"
}
},
"103:121": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 960
},
"_meta": {
"title": "Width"
}
},
"103:122": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 960
},
"_meta": {
"title": "Height"
}
},
"103:128": {
"class_type": "GetVideoComponents",
"inputs": {
"video": [
"127",
0
]
},
"_meta": {
"title": "Get Video Components"
}
},
"103:129": {
"class_type": "ImageFromBatch",
"inputs": {
"batch_index": 0,
"image": [
"103:128",
0
],
"length": 121
},
"_meta": {
"title": "Get Image from Batch"
}
},
"103:13": {
"class_type": "SamplerCustomAdvanced",
"inputs": {
"guider": [
"103:91",
0
],
"latent_image": [
"103:115",
2
],
"noise": [
"103:25",
0
],
"sampler": [
"103:16",
0
],
"sigmas": [
"103:17",
0
]
},
"_meta": {
"title": "SamplerCustomAdvanced"
}
},
"103:131": {
"class_type": "ResizeImageMaskNode",
"inputs": {
"input": [
"103:129",
0
],
"resize_type": "scale dimensions",
"resize_type.crop": "center",
"resize_type.height": [
"103:122",
0
],
"resize_type.width": [
"103:121",
0
],
"scale_method": "lanczos"
},
"_meta": {
"title": "Resize Image/Mask"
}
},
"103:16": {
"class_type": "KSamplerSelect",
"inputs": {
"sampler_name": "euler"
},
"_meta": {
"title": "KSamplerSelect"
}
},
"103:17": {
"class_type": "BasicScheduler",
"inputs": {
"denoise": 1,
"model": [
"103:12",
0
],
"scheduler": "simple",
"steps": 20
},
"_meta": {
"title": "BasicScheduler"
}
},
"103:25": {
"class_type": "RandomNoise",
"inputs": {
"noise_seed": 908966882718651
},
"_meta": {
"title": "RandomNoise"
}
},
"103:44": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"103:11",
0
],
"text": "Change the ocean background in the video to a grassland and make the horse run on the grassland."
},
"_meta": {
"title": "CLIP Text Encode (Positive Prompt)"
}
},
"103:67": {
"class_type": "ModelSamplingSD3",
"inputs": {
"model": [
"103:12",
0
],
"shift": 7
},
"_meta": {
"title": "ModelSamplingSD3"
}
},
"103:8": {
"class_type": "VAEDecode",
"inputs": {
"samples": [
"103:13",
0
],
"vae": [
"103:10",
0
]
},
"_meta": {
"title": "VAE Decode"
}
},
"103:91": {
"class_type": "CFGGuider",
"inputs": {
"cfg": 6,
"model": [
"103:67",
0
],
"negative": [
"103:115",
1
],
"positive": [
"103:115",
0
]
},
"_meta": {
"title": "CFG Guider"
}
},
"103:93": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"103:11",
0
],
"text": "blurry, low quality, distorted, ugly, watermark, text"
},
"_meta": {
"title": "CLIP Text Encode (Negative Prompt)"
}
}
})
# run() also passes api_key to the partner nodes in this workflow.
job = client.run(workflow, api_key=api_key)よくある質問
You can upload both video and image files using the LoadVideo node, as the workflow supports various common formats.
Your text instructions should be clear and concise, specifying the desired outcome to help the Capybara model generate accurate results.
Yes, you can configure the resolution settings based on the Capybara Resolution Configuration table provided in the workflow notes.
Consider increasing the number of steps in the inference process, as suggested in the author notes, to improve the quality of the generated video.