Capybara: Video Edit
Upload an image or video and a text instruction to perform generation or editing tasks. Generate a new or edited image or video based on the provided instruction.
简介
The Capybara: Video Edit workflow is a powerful tool designed for users looking to perform video editing and generation tasks using AI. At its core, this workflow leverages the Capybara model, which is adept at interpreting text instructions to modify or create video content. The workflow begins with the LoadVideo node, which allows users to upload their desired video or image file.
Once the media is loaded, users can input specific text instructions that dictate the editing or generation task they wish to perform. The Capybara model processes these instructions to produce the desired output, which can range from simple edits to entirely new video sequences.
Technically, the workflow is structured to facilitate seamless integration between user inputs and AI-driven video processing. The MarkdownNote node serves as a guide, providing users with essential information and tips for optimal usage. After processing, the SaveVideo node ensures that the output is stored efficiently, ready for review or further use. This workflow is particularly useful for content creators, marketers, and educators who need to quickly generate or edit video content based on specific requirements, without the need for extensive manual editing.
你需要提供
- 正在读取工作流
在 ComfyUI 中运行。 此工作流无法在本页面运行。你可以在 Comfy Cloud 中打开（已预先加载所需内容），或下载后在自己的 ComfyUI 中使用。
在 Cloud 中试用查看工作流内部
节点图的只读预览。在 Comfy Cloud 中打开或下载后即可编辑节点。
Loading the graph
Open full-size workflow preview- 作者
- ComfyUI
需要自定义节点
- 正在读取工作流
通过代码使用此工作流
尽可能根据工作流图和输入自动生成代码，否则显示已保存的示例。运行代码可能产生费用，请先核对凭据和端点。
What you need
- Runs on
- Comfy Cloud
- 密钥
- COMFY_API_KEY
需要付费的 Cloud 方案和额度。
# Python 3.10+: pip install comfy-sdk==0.4.0
import os
from comfy_sdk import Comfy
api_key = os.environ.get("COMFY_API_KEY", "").strip()
if not api_key: raise RuntimeError("Set COMFY_API_KEY")
os.environ.setdefault("COMFY_BASE_URL", "https://cloud.comfy.org")
client = Comfy(api_key=api_key)
workflow = client.workflows.from_json({
"114": {
"class_type": "SaveVideo",
"inputs": {
"filename_prefix": "video/ComfyUI",
"format": "auto",
"format.codec": "auto",
"video": [
"103:113",
0
]
},
"_meta": {
"title": "Save Video"
}
},
"127": {
"class_type": "LoadVideo",
"inputs": {
"file": "horse_running.mp4"
},
"_meta": {
"title": "Load Video"
}
},
"103:10": {
"class_type": "VAELoader",
"inputs": {
"vae_name": "hunyuanvideo15_vae_fp16.safetensors"
},
"_meta": {
"title": "Load VAE"
}
},
"103:11": {
"class_type": "DualCLIPLoader",
"inputs": {
"clip_name1": "qwen_2.5_vl_7b.safetensors",
"clip_name2": "byt5_small_glyphxl_fp16.safetensors",
"device": "default",
"type": "hunyuan_video_15"
},
"_meta": {
"title": "Load CLIP (Dual)"
}
},
"103:113": {
"class_type": "CreateVideo",
"inputs": {
"audio": [
"103:128",
1
],
"bit_depth": "auto",
"codec": "none",
"color_space": "sRGB",
"fps": 24,
"images": [
"103:8",
0
]
},
"_meta": {
"title": "Create Video"
}
},
"103:115": {
"class_type": "HunyuanVideo15ImageToVideo",
"inputs": {
"batch_size": 1,
"clip_vision_output": [
"103:119",
0
],
"height": [
"103:122",
0
],
"length": 121,
"negative": [
"103:93",
0
],
"positive": [
"103:44",
0
],
"start_image": [
"103:131",
0
],
"vae": [
"103:10",
0
],
"width": [
"103:121",
0
]
},
"_meta": {
"title": "HunyuanVideo15ImageToVideo"
}
},
"103:117": {
"class_type": "CLIPVisionLoader",
"inputs": {
"clip_name": "sigclip_vision_patch14_384.safetensors"
},
"_meta": {
"title": "Load CLIP Vision"
}
},
"103:119": {
"class_type": "CLIPVisionEncode",
"inputs": {
"clip_vision": [
"103:117",
0
],
"crop": "center",
"image": [
"103:131",
0
]
},
"_meta": {
"title": "CLIP Vision Encode"
}
},
"103:12": {
"class_type": "UNETLoader",
"inputs": {
"unet_name": "capybara_v0.1.safetensors",
"weight_dtype": "default"
},
"_meta": {
"title": "Load Diffusion Model"
}
},
"103:121": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 960
},
"_meta": {
"title": "Width"
}
},
"103:122": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 960
},
"_meta": {
"title": "Height"
}
},
"103:128": {
"class_type": "GetVideoComponents",
"inputs": {
"video": [
"127",
0
]
},
"_meta": {
"title": "Get Video Components"
}
},
"103:129": {
"class_type": "ImageFromBatch",
"inputs": {
"batch_index": 0,
"image": [
"103:128",
0
],
"length": 121
},
"_meta": {
"title": "Get Image from Batch"
}
},
"103:13": {
"class_type": "SamplerCustomAdvanced",
"inputs": {
"guider": [
"103:91",
0
],
"latent_image": [
"103:115",
2
],
"noise": [
"103:25",
0
],
"sampler": [
"103:16",
0
],
"sigmas": [
"103:17",
0
]
},
"_meta": {
"title": "SamplerCustomAdvanced"
}
},
"103:131": {
"class_type": "ResizeImageMaskNode",
"inputs": {
"input": [
"103:129",
0
],
"resize_type": "scale dimensions",
"resize_type.crop": "center",
"resize_type.height": [
"103:122",
0
],
"resize_type.width": [
"103:121",
0
],
"scale_method": "lanczos"
},
"_meta": {
"title": "Resize Image/Mask"
}
},
"103:16": {
"class_type": "KSamplerSelect",
"inputs": {
"sampler_name": "euler"
},
"_meta": {
"title": "KSamplerSelect"
}
},
"103:17": {
"class_type": "BasicScheduler",
"inputs": {
"denoise": 1,
"model": [
"103:12",
0
],
"scheduler": "simple",
"steps": 20
},
"_meta": {
"title": "BasicScheduler"
}
},
"103:25": {
"class_type": "RandomNoise",
"inputs": {
"noise_seed": 908966882718651
},
"_meta": {
"title": "RandomNoise"
}
},
"103:44": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"103:11",
0
],
"text": "Change the ocean background in the video to a grassland and make the horse run on the grassland."
},
"_meta": {
"title": "CLIP Text Encode (Positive Prompt)"
}
},
"103:67": {
"class_type": "ModelSamplingSD3",
"inputs": {
"model": [
"103:12",
0
],
"shift": 7
},
"_meta": {
"title": "ModelSamplingSD3"
}
},
"103:8": {
"class_type": "VAEDecode",
"inputs": {
"samples": [
"103:13",
0
],
"vae": [
"103:10",
0
]
},
"_meta": {
"title": "VAE Decode"
}
},
"103:91": {
"class_type": "CFGGuider",
"inputs": {
"cfg": 6,
"model": [
"103:67",
0
],
"negative": [
"103:115",
1
],
"positive": [
"103:115",
0
]
},
"_meta": {
"title": "CFG Guider"
}
},
"103:93": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"103:11",
0
],
"text": "blurry, low quality, distorted, ugly, watermark, text"
},
"_meta": {
"title": "CLIP Text Encode (Negative Prompt)"
}
}
})
# run() also passes api_key to the partner nodes in this workflow.
job = client.run(workflow, api_key=api_key)常见问答
You can upload both video and image files using the LoadVideo node, as the workflow supports various common formats.
Your text instructions should be clear and concise, specifying the desired outcome to help the Capybara model generate accurate results.
Yes, you can configure the resolution settings based on the Capybara Resolution Configuration table provided in the workflow notes.
Consider increasing the number of steps in the inference process, as suggested in the author notes, to improve the quality of the generated video.