The Wan 2.1 ControlNet workflow is designed to generate videos that are guided by pose, depth, and edge controls. This workflow leverages the capabilities of the Wan2.1 model, which is known for its robust handling of video-to-video transformations. The core of the workflow is built around several key nodes such as KSampler, CLIPTextEncode, and VAEDecode, which work together to process input video frames and apply stylistic transformations based on user-defined prompts. The use of ControlNet allows for precise control over video characteristics, making it possible to manipulate aspects like pose and depth with high accuracy.
Technically, the workflow begins by loading models and preparing the initial frame using nodes like CLIPVisionLoader and LoadImage. The WanFunControlToVideo node is crucial as it integrates ControlNet's functionalities to guide the video transformation process. The Sampling & Decode group, which includes nodes like ModelSamplingSD3 and UNetTemporalAttentionMultiply, ensures that the output video maintains high quality and coherence across frames. This workflow is particularly useful for creators looking to produce stylized videos with specific visual effects, offering a high degree of customization and control over the final output.
API
Use this workflow from code
Every Comfy workflow is a JSON graph. The payload below is this workflow, exactly as ComfyUI runs it — fetch it from the URL, keep it in version control, or load it in ComfyUI and run it node by node.
Run it from TypeScript or Python with the Comfy SDK. The same code targets Comfy Cloud or a ComfyUI you host yourself — only the base URL changes.
// Install (beta)
npm i @comfyorg/sdk
// Run "Wan 2.1 ControlNet" (TypeScript)
import { Comfy } from "@comfyorg/sdk";
const client = new Comfy({ apiKey: "comfyui-..." });
// This workflow, exported in API format (see note below)
const wf = await client.workflows.fromFile("wan2.1_fun_control_api.json");
const asset = client.assets.fromFile("input.png");
wf.setInput("52", "image", asset); // LoadImage
wf.setInput("6", "text", "a women sticking out her toungue while mkaing wierd faces,(clay style:1.1)"); // CLIPTextEncode
const job = await client.run(wf);
await job.getOutputs("78")[0].toFile("output.png"); // SaveVideoThe SDK takes a workflow in API format: open this workflow in ComfyUI and use File → Export Workflow (API).
Comfy Cloud API access requires a plan with an API key.
FAQ
