Qwen-Image ControlNet Model Patch
Control image generation using Qwen-Image ControlNet models. Supports canny, depth, and inpainting controls through model patching.
About
The Qwen-Image ControlNet Model Patch workflow is designed to provide enhanced control over image generation through the use of Qwen-Image models. This workflow integrates several advanced nodes, including KSampler, CLIPTextEncode, and QwenImageDiffsynthControlnet, to allow for precise manipulation of image attributes such as canny, depth, and inpainting.
By leveraging the ModelPatchLoader node, users can apply patches to models, enabling dynamic adjustments and improved image quality. This workflow is particularly useful for artists and designers who need to generate images with specific stylistic or thematic elements, as it allows for detailed customization using ControlNet techniques.
Technically, the workflow operates by first loading the necessary models and reference images, then processing these inputs through a series of nodes that encode, decode, and apply the desired transformations. The VAELoader and UNETLoader nodes play a crucial role in managing the image data, while the CLIPLoader and CLIPTextEncode nodes handle text-based prompts to guide the image generation process. The integration of the QwenImageDiffsynthControlnet node allows for the application of ControlNet models to refine image details, making this workflow a powerful tool for generating high-quality, customized images.
What you provide
- Reading the workflow
Runs in ComfyUI. This workflow can't run on this page. Open it in Comfy Cloud with everything loaded, or download it for your own ComfyUI.
Try in CloudInside the workflow
A read-only preview of the node graph. Open it in Comfy Cloud or download it to edit the nodes.
Loading the graph
Open full-size workflow preview- Author
- ComfyUI
Requires custom nodes
- Reading the workflow
Use this workflow from code
Generated from the workflow graph and inputs when possible; stored examples are shown otherwise. Review credentials and endpoints before running code, which may incur charges.
What you need
- Runs on
- Comfy Cloud
- Key
- COMFY_API_KEY
Needs a paid Cloud plan and credits.
# Python 3.10+: pip install comfy-sdk==0.4.0
import os
from comfy_sdk import Comfy
api_key = os.environ.get("COMFY_API_KEY", "").strip()
if not api_key: raise RuntimeError("Set COMFY_API_KEY")
os.environ.setdefault("COMFY_BASE_URL", "https://cloud.comfy.org")
client = Comfy(api_key=api_key)
workflow = client.workflows.from_json({
"3": {
"class_type": "KSampler",
"inputs": {
"cfg": 2.5,
"denoise": 1,
"latent_image": [
"76",
0
],
"model": [
"69",
0
],
"negative": [
"7",
0
],
"positive": [
"6",
0
],
"sampler_name": "euler",
"scheduler": "simple",
"seed": 91832422759220,
"steps": 20
},
"_meta": {
"title": "KSampler"
}
},
"6": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"38",
0
],
"text": "Conceptual makeup, a fairy girl with pink hair, pink shimmery scales dotted at the corners of her eyes, starry eyeshadow, makeup painting, thin eyebrows, three-dimensional features, a glossy finish, dazzling gold powder, silver fine glitter, a sense of layering and depth in the makeup, decorated with gold ornaments, pearlescent eyeshadow, dreamy makeup, soft pastels and subtle sparkles, a mysterious and fantasy-filled atmosphere, high-end makeup, dappled light on the face, soft lighting, optimal shadows, complex depth of field, dramatic lighting, clear focus, 8k, high quality, Fujifilm filter, surreal, a dreamy pastel wonderland, bright colors, a starry pink background, realistic.\n"
},
"_meta": {
"title": "CLIP Text Encode (Positive Prompt)"
}
},
"7": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"38",
0
],
"text": " "
},
"_meta": {
"title": "CLIP Text Encode (Negative Prompt)"
}
},
"8": {
"class_type": "VAEDecode",
"inputs": {
"samples": [
"3",
0
],
"vae": [
"39",
0
]
},
"_meta": {
"title": "VAE Decode"
}
},
"37": {
"class_type": "UNETLoader",
"inputs": {
"unet_name": "qwen_image_fp8_e4m3fn.safetensors",
"weight_dtype": "default"
},
"_meta": {
"title": "Load Diffusion Model"
}
},
"38": {
"class_type": "CLIPLoader",
"inputs": {
"clip_name": "qwen_2.5_vl_7b_fp8_scaled.safetensors",
"device": "default",
"type": "qwen_image"
},
"_meta": {
"title": "Load CLIP"
}
},
"39": {
"class_type": "VAELoader",
"inputs": {
"vae_name": "qwen_image_vae.safetensors"
},
"_meta": {
"title": "Load VAE"
}
},
"60": {
"class_type": "SaveImage",
"inputs": {
"filename_prefix": "ComfyUI",
"images": [
"8",
0
]
},
"_meta": {
"title": "Save Image"
}
},
"66": {
"class_type": "ModelSamplingAuraFlow",
"inputs": {
"model": [
"37",
0
],
"sampling": "flow",
"shift": 3.1000000000000005
},
"_meta": {
"title": "ModelSamplingAuraFlow"
}
},
"69": {
"class_type": "QwenImageDiffsynthControlnet",
"inputs": {
"end_percent": 1,
"image": [
"72",
0
],
"model": [
"66",
0
],
"model_patch": [
"70",
0
],
"start_percent": 0,
"strength": 1,
"vae": [
"39",
0
]
},
"_meta": {
"title": "Apply Qwen Image DiffSynth ControlNet"
}
},
"70": {
"class_type": "ModelPatchLoader",
"inputs": {
"name": "qwen_image_canny_diffsynth_controlnet.safetensors"
},
"_meta": {
"title": "Load Model Patch"
}
},
"71": {
"class_type": "LoadImage",
"inputs": {
"image": "image_qwen_image_controlnet_patch_input_image.png"
},
"_meta": {
"title": "Load Image"
}
},
"72": {
"class_type": "Canny",
"inputs": {
"high_threshold": 0.2,
"image": [
"75",
0
],
"low_threshold": 0.1
},
"_meta": {
"title": "Detect Edges (Canny)"
}
},
"73": {
"class_type": "PreviewImage",
"inputs": {
"images": [
"72",
0
]
},
"_meta": {
"title": "Preview Image"
}
},
"75": {
"class_type": "ImageScaleToTotalPixels",
"inputs": {
"image": [
"71",
0
],
"megapixels": 1.68,
"resolution_steps": 1,
"upscale_method": "area"
},
"_meta": {
"title": "Scale Image to Total Pixels"
}
},
"76": {
"class_type": "VAEEncode",
"inputs": {
"pixels": [
"75",
0
],
"vae": [
"39",
0
]
},
"_meta": {
"title": "VAE Encode"
}
}
})
# run() also passes api_key to the partner nodes in this workflow.
job = client.run(workflow, api_key=api_key)Questions & answers
Try increasing the shift value in the KSampler node to enhance image clarity and brightness.
Yes, you can upload your own models using the ModelPatchLoader node to customize the image generation process.
You can adjust the level of detail by modifying the settings in the KSampler and ModelSamplingAuraFlow nodes.
Yes, you can generate images based solely on text prompts, but using a reference image can enhance the accuracy and detail of the output.