Kling2.6: Motion Control

Apply precise character actions and expressions from a reference video to your character image with synchronized motion control.

Download Workflow Try in Cloud

Input

Character

Motion video

Prompt

Orientation from

Keep original sound

Advanced settings

Mode

Run

Output

00:00

Example

Call this model from your code

The request below uses the same prepared graph and settings as the Playground. Each submission can start a paid run; do not automatically retry a submission whose outcome is unknown.

Get API key

What you need

Runs on: Comfy Cloud
Key: COMFY_API_KEY
Your files: Uploaded by the code from their urls

Needs a paid Cloud plan and credits.

Download API JSON

# Python 3.10+: pip install comfy-sdk==0.4.0
import os

from comfy_sdk import Comfy

api_key = os.environ.get("COMFY_API_KEY", "").strip()
if not api_key: raise RuntimeError("Set COMFY_API_KEY")
os.environ.setdefault("COMFY_BASE_URL", "https://cloud.comfy.org")
client = Comfy(api_key=api_key)
workflow = client.workflows.from_json({
    "2": {
        "class_type": "LoadVideo",
        "inputs": {
            "file": "UPLOADED_motion_FILENAME"
        },
        "_meta": {
            "title": "Load Video"
        }
    },
    "8": {
        "class_type": "KlingMotionControl",
        "inputs": {
            "character_orientation": "video",
            "keep_original_sound": True,
            "mode": "pro",
            "model": "kling-v2-6",
            "prompt": "A sunlit botanical glasshouse in the late morning, with tall palms, hanging ferns and climbing vines rising toward a white iron-framed glass roof. The character dances on a wide stone path between raised beds of tropical plants, the whole body in view from head to feet. Warm sunlight streams through the glass panes and casts soft geometric shadows across the floor, while fine mist from overhead sprinklers drifts through the light beams and catches small rainbow glints. Lush green leaves sway gently behind the dancer, and a few orange butterflies flutter past. Natural, true-to-life colour, gentle shallow depth of field, crisp detail on the clothing and face, smooth realistic motion that follows the dancer's rhythm, a steady camera at chest height with a slow, subtle push-in.",
            "reference_image": [
                "11",
                0
            ],
            "reference_video": [
                "2",
                0
            ]
        },
        "_meta": {
            "title": "Kling Motion Control"
        }
    },
    "9": {
        "class_type": "SaveVideo",
        "inputs": {
            "filename_prefix": "video/kling-motion-control",
            "format": "auto",
            "format.codec": "auto",
            "video": [
                "8",
                0
            ]
        },
        "_meta": {
            "title": "Save Video"
        }
    },
    "11": {
        "class_type": "LoadImage",
        "inputs": {
            "button": {
                "clicked": False,
                "computedDisabled": False,
                "computedHeight": 24,
                "label": "choose file to upload",
                "last_y": 70,
                "name": "upload",
                "options": {
                    "canvasOnly": True,
                    "serialize": False
                },
                "type": "button",
                "y": 70
            },
            "image": "UPLOADED_character_FILENAME"
        },
        "_meta": {
            "title": "Load Image"
        }
    }
})
workflow.set_input("11", "image", client.assets.from_url("https://comfy.org/catalog-media/2136161c-7b56-44c1-8f25-cbfa8cbe01fc/42487951-1e24-496a-b42d-4ebfd3726efc"))
workflow.set_input("2", "file", client.assets.from_url("https://comfy.org/catalog-media/e54a3841-7438-40f4-8376-b158303bc3e4/a03c1d74-794c-4884-9c46-7cda6f4d561d"))

# run() also passes api_key to the partner nodes in this workflow.
job = client.run(workflow, api_key=api_key)
for output in job.get_outputs("9"):
    output.to_file(output.name)

API documentation

Try an example

About this workflow

The 'Kling2.6: Motion Control' workflow is designed to help users apply precise character actions and expressions from a reference video to a character image. This workflow leverages the capabilities of the Kling 2.6 model to synchronize motion control, making it highly effective for animators and video editors who need to create realistic character animations.

The workflow begins with the LoadVideo node, which imports the reference video containing the desired motions. The KlingMotionControl node is the core of this workflow, where the motion data is extracted and applied to the character image loaded via the LoadImage node. Finally, the SaveVideo node outputs the animated character video, ensuring that the character's movements are in perfect harmony with the reference video.

This workflow is particularly useful for its ability to maintain the integrity of character expressions and actions, ensuring that the animation looks natural and fluid. By using the Kling model, which is renowned for its precise motion capture capabilities, the workflow allows for detailed motion control that can be customized to fit various animation needs. The inclusion of the MarkdownNote node provides users with the ability to annotate and document their workflow process, making it easier to replicate or modify for future projects.

Kling2.6: Motion Control

Input

Inside the workflow

Call this model from your code

What you need

Try an example

About this workflow

Questions & answers

Input

Inside the workflow

Call this model from your code

What you need

Try an example

About this workflow

Questions & answers

Can I use any video as a reference for motion control?

Do I need to adjust the aspect ratio of the image and video?

What if the character image is only half-body?

How can I document my workflow process?