LTX 2.3_ID_lora_CelebVHQ
Сохраните стабильность голоса в вашем open source пайплайне генерации видео с Qwen 3 TTS и LTX 2.3 с CelebHVQ ID-LoRA.
Внутри рабочего процесса
Предварительный просмотр исходного шаблона только для чтения. Откройте его в Cloud, чтобы изучить ноды и внести изменения.
Loading the graph
Open full-size workflow previewФайлы в этом рабочем процессе
- Выход
- Видео
- Автор
- ComfyUI
Вызовите эту модель из своего кода
Приведённый ниже запрос использует тот же подготовленный граф и настройки, что и Playground. Каждая отправка может запустить платный запуск; не повторяйте автоматически отправку, результат которой неизвестен.
What you need
- Runs on
- Comfy Cloud
- Ключ
- COMFY_API_KEY
- Ваши файлы
- Загружаются кодом по их URL
Требуются платный тариф Cloud и кредиты.
# Python 3.10+: pip install comfy-sdk==0.4.0
import os
from comfy_sdk import Comfy
api_key = os.environ.get("COMFY_API_KEY", "").strip()
if not api_key: raise RuntimeError("Set COMFY_API_KEY")
os.environ.setdefault("COMFY_BASE_URL", "https://cloud.comfy.org")
client = Comfy(api_key=api_key)
workflow = client.workflows.from_json({
"440": {
"class_type": "LoadImage",
"inputs": {
"image": "UPLOADED_image_FILENAME"
},
"_meta": {
"title": "Load Image"
}
},
"441": {
"class_type": "GeminiNode",
"inputs": {
"images": [
"440",
0
],
"model": "gemini-3-1-pro",
"prompt": "A greenhouse gardener talking straight into the camera, warm and a little amused, about why he starts every morning by repotting one plant before anyone else arrives. He says the quiet of the glasshouse at sunrise, the smell of wet soil and the small job done with his hands set the pace for his whole day, and he ends by inviting the viewer to try one slow, simple task tomorrow morning before checking their phone. Casual, honest, handheld testimonial feel, soft birdsong and a distant sprinkler in the background, natural pauses and a small laugh near the end.",
"seed": 1101794556747325,
"system_prompt": "Role: You are a professional prompt writer for ID-LoRA prompt generation.\n\nTask: Generate a two-part output from the user’s input.\n\nStrict Output Constraint (IMPORTANT):\n\nReturn the output as RAW PLAIN TEXT only.\nDO NOT use Markdown code blocks.\nDO NOT use any headers, labels, or bold text outside of the generated two-part output.\nUse ONLY the separator --- to divide the two parts.\nPart 1: spoken dialogue/performance text only, optimized to sound natural when spoken aloud.\nPart 2: the same concept rewritten in the exact ID-LoRA tagged format using these three sections only:\n[VISUAL]: ...\n[SPEECH]: ...\n[SOUNDS]: ...\nDo NOT output anything before Part 1 or after Part 2.\n\nGlobal Length Rule:\n\nKeep the spoken content short enough to produce about 10 seconds of speech unless the user explicitly asks otherwise.\nAim for roughly 20 to 35 spoken words in Part 1.\nCondense long inputs aggressively while preserving the core meaning, tone, and key message.\nPrioritize brevity, clarity, speakability, and a strong opening hook over completeness.\nRemove repetition, side details, filler, and nonessential context.\nIf the input is too long, compress it into a concise spoken version rather than preserving everything.\n\nPart 1 Rules:\n\nPart 1 must contain ONLY the words meant to be spoken aloud.\nDo NOT begin Part 1 with visual description, scene-setting narration, character description, or action description.\nStart immediately with the most attention-grabbing, high-impact spoken hook.\nThe hook should appear as early as possible, ideally in the first sentence or phrase.\nDo NOT describe what the character looks like, where they are, what they are wearing, what they are holding, or what is happening visually unless that information is spoken by the character as part of the dialogue itself.\nPart 1 should read like a real spoken performance, monologue, ad read, or direct address.\nUse punctuation, pauses, ellipses, and selective capitalization to improve spoken delivery when useful.\nDo NOT use SSML.\nDo NOT use non-voice tags such as music, ambience, sound effects, camera notes, or technical markup.\nPrefer one clear hook, one core message or benefit, and one short closing beat.\n\nPart 2 Rules:\n\nPart 2 must contain exactly three tagged sections in this exact order:\n[VISUAL]:\n[SPEECH]:\n[SOUNDS]:\n\n[VISUAL] Rules:\nDescribe the shot type, subject appearance, clothing, setting, lighting, framing, and visible actions.\nBe descriptive enough to guide generation clearly, but keep it compact and production-useful.\nExplicitly indicate that the person is speaking or talking to camera so the line is generated as on-screen speech rather than voice-over.\nDefault toward believable UGC-style behavior unless the user asks otherwise:\ndirect-to-camera delivery\nhandheld or phone-like framing\nselfie or testimonial feel\nnatural gestures\nauthentic facial expression\nslight body movement\ncasual presenting or showing when relevant\nAvoid cinematic, polished, theatrical, or overly staged action unless the user explicitly asks for that.\n\n[SPEECH] Rules:\nWrite the exact words the person should say.\nThis must be the literal transcript, not a summary.\nKeep it closely aligned with Part 1, ideally verbatim except for minor punctuation cleanup if needed.\nDo NOT add scene description, camera notes, or sound cues inside [SPEECH].\n\n[SOUNDS] Rules:\nDescribe both the vocal delivery and the ambient/background audio.\nInclude speaker qualities such as tone, volume, pace, energy, and mic proximity or distance.\nInclude relevant environmental sounds, room tone, music, nature sounds, or other ambience when appropriate.\nKeep the audio grounded, coherent with the visual scene, and not overly busy unless requested.\n\nConsistency Rules:\n\nPart 1 and Part 2 must describe the same idea, message, tone, and scene.\n[SPEECH] in Part 2 should match Part 1 as closely as possible.\n[VISUAL], [SPEECH], and [SOUNDS] must feel like one unified prompt, not separate concepts.\nDo not introduce unrelated ideas, props, settings, or actions that were not implied by the user’s request.\nIf the user provides exact wording, preserve it in Part 1 and [SPEECH] unless the user asks for rewriting or shortening.\n\nSafety and Quality Rules:\n\nDo not introduce sensitive, explicit, hateful, political, or unsafe content that was not already in the user input.\nEnsure the final output always contains exactly two parts separated by ---.\nEnsure Part 2 always uses the exact three ID-LoRA tags and includes all three of them once."
},
"_meta": {
"title": "Google Gemini"
}
},
"443": {
"class_type": "PreviewAny",
"inputs": {
"source": [
"441",
0
]
},
"_meta": {
"title": "Preview as Text"
}
},
"473": {
"class_type": "PreviewAny",
"inputs": {
"source": [
"476",
0
]
},
"_meta": {
"title": "Preview as Text"
}
},
"474": {
"class_type": "PreviewAny",
"inputs": {
"source": [
"475",
0
]
},
"_meta": {
"title": "Preview as Text"
}
},
"475": {
"class_type": "RegexExtract",
"inputs": {
"case_insensitive": True,
"dotall": False,
"group_index": 1,
"mode": "First Group",
"multiline": False,
"regex_pattern": "(?sm)^(.*?)^---$(.*)",
"string": [
"441",
0
]
},
"_meta": {
"title": "Extract Text"
}
},
"476": {
"class_type": "RegexExtract",
"inputs": {
"case_insensitive": True,
"dotall": False,
"group_index": 1,
"mode": "First Group",
"multiline": False,
"regex_pattern": "(?s)(?<=\\n---\\n)([\\s\\S]*)$",
"string": [
"441",
0
]
},
"_meta": {
"title": "Extract Text"
}
},
"479": {
"class_type": "SaveVideo",
"inputs": {
"filename_prefix": "video/ComfyUI",
"format": "auto",
"format.codec": "auto",
"video": [
"662:653",
0
]
},
"_meta": {
"title": "Save Video"
}
},
"621": {
"class_type": "FB_Qwen3TTSCustomVoice",
"inputs": {
"attention": "auto",
"custom_model_path": "",
"custom_speaker_name": "",
"device": "cuda",
"instruct": [
"473",
0
],
"language": "English",
"max_new_tokens": 2048,
"model_choice": "1.7B",
"precision": "bf16",
"repetition_penalty": 1.05,
"seed": 193948051206350,
"speaker": "Aiden",
"temperature": 1,
"text": [
"474",
0
],
"top_k": 20,
"top_p": 0.8,
"unload_model_after_generate": False
},
"_meta": {
"title": "🎵 Qwen3-TTS CustomVoice"
}
},
"622": {
"class_type": "SaveAudioMP3",
"inputs": {
"audio": [
"621",
0
],
"filename_prefix": "audio/ComfyUI",
"quality": "V0"
},
"_meta": {
"title": "Save Audio (MP3) (DEPRECATED)"
}
},
"662:293": {
"class_type": "LoraLoaderModelOnly",
"inputs": {
"lora_name": "ltx-2.3-22b-distilled-lora-384.safetensors",
"model": [
"662:317",
0
],
"strength_model": 0.5
},
"_meta": {
"title": "Load LoRA"
}
},
"662:313": {
"class_type": "LatentUpscaleModelLoader",
"inputs": {
"model_name": "ltx-2.3-spatial-upscaler-x2-1.1.safetensors"
},
"_meta": {
"title": "Load Latent Upscale Model"
}
},
"662:317": {
"class_type": "CheckpointLoaderSimple",
"inputs": {
"ckpt_name": "ltx-2.3-22b-dev-fp8.safetensors"
},
"_meta": {
"title": "Load Checkpoint"
}
},
"662:318": {
"class_type": "LTXAVTextEncoderLoader",
"inputs": {
"ckpt_name": "ltx-2.3-22b-dev-fp8.safetensors",
"device": "default",
"text_encoder": "gemma_3_12B_it_fp4_mixed.safetensors"
},
"_meta": {
"title": "Load LTXV Audio Text Encoder"
}
},
"662:335": {
"class_type": "LTXVAudioVAELoader",
"inputs": {
"ckpt_name": "ltx-2.3-22b-dev-fp8.safetensors"
},
"_meta": {
"title": "Load LTXV Audio VAE"
}
},
"662:346": {
"class_type": "LoraLoaderModelOnly",
"inputs": {
"lora_name": "ltx-2.3-id-lora-celebvhq-3k.safetensors",
"model": [
"662:293",
0
],
"strength_model": 1
},
"_meta": {
"title": "Load LoRA"
}
},
"662:348": {
"class_type": "LTXVEmptyLatentAudio",
"inputs": {
"audio_vae": [
"662:335",
0
],
"batch_size": 1,
"frame_rate": [
"662:652",
1
],
"frames_number": [
"662:657",
1
]
},
"_meta": {
"title": "Empty LTXV Latent Audio"
}
},
"662:349": {
"class_type": "LTXVReferenceAudio",
"inputs": {
"audio_vae": [
"662:335",
0
],
"end_percent": 1,
"identity_guidance_scale": 3,
"model": [
"662:346",
0
],
"negative": [
"662:642",
0
],
"positive": [
"662:636",
0
],
"reference_audio": [
"621",
0
],
"start_percent": 0
},
"_meta": {
"title": "LTXV Reference Audio (ID-LoRA)"
}
},
"662:624": {
"class_type": "RandomNoise",
"inputs": {
"noise_seed": 42
},
"_meta": {
"title": "RandomNoise"
}
},
"662:625": {
"class_type": "RandomNoise",
"inputs": {
"noise_seed": 953776337212690
},
"_meta": {
"title": "RandomNoise"
}
},
"662:626": {
"class_type": "LTXVConcatAVLatent",
"inputs": {
"audio_latent": [
"662:639",
1
],
"video_latent": [
"662:633",
0
]
},
"_meta": {
"title": "Concat AV Latent"
}
},
"662:627": {
"class_type": "KSamplerSelect",
"inputs": {
"sampler_name": "euler_cfg_pp"
},
"_meta": {
"title": "KSamplerSelect"
}
},
"662:628": {
"class_type": "ManualSigmas",
"inputs": {
"sigmas": "0.85, 0.7250, 0.4219, 0.0"
},
"_meta": {
"title": "ManualSigmas"
}
},
"662:629": {
"class_type": "CFGGuider",
"inputs": {
"cfg": 1,
"model": [
"662:293",
0
],
"negative": [
"662:631",
1
],
"positive": [
"662:631",
0
]
},
"_meta": {
"title": "CFG Guider"
}
},
"662:630": {
"class_type": "SamplerCustomAdvanced",
"inputs": {
"guider": [
"662:643",
0
],
"latent_image": [
"662:659",
0
],
"noise": [
"662:625",
0
],
"sampler": [
"662:634",
0
],
"sigmas": [
"662:638",
0
]
},
"_meta": {
"title": "SamplerCustomAdvanced"
}
},
"662:631": {
"class_type": "LTXVCropGuides",
"inputs": {
"latent": [
"662:639",
0
],
"negative": [
"662:637",
1
],
"positive": [
"662:637",
0
]
},
"_meta": {
"title": "LTXVCropGuides"
}
},
"662:632": {
"class_type": "LTXVLatentUpsampler",
"inputs": {
"samples": [
"662:639",
0
],
"upscale_model": [
"662:313",
0
],
"vae": [
"662:317",
2
]
},
"_meta": {
"title": "LTXVLatentUpsampler"
}
},
"662:633": {
"class_type": "LTXVImgToVideoInplace",
"inputs": {
"bypass": False,
"image": [
"662:649",
0
],
"latent": [
"662:632",
0
],
"strength": 1,
"vae": [
"662:317",
2
]
},
"_meta": {
"title": "LTXVImgToVideoInplace"
}
},
"662:634": {
"class_type": "KSamplerSelect",
"inputs": {
"sampler_name": "euler_ancestral_cfg_pp"
},
"_meta": {
"title": "KSamplerSelect"
}
},
"662:635": {
"class_type": "LTXVAudioVAEDecode",
"inputs": {
"audio_vae": [
"662:335",
0
],
"samples": [
"662:641",
1
]
},
"_meta": {
"title": "LTXV Audio VAE Decode"
}
},
"662:636": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"662:318",
0
],
"text": [
"662:660",
0
]
},
"_meta": {
"title": "CLIP Text Encode (Prompt)"
}
},
"662:637": {
"class_type": "LTXVConditioning",
"inputs": {
"frame_rate": [
"662:652",
0
],
"negative": [
"662:349",
2
],
"positive": [
"662:349",
1
]
},
"_meta": {
"title": "LTXVConditioning"
}
},
"662:638": {
"class_type": "ManualSigmas",
"inputs": {
"sigmas": "1.0, 0.99375, 0.9875, 0.98125, 0.975, 0.909375, 0.725, 0.421875, 0.0"
},
"_meta": {
"title": "ManualSigmas"
}
},
"662:639": {
"class_type": "LTXVSeparateAVLatent",
"inputs": {
"av_latent": [
"662:630",
0
]
},
"_meta": {
"title": "Separate AV Latent"
}
},
"662:640": {
"class_type": "SamplerCustomAdvanced",
"inputs": {
"guider": [
"662:629",
0
],
"latent_image": [
"662:626",
0
],
"noise": [
"662:624",
0
],
"sampler": [
"662:627",
0
],
"sigmas": [
"662:628",
0
]
},
"_meta": {
"title": "SamplerCustomAdvanced"
}
},
"662:641": {
"class_type": "LTXVSeparateAVLatent",
"inputs": {
"av_latent": [
"662:640",
0
]
},
"_meta": {
"title": "Separate AV Latent"
}
},
"662:642": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"662:318",
0
],
"text": "pc game, console game, video game, cartoon, childish, ugly"
},
"_meta": {
"title": "CLIP Text Encode (Prompt)"
}
},
"662:643": {
"class_type": "CFGGuider",
"inputs": {
"cfg": 1,
"model": [
"662:349",
0
],
"negative": [
"662:637",
1
],
"positive": [
"662:637",
0
]
},
"_meta": {
"title": "CFG Guider"
}
},
"662:644": {
"class_type": "VAEDecodeTiled",
"inputs": {
"overlap": 64,
"samples": [
"662:641",
0
],
"temporal_overlap": 4,
"temporal_size": 4096,
"tile_size": 768,
"vae": [
"662:317",
2
]
},
"_meta": {
"title": "VAE Decode (Tiled)"
}
},
"662:645": {
"class_type": "LTXVImgToVideoInplace",
"inputs": {
"bypass": False,
"image": [
"662:649",
0
],
"latent": [
"662:656",
0
],
"strength": 0.7,
"vae": [
"662:317",
2
]
},
"_meta": {
"title": "LTXVImgToVideoInplace"
}
},
"662:646": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 1280
},
"_meta": {
"title": "Width"
}
},
"662:647": {
"class_type": "SolidMask",
"inputs": {
"height": [
"662:658",
0
],
"value": 0,
"width": [
"662:646",
0
]
},
"_meta": {
"title": "Create Solid Mask"
}
},
"662:648": {
"class_type": "ResizeImagesByLongerEdge",
"inputs": {
"images": [
"662:661",
0
],
"longer_edge": 1536
},
"_meta": {
"title": "Resize Images by Longer Edge (DEPRECATED)"
}
},
"662:649": {
"class_type": "LTXVPreprocess",
"inputs": {
"image": [
"662:648",
0
],
"img_compression": 18
},
"_meta": {
"title": "LTXV Preprocess"
}
},
"662:650": {
"class_type": "ComfyMathExpression",
"inputs": {
"expression": "a/2",
"values.a": [
"662:658",
0
]
},
"_meta": {
"title": "Math Expression"
}
},
"662:651": {
"class_type": "ComfyMathExpression",
"inputs": {
"expression": "a/2",
"values.a": [
"662:646",
0
]
},
"_meta": {
"title": "Math Expression"
}
},
"662:652": {
"class_type": "ComfyMathExpression",
"inputs": {
"expression": "a",
"values.a": [
"662:655",
0
]
},
"_meta": {
"title": "Math Expression"
}
},
"662:653": {
"class_type": "CreateVideo",
"inputs": {
"audio": [
"662:635",
0
],
"bit_depth": "auto",
"codec": "none",
"color_space": "sRGB",
"fps": [
"662:652",
0
],
"images": [
"662:644",
0
]
},
"_meta": {
"title": "Create Video"
}
},
"662:654": {
"class_type": "PrimitiveFloat",
"inputs": {
"value": 9
},
"_meta": {
"title": "Duration"
}
},
"662:655": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 25
},
"_meta": {
"title": "Frame Rate"
}
},
"662:656": {
"class_type": "EmptyLTXVLatentVideo",
"inputs": {
"batch_size": 1,
"height": [
"662:650",
1
],
"length": [
"662:657",
1
],
"width": [
"662:651",
1
]
},
"_meta": {
"title": "Empty LTXV Latent Video"
}
},
"662:657": {
"class_type": "ComfyMathExpression",
"inputs": {
"expression": "a * b + 1",
"values.a": [
"662:654",
0
],
"values.b": [
"662:655",
0
]
},
"_meta": {
"title": "Math Expression"
}
},
"662:658": {
"class_type": "PrimitiveInt",
"inputs": {
"value": 720
},
"_meta": {
"title": "Height"
}
},
"662:659": {
"class_type": "LTXVConcatAVLatent",
"inputs": {
"audio_latent": [
"662:348",
0
],
"video_latent": [
"662:645",
0
]
},
"_meta": {
"title": "Concat AV Latent"
}
},
"662:660": {
"class_type": "PrimitiveStringMultiline",
"inputs": {
"value": [
"473",
0
]
},
"_meta": {
"title": "Prompt"
}
},
"662:661": {
"class_type": "ResizeImageMaskNode",
"inputs": {
"input": [
"440",
0
],
"resize_type": "scale dimensions",
"resize_type.crop": "center",
"resize_type.height": [
"662:658",
0
],
"resize_type.width": [
"662:646",
0
],
"scale_method": "lanczos"
},
"_meta": {
"title": "Resize Image/Mask"
}
}
})
workflow.set_input("440", "image", client.assets.from_url("https://comfy.org/catalog-media/5d0fa42a-34e1-4555-877a-6cd998ac071c/c529f8e8-bfdf-488f-8f98-436a881972a9"))
# run() also passes api_key to the partner nodes in this workflow.
job = client.run(workflow, api_key=api_key)
for output in job.get_outputs("479"):
output.to_file(output.name)Попробовать пример
Об этом рабочем процессе
Этот учебный рабочий процесс ComfyUI превращает одно изображение в короткое видео с озвучкой, с устойчивой визуальной идентичностью и воспринимаемым совпадением движения губ. Он сочетает Lightricks LTX 2.3 для генерации видео с ID-LoRA (CelebVHQ), чтобы лицо и стиль субъекта оставались согласованными от кадра к кадру, а Qwen 3 TTS создаёт соответствующую голосовую дорожку. Пайплайн разделён на понятные группы: Load Image, Create Prompts, Prompt Extraction, Create Voice и Create Video.
Внутри GeminiNode анализирует загруженное изображение (через LoadImage) и набрасывает два взаимодополняющих текста: реплику для речи с подсказками по экспрессии и описание сцены/внешности. Затем RegexExtract парсит их в формат, готовый для ID-LoRA, с тремя размеченными секциями: [VISUAL], [SPEECH] и [SOUNDS]. FB_Qwen3TTSCustomVoice использует [SPEECH] и стилевые подсказки из [SOUNDS], чтобы синтезировать голосовую дорожку, которая сохраняется через SaveAudioMP3. Видеонода LTX 2.3 (пользовательская нода в этом графе) принимает структурированный промпт вместе с референсным изображением и рендерит говорящий ролик, а ID-LoRA CelebVHQ улучшает удержание идентичности в динамичных, «в полевых условиях» сценариях. PreviewAny помогает быстро итератировать, а SaveVideo записывает финальный клип. Локальные пользователи могут опционально загрузить LTX spatial upscaler (ltx-2.3-spatial-upscaler-x2-1.1.safetensors) для более чистых деталей на повышенном разрешении.
Вопросы и ответы
Он помогает LTX 2.3 сохранять идентичность субъекта и черты лица между кадрами, особенно в динамичных «in-the-wild» сценах. Лучше всего работает с чётким фронтальным изображением и хорошо структурированным промптом, где отдельно идут [VISUAL], [SPEECH] и [SOUNDS].
Рабочий процесс выравнивает воспринимаемые движения рта за счёт явного текста в [SPEECH] и соответствующего TTS-аудио. Это не покадровый lip-sync, но короткие фразы, чёткая пунктуация и подсказки по экспрессии улучшают совпадение. При необходимости можно подогнать тайминг на монтаже.
Да. Замените LoRA в ноде LTX 2.3. CelebVHQ настроен на «in-the-wild» движение; TalkVid обычно сильнее для крупного плана в студийных talking-head сценах. Для пользовательских LoRA оставьте тот же формат структурированного промпта и тестируйте разные значения strength.
Опционально. Если вы положите ltx-2.3-spatial-upscaler-x2-1.1.safetensors в latent_upscale_models, рабочий процесс сможет рендерить более резкий результат на более высоком разрешении. Если VRAM ограничен, отключите апскейл или рендерьте на меньшем базовом разрешении.