Building a Local Novel-to-Video Pipeline with FLUX, Wan2.2, and MuseTalk on a Single GPU

Short summary

Details a local pipeline called iTube that converts entire novels into narrated, lip-synced motion video on a single 16GB RTX 4060Ti using FLUX, Wan2.2, MuseTalk, PuLID, and ComfyUI. The key insight is that audio drives everything: spoken line length determines scene duration, and visuals are stretched or trimmed to match. The real engineering challenge is routing around each model's failure modes, such as Wan animating mouths on silent scenes or conjuring people into empty landscapes.

• •Built iTube pipeline converting full novels to narrated lip-synced video on a single 16GB GPU
• •Audio-first architecture: speech length determines scene duration, visuals adapt to match
• •Core challenge is routing around each model's specific failure modes with targeted script logic

Generated with AI, which can make mistakes.