A ComfyUI node pack that integrates FreeLong (NeurIPS 2024) spectral blending technology specifically for enhancing video generation with the Wan 2.2 model. It aims to improve motion consistency and prompt adherence in video outputs, particularly when generating longer sequences.
- Enhanced motion coherence within 81 frames, resulting in smoother transitions and reduced motion reversal.
- Facilitates chunked video generation, allowing for reliable continuation and clean anchors across multiple segments.
- Supports both image-to-video (i2v) and text-to-video (t2v) workflows, with particular advantages for i2v due to its anchor-based mechanisms.
Context
This tool is designed to enhance the ComfyUI workflow for video generation by implementing advanced spectral blending techniques. Its primary goal is to ensure consistent motion and adherence to prompts when generating videos using the Wan 2.2 model, addressing common challenges in video coherence and quality.
Key Features & Benefits
The FreeLong implementation provides several practical features:
- Frequency-aware attention blending: This technique significantly reduces motion reversal and subject drift, allowing for a more coherent visual narrative.
- Chunked generation capability: By processing video in segments of 81 frames, the tool ensures that transitions between chunks are seamless, which is crucial for longer video sequences.
- Motion scale adjustments: Users can manipulate the speed of motion through parameters, enhancing the versatility of video outputs.
Advanced Functionalities
The tool includes specialized nodes such as:
- WanFreeLongEnforcer: This offers stricter motion locking for complex scenes, ensuring that even intricate movements maintain their intended trajectory.
- WanMotionScaleAdvanced: An experimental node that provides additional controls for motion scaling, potentially improving coherence beyond the standard 81 frames.
Practical Benefits
By employing this tool, users can expect an improved workflow characterized by:
- Greater control over motion dynamics, leading to higher-quality video outputs.
- Enhanced efficiency in generating longer videos without sacrificing coherence, thanks to the chunking method.
- The ability to experiment with various prompts and scene changes across video segments, enriching the storytelling potential of the generated content.
Credits/Acknowledgments
The development of this tool is based on the FreeLong paper from NeurIPS 2024, with implementation contributions from the original authors and specific acknowledgments to @onerok for enhancements related to end-frame guidance. The project is licensed under the MIT license.