MotionVideoSearch is a Python-based tool designed to extract motion-based embeddings from video files, enabling rapid searches for video segments using motion images. It leverages a vector database (FAISS) for efficient similarity searching and integrates seamlessly with ComfyUI for enhanced user experience.
- Offers a command-line interface (CLI) built with Typer for easy interaction.
- Utilizes PyTorch for extracting motion embeddings from video segments, which are then stored in a FAISS index.
- Provides custom nodes for ComfyUI, allowing users to search and manipulate motion images within the node graph environment.
Context
MotionVideoSearch is a specialized tool within the ComfyUI ecosystem that focuses on indexing videos based on their motion characteristics. Its primary goal is to facilitate the retrieval of specific video segments that match a given motion image, thereby enhancing the workflow for users dealing with video content.
Key Features & Benefits
- Video Ingestion: This feature allows users to break down videos into smaller segments, generating corresponding motion embeddings that are stored in a FAISS index. This process ensures that video content can be efficiently searched based on motion criteria.
- Search Functionality: Users can input a motion image to query the FAISS index, retrieving the most relevant video segments ranked by their similarity. This capability is crucial for applications that require quick access to specific video content based on visual motion.
- Concurrency Support: The tool allows for organizing indexing tasks across multiple subdirectories, enabling users to process large volumes of videos concurrently and later merge them into a single comprehensive index.
Advanced Functionalities
- Watermark Detection and Removal: By integrating a separate repository, users have the option to automatically detect and remove watermarks from video frames during the embedding process. This feature is particularly useful for maintaining the integrity of the indexed content.
- ComfyUI Nodes: The inclusion of custom nodes,
IG_MotionVideoSearchandIG_MotionVideoFrame, enhances the functionality within ComfyUI. The former retrieves top-ranked video URLs based on a motion image, while the latter condenses multiple frames into a single representation, aiding in the visualization of motion features.
Practical Benefits
MotionVideoSearch significantly streamlines the workflow for users working with video data in ComfyUI. By automating the process of embedding extraction and providing a robust search mechanism, it enhances control over video content, improves the quality of searches, and increases overall efficiency in managing large video databases.
Credits/Acknowledgments
The tool was developed by IDGallagher and is licensed under the MIT License. Contributions and suggestions for improvements are encouraged, allowing the community to enhance the functionality and usability of the project.