This custom node for ComfyUI facilitates the scraping and downloading of images and videos from Midjourney showcase pages. It employs undetected_chromedriver to navigate around anti-scraping protections but necessitates session cookies from a logged-in browser to operate effectively.
- Scrapes both images and videos from Midjourney showcase URLs, ensuring a comprehensive retrieval of content.
- Converts
.webpimage and video thumbnails into full.pngimages and.mp4videos, enhancing accessibility and usability of the downloaded files. - Offers customization options for the number of items to scrape, output directory, and browser settings, allowing users to tailor the scraping process to their needs.
Context
This tool serves as a dedicated node within ComfyUI designed specifically for extracting media content from Midjourney's showcase pages. Its primary aim is to streamline the process of gathering visual assets while bypassing restrictions that typically hinder automated scraping efforts.
Key Features & Benefits
The scraper's ability to handle both images and videos makes it versatile for users looking to compile a variety of media types. The conversion of thumbnails into standard formats like .png and .mp4 ensures that users receive high-quality files that are ready for immediate use. Additionally, the customization options provide flexibility, allowing users to define specific parameters for their scraping tasks.
Advanced Functionalities
This tool incorporates a retry mechanism that checks file sizes to confirm that downloads are complete and not corrupted, which is crucial for maintaining the integrity of the media files. The ability to run the browser in a headless mode offers efficiency by allowing scraping to occur without the need for a visible user interface.
Practical Benefits
By integrating this node into ComfyUI, users can significantly enhance their workflow when collecting media from Midjourney. The tool not only saves time by automating the scraping process but also ensures that users have control over the quality and quantity of the downloaded content, ultimately improving overall efficiency.
Credits/Acknowledgments
The tool is developed as an open-source contribution, utilizing the undetected_chromedriver library to facilitate its scraping capabilities. Users are encouraged to acknowledge the original authors and contributors involved in its development.