Show HN: Hotshot – 4 Person Team Builds a State of the Art Video Model

A four-person team has developed Hotshot, a large-scale diffusion transformer model designed for text-to-video generation. Hotshot is noted for its prompt alignment, consistency, and ability to produce videos of up to 10 seconds at 720p resolution. The team has trained three models over the past 13 months, starting with Hotshot-XL, which generated 1-second videos, and progressing to Hotshot Act-One, which produced 3-second videos. The latest model, Hotshot, has shown a 70% preference rate among users compared to other text-to-video models. The team faced significant challenges in data engineering, requiring the scaling of their dataset to 600 million video clips and 1 billion images. They also developed a video captioner to enhance temporal understanding, which involved managing thousands of GPUs and optimizing their training processes. The training phase revealed complexities in infrastructure management and optimization, with the team likening the experience to launching a rocket. They implemented various strategies to handle GPU failures and data streaming issues, ultimately compressing their dataset to ensure efficient training. The entire process took four months and millions of GPU hours, showcasing the team's commitment to pushing the boundaries of AI-generated video content.

- Hotshot is a text-to-video model capable of generating 10-second videos at 720p.

- The team trained three models over 13 months, with Hotshot showing a 70% user preference over competitors.

- Significant challenges included scaling datasets and managing thousands of GPUs.

- The training process involved complex infrastructure and optimization strategies.

- The project took four months and millions of GPU hours to complete.