Diverse ML Systems at Netflix

Netflix leverages data science and machine learning extensively, with the Machine Learning Platform (MLP) team providing tools around Metaflow to support diverse ML systems. Metaflow offers human-friendly APIs and integrations to production systems, enabling smooth transition from prototype to production. The Fast Data library facilitates fast and scalable data access from the Netflix data warehouse. Compute tasks benefit from the centralized compute platform, Titus, enhancing scalability and reproducibility. Maestro serves as the orchestration layer, supporting event-triggering and scalability for Metaflow projects. Deployment involves integrating Metaflow projects with other production systems, either through shared tables or low-latency APIs. An example use case involves building model explainers for model explainability. The system at Netflix supports content decision making for a global audience, with sophisticated models and data pipelines managed efficiently by a small team. The deployment pattern includes caching infrastructure for precomputed predictions, illustrated by a content performance visualization application.