What you'll learn

Energy wastage by residential buildings is a significant contributor to total worldwide energy consumption. Quby, an Amsterdam based technology company, offers solutions to empower homeowners to stay in control of their electricity, gas, and water usage. Using Europe's largest energy data-set, consisting of petabytes of IoT data, the company has developed AI powered products that are used by hundreds of thousands of users on a daily basis. In this talk, Erni Durdevic will take you on a tour of how Quby leverages the full Databricks stack to quickly prototype, validate, scale and launch data science products. We will explore the technical workflow of a Data Science project from end to end. Starting from developing a notebook prototype and tracking model performance with MLFlow, we move towards production-grade Databricks jobs with a CI/CD pipeline and monitoring system in place. We will see how Quby manages more than 1 million models in production, how Delta Lake allows batch and streaming on the same IoT data, and the impact these tools have had on the team itself.