What you'll learn

With exponential growth in data, diverse sources, faster processing, and changing business needs, traditional ETL tools struggle to meet modern pipeline demands. This is where Azure Databricks and Azure Data Factory come in. Databricks handles large-scale data and enables faster, more reliable ETL pipelines, while Data Factory ingests data from multiple sources and orchestrates workflows efficiently.

In this course, Build and Run ETL Pipelines with Azure Databricks and Azure Data Factory, you’ll gain the ability to build scalable and reliable ETL pipelines using Azure Databricks and Azure Data Factory.

First, you’ll learn how Azure Databricks and Azure Data Factory can help you build common ETL patterns. Then, you’ll see how to build ETL pipelines using Azure Databricks - extracting data from multiple sources, like Azure Data Lake Store; cleaning and transforming the data using Spark; and loading the processed data in Data Lake or as Databricks Tables in Delta Lake format. Next, you’ll explore how to do performance optimization in Azure Databricks. Finally, you’ll discover how to automate and orchestrate ETL pipelines using Azure Databricks and Azure Data Factory. When you’re finished with this course, you’ll have the skills and knowledge of Azure Databricks and Azure Data Factory needed to build scalable, reliable, and performant ETL pipelines.