Deploying Data Pipelines in Microsoft Azure

In this course, you will learn foundational knowledge needed to apply CI/CD methodologies to your pipeline creation process in Azure Data Factory to deploy robust and well-tested data pipelines to production.
Course info
Rating
(61)
Level
Intermediate
Updated
Dec 12, 2019
Duration
1h 28m
Table of contents
Course Overview
Getting Started Deploying Data Pipelines in Azure
Creating the Data Pipeline Deployment Infrastructure
Creating Azure Data Factory Environments
Integrating Azure Data Factory Pipelines with Source Control Using Azure DevOps
Deploying Data Pipelines Using ARM Templates and Azure DevOps
Implementing Continuous Integration and Delivery of Azure Data Factory Pipelines Using Azure DevOps
Description
Course info
Rating
(61)
Level
Intermediate
Updated
Dec 12, 2019
Duration
1h 28m
Description

Data engineers working with Azure Data Factory can take advantage of Continuous Integration and Continuous Delivery practices to deploy robust and well-tested data pipelines to production.

In this course, Deploying Data Pipelines in Microsoft Azure, you will learn foundational knowledge to apply CI/CD methodologies to your data pipeline creation process.

First, you will learn to create the right environments to fall into the pit of success when creating data pipelines in ADF.

Next, you will discover how to deploy data pipelines using ADF visual tools and ARM templates.

Finally, you will explore how to create a release pipeline in Azure DevOps to automate the deployment process between three distinct environments: development, staging, and production.

When you are finished with this course, you will have the skills and knowledge to apply CI/CD practices to your data pipeline creation process, effortlessly.

About the author
About the author

Marcelo has 18 years' experience architecting solutions and developing software for the web, cloud, and mobile using C#, .NET/Core, Azure, Xamarin, JavaScript and modern front-end frameworks.

More from the author
Integrating Data in Microsoft Azure
Advanced
2h 23m
Jul 17, 2020
Confluence Cloud: Site Administration
Intermediate
1h 38m
May 7, 2020
Section Introduction Transcripts
Section Introduction Transcripts

Course Overview
Hi everyone. My name is Marcelo Pastorino, and welcome to my course, Deploying Data Pipelines in Microsoft Azure. I am a software developer and solutions architect with 20 years of commercial experience designing and developing software, services, and applications that run in the cloud, web, and mobile devices. Continuous integration, delivery, and deployment are a set of practices that allow software developers to continuously deliver value and also a great way to accelerate the feedback loop with customers. Did you know that that data engineers working with Azure Data Factory can take advantage of the same methodologies to deliver robust, well-tested data pipelines to production? In this course, you are going to learn about such practices and how to incorporate them into your Azure Data Factory pipeline creation process. Some of the major topics that we will cover include creating the infrastructure needed to support multiple deployment environments, integrating Azure Data Factory with a source controlled system, deploying data pipelines using Azure Data Factory visual tools and ARM templates, and also deploying data pipelines using a fully automated release pipeline in Azure DevOps. By the end of this course, you will have the skills and knowledge to apply CI and CD practices to your Azure Data Factory pipeline creation process. Before beginning the course, you should be familiar with Azure Data Factory and other Azure services, such as Azure Storage and Azure Key Vault, as well as having some familiarity with Git version control system. I hope you'll join me on this journey to learn with the Deploying Data Pipelines in Microsoft Azure course, right here at Pluralsight.