Microsoft Exam DP-203 Data Engineering on Microsoft Azure

Paths

Microsoft Exam DP-203 Data Engineering on Microsoft Azure

Authors: Axel Sirota, Bismark Adomako, Thomas LeBlanc, Ifedayo Bamikole, Reza Salehi, Tapan Ghatalia, Niraj Joshi, Ana Voicu, Emilio Melo

Microsoft Azure offers a spread of services dedicated to addressing common business data engineering problems. This skill teaches how these Azure services work together to enable... Read more

Design and Develop Data Processing

Design and Implement Data Security

Monitor and Optimize Data Storage and Data Processing

Pre-requisites

This path is intended for learners familiar with Data Engineering Principles, Workloads and Best Practices.

Microsoft Exam DP-900 (Azure Data Fundamentals).

Beginner

The courses in this section address design elements for data engineering. For example, this section of the skill teaches how to design effective storage solutions, partitioning, security and serving layer principles for Microsoft Azure.

Design Principles for Effective Storage Solutions

by Axel Sirota

May 10, 2021 / 1h 1m

1h 1m

Start Course
Description

It is crucial to have you data available at all times. In this course, Design Principles for Effective Storage Solution, you’ll learn to choose and design a storage strategy to ensure your data is always available, resilient, and useful. First, you’ll explore how to choose a data storage solution. Next, you’ll discover Azure storage and Azure Data Lake Storage V2. Finally, you’ll learn Azure CosmosDB and Azure Synapse Analytics as storage solutions. When you’re finished with this course, you’ll have the skills and knowledge of storage in Azure needed to design an effective storage solution.

Table of contents
  1. Course Overview
  2. Understanding Key Concepts for Storage
  3. Design Effective File Storage Solution with Microsoft Azure
  4. Design Effective SQL and NoSQL Solutions with Microsoft Azure

Design Principles for Partitioning with Azure

by Axel Sirota

May 7, 2021 / 52m

52m

Start Course
Description

Partitioning is one of the key solutions to scale a data platform both in throughput and availability. In this course, Design Principles for Partitioning with Azure, you’ll learn to architecture partition strategies for each Azure Data Service. First, you’ll explore what partition is and when it is useful. Next, you’ll discover how to design partitioning for performance. Finally, you’ll learn how to implement partitioning and distribution in each Azure service. When you’re finished with this course, you’ll have the skills and knowledge of partition strategies needed to scale your organization’s data platform effortlessly.

Table of contents
  1. Course Overview
  2. Understanding Key Concepts for Designing a Partition Strategy
  3. Implement Partition Design Strategy for Effective Performance
  4. Evaluate Partition Strategy with Microsoft Azure

Design Principles for Serving Layer in Microsoft Azure

by Bismark Adomako

Jul 22, 2021 / 48m

48m

Start Course
Description

As a data professional, knowing how data is served to you helps when developing for or at the serving layer. In this course, Design Principles for Serving Layer in Microsoft Azure, you’ll learn to design and optimize analytical workloads. First, you’ll explore the analytical data stores in Azure. Next, you’ll discover key selection criteria for analytical data stores. Finally, you’ll learn how to design metastores in Azure Synapse Analytics and Azure Databricks. When you’re finished with this course, you’ll have the skills and knowledge of a data professional needed to design your presentation layer.

Table of contents
  1. Course Overview
  2. Evaluating Analytical Data Stores in Azure
  3. Design and Optimize Analytical Workloads in Azure

Design Principles for Implementing Batch Processing Solutions

by Axel Sirota

May 13, 2021 / 1h 16m

1h 16m

Start Course
Description

Batch processing is key to big Data. but such requests may take several hours and may have many different challenges to be implemented correctly. In this course, Design Principles for Implementing Batch Processing Solution, you’ll learn to design optimal batch processing solutions in Azure. First, you’ll explore what batch processing really is.. Next, you’ll discover how Azure offers services for each part of a batch architecture. Finally, you’ll learn how to implement these pipelines in Azure Data Factory. When you’re finished with this course, you’ll have the skills and knowledge of batch processing in Azure needed to to design effective and optimal batch processing solutions in Azure for your own needs

Table of contents
  1. Course Overview
  2. Understanding Key Concepts for Batch Processing
  3. Build a Batch Processing Solution with Microsoft Azure
  4. Implementing Batch Processing with Azure Data Factory

Design Principles for Implementing Stream Processing Solution

by Thomas LeBlanc

Jul 21, 2021 / 33m

33m

Start Course
Description

This course will look at consuming a stream of data in Azure. In this course, Design Principles for Implementing Stream Processing Solutions, you will learn design principles for Azure services that ingest streaming data. First, you will explore the Azure services to connect to stream processing like Event Hub. Next, you will discover how to ingest streaming data with Data Factory and Databricks. Finally, you will learn the design and test methodologies for cleansing the stream data that loads into analytics engines like Azure Stream Analytics. This will help with principles for machine and deep learning models. When you’re finished with this course, you’ll have the skills and knowledge of streaming design principles needed to create and manage a streaming solution.

Table of contents
  1. Course Overview
  2. Stream Processing and Storage
  3. Stream Consistency and Analytics

Design Principles for Implementing Data Security with Microsoft Azure

by Bismark Adomako

May 26, 2021 / 1h 3m

1h 3m

Start Course
Description

There's no easy approach for security and no solution that solves all your problems from a security perspective. In this course, Design Principles for Implementing Data Security with Microsoft Azure, you’ll learn to consider key architectural security when designing for the cloud. First, you’ll explore cloud defense in depth. Next, you’ll discover identity management and protecting your infrastructure on Microsoft Azure. Finally, you’ll learn how to secure your Azure data storage and encrypt your data. When you’re finished with this course, you’ll have the skills and knowledge of data and security engineering needed to secure your cloud architecture.

Table of contents
  1. Course Overview
  2. Defense in Depth
  3. Identity Management
  4. Data Security and Encryption

Intermediate

The courses in this section address implementation of security and data auditing for creating balanced workloads.

Implementing Data Storage with Azure Data Lake

by Bismark Adomako

Jul 22, 2021 / 59m

59m

Start Course
Description

Globomantics Corporation is interested in using Azure to analyze all of their business data. In this course, Implementing Data Storage with Azure Data Lake, you’ll learn to design, build, and monitor an Azure Data Lake solution. First, you’ll explore Azure Data Lake Storage. Next, you’ll discover operationalizing data movement and preparation with Azure Data Factory. Finally, you’ll learn how to monitor and troubleshoot the implemented solution. When you’re finished with this course, you’ll have the skills and knowledge of a data engineer needed to design and implement a data storage solution with Azure Data Lake.

Table of contents
  1. Course Overview
  2. Introduction to Azure Data Lake Storage
  3. Operationalize Azure Data Factory
  4. Monitoring Azure Data Lake Storage

Implement Security on Azure Synapse

by Ifedayo Bamikole

Jun 7, 2021 / 39m

39m

Start Course
Description

Security is a large portion when planning building data strategy. In this course, Implementing Security in Azure Synapse, you’ll learn the different options available to secure your data in your Azure environment using Azure Synapse. First, you’ll explore how to implement custom keys. Next, you’ll discover how to mask your data with data masking features. Finally, you’ll learn how to implement row level and column level security. When you’re finished with this course, you’ll have the skills and knowledge of Implementing Security in Azure Synapse which will help create a well protected Data Warehouse environment.

Table of contents
  1. Course Overview
  2. Encrypt Data at Rest and in Motion
  3. Encryption with a Custom Key
  4. Implement Data Masking
  5. Implement Row-level and Column-level Security

Implement Security on Azure Data Lakes

by Reza Salehi

Apr 27, 2021 / 42m

42m

Start Course
Description

Keeping the data secure is vital when using a public cloud platform. In this course, Implement Security on Azure Data Lakes, you’ll learn to secure your Azure Data Lake Gen2 storage instance. First, you’ll explore setting up data retention policies to delete sensitive data after they expire. Next, you’ll discover implementing RBAC for Azure Data Lake . Finally, you’ll learn how to use POSIX-like ACLs to define permissions at the folder and file level. When you’re finished with this course, you’ll have the skills and knowledge of Azure Data Lake Gen2 security needed to prepare for Microsoft exam DP-203.

Table of contents
  1. Course Overview
  2. Azure Data Lake Gen2 Data Retention
  3. Azure Data Lake Gen2 Authorization

Implement Data Auditing with Azure Data Lake

by Tapan Ghatalia

Apr 15, 2021 / 34m

34m

Start Course
Description

Sensitive data masking should be of prime importance for efficient data governance models to be followed. In this course, Implement Data Auditing with Azure Data Lake, you’ll learn to apply efficient encryption, data masking, and secure endpoints over Azure Cloud Portal for specific services. First, you’ll explore how to design security for source data access. Next, you’ll discover designing security for data policies and standards. Finally, you’ll learn how to authenticate the service principle to mount data lake on Azure DataBricks. When you’re finished with this course, you’ll have the skills and knowledge of the importance of data masking, efficient encryption policies, management of key Vault, and policy-driven data lake architecture.

Table of contents
  1. Course Overview
  2. Implementing Data Masking
  3. Implementing Data Encryption and Authorization
  4. Designing a Solution with Secure Endpoints

Advanced

The final section of this skill covers monitoring your Microsoft Azure data environment performance. Specific attention is given to monitoring databases, monitoring data pipelines, configuring alerts, and optimizing your solutions.

Implement Partitioning with Azure

by Niraj Joshi

Apr 21, 2021 / 28m

28m

Start Course
Description

Partitioning is an important strategy to segregate the data based on the partition key and distribute the data evenly across partitions for efficient querying and analysis.

In this course, Implement Partitioning with Azure, you’ll learn to apply efficient partitioning, sharding, and data distribution techniques over Azure Cloud Portal for specific services:

  1. explore how to design non-relational data stores
  2. discover designing relational data stores
  3. learn how to implement partitioning and sharding strategies over Azure Synapse
When you’re finished with this course, you’ll have the skills and knowledge of data distribution strategies across multiple cloud stores and even implement the global distribution of the data to increase throughput, the performance of the data stores, and availability of the dataset using the Azure Cloud Portal.

Table of contents
  1. Course Overview
  2. Partitioning in Non-relational Cloud Data Stores
  3. Partitioning in Relational Cloud Data Stores
  4. Partitioning in Azure Synapse

Create and Monitor Data Pipelines for a Batch Processing Solution

by Bismark Adomako

Jun 11, 2021 / 1h 29m

1h 29m

Start Course
Description

As a data specialist, you may be required to design and implement an end-to-end data pipeline for a batch processing solution. In this course, Create and Monitor Data Pipelines for a Batch Processing Solution, you’ll learn to design and implement data pipelines for a batch processing solution. First, you’ll explore the available data storages on Azure. Next, you’ll discover and develop batch processing solutions using Azure Data Factory and the available data storages. Finally, you’ll learn how to automate the data processing process and how to monitor for optimization and efficiency. When you’re finished with this course, you’ll have the skills and knowledge of a data professional needed to build and monitor end-to-end data pipelines.

Table of contents
  1. Course Overview
  2. Working with Data Storage
  3. Creating and Orchestrating Data Movement
  4. Design and Implement the Serving Layer
  5. Monitoring Data Storage and Processing

Create and Monitor Data Pipelines for a Stream Processing Solution

by Ana Voicu

Jun 9, 2021 / 1h 6m

1h 6m

Start Course
Description

Azure gives you all of the resources you need to create and monitor data pipelines and find the data processing solutions you need. In this course, Create and Monitor Data Pipelines for a Stream Processing Solution, you'll learn how to work with stream processing pipelines in Azure in order to create, configure, monitor, and troubleshoot Azure Stream Analytics jobs. First, you'll discover the components of a stream analytics job and why each of them are important. Next, you'll explore how to handle the data coming from the pipeline (insert and update), how to visualize the data in different time windows, and how to troubleshoot such a job. Finally, you'll learn how to improve the performance of your job, by choosing the number of streaming units, parallelizing the query, and using repartitioning to scale your query. By the end of this course, you'll have a better understanding of what it means to create and monitor real-time data processing solutions in Azure.

Table of contents
  1. Course Overview
  2. Creating a Stream Processing Solution with Azure Technology
  3. Optimizing Your Stream Analytics Job

Optimize Data Pipelines with Azure Services

by Emilio Melo

Jun 1, 2021 / 50m

50m

Start Course
Description

Nowadays data is everywhere, and companies are increasingly interested in transforming this data on insights that can help them become more efficient and productive. In this course, Optimize Data Pipelines with Azure Services, you’ll learn to manage and optimize pipelines on Azure. First, you’ll explore how to manage pipelines. Next, you’ll discover how and when to execute them through triggers. Finally, you’ll learn how to perform error-handling and retries for your pipelines. When you’re finished with this course, you’ll have the skills and knowledge of data pipelines on Azure Services needed for the exam DP-203.

Table of contents
  1. Course Overview
  2. Managing Azure Data Pipelines
  3. Pipeline Scheduling and Error-handling