Modernizing Data Lakes and Data Warehouses with GCP
By Google Cloud
The two key components of any data pipeline are data lakes and warehouses. This course highlights use-cases for each type of storage and dives into the available data lake and warehouse solutions on Google Cloud Platform in technical detail.
Course info
Level
Intermediate
Updated
Jan 26, 2021
Duration
3h 34m
Table of contents
Introduction
Introduction to Data Engineering
Explore the role of a data engineer
7m
Analyze data engineering challenges
9m
Intro to BigQuery
3m
Data Lakes and Data Warehouses
6m
Demo:Federated Queries with BigQuery
7m
Transactional Databases vs Data Warehouses
5m
Partner effectively with other data teams
6m
Manage data access and governance
2m
Demo:Finding PII in your dataset with DLP API
2m
Build production-ready pipelines
2m
Review GCP customer case study
4m
Recap
1m
Getting Started With GCP And Qwiklabs
4m
Lab Intro:Using BigQuery to do Analysis
0m
Lab: Using BigQuery to do Analysis
0m
Building a Data Lake
Introduction to Data Lakes
11m
Data Storage and ETL options on GCP
5m
Building a Data Lake using Cloud Storage
11m
Demo:Optimizing cost with Google Cloud Storage classes and Cloud Functions
7m
Securing Cloud Storage
6m
Storing All Sorts of Data Types
5m
Demo:Running federated queries on Parquet and ORC files in BigQuery
4m
Storing Relational Data in the Cloud
1m
Cloud SQL as a relational Data Lake
8m
Lab:Loading Taxi Data into Cloud SQL
0m
Lab: Loading Taxi Data into Google Cloud SQL
0m
Building a data warehouse
The Modern Data Warehouse
4m
Intro to BigQuery
1m
Demo:Querying TB of Data in seconds
7m
Getting Started
9m
Loading Data
11m
Lab Intro:Loading Data into BigQuery
0m
Lab: Loading data into BigQuery
0m
Exploring Schemas
0m
Demo:Exploring Schemas
10m
Schema Design
3m
Nested and Repeated Fields
8m
Demo:Nested and Repeated Fields
16m
Lab Intro:Working with JSON and Array Data in BigQuery
0m
Lab: Working with JSON and Array data in BigQuery
0m
Optimizing with Partitioning and Clustering
5m
Demo:Creating Partitioned Tables
7m
Demo:Partitioning and Clustering
6m
Preview:Transforming Batch and Streaming Data
3m
Recap
2m
Summary
Description
Course info
Level
Intermediate
Updated
Jan 26, 2021
Duration
3h 34m
Description
The two key components of any data pipeline are data lakes and warehouses. This course highlights use-cases for each type of storage and dives into the available data lake and warehouse solutions on Google Cloud Platform in technical detail. Also, this course describes the role of a data engineer, the benefits of a successful data pipeline to business operations, and examines why data engineering should be done in a cloud environment. Learners will get hands-on experience with data lakes and warehouses on Google Cloud Platform using QwikLabs.
About the author
About the author
Build, innovate, and scale with Google Cloud Platform.
More from the author
More courses by Google Cloud