Modernizing Data Lakes and Data Warehouses with GCP
The two key components of any data pipeline are data lakes and warehouses. This course highlights use-cases for each type of storage and dives into the available data lake and warehouse solutions on Google Cloud Platform in technical detail.
What you'll learn
The two key components of any data pipeline are data lakes and warehouses. This course highlights use-cases for each type of storage and dives into the available data lake and warehouse solutions on Google Cloud Platform in technical detail. Also, this course describes the role of a data engineer, the benefits of a successful data pipeline to business operations, and examines why data engineering should be done in a cloud environment. Learners will get hands-on experience with data lakes and warehouses on Google Cloud Platform using QwikLabs.
Table of contents
- Explore the role of a data engineer 7m
- Analyze data engineering challenges 9m
- Intro to BigQuery 3m
- Data Lakes and Data Warehouses 6m
- Demo:Federated Queries with BigQuery 7m
- Transactional Databases vs Data Warehouses 5m
- Partner effectively with other data teams 6m
- Manage data access and governance 2m
- Demo:Finding PII in your dataset with DLP API 2m
- Build production-ready pipelines 2m
- Review GCP customer case study 4m
- Recap 1m
- Getting Started With GCP And Qwiklabs 4m
- Lab Intro:Using BigQuery to do Analysis 0m
- Lab: Using BigQuery to do Analysis 0m
- Introduction to Data Lakes 11m
- Data Storage and ETL options on GCP 5m
- Building a Data Lake using Cloud Storage 11m
- Demo:Optimizing cost with Google Cloud Storage classes and Cloud Functions 7m
- Securing Cloud Storage 6m
- Storing All Sorts of Data Types 5m
- Demo:Running federated queries on Parquet and ORC files in BigQuery 4m
- Storing Relational Data in the Cloud 1m
- Cloud SQL as a relational Data Lake 8m
- Lab:Loading Taxi Data into Cloud SQL 0m
- Lab: Loading Taxi Data into Google Cloud SQL 0m
- The Modern Data Warehouse 4m
- Intro to BigQuery 1m
- Demo:Querying TB of Data in seconds 7m
- Getting Started 9m
- Loading Data 11m
- Lab Intro:Loading Data into BigQuery 0m
- Lab: Loading data into BigQuery 0m
- Exploring Schemas 0m
- Demo:Exploring Schemas 10m
- Schema Design 3m
- Nested and Repeated Fields 8m
- Demo:Nested and Repeated Fields 16m
- Lab Intro:Working with JSON and Array Data in BigQuery 0m
- Lab: Working with JSON and Array data in BigQuery 0m
- Optimizing with Partitioning and Clustering 5m
- Demo:Creating Partitioned Tables 7m
- Demo:Partitioning and Clustering 6m
- Preview:Transforming Batch and Streaming Data 3m
- Recap 2m