Designing Scalable Data Architectures on the Google Cloud
This course focuses on the end-to-end design of a cloud architecture, specifically from the perspective of optimizing that architecture for Big Data processing, real-time analytics, and real-time prediction using ML and AI.
What you'll learn
The Google Cloud Platform offers up a very large number of services, for every important aspect of public cloud computing. This array of services and choices can often seem intimidating - even a practitioner who understands several important services might have trouble connecting the dots, as it were, and fitting together those services in meaningful ways. In this course, Designing Scalable Data Architectures on the Google Cloud, you will gain the ability to design lambda and kappa architectures that integrate batch and streaming, plan intelligent migration and disaster-recovery strategies, and pick the right ML workflow for your enterprise. First, you will learn why the right choice of stream processing architecture is becoming key to the entire design of a cloud-based infrastructure. Next, you will discover how the Transfer Service is an invaluable tool in planning both migration and disaster-recovery strategies on the GCP. Finally, you will explore how to pick the right Machine Learning technology for your specific use. When you’re finished with this course, you will have the skills and knowledge of the entire cross-section of Big Data and Machine Learning offerings on the GCP to build cloud architectures that are optimized for scalability, real-time processing, and the appropriate use of Deep Learning and AI technologies.
Table of contents
- Module Overview 2m
- Prerequisites and Course Outline 3m
- Changing Architectural Considerations in the Real World 4m
- Discussion: Using and Building ML Models, Querying Data 4m
- Discussion: Training Serving Skew, a Multi Cloud World, Lambda, and Kappa Architectures 5m
- Lambda and Kappa Architectures 5m
- Setting up a GCS Bucket, PubSub Topic, and BigQuery Table 6m
- Implementing Integrated Processing for Batch and Streaming 6m
- Executing the Pipeline for Batch and Stream Processing 5m
- Module Overview 1m
- Migrating Structured Data 4m
- Storage Options on the GCP 3m
- Migrating Unstructured Data 3m
- Transfer Data from a Regional to a Multi Regional Bucket 5m
- Creating and Uploading Data in an S3 Bucket 4m
- Transferring Data from S3 to GCS Buckets 4m
- Transferring Data from an Externally Accessible URL 4m
- Object Lifecycle Management 5m
- Planning for Disaster 2m
- Module Summary 2m
- Module Overview 1m
- Google Cloud AI Products: Making ML Accessible to All 5m
- Cloud ML Engine for Building and Training Custom Machine Learning Models 3m
- Introducing Dataprep Flows and Recipes to Wrangle Data 9m
- Cleaning and Formatting Data 6m
- Preparing Data for Machine Learning 5m
- Building and Serializing a Linear Regression Model Using Scikit Learn 3m
- Deploying a Model and Using Cloud ML Engine for Prediction 4m
- Summary and Further Study 2m
About the author
Janani has a Masters degree from Stanford and worked for 7+ years at Google. She was one of the original engineers on Google Docs and holds 4 patents for its real-time collaborative editing framework. After spending years working in tech in the Bay Area, New York, and Singapore at companies such as Microsoft, Google, and Flipkart, Janani finally decided to combine her love for technology with her passion for teaching. She is now the co-founder of Loonycorn, a content studio focused on providing ... more