Designing Scalable Data Architectures on the Google Cloud

This course focuses on the end-to-end design of a cloud architecture, specifically from the perspective of optimizing that architecture for Big Data processing, real-time analytics, and real-time prediction using ML and AI.
Course info
Level
Advanced
Updated
Jan 16, 2019
Duration
1h 59m
Table of contents
Course Overview
Implementing Integrated Batch and Streaming Architectures on the GCP
Designing Migration and Disaster Recovery Strategies on the GCP
Designing Robust ML Workflows on the GCP
Description
Course info
Level
Advanced
Updated
Jan 16, 2019
Duration
1h 59m
Description

The Google Cloud Platform offers up a very large number of services, for every important aspect of public cloud computing. This array of services and choices can often seem intimidating - even a practitioner who understands several important services might have trouble connecting the dots, as it were, and fitting together those services in meaningful ways. In this course, Designing Scalable Data Architectures on the Google Cloud, you will gain the ability to design lambda and kappa architectures that integrate batch and streaming, plan intelligent migration and disaster-recovery strategies, and pick the right ML workflow for your enterprise. First, you will learn why the right choice of stream processing architecture is becoming key to the entire design of a cloud-based infrastructure. Next, you will discover how the Transfer Service is an invaluable tool in planning both migration and disaster-recovery strategies on the GCP. Finally, you will explore how to pick the right Machine Learning technology for your specific use. When you’re finished with this course, you will have the skills and knowledge of the entire cross-section of Big Data and Machine Learning offerings on the GCP to build cloud architectures that are optimized for scalability, real-time processing, and the appropriate use of Deep Learning and AI technologies.

About the author
About the author

A problem solver at heart, Janani has a Masters degree from Stanford and worked for 7+ years at Google. She was one of the original engineers on Google Docs and holds 4 patents for its real-time collaborative editing framework.

More from the author
Scraping Your First Web Page with Python
Beginner
2h 39m
Nov 5, 2019
More courses by Janani Ravi
Section Introduction Transcripts
Section Introduction Transcripts

Course Overview
Hi! My name is Janani Ravi, and welcome to this course on Designing Scalable Data Architectures on the Google Cloud. A little about myself. I have a master's degree in electrical engineering from Stanford and have worked at companies such as Microsoft, Google, and Flipkart. At Google, I was one of the first engineers working on real-time collaborative editing in Google Docs, and I hold four patents for its underlying technologies. I currently work on my own startup, Loonycorn, a studio for high-quality video content. The GCP offers up a very large number of services for every important aspect of public cloud computing. This array of services and choices can often seem intimidating. Even a practitioner who understands several important services might have trouble connecting the dots, as it were, and fitting together those services in meaningful ways. At such terms, a need is felt for end-to-end integrated examples which span different services and focus on how they all fit together into the big picture. In this course, you will gain the ability to design Lambda and Kappa Architectures that integrate batch and streaming, plan intelligent migration and disaster recovery strategies, and pick the right machine learning workflow for your enterprise. First, you will learn why the right choice of stream processing architecture is becoming key to the entire design of a cloud-based infrastructure. This has to do with the need for real-time data processing and a specific issue in machine learning modeling known as training-serving skew. Increasingly, organizations are confronted by ML models that work well in the lab but perform poorly in real-time prediction. This is often because real-time data is not processed as accurately and cleanly as batch data. You will learn two specific architectural patterns, Lambda and Kappa Architectures, that seek to mitigate this. Next, you will discover how the Transfer Service is an invaluable tool in planning both migration and disaster recovery strategies on the GCP. Currently, the GCP does not offer an equivalent of the AWS Data Migration Service, so when you migrate to the GCP, it is rational to first move all your structured and unstructured data to cloud storage rather than committing upfront to a specific structured storage solution. The Transfer Service is great for this specific use case. Finally, you will explore how to pick the right machine learning technology for your specific use. You can leverage pre-trained ML APIs if you don't need to build a custom model at all. Or you could leverage auto-ML to implement transfer learning, which retrains a sophisticated model to your specific data. You can even build ML models in SQL using BigQuery ML. When you're finished with this course, you will have the skills and knowledge of the entire cross-section of big data and machine learning offerings on the GCP to build cloud architectures that are optimized for scalability, real-time processing, and the appropriate use of deep learning and AI technologies.