Deploying TensorFlow Models to AWS, Azure, and the GCP

This course will help the data scientist or engineer with a great ML model, built in TensorFlow, deploy that model to production locally or on the three major cloud platforms; Azure, AWS, or the GCP.
Course info
Rating
(10)
Level
Intermediate
Updated
Apr 30, 2018
Duration
2h 11m
Table of contents
Course Overview
Using TensorFlow Serving
Containerizing TensorFlow Models Using Docker on Microsoft Azure
Deploying TensorFlow Models on Amazon AWS
Deploying TensorFlow Models on the Google Cloud Platform
Description
Course info
Rating
(10)
Level
Intermediate
Updated
Apr 30, 2018
Duration
2h 11m
Description

Deploying and hosting your trained TensorFlow model locally or on your cloud platform of choice - Azure, AWS or, the GCP, can be challenging. In this course, Deploying TensorFlow Models to AWS, Azure, and the GCP, you will learn how to take your model to production on the platform of your choice. This course starts off by focusing on how you can save the model parameters of a trained model using the Saved Model interface, a universal interface for TensorFlow models. You will then learn how to scale the locally hosted model by packaging all dependencies in a Docker container. You will then get introduced to the AWS SageMaker service, the fully managed ML service offered by Amazon. Finally, you will get to work on deploying your model on the Google Cloud Platform using the Cloud ML Engine. At the end of the course, you will be familiar with how a production-ready TensorFlow model is set up as well as how to build and train your models end to end on your local machine and on the three major cloud platforms. Software required: TensorFlow, Python.

About the author
About the author

A problem solver at heart, Janani has a Masters degree from Stanford and worked for 7+ years at Google. She was one of the original engineers on Google Docs and holds 4 patents for its real-time collaborative editing framework.

More from the author
Analyzing Data with Qlik Sense
Intermediate
2h 11m
Jun 17, 2019
Using PyTorch in the Cloud: PyTorch Playbook
Intermediate
2h 21m
Apr 25, 2019
More courses by Janani Ravi
Section Introduction Transcripts
Section Introduction Transcripts

Course Overview
Hi, my name is Janani Ravi, and welcome to this course on deploying TensorFlow models to AWS, Azure, and the GCP. A little about myself. I have a Masters in electrical engineering from Stanford, and have worked at companies such as MicroSoft, Google and Flipkart. At Google I was one of the first engineers working on the real-time collaborative editing in Google Docs, and I hold four patents for its underlying technologies. I currently work on my own startup, Loonycorn, a studio for high quality video content. This course will help you deploy and host your trained TensorFlow model locally, or on the cloud platform of your choice, Azure, AWS, or the GCP. This course starts off by focusing on how you can save the model parameters of a trained model using the save model interface, a universal interface for all TensorFlow models. Saved models can then be deployed in an on-premise datacenter using the TF model server, which deploys and hosts the model locally. You'll then learn how to scale the locally hosted model, by packaging all dependencies in a Docker container. But you might want to work with TensorFlow on AWS. You'll then get introduced to the AWS SageMaker service, the fully managed ML service offered by Amazon. SageMaker makes it very simple to run distributed training on the cloud, and deploy your model on multiple instances. Or, you might want to work with the GCP. You'll then study how you can deploy your model on the Google cloud platform using the cloud ML engine. Cloud MLE abstracts away the process of distributed training and deployment behind the very simple GCloud command line tools. At the end of this course, you will have learned how a production ready TensorFlow model is set up, and you'll learn to build and train your models end to end, on your local machine, and on the three major cloud platforms.

Using TensorFlow Serving
Hi, and welcome to this course on deploying TensorFlow Models to AWS, Azure, and the GCP. You've built an amazing machine learning model for your used keys, it's now time to deploy and host it somewhere so that it can be used for predictions. This course will show you what options you have and how you can deploy TensorFlow models on the various cloud platforms. This module focuses on TensorFlow serving, which is one way to deploy TensorFlow models to production. This can be set up on any virtual machine even on your local machine. You first build and train your machine learning model anywhere that you want to and then you save it with the SavedModel interface. That TensorFlow models server will then pick up this model from where you've saved it. This model then can then be accessed for predictions on a particular IP address and port. What you'll see in this module is that TF serving is kind of hard to use and hard to set up. TF serving was launched way back in 2015 that's when managed ML services on the cloud were not very well developed. But since then, many platforms such as AWS SageMaker and GCPs Cloud MLE have come to the fore. You'll find that deploying your TensorFlow models using these manage services are far easier than TF serving.

Containerizing TensorFlow Models Using Docker on Microsoft Azure
Hi, and welcome to this module on Containerizing TensorFlow models using Docker on Microsoft Azure. Containerizing our models will allow us to port it to any platform. And it so happens that is you want to bring your own TensorFlow models to Azure, Docker containers is a great way to do this. Microsoft Azure is a cloud platform that is slowly gaining popularity and it offers powerful solutions for ML models both in Infra-as-a-Service solutions and Platform-as-a-Service solutions. It makes sense for you to use these solutions from Microsoft if what you're looking for is end-to-end development on Azure. However, if you've built your TensorFlow model elsewhere, and you want to bring it to Azure, an excellent way to do it would be using Docker containers and that's what we'll see in this module. In this environment where portability is extremely important, an easy way to lug around your custom code and packages is by using containers. Containers are lightweight units of deployment. It's increasingly seeming like organizations would prefer not to put all their eggs in one basket and stick to one cloud platform which means hybrid multi-cloud use cases are becoming more popular and this explains why Docker containers are slowly gaining popularity.

Deploying TensorFlow Models on Amazon AWS
Welcome to this model where we'll deploy our TensorFlow churn prediction model on AWS SageMaker. You'll see how much easier it is to build and train machine learning models when you use a cloud platform such as SageMaker. SageMaker is a fully managed machine learning service offered by AWS. It allows you to build, train and deploy machine learning models on the AWS cloud. AWS offers supports for more than just TensorFlow models. We focus our attention on TensorFlow in this model. SageMaker offers an integrated Jupyter notebook instance which you can use to prototype and develop your models. This is also great for data exploration. When you use SageMaker, you see the power of using a cloud platform-managed service. You'll see how easy it is to run distributed training for your TensorFlow models on the cloud.

Deploying TensorFlow Models on the Google Cloud Platform
Hi and welcome to this module where we'll see how we can deploy our TensorFlow model on the Google Cloud Platform using Google's fully managed service, Cloud MLE, or ML Engine. TensorFlow was originally open source by Google engineers, so it makes sense that Google Cloud Platform has great support for distributed training and serving of TensorFlow modelings. This it does using Google Cloud ML Engine, which is a managed service for both, training and prediction. Cloud MLE offers two specific services. The first is the ML Engine training service, which is responsible for distributed training of your model. The training service splits the training process across multiple machines. Which means, training that originally took days, can now be done in a few hours. Once you've deployed your fully trained model and have it hosted on the GCP, the ML Engine prediction service supports either batch or online prediction. Cloud ML Engine on the GCP has been specifically built to support TensorFlow models. It doesn't support other deep learning frameworks such as Apache MXNet. If you already have your training dataset on the GCP or you plan to use other GCP services such as BigQuery, Dataflow, et cetera, then using Cloud ML Engine to build and train your model makes total sense. ML Engine is great if you want to integrate with other GCP services.