Building Deep Learning Models on Databricks
In this course, you will learn to train neural network models using TensorFlow and PyTorch, perform distributed training using the Horovod framework, and perform hyperparameter tuning using Hyperopt.
What you'll learn
The Databricks Data Lakehouse platform offers a managed environment to train and compare your deep learning models, perform hyperparameter tuning, and productionize and serve your models.
In this course, Building Deep Learning Models on Databricks, you will learn to use Bamboolib for no-code data analysis and transformations.
First, you will build deep learning models using TensorFlow 2.0 and Keras, and will create a workspace experiment to manage your runs and use autologging to track model parameters, metrics, and artifacts.
Next, you will compare multiple runs to find the best-performing model using the MLflow UI.
Then, you will see that in order to have support for autologging in MLflow you need to use the PyTorch Lightning framework to design and train your model. You will also register your model with the model registry and use it for batch inference, deploy a Classic MLflow endpoint to serve model predictions, and use the Horovod framework for distributed training of your model.
Finally, you will learn how you can use the Hyperopt tool for hyperparameter tuning of your deep learning models, and will run hyperparameter tuning in a distributed fashion on a Spark cluster using the SparkTrials class.
When you are finished with this course, you will have the skills and knowledge to build and train deep machine learning models on Databricks using MLflow to manage your machine learning workflow.
Table of contents
- Quick Overview of TensorFlow and Keras 4m
- Demo: Setting up the Databricks Environment 6m
- Demo: Exploring and Preprocessing Data 5m
- Demo: Training a Regression Model Using TensorFlow 3m
- Demo: Autologging Model Parameters 8m
- Demo: Comparing Multiple Model Runs 6m
- Demo: Using Bamboolib for Loading and Exploring Data 5m
- Demo: Data Cleaning Using Bamboolib - I 4m
- Demo: Data Cleaning Using Bamboolib - II 4m
- Demo: Preprocessing Data for Deep Learning 2m
- Demo: Training Multiple Models Using Multiple Runs 5m
- Demo: Registering Models and Specifying Stages 3m
- Demo: Creating a Delta Table for Batch Inference 3m
- Demo: Using Model for Batch Inference 4m
- A Quick Overview of PyTorch 4m
- Demo: Preprocessing Data for Classification 4m
- Demo: Training a PyTorch Model Using MLflow Tracking 8m
- Demo: Use Classic Serving to Serve Model Predictions 7m
- Demo: Loading Image Data for Classification 3m
- Demo: Training a PyTorch Image Classification Model 5m
- Horovod for Distributed Training 3m
- Demo: Setting up Functions for Single Node Training 7m
- Demo: Performing Distributed Training Using the Horovod Runner 6m