Getting Started with NLP Deep Learning Using PyTorch and fastai

This course will teach you how to start using fastai library and PyTorch to obtain near-state-of-the-art results with Deep Learning NLP for text classification. It will give you a theoretical background and show how to take models to production.
Course info
Level
Intermediate
Updated
Mar 21, 2019
Duration
2h 13m
Table of contents
Course Overview
Exploring the fastai Library
Setting up a Development Environment
Building a Text/Topic Classifier with Transfer Learning
Using Deep Learning for NLP
Going from Prototype to Production
Building a Custom Language Model from Scratch
Recapping and Next Steps
Description
Course info
Level
Intermediate
Updated
Mar 21, 2019
Duration
2h 13m
Description

In this course, Getting Started with NLP Deep Learning Using PyTorch and fastai, we'll have a look at the amazing fastai library, built on top of the PyTorch Deep Learning Framework, to learn how to perform Natural Language Processing (NLP) with Deep Neural Networks, and how to achieve some of the most recent state-of-the-art results in text classification. First, we’ll learn how to train a model for text classification very quickly, thanks to the fastai library and transfer learning. Next, we'll explore some of the theory behind Deep Learning NLP techniques, and how to deploy our models to production in Microsoft Azure. Finally, we’ll discover how to train a custom language model from scratch. When you’re finished with this course, you’ll know why fastai and PyTorch are great frameworks, how to train deep learning models for NLP tasks on your own datasets, and how to bring them to production.

About the author
About the author

Gianni is an R&D Senior Software Engineer in Deltatre's Innovation Lab, based in Italy. A Microsoft MVP since 2011, he has been focused on emerging technologies, AI, and Virtual/Augmented/Mixed Reality since 2013.

More from the author
Section Introduction Transcripts
Section Introduction Transcripts

Course Overview
Hi everyone. My name is Gianni Rosa Gallina, and welcome to my course, Getting Started with NLP Deep Learning Using PyTorch and fastai. I am an R&D specialist and senior software engineer in the Innovation Lab of Deltatre, based in Italy. In this course, we are going to have a look at the amazing fastai library built on top of PyTorch deep learning framework to learn how to perform natural language processing with deep learning neural networks, how to achieve some of the most recent state-of-the-art results in this field. Some of the major topics that we will cover include data preparation for deep learning NLP, from raw text to embeddings, learn how to leverage transfer learning in NLP for classification tasks, understanding and building a custom language model, going from prototype to production once we are satisfied with our models. By the end of this course, you'll know why fastai and PyTorch are great frameworks, how it's quick and easy to train deep learning models for natural language processing tasks on your own dataset and bring them to production. Before beginning the course, you should be familiar with general machine learning and deep learning concepts, some basic Python programming and dev tools, PyTorch basics, and Microsoft Azure Machine Learning services. I hope you will join me on this journey to learn some NLP deep learning concepts with the Getting Started with NLP Deep Learning Using PyTorch and fastai course, at Pluralsight.

Using Deep Learning for NLP
Natural language processing is one of the most important fields of study and research and has seen a phenomenal rise in interest in the past years. With traditional machine learning techniques, the basics of NLP are widely known and easy to grasp. But things start to get tricky when the text data becomes huge and unstructured. That's where deep learning comes in. In the last years, deep learning has proven its usefulness in computer vision tasks like image classification, object detection, and segmentation. With NLP applications such as text classification, sentiment analysis, translation, or text generation, until recently, deep learning was not producing interesting results compared to traditional techniques. Applying deep learning to text is relatively new with the first amazing state-of-the-art results that in back in late 2017, and some breakthrough discoveries and new techniques have been introduced in 2018, marking the ImageNet moment from NLP as defined by NLP's researcher, Sebastian Ruder. Research is ongoing and very fast paced, continuously producing new state-of-the-art results almost every month. This is a very interesting moment in time to start exploring, studying, and applying machine learning to text or data. No way to get bored. In this module, we'll have a quick look at the theory behind the deep learning and on preprocessing techniques we learned and applied in module 4, very easily, thanks to fastai library. The aim of this module is just giving some easy-to-understand contextual information, not teaching you the internals and underlying math concepts. So, here, I expect you to know already some theory about machine learning, neural networks, and deep neural networks. If you're interested in much details, I'll give you some references in the last module of the course.

Building a Custom Language Model from Scratch
In module 4, we learned how to build a text classifier from scratch, leveraging transfer learning. One of the key components of that process was a pre-trained language model. In the specific, we used an English model trained on Wikitext-103. But what if we want to build a classifier in another language which has no pre-trained models available? This module will explain all the basic steps to train one from scratch, and we'll use Italian, my language, as example, but you can use the very same steps to build for your own. As we'll see, we already know everything to do it. It's just a matter of a few scripts and some processing time. From a command prompt or PowerShell, let's launch Jupyter Notebook, but first, remember to enable the virtual environment we created in the previous steps. Leave the shell open, as we'll need it in a moment. In the browser, open the provided notebook, Train a language model from scratch. You can find it in the course material for module 7.