Getting Started with NLP Deep Learning Using PyTorch and fastai

Course Overview
Hi everyone. My name is Gianni Rosa Gallina, and welcome to my course, Getting Started with NLP Deep Learning Using PyTorch and fastai. I am an R&D specialist and senior software engineer in the Innovation Lab of Deltatre, based in Italy. In this course, we are going to have a look at the amazing fastai library built on top of PyTorch deep learning framework to learn how to perform natural language processing with deep learning neural networks, how to achieve some of the most recent state-of-the-art results in this field. Some of the major topics that we will cover include data preparation for deep learning NLP, from raw text to embeddings, learn how to leverage transfer learning in NLP for classification tasks, understanding and building a custom language model, going from prototype to production once we are satisfied with our models. By the end of this course, you'll know why fastai and PyTorch are great frameworks, how it's quick and easy to train deep learning models for natural language processing tasks on your own dataset and bring them to production. Before beginning the course, you should be familiar with general machine learning and deep learning concepts, some basic Python programming and dev tools, PyTorch basics, and Microsoft Azure Machine Learning services. I hope you will join me on this journey to learn some NLP deep learning concepts with the Getting Started with NLP Deep Learning Using PyTorch and fastai course, at Pluralsight.

Using Deep Learning for NLP
Natural language processing is one of the most important fields of study and research and has seen a phenomenal rise in interest in the past years. With traditional machine learning techniques, the basics of NLP are widely known and easy to grasp. But things start to get tricky when the text data becomes huge and unstructured. That's where deep learning comes in. In the last years, deep learning has proven its usefulness in computer vision tasks like image classification, object detection, and segmentation. With NLP applications such as text classification, sentiment analysis, translation, or text generation, until recently, deep learning was not producing interesting results compared to traditional techniques. Applying deep learning to text is relatively new with the first amazing state-of-the-art results that in back in late 2017, and some breakthrough discoveries and new techniques have been introduced in 2018, marking the ImageNet moment from NLP as defined by NLP's researcher, Sebastian Ruder. Research is ongoing and very fast paced, continuously producing new state-of-the-art results almost every month. This is a very interesting moment in time to start exploring, studying, and applying machine learning to text or data. No way to get bored. In this module, we'll have a quick look at the theory behind the deep learning and on preprocessing techniques we learned and applied in module 4, very easily, thanks to fastai library. The aim of this module is just giving some easy-to-understand contextual information, not teaching you the internals and underlying math concepts. So, here, I expect you to know already some theory about machine learning, neural networks, and deep neural networks. If you're interested in much details, I'll give you some references in the last module of the course.

Building a Custom Language Model from Scratch
In module 4, we learned how to build a text classifier from scratch, leveraging transfer learning. One of the key components of that process was a pre-trained language model. In the specific, we used an English model trained on Wikitext-103. But what if we want to build a classifier in another language which has no pre-trained models available? This module will explain all the basic steps to train one from scratch, and we'll use Italian, my language, as example, but you can use the very same steps to build for your own. As we'll see, we already know everything to do it. It's just a matter of a few scripts and some processing time. From a command prompt or PowerShell, let's launch Jupyter Notebook, but first, remember to enable the virtual environment we created in the previous steps. Leave the shell open, as we'll need it in a moment. In the browser, open the provided notebook, Train a language model from scratch. You can find it in the course material for module 7.