Scaling Python Data Applications with Dask

Learn how to work with very large datasets without leaving familiar and rich Python data ecosystem. This course will teach you how to leverage power of Dask library in order to handle data that is too big for regular tools like Pandas or NumPy.
Course info
Level
Intermediate
Updated
Jun 21, 2019
Duration
1h 21m
Table of contents
Description
Course info
Level
Intermediate
Updated
Jun 21, 2019
Duration
1h 21m
Description

Working with so-called ‘Big Data’ can be a daunting Dask and many tools that solve this problem have a very steep learning curve. Also, developers familiar with Python may not want to resort to solutions built on another technology stack. In this course, Scaling Python Data Applications with Dask, you will gain the ability to work with very large datasets using a Python-native and approachable tool. First, you will learn how to use Dask when your application written using standard Python stops working because of the growing size of the data. Next, you will discover how Dask works underneath and what techniques it uses to make processing large datasets in various scenarios possible and accessible. Finally, you will explore how to exchange Pandas and NumPy for their Big Data variants, with practically no changes to the code. When you’re finished with this course, you will have the skills and knowledge of Dask needed to confidently write data applications that scale, using exclusively Python stack.

About the author
About the author

Paweł is a software engineer passionate about knowledge sharing. He's especially focused on processing and exploring data sets (be it big or small) and is always searching for emerging tools that will make working with data simpler in the future.

More from the author
Advanced Pandas
Intermediate
2h 30m
Jul 12, 2018
Pandas Fundamentals
Beginner
1h 21m
Oct 10, 2017
Section Introduction Transcripts
Section Introduction Transcripts

Course Overview
Hi everyone. My name is Pawel Kordek, and welcome to my course, Scaling Python Data Applications with Dask. I am a big data engineer at Farfetch, where I develop data applications, helping both the business and the customers. Python, along with tools like Pandas and NumPy, is a tool of choice for many data analysts and scientists who value its ease of use, expressiveness, and possibility to quickly iterate on ideas. Unfortunately, most of Python tools do not scale to higher data volumes easily. In this course, we are going to discover the Dask library that helps in scaling of Python applications, especially data-intensive ones, be it on single or multiple machines. Some of the major topics that we will cover include use cases and operation principles, ways to scale basic Python applications, scalable versions of Pandas and NumPy APIs, and ways to scale beyond a single machine. By the end of this course, you will have a solid background and confidence to write 100% Python data processing applications that can keep up with the ever-increasing data volumes. Before beginning this course, you should be familiar with Python and have basic experience with Pandas and NumPy libraries. I hope you'll join me on this journey to learn Dask with the Scaling Python Data Applications with Dask course, at Pluralsight.