Expanded

Scaling Python Data Applications with Dask

Learn how to work with very large datasets without leaving familiar and rich Python data ecosystem. This course will teach you how to leverage power of Dask library in order to handle data that is too big for regular tools like Pandas or NumPy.
Course info
Rating
(19)
Level
Intermediate
Updated
Jun 21, 2019
Duration
1h 21m
Table of contents
Description
Course info
Rating
(19)
Level
Intermediate
Updated
Jun 21, 2019
Duration
1h 21m
Description

Working with so-called ‘Big Data’ can be a daunting task and many tools that solve this problem have a very steep learning curve. Also, developers familiar with Python may not want to resort to solutions built on another technology stack. In this course, Scaling Python Data Applications with Dask, you will gain the ability to work with very large datasets using a Python-native and approachable tool. First, you will learn how to use Dask when your application written using standard Python stops working because of the growing size of the data. Next, you will discover how Dask works underneath and what techniques it uses to make processing large datasets in various scenarios possible and accessible. Finally, you will explore how to exchange Pandas and NumPy for their Big Data variants, with practically no changes to the code. When you’re finished with this course, you will have the skills and knowledge of Dask needed to confidently write data applications that scale, using exclusively Python stack.

About the author
About the author

Paweł is a software engineer passionate about knowledge sharing. He's especially focused on processing and exploring data sets (be it big or small) and is always searching for emerging tools that will make working with data simpler in the future.

More from the author
Deploying a Kafka Cluster
Beginner
2h 30m
Jun 10, 2020
Advanced Pandas
Intermediate
2h 30m
Jul 12, 2018
Section Introduction Transcripts
Section Introduction Transcripts

Course Overview
[Autogenerated] Hi, everyone. My name is Pablo Kordic, and welcome to my core scaling Python data applications with dusk. I am a big data engineered Farfetch where I developed data applications, helping both the business and the customers. Python, along with tools like pandas and NumPy, is a tool of choice for many data analysts and scientists who valued these off Hughes expressiveness and possibility to quickly iterated on ideas. Unfortunately, most of python tools do not scale to higher data volumes easily. In this course, we're going to discover the dusk library that helps in scaling off Python applications, especially data intensive ones paid on a single or multiple machines. Some of the major topics that will cover include use cases and operation principles, ways to scale basic python applications, scalable versions of pandas and NumPy APIs, and ways to scale beyond a single machine. By the end of this course, you will have a solid background and confidence to write 100% python data processing applications that can keep up with the ever increasing data volumes. Before beginning this course, you should be familiar with python and have basic experienced with pandas and NumPy libraries. I hope you'll join me on this journey to learn dusk with these scaling python data applications with dusk course thead Pluralsight