All courses
Resource hub
Featured resource
Tech Upskilling Playbook 2025
Tech Upskilling Playbook

Build future-ready tech teams and hit key business milestones with seven proven plays from industry leaders.

Learn more
Hamburger Icon
  • Path icon Learning Path
    • Libraries: This path is only available in the libraries listed. To access this path, purchase a license for the corresponding library.
  • Data
    •

Apache Spark for Data Scientists

11 Courses
5 Labs
9 Hours
Skill IQ

Apache Spark is a powerful, open-source framework that enables data scientists to efficiently process, analyze, and transform large-scale data. This learning path equips you with the essential skills necessary to perform data science tasks using Apache Spark. In this path, you'll learn everything from Spark fundamentals, DataFrames, and SQL to advanced data transformations, machine learning, and structured streaming.

Get started

Content in this path

Apache Spark for Data Scientists

Start watching the courses below to get started.

Course

Apache Spark Architecture and Fundamentals

  • by Saravanan Dhandapani
  • 24m
  • Dec 26, 2024
Course

Work with RDDs, DataFrames, and Datasets in Apache Spark

  • by Raphael Alampay
  • 17m
  • Feb 20, 2025
Course

Transform Data Using PySpark

  • by Pinal Dave
  • 43m
  • Dec 11, 2024
Course

Transform Data Using Spark SQL

  • by Anand Saravanan
  • 14m
  • Jan 16, 2025
Course

Transform Data Using the Pandas API in Apache Spark

  • by Bismark Adomako
  • 12m
  • Dec 31, 2025
Course

Structured Streaming in Apache Spark

  • by Janani Ravi
  • 59m
  • Feb 28, 2025
Course

Window and Join Operations in Apache Spark

  • by Pinal Dave
  • 18m
  • Dec 13, 2024
Course

Performance Optimization in Apache Spark

  • by Pinal Dave
  • 39m
  • Jan 07, 2025
Course

Machine Learning with Apache Spark

  • by Avdhesh Gaur
  • 25m
  • Dec 23, 2024
Course

Monitor Apache Spark Clusters

  • by Harsh Karna
  • 23m
  • Feb 15, 2025
Course

Graph Operations in Apache Spark Using GraphFrames

  • by Janani Ravi
  • 25m
  • Feb 20, 2025

Hands-on Practice with Apache Spark

The following labs will help you get practical experience with Apache Spark.

Lab

Transformations and Actions with RDDs in Apache Spark

  • 37m
  • Sep 02, 2025
Lab

Basic Data Manipulation with PySpark

  • by Pinal Dave
  • 46m
  • Aug 21, 2025
Lab

Advanced Data Manipulation with PySpark

  • by Pinal Dave
  • 54m
  • Sep 02, 2025
Lab

Perform Data Cleaning Operations with PySpark

  • by Chhaya Wagmi
  • 54m
  • Aug 31, 2025
Lab

Create and Use User-defined Functions with PySpark

  • by Pinal Dave
  • 40m
  • Aug 06, 2025
Try this learning path for free
Access this learning path and other top-rated tech content with a free trial.
Free individual trial Free team trial
Have questions? Get them answered now.
Start a live chat
What You'll Learn
  • Apache Spark Architecture and Fundamentals
  • How to work with RDDs, DataFrames, and datasets in Spark
  • How to transform data in Spark using PySpark, Spark SQL, and the Pandas API
  • How to process real-time data streams using structured streaming in Spark
  • How to use window and join operations in Spark
  • How to optimize performance in Spark
  • How to monitor Spark clusters
  • How to perform machine learning operations in Spark
  • How to use graph operations in Spark
Prerequisites
  • Learners interested in this path should have basic programming knowledge in Python or Scala. Learners should also be familiar with SQL and fundamental data processing, machine learning, and big data concepts.
Related topics
  • Apache Spark
  • Data Science
  • Big Data
  • Machine Learning
  • Data Transformation
  • SQL
Not sure where to start?
With over 500 assessments to choose from, you can see where your skills stand and receive adaptive learning recommendations to fill knowledge gaps in as little as 10 minutes.
Learn more

Learn with the best

Saravanan Dhandapani
Saravanan Dhandapani
I have worked in IT design, development, and architecture for over a decade for some of the top fortune 100 companies. I have designed and architected enterprise applications and developed scalable and portable software. I am a Google Certified Professional Architect. Critical areas where I have worked are architecture and design using Java, ESB, Tomcat, ReactJS, JavaScript, Linux, Oracle, SVN, GIT, and so on, and cloud technologies, including AWS and GCP.
Raphael Alampay
Raphael Alampay
Raphael Alampay is the co-founder of Cloudband Solutions Co., a software development and consultancy company that caters to custom based software for SME around the globe. Using time and tested technology such as Java, Ruby on Rails, Python, PostgreSQL and Linux, he has a passion for creating applications that solve real world problems making businesses more efficient and innovative at the same time. His craft in software is largely based on the philosophy of "kaizen" which means continuous impr...
Pinal Dave
Pinal Dave
Pinal Dave is an SQL Server Performance Tuning Expert and independent consultant with over 24 years of hands-on experience. He holds a Master of Science degree and numerous database certifications. Pinal has authored 14 SQL Server database books, 91 Pluralsight courses and 11 Code Labs. To freely share his knowledge and help others build their expertise, Pinal has also written more than 5,800 database tech articles on his blog at https://blog.sqlauthority.com.
Anand Saravanan
Anand Saravanan
Anand is a software engineer and is enthusiastic about AI and its multidisciplinary applications. He has always had a passion for technology and how it can be used to help people and businesses. Excellence in various programming languages such as C++, Java, R, Python, SQL, Hadoop, etc., he is a man who has an itch to constantly learn and share his knowledge. He has worked with Fortune 500 companies in various sectors to help them effectively use the data at hand to help make better business dec...
Bismark Adomako
Bismark Adomako
Bismark believes that education is no longer a one-time investment, but instead a lifelong pursuit and that great authors and mentors can play an invaluable role in finding and developing a fulfilling career. He holds a Bachelor of Science degree in Computer Engineering specializing in Software Engineering, Artificial Intelligence and Distributed computing. He taught courses in robotics and computer vision as a student leader whiles in school, and interned and consulted for the British Counci...
Janani Ravi
Janani Ravi
Janani has a Masters degree from Stanford and worked for 7+ years at Google. She was one of the original engineers on Google Docs and holds 4 patents for its real-time collaborative editing framework. After spending years working in tech in the Bay Area, New York, and Singapore at companies such as Microsoft, Google, and Flipkart, Janani finally decided to combine her love for technology with her passion for teaching. She is now the co-founder of Loonycorn, a content studio focused on providing ...
Avdhesh Gaur
Avdhesh Gaur
Avdhesh Gaur is a Senior Data Scientist, an instructor, an author, a storyteller and consultant as well when it comes to bringing insights out of data and presenting the story behind. His career spans more than 10 years with a focus on taking down various business requirements by building Data-driven BI models using Tableau, SQL, Hive, and numerous Statistical data analysis Techniques. ​ Avdhesh Gaur got one of his submissions to Tableau accepted on 7th January 2019. On average, only 1 out of ev...
Harsh Karna
Harsh Karna
Harsh is a skilled software engineer with over 4 years of experience in Data Engineering and Software Development. He has strong expertise in big data technologies like Hadoop, Spark, and SQL, along with cloud platforms like AWS and Azure, where he has built robust ETL and data frameworks on platforms like Snowflake and Databricks. Proficient in programming languages such as Python, SQL, and Java, Harsh also has experience in Django and Flask frameworks. His work includes designing and implement...
Chhaya Wagmi
Chhaya Wagmi
Written content author.

Join our learners and upskill
in leading technologies