Path icon Learning Paths
Skills

Apache Spark on Databricks

  • Number of Courses7 courses
  • Duration14 hours
  • Skill IQ available Skill IQ

Apache Spark on Databricks is a unified analytics platform that combines the powerful data processing capabilities of Apache Spark with the collaborative and managed environment of Databricks, enabling scalable and efficient big data processing, real-time analytics, and machine learning applications in a cloud-native architecture. This learning path is intended to give learners foundational skills to start working with Apache Spark on Databricks for these purposes.

Courses in this path

Beginner

You will learn Spark transformations, actions, visualizations, and functions leveraging the Databricks API. You will also learn how to transform and aggregate batch data using Spark with built-in and user defined functions, and perform windowing and join operations on batch data.

Getting Started with Apache Spark on Databricks

  • by Janani Ravi
  • 1h 52m Duration
  •  (59)

Handling Batch Data with Apache Spark on Databricks

  • by Janani Ravi
  • 2h 22m Duration
  •  (22)

Intermediate

You will learn how to use Spark abstractions for streaming data and perform transformations on streaming data using the Spark streaming APIs on Databricks as well as how to leverage windowing, watermarking and join operations on streaming data in Spark for your specific use-cases.

Processing Streaming Data with Apache Spark on Databricks

  • by Janani Ravi
  • 2h 1m Duration
  •  (24)

Windowing and Join Operations on Streaming Data with Apache Spark on Databricks

  • by Janani Ravi
  • 2h 2m Duration

Advanced

You will understand and implement important techniques for predictive analytics such as regression and classification using Apache Spark MLlib APIs on Databricks as well as learn how to implement graph algorithms such as Triangle Count and PageRank and visualize them using the GraphFrames API on Spark Databricks. You will also learn how to optimize the performance of Spark clusters by identifying and mitigating various performance issues such as data ingestion problems and leveraging the new features offered by Spark 3.

Predictive Analytics Using Apache Spark MLlib on Databricks

  • by Janani Ravi
  • 1h 57m Duration

Executing Graph Algorithms with GraphFrames on Databricks

  • by Janani Ravi
  • 1h 34m Duration

Optimizing Apache Spark on Databricks

  • by Janani Ravi
  • 2h 32s Duration
  •  (25)

Learn with the best

Janani Ravi
Janani Ravi
Janani has a Masters degree from Stanford and worked for 7+ years at Google. She was one of the original engineers on Google Docs and holds 4 patents for its real-time collaborative editing framework. After spending years working in tech in the Bay Area, New York, and Singapore at companies such as Microsoft, Google, and Flipkart, Janani finally decided to combine her love for technology with her passion for teaching. She is now the co-founder of Loonycorn, a content studio focused on providing ... more

Join our learners and upskill
in leading technologies