Executing Graph Algorithms with GraphFrames on Databricks
This course will teach you how to create and represent graph data using GraphFrames in Apache Spark and implement graph algorithms such as Shortest Path and PageRank on Azure Databricks.
What you'll learn
The Spark unified analytics engine is one of the most popular frameworks for big data analytics and processing. The GraphFrames package in Apache Spark allows you to represent graphs using a DataFrame-based API. GraphFrames also supports a number of graph algorithms such as Shortest Path, PageRank, Breadth-first search, and connected components.
In this course, Executing Graph Algorithms with GraphFrames on Databricks, you will explore how graphs can be used to model entities and relationships in the real world. First, you will learn about the different kinds of graphs such as directed and undirected graphs, weighted and unweighted graphs. Then, you will discover how graphs can be represented using the GraphFrames API in Apache Spark and how you can compute the properties of a graph such as indegree and outdegree of a vertex and perform filtering operations on vertices and edges.
Next, you will see how you can perform motif searches using GraphFrames in order to detect structural patterns in the graph. After that, you will learn how to use a domain-specific language for motif finding and run stateless and stateful queries on simple as well as complex real-world graphs.
Finally, you will explore the variety of graph algorithms supported by the GraphFrames API including Breadth-first search, Shortest Path, triangle count, connected and strongly connected components, and PageRank.
When you are finished with this course, you will have the skills and knowledge of graph algorithms in Spark needed to implement graph algorithms using the GraphFrames API provided by Spark.
Table of contents
- Version Check 0m
- Prerequisites and Course Outline 2m
- Graphs to Model Relationships 3m
- Directed and Undirected Graphs 2m
- Adjacent Nodes, Paths, Degree of a Node 3m
- GraphFrames on Databricks 3m
- Demo: Installing GraphFrames and NetworkX 3m
- Demo: Creating GraphFrames 6m
- Demo: Indegree and Outdegree 2m
- Demo: Filtering Vertices and Edges 6m
- Breadth-first and Depth-first Traversal 6m
- Demo: Breadth-first Search 4m
- Shortest Path 4m
- Demo: Shortest Path 3m
- Demo: Triangle Count 3m
- Connected Components and Strongly Connected Components 2m
- Demo: Connected Components and Strongly Connected Components 4m
- PageRank 2m
- Demo: PageRank 5m
- Summary and Further Study 2m