- Course
- Cloud
- Data
Working with Azure Databricks Programmatically
To carry out large-scale operations on Databricks, you'll need to develop apps or scripts which can interact with this big data service. This course looks into the Databricks CLI, its REST API, and the dbutils library to automate such interactions.
What you'll learn
While nearly every task in Databricks can be accomplished from its Web UI, to perform operations of any complexity at scale, you need to interact with this service from a script or an application. In this course, Working with Azure Databricks Programmatically, you'll learn about the Databricks command-line interface (CLI). First, you'll explore how we can create and use a personal access token for authentication, and how to construct CLI commands to perform a variety of workspace operations. Then, you'll explore the use of the Databricks REST API. Finally, you'll discover the versatility of the dbutils library in order to interact with the Databricks file system from a Python application. Once you complete this course, you'll have a clear understanding of how interactions with a Databricks service can be automated using the Databricks CLI, the REST API, and the dbutils package.
Table of contents
- Course Prerequisites and Outline | 2m 15s
- Interacting with Databricks | 4m 35s
- Automating Databricks Interactions | 6m 6s
- Demo: Downloading and Setting up the Databricks CLI | 7m 41s
- Demo: Viewing Cluster Data with the CLI | 6m 5s
- Demo: Deleting Clusters with the CLI | 3m 36s
- Demo: Creating and Editing Clusters | 6m 17s
About the author
An engineer at heart, I am drawn to any interesting technical topic. Big Data, ML and Cloud are presently my topics of interest.