What you'll learn

Administering a Databricks workspace involves a wide variety of tasks, but one with far-reaching consequences which is often neglected is the management of clusters and cluster policies. In this course, Administering Clusters and Configuring Policies with Databricks Service, you'll explore how to effectively manage clusters and policies in Databricks which helps ensure teams run their jobs in a consistent environment while keeping costs down to manageable levels. First, you'll learn the need for cluster policies, and specifically how these can help control costs while still allowing users to run their jobs efficiently. Next, you'll explore how rules can be set to restrict access to specific Databricks resources, such as instance pools, clusters, and notebooks. Then, you'll discover how cluster policies can be framed and applied programmatically. Finally, you'll learn how to generate a personal access token to interact with Databricks using the REST API, and cover how this can be used to define policies, as well as clusters built by applying them.

Once you complete this course, you'll recognize exactly why cluster policies are required, how these can be defined and rolled out in an organization, and how these can be managed with the Databricks REST API.