Administering Clusters and Configuring Policies with Databricks Service
Clusters are at the heart of most tasks in Databricks, and configuring them precisely is crucial to ensuring optimum performance while also managing costs. This course covers how cluster policies can be used to help meet these goals.
What you'll learn
Administering a Databricks workspace involves a wide variety of tasks, but one with far-reaching consequences which is often neglected is the management of clusters and cluster policies. In this course, Administering Clusters and Configuring Policies with Databricks Service, you'll explore how to effectively manage clusters and policies in Databricks which helps ensure teams run their jobs in a consistent environment while keeping costs down to manageable levels. First, you'll learn the need for cluster policies, and specifically how these can help control costs while still allowing users to run their jobs efficiently. Next, you'll explore how rules can be set to restrict access to specific Databricks resources, such as instance pools, clusters, and notebooks. Then, you'll discover how cluster policies can be framed and applied programmatically. Finally, you'll learn how to generate a personal access token to interact with Databricks using the REST API, and cover how this can be used to define policies, as well as clusters built by applying them.
Once you complete this course, you'll recognize exactly why cluster policies are required, how these can be defined and rolled out in an organization, and how these can be managed with the Databricks REST API.
Table of contents
- Course Prerequisites and Outline 2m
- The Need for Cluster Policies 5m
- Applying Cluster Policies 5m
- Demo: Creating a Simple Cluster Policy 5m
- Demo: Hidden and Default Attributes 6m
- Demo: Allowing and Blocking Cluster Attribute Values 4m
- Demo: Modifying Cluster Policy Permissions 6m
- Demo: Creating a Cluster from a Policy 3m