What you'll learn

A data pipeline that only runs when someone remembers to click a button is not ready for production. In this course, Automate Data Pipelines with Dagster, you'll gain the ability to make your pipelines run automatically and reliably. First, you'll explore how to create jobs that select and run specific sets of assets with consistent configuration. Next, you'll discover how to schedule pipeline runs on a time-based cadence and troubleshoot schedules that fail to fire. Finally, you'll learn how to use sensors for event-driven automation and partitions for incremental processing and historical backfills. When you're finished with this course, you'll have the skills and knowledge of Dagster automation needed to run data pipelines that execute on their own, respond to new data, and process history efficiently.