What you'll learn

Many data platforms begin with simple append-only ingestion, but real-world pipelines quickly become more complex. Teams need to handle updates, deletes, late-arriving records, incremental reads, and recovery from bad writes without relying on fragile manual workflows.

In this course, Ingestion and Incremental Processing with Open Table Formats, you’ll gain the ability to understand and explain how open table formats support modern ingestion and incremental processing patterns.

First, you’ll explore how writers append, update, and delete data across batch, microbatch, and streaming workflows. Next, you’ll discover how upserts, MERGE operations, and ingestion design tradeoffs differ across formats and workload types. Finally, you’ll learn how snapshots, time travel, rollback, and incremental reads help build reliable downstream processing pipelines.

When you’re finished with this course, you’ll have the skills and knowledge of ingestion and incremental processing with open table formats needed to reason about practical data platform design decisions.