What you'll learn

Kafka’s flexibility comes with responsibility — poor design choices can cause bottlenecks, data loss, or high storage costs. In this course, Design Topics, Schemas, and Retention Policies in Kafka, you’ll gain the ability to design scalable, consistent, and efficient Kafka data pipelines. First, you’ll explore how to design Kafka topics and partitions for performance, parallelism, and durability. Next, you’ll discover how to apply effective keying and partitioning strategies to optimize ordering and data distribution. Then, you’ll learn how to define and manage message schemas to ensure consistent and evolvable data contracts. Finally, you’ll understand how to configure retention, compaction, and data lifecycle policies to balance cost and replay capability. When you’re finished with this course, you’ll have the skills and knowledge to architect Kafka topics, schemas, and retention strategies that perform well at scale in real-world systems.