Kafka: Build, Deploy, and Monitor Your First Real-world Application
This course targets data developers who are looking to learn how to funnel all their disparate incoming data through a Kafka pipeline wanting to better understand and maintain it.
What you'll learn
Companies nowadays are finding it critical to be able to gather and analyze massive amounts of disparately sourced data, quickly and reliably. In this course, Kafka: Build, Deploy, and Monitor Your First Real-world Application, you'll learn how it can be more than a superior, distributed messaging system, but a platform to handle all your stream processing needs. First, you’ll review the basics; topics, producers, consumers, and beyond. Then, you’ll discover how to operationalize your data pipelines. Finally, you'll learn how easily maintaining and monitoring the data pipeline throughout a full production lifecycle. By the end of this course, you’ll have learned how to use Kafka for many of your data needs, merging the worlds of data consuming, processing, and devops.
Table of contents
- Introduction 2m
- Kafka's Design: A Perfect Fit for Microservices 5m
- Oh the Things You Can Do with Kafka 3m
- Kafka's Architecture Overview 9m
- Kafka's Heart: The Broker 3m
- Kafka's Failure Manager: Zookeeper 5m
- Keeping the Data Alive: Data Fault Tolerance 7m
- Demo: Starting up in 1 2 3 5m
- Kafka's Veins and Arteries: Producers and Consumers 6m
- Demo: Putting it Together - Producing Data 13m
- Demo: Putting it Together - Consuming Data 11m
- Resources 1m
- Summary 2m
- Introduction 1m
- Configuration Overload Understood 3m
- Configuring the Broker: Core Configuration and Managment 5m
- Demo: Altering Configurations 1m
- Demo: Scaling the Brokers 9m
- Configuring the Broker: Parallelizing via Partitions 7m
- Configuring the Broker: Fault Tolerance via Replication 7m
- Demo: Increasing the Replication Factor 6m
- Configuring the Broker: Data Retention 10m
- Resources 1m
- Summary 1m
- Introduction 1m
- Configuring the Producer 11m
- Demo: Configuring the Producer 2m
- Configuring the Consumer 9m
- Consumer Offset Tracking 7m
- Consumer Offset Retrieval 4m
- Common Production and Reliability Concerns 9m
- Empowerment Through Custom Messages: Serializers and Deserializers 3m
- Demo: Adding Types Via Serialization (Part 1) 7m
- Demo: Adding Types Via Serialization (Part 2) 6m
- Resources 1m
- Summary 1m
About the author
Justin is a software journeyman, continuously learning and honing his skills. Most of his early professional career was spent in C# and MSSQL, but he loves learning about many different languages, especially Scala. This passion for Scala led him to join the Lightbend (formerly Typesafe) team, diving even deeper into the Scala ecosphere. And, as much as he loves to learn, he also loves to spread his knowledge through teaching and helping others. He is a very active answerer on StackOverflow... more