This course is a technical introduction to Kafka Connect and how to use it to quickly transfer data in and out of the Apache Kafka ecosystem, to and from common data systems like databases, key-value stores, search indexes, and file systems.
You may be wondering why the word "Connect" has suddenly sprung up next to "Kafka". Isn’t Kafka a Distributed Streaming Platform? Well, Kafka is more than that. Apache Kafka is an entire ecosystem and Kafka Connect is a part of it. In this course, Kafka Connect Fundamentals, you will gain the ability to create your own real-time ETL pipelines from and to Apache Kafka. First, you will learn what the ETL model is and how to set up your own ETL pipeline using Kafka Connect. Next, you will discover the inner details of Kafka Connect by exploring its architecture. Finally, you will explore how to successfully manage your Kafka Connect installation in a production environment. When you are finished with this course, you will have the skills and knowledge of Kafka Connect needed to set up, build, and maintain your own Kafka Connect installation.
Bogdan Sucaciu is a Software Engineer at Axual in The Netherlands, where he is taking part in building a Streaming Platform designed to share information in real-time, enabling instant processing of incoming events.
He has several years of experience "cooking" software with JVM-based languages, some flavors of web technologies and garnishing with automated testing.
Course Overview Hi. My name is Bogdan Sucaciu, and welcome to my course, Kafka Connect Fundamentals. I'm a tech lead at Axual in the Netherlands where I'm taking part in building a streaming platform designed to share information in real time. You may be wondering where the word Connect has suddenly sprung up in Kafka Connect. Isn't Kafka a distributed streaming platform? Well, Kafka is more than that. Apache Kafka is an entire ecosystem, and Kafka Connect is part of it. In this course, we're going to explore how we can leverage Kafka Connect to quickly transfer data in and out of the Apache Kafka ecosystem to and from common data systems like databases, key value stores, search indexes, and file systems. Some of the major topics that we'll cover include building real‑time ETL pipelines using Apache Kafka, deep dive into Kafka Connect's architecture, creating and customizing your own connector, and also some tips and tricks that will help you to run Kafka Connect in production. By the end of this course, you should be comfortable to set up your own data processing pipeline using Apache Kafka and Kafka Connect and safely run it without any surprises. Before beginning the course, you should already be familiar with Apache Kafka concepts and its ecosystem. Also, I'll be using Docker continuously to keep the differences between systems to a minimum so it shouldn't matter if you use Windows, Linux or Mac. Finally, we're going to build our own connector and customize it using Java. So, a basic understanding of the Java programming language would definitely help you to follow along. I hope you'll join me on this journey to learn Kafka Connect with the Kafka Connect Fundamentals course, as Pluralsight.