What you'll learn

Data is often the most valuable asset an organization has, but how do you process data in real time, at scale, when data can be coming in at huge volumes from disparate sources in varying formats?

In this course, Real-time Data Processing for Data Engineers, you’ll gain the ability to build and contribute to professional data pipelines capable of processing big data at scale.

First, you’ll explore the fundamentals of data processing at scale, including what data pipelines are, how they are implemented, and how they vary from batch data processing.

Next, you’ll discover key data pipeline tooling such as Apache Kafka and Spark, and take a look at their strengths and where they fit into data pipelines.

Finally, you’ll learn how to implement data pipelines, first by reviewing an example scenario, and then via a coding example where you will implement a working data pipeline in minutes.

When you’re finished with this course, you’ll have the skills and knowledge of real-time data processing needed to effectively process real-time data at scale.