Expanded

Complex Event Processing Using Apache Flink

The Complex Event Processing is built on top of the core Flink libraries. It allows you to detect and process patterns in an infinite stream of events, or within a time window. This course will teach you to handle complex event processing for streaming data using Apache Fink.
Course info
Level
Advanced
Updated
May 12, 2021
Duration
2h 28m
Table of contents
Course Overview
Getting Started with Complex Event Processing
Understanding and Applying Combining Patterns
Detecting and Processing Complex Patterns
Description
Course info
Level
Advanced
Updated
May 12, 2021
Duration
2h 28m
Description

The Apache Flink library supports complex event processing using the Complex Event Processing or the CEP library which is built on top of the Flink core APIs.

In this course, Complex Event Processing Using Apache Flink, you'll learn about complex event processing involving defining patterns that you want to detect and identify in the input stream, either across the stream as a whole or within a certain time window.

First, you'll be introduced to the Pattern API which allows you to define patterns that you want to detect in input streams. You'll create patterns using singleton patterns that match only one event in the input stream, you'll also use looping patterns created using quantifiers that can match multiple events in the input stream.

Next, you'll see how you can specify different kinds of conditions to accept events into your pattern. Simple conditions only look at the properties of an event to perform matching, iterative conditions can look at the previous matched events and even compute statistics on the input stream to accept matching events. You'll see how more complex conditions can be created using combining operations with logical 'AND' and logical 'OR.' You'll also explore the contiguity constraints that we can define on patterns including strict contiguity, relaxed contiguity, and non-deterministic relaxed contiguity.

Finally, you'll see how you can control how many times a particular event matches a pattern using the after match skip strategy. You'll understand the differences between the no skip, skip to next, skip past last event, skip to first, and skip to last strategies. You'll also see how you can perform pattern matching within a time window, and handle partial timed-out matches.

When you're finished with this course, you'll have the skills and the ability to detect patterns in input streams using the pattern API in the Apache Flink Complex Event Processing library.

About the author
About the author

A problem solver at heart, Janani has a Masters degree from Stanford and worked for 7+ years at Google. She was one of the original engineers on Google Docs and holds 4 patents for its real-time collaborative editing framework.

More from the author
Summarizing Data and Deducing Probabilities
Intermediate
2h 50m
Jul 8, 2021
More courses by Janani Ravi
Section Introduction Transcripts
Section Introduction Transcripts

Course Overview
Hi, my name is Janani Ravi, and welcome to this course on Complex Event Processing Using Apache Flink. A little about myself. I have a master's degree in electrical engineering from Stanford and have worked at companies such as Microsoft, Google, and Flipkart. I currently work on my own startup, Loonycorn, a studio for high‑quality video content. Complex event processing involves defining patterns which you want to detect and identify in the input stream, either across the stream as a whole or within a certain time window. First, you will be introduced to the pattern API which allows you to define patterns you want to detect in input streams. You will create patterns using singleton patterns that match only one event in the input stream. You'll also use looping patterns created using quantifiers. Looping patterns can match multiple events in the input stream. Next, you will see how you can specify different kinds of conditions to accept events into your pattern, simple conditions which look only at the properties of an event to perform matching, and iterative conditions that can look at previous matched events and even compute statistics over previous matched events. You will see how more complex conditions can be created using logical AND and logical OR operations. You will explore contiguity constraints that we can define on patterns, including strict contiguity, relaxed contiguity, and non‑deterministic relaxed contiguity. Finally, you will see how you can control how many times a particular event matches a pattern using the after match skip strategy. You'll understand the differences between no skip, skip to next, skip past last event, skip to first, and skip to last strategies. You'll also see how we can perform pattern matching within a time window and handle partially timed‑out matches. When you're finished with this course, you will have the skills and ability to detect patterns and input streams using the Apache FlinkCEP library.