Complex Event Processing Using Apache Flink
The Complex Event Processing is built on top of the core Flink libraries. It allows you to detect and process patterns in an infinite stream of events, or within a time window. This course will teach you to handle complex event processing for streaming data using Apache Fink.
What you'll learn
The Apache Flink library supports complex event processing using the Complex Event Processing or the CEP library which is built on top of the Flink core APIs.
In this course, Complex Event Processing Using Apache Flink, you'll learn about complex event processing involving defining patterns that you want to detect and identify in the input stream, either across the stream as a whole or within a certain time window.
First, you'll be introduced to the Pattern API which allows you to define patterns that you want to detect in input streams. You'll create patterns using singleton patterns that match only one event in the input stream, you'll also use looping patterns created using quantifiers that can match multiple events in the input stream.
Next, you'll see how you can specify different kinds of conditions to accept events into your pattern. Simple conditions only look at the properties of an event to perform matching, iterative conditions can look at the previous matched events and even compute statistics on the input stream to accept matching events. You'll see how more complex conditions can be created using combining operations with logical 'AND' and logical 'OR.' You'll also explore the contiguity constraints that we can define on patterns including strict contiguity, relaxed contiguity, and non-deterministic relaxed contiguity.
Finally, you'll see how you can control how many times a particular event matches a pattern using the after match skip strategy. You'll understand the differences between the no skip, skip to next, skip past last event, skip to first, and skip to last strategies. You'll also see how you can perform pattern matching within a time window, and handle partial timed-out matches.
When you're finished with this course, you'll have the skills and the ability to detect patterns in input streams using the pattern API in the Apache Flink Complex Event Processing library.
Table of contents
- Version Check 0m
- Prerequisites and Course Outline 3m
- Pattern Matching Using Flink CEP 6m
- Demo: Configuring a Maven Project to Use Flink CEP 5m
- Demo: Pattern Detection and Selection Using the Pattern Select Function 6m
- Demo: Pattern Detection and Selection Using the Pattern Process Function 2m
- Demo: Patterns with Conditions 3m
- Demo: Pattern Sequences with Multiple Individual Patterns 6m
- Demo: Pattern Matching with Complex Entity Streams 7m
- Demo: Specifying Looping Patterns Using Quantifiers 8m
- Demo: Exploring Different Looping Patterns 9m
- Simple Conditions and Iterative Conditions 3m
- Demo: Simple Conditions 4m
- Demo: Sub Type Conditions 3m
- Demo: Iterative Conditions 5m
- Demo: Looping Patterns and Iterative Conditions 6m
- Demo: Combining Conditions Using Logical AND And Logical OR 4m
- Demo: Stopping Conditions for Looping Patterns 3m
- Contiguity Constraints 2m
- Demo: Strict Contiguity, Relaxed Contiguity, and Non Deterministic Relaxed Contiguity 5m
- Demo: Not Next and Not Followed by Contiguity Constraints 5m
- Demo: Contiguity Constraints for Looping Patterns 8m
- Demo: Applying Contiguity Constraints on Complex Entities 7m
- After Match Skip Strategies 3m
- Demo: Groups of Patterns 4m
- Demo: Control Pattern Matches Using After Match Skip Strategies 8m
- Demo: Disambiguating between Skip Past Last Event, Skip to First, and Skip to Last 5m
- Time in the CEP Library 2m
- Demo: Extracting Event Time from Input Streams 6m
- Demo: Pattern Matching with in an Event Time Window 2m
- Demo: Handling Timed out Partial Matches 4m
- Summary and Further Study 2m