Developing Stream Processing Applications with AWS Kinesis

AWS Kinesis is a powerful, real-time, elastic, reliable service for stream processing. This course will teach you how to build stream processing applications using AWS Kinesis, stream processing services, and Big Data frameworks.
Course info
Rating
(17)
Level
Intermediate
Updated
Mar 1, 2018
Duration
3h 32m
Table of contents
Course Overview
Kinesis Fundamentals
Developing Applications Using Kinesis Client Library
Implementing Advanced Kinesis Consumers
Funneling Data with Kinesis Firehose
Implementing Stream Analysis Applications Using Streaming SQL
Description
Course info
Rating
(17)
Level
Intermediate
Updated
Mar 1, 2018
Duration
3h 32m
Description

The landscape of the Big Data field is changing. Previously, you could get away with processing incoming data for hours or even days. Now you need to do it in minutes or even seconds. These challenges require new solutions, new architectures, and new tools. In Developing Stream Processing Applications with AWS Kinesis, you will learn the ins and outs of AWS Kinesis. First, you will learn how it works, how to scale it up and down, and how to write applications with it. Next, you will explore how to use a variety of tools to work with it such as Kinesis Client Library, Kinesis Connector Library, Apache Flink, and AWS Lambda. Finally, you will discover how to use more high-level Kinesis products such as Kinesis Firehose and how to write streaming applications using SQL queries with Kinesis Analytics. When you are finished with this course, you will have an in-depth knowledge of AWS Kinesis that will help you to build your streaming applications.

About the author
About the author

Ivan is a software development engineer at Amazon in CloudWatch team. Before he joined Amazon he was working at Samsung R&D. In his free time, he contributes to Apache Flink.

More from the author
AWS DynamoDB Deep Dive
Intermediate
3h 21m
Oct 2, 2017
Section Introduction Transcripts
Section Introduction Transcripts

Course Overview
Hi everyone. My name is Ivan Mushketyk, and welcome to my course, Developing Stream Processing Applications with AWS Kinesis. I am a Principal Software Engineer at Viasat and an open-source contributor, but before I was working at Amazon and Samsung R&D. The field of big data is going through a revolution. Stream processing is a new norm in companies who process incoming data in minutes or even seconds just to stay competitive. This course is an in-depth course about AWS Kinesis, a powerful stream-processing solution from Amazon. In this course, we will start with the very basics of stream processing with Kinesis and learn how to build complex stream-processing applications. Some of the major topics that this course will cover include what AWS Kinesis can be used for and how it works, how to build applications with Kinesis, how to use high-level tools with AWS Kinesis, and how to build applications using streaming SQL. By the end of the course, you will know the ins and outs of AWS Kinesis, and you will be able to build your stream-processing applications with AWS. You don't need to know big data or stream processing for this course, you only need to know Core Java and basics of AWS to get started. From here, you should feel comfortable diving into stream processing with courses on big data frameworks like Apache Flink or Apache Spark. I hope you will join me on this journey to learn AWS Kinesis with the Developing Stream Processing Applications with AWS Kinesis course, at Pluralsight.

Developing Applications Using Kinesis Client Library
Hi. I'm Ivan Mushketyk with Pluralsight, and welcome to the next module of this course called Developing Applications Using Kinesis Client Library. And in this module, we are finally going to implement something practical with AWS Kinesis. And specifically we are going to build a simple application that will read a stream of tweets from Twitter and calculate how many tweets were written in each language in every minute. In this example, there was just 1 tweet written at 13:45, but there were 2 tweets written at 13:46. This will allow us to get some sort of metadata about what's happening in Twitter at the moment, who is writing tweets right now. To implement this simple example, we are going to use some interesting technologies. First of all, we are going to read data from actual Twitter. We are going to work with actual tweets. To write data to Kinesis, we are going to use a library implemented by AWS called Kinesis Producer Library. And then we'll use another library developed by Amazon called Kinesis Client Library that we will use to read data from Kinesis and process them and calculate our statistics, and then we will pretend that we are writing data into a database. And you can ask why don't we just use a real database? And the reason for this is that if you don't know the database that I'm going to use, then it will be confusing to you and it won't help us to understand how to use Kinesis. This application may sound simple, but there are some tricky parts, so we won't try to implement it altogether. We will take baby steps and implement one simple part at a time. Okay, let's go straight to that.