Event Hubs facilitate big data applications by moving large volumes of data efficiently. This course teaches you the core concepts and design patterns of Event Hubs, starting with a simple example and working toward an Internet of Things application.
At the core of big data is a thorough knowledge of large-scale data transfer mechanisms. In this course, Getting Started with Azure Event Hubs with C#, you'll learn how to leverage Event Hubs to transfer data from multiple sources in parallel, at hyper scale. First, you'll discover data transfer design patterns. Next, you'll explore integrating your .NET applications with Event Hubs through state-of-the-art web protocols. Finally, you'll experience building well-balanced applications composed of parallel tiers and exhibiting fluid data-flow. When you’re finished with this course, you'll have a foundational understanding of hyperscale systems that will help you as you scale your applications to handle exponentially increasing data volumes at a global scale.
Paul is a Microsoft .NET MVP with over 15 years of experience in software development. Paul regularly speaks at conferences and meetups, is an active blogger, and open-source contributor (#1 in Ireland, C#). Paul focuses on performance-optimisation and develops real-time systems that handle large volumes of data.
Section Introduction Transcripts
Section Introduction Transcripts
Course Overview Hi everybody, my name is Paul Mooney, and welcome to my course, Getting Started with Azure Event Hubs with C#. I'm a Microsoft MVP and freelance software architect specializing in high-performance systems, and web security. Azure Event Hubs are a PaaS, that's a Platform as a Service system that facilitates big data. Event Hubs are a temporal buffer acting as an integration point between client and server. In this course, we're going to cover design patterns for big data transmission, Azure Event Hubs core concepts, and achieving balanced throughput at hyperscale. By the end of this course, you'll know all about efficiently moving large volumes of data from multiple sources, at speed. You'll understand concepts, such as data partitioning and parallel design, and we'll be well on your way to designing and building multi-tier applications that exhibit fluid and balanced data flow at global scale. Before beginning this course, you should be reasonably familiar with cloud concepts such as PaaS or Platform as a Service. You should also know beginner to intermediate knowledge of the C# programming language. I hope you'll join me on this journey to learn about Azure Event Hubs with the Getting Started with Azure Event Hubs with C# course, at Pluralsight.
Understanding the Big Picture Hi, this is Paul Mooney. You're very welcome to my course, Getting Started with Azure Event Hubs with C#. So what will you learn from this course? That it's the first step in becoming a data specialist, you'll learn to control fast-moving data, and to organize dissimilar data chunks in real-time. Let's talk about how you'll learn in this course. We're going to build an IoT, that is an Internet of Things simulation. We're going to transmit and analyze telemetry in real-time, we're going to manage data transited at varying speeds. There are a couple of things you'll need for this course. First, you'll need a Microsoft Azure account. If you don't have one you can set one up for free online. Second, you'll need a Visual Studio 2015/17 subscription. If you don't have a paid edition, community edition is just fine. Finally, you'll need a reasonably powerful machine, something in the region of a 1 GHz processor, 512 MB of RAM, with 4. 5 GB disc free space.
Building a Simple Application Hi guys, this is Paul Mooney, and this is Building a Simple Application. In this module we're going to talk about the publish/subscribe design pattern, the underlying architecture leveraged by Event Hubs. We'll also build a simple end-to-end C# application that implements the publish/subscribe design pattern. Specifically, we'll implement the publisher, which transmits events to a broker, in our case an Event Hub, and a subscriber, which consumes the events. We'll also touch on more advanced concepts, such as parallel processing and partitioning.
Simulating the Internet of Things Hi guys, I'm Paul Mooney. You're very welcome to module 3 of my course, Getting Started with Azure Event Hubs with C#. This is simulating the Internet of Things. In this module we're going to simulate IoT devices. We'll transmit device telemetry through an Event Hub and we'll implement two unique consumer types, a fast consumer and a slow consumer. Our fast consumer will analyze telemetry in real-time. Our slow consumer, on the other hand, will store the same telemetry in a SQL database. We'll expand on the partitioned-design concept, comparing against its counterpart, the competing consumer design. We'll increase traffic volume, that is our IoT device telemetry traffic, to more realistic levels than our initial example in the previous module. And we'll monitor our Event Hub in the Azure portal. Finally, we'll scale our application to improve performance, increase throughput, and achieve a concept known as embarrassingly parallel design, where each application tier, that is the publisher, broker, and consumer can scale independently, but can also scale to complement adjacent tiers.
Event Hubs in Depth Hi this is Paul Mooney here, you're very welcome to my course, Getting Started with Azure Event Hubs with C#. This is Event Hubs in Depth. In this module we're going to examine Event Hub connections in more detail. We'll discuss AMQP, the Advanced Message Queuing Protocol, the underlying connection mechanism leveraged by Event Hubs comparing it to HTTPS. We'll talk about check-pointing and failover, discussing how our application will react to failure and how it will recover. We'll cover consumer groups and add multiple types of consumer to our IoT simulation, where each consumer will stream events from the same Event Hub independently of other consumers, and at their own pace. Specifically, we'll add two consumer types, a fast reader that will analyze events in real-time, and a slow reader that will persist each event to durable storage in the form of an Azure SQL database. Finally, we'll discuss a number of alternatives to Event Hub as a service, and discuss the pros and cons of each.