Architecting Stream Processing Solutions Using Google Cloud Pub/Sub
Course info
Course info
Description
As data warehousing and analytics become more and more integrated into the business models of companies, the need for real-time analytics and data processing has grown. Stream processing has quickly gone from being nice-to-have to must-have. In this course, Architecting Stream Processing Solutions Using Google Cloud Pub/Sub, you will gain the ability to ingest and process streaming data on the Google Cloud Platform, including the ability to take snapshots and replay messages. First, you will learn the basics of a Publisher-Subscriber architecture. Publishers are apps that send out messages, these messages are organized into Topics. Topics are associated with Subscriptions, and Subscribers need to listen in on subscriptions. Each subscription is a message queue, and messages are held in that queue until at least one subscriber per subscription has acknowledged the message. This is why Pub/Sub is said to be a reliable messaging system. Next, you will discover how to create topics, as well as how to push and pull subscriptions. As their names would suggest, push and pull subscriptions differ in who controls the delivery of messages to the subscriber. Finally, you will explore how to leverage advanced features of Pub/Sub such as creating snapshots, and seeking to a specific timestamp, either in the past or in the future. You will also learn the precise semantics of creating snapshots and the implications of turning on the “retain acknowledged messages” option on a subscription. When you’re finished with this course, you will have the skills and knowledge of Google Cloud Pub/Sub needed to effectively and reliably process streaming data on the GCP.
About the author
An engineer and tinkerer, Vitthal has worked at Google, Credit Suisse, and Flipkart and studied at Stanford and INSEAD. He has worn many hats, each of which has involved writing code and building models. He is passionately devoted to his hobby of laughing at his own jokes.
More from the author
Section Introduction Transcripts
Course Overview
Hi! My name is Vitthal Srinivasan, and I'd like to welcome you to this course on Architecting Stream Processing Solutions Using Google Cloud Pub/Sub. A little bit about myself. I have master's degrees in Financial Math and Electrical Engineering from Stanford University and have previously worked in companies such as Google in Singapore and Credit Suisse in New York. I am now co-founder at Loonycorn, a studio for high-quality video content based in Bangalore, India. As data warehousing and analytics become more and more integrated into the business models of companies, the need for real-time analytics and processing has grown. Stream processing has quickly gone from being nice-to-have to must-have. In this course, you will gain the ability to ingest and process streaming data on the GCP including the ability to take snapshots and replay messages. First, you will learn the basics of a publisher subscriber architecture. Publishers are apps that send out messages. Those messages are organized into topics, which are associated with subscriptions. Subscribers in turn need to listen in on those subscriptions. Each subscription is a message queue. Messages are held in that queue until at least one subscriber has acknowledged them. This is what makes Pub-Sub a reliable messaging system. In addition, it offers global replication. We will learn all of these features of Pub-Sub. Next, you will discover how to create topics, as well as about push and pull subscriptions. As their names will suggest, push and pull subscriptions differ in who controls the delivery of messages. In a push subscription, the service sends out the messages to subscribers and keeps retrying until an acknowledgement has been received. In pull subscriptions, on the other hand, subscribers must provide a web hook, which is invoked when the message comes in. Finally, you will explore how to leverage advanced features of Pub-Sub such as creating snapshots and seeking to a specific timestamp either in the past or in the future. You will also learn the precise semantics of creating snapshots and the implications of turning on the retain acknowledged messages option in a subscription. When you're finished with this course, you will have the skills and the knowledge of Google Cloud Pub-Sub needed to effectively and reliably process streaming data on the GCP.