Analyzing Machine Data with Splunk

Splunk is one of the most used applications for analyzing unstructured data in the data center. This course will teach you the basics of setting up Splunk, writing Splunk queries, and running Splunk with Hadoop.
Course info
Rating
(58)
Level
Beginner
Updated
Nov 4, 2016
Duration
2h 38m
Table of contents
Description
Course info
Rating
(58)
Level
Beginner
Updated
Nov 4, 2016
Duration
2h 38m
Description

Today there are more devices generating data than ever before, but making sense of this data is nearly impossible. In this course, Analyzing Machine Data with Splunk, you'll learn foundational knowledge of/gain the ability to utilize Splunk to analyze your devices both in and outside of the data center. First, you will learn how to set up your own Splunk development environment. Next, you'll learn to use Splunk forwarders to send data back to your Splunk development environment. Finally, you'll learn how to move data from Hadoop Distributed File System (HDFS) into Splunk. When you're finished with this course, you will have the skills and knowledge of Splunk needed to analyze your machine data.

About the author
About the author

Thomas is a Senior Software Engineer and Certified ScrumMaster. He spends most of his time working with the Hortonwork Data Platform and Agile Coaching.

More from the author
Enterprise Skills in Hortonworks Data Platform
Intermediate
1h 37m
21 Sep 2018
Getting Started with HDFS
Beginner
2h 48m
16 Feb 2016
Section Introduction Transcripts
Section Introduction Transcripts

Course Overview
Hi everyone, my name is Thomas Henson, and welcome to my course, Analyzing Machine Data with Splunk. I'm a big data evangelist and Pluralsight author. Most of my time is spent in a data center helping customers learn how to use HADOOP for their data. Splunk has been called the easy button for machine data, and has about 12, 000 customer worldwide who are all analyzing machine data and unstructured data with Splunk. In this course, we're going to dive into Splunk to learn the basics. Some of the major topics that we're going to cover include, setting up your first Splunk development environment, learning to write Splunk queries with their SPL language, downloading applications from the Splunkbase, and then integrating those into your own environment, and running Splunk on Hadoop with Hunk. That's right, I'm going to show you how to integrate data from your Hadoop environment into your Splunk environment. By the end of this course, you'll know the basics on Splunk and how to analyze machine data in your Splunk environment, also you'll be able to identify new data points in your organizations that can be analyzed with Splunk, whether it be an IT security, DevOps, or even business analytics. I hope you'll join me on this journey to learn Splunk with Analyzing Machine Data Splunk course, here at Pluralsight.

Setting up the Splunk Environment
Hi, I'm Thomas Henson with Pluralsight. Now that we know what Splunk is and we know some of the use cases around when we would use Splunk, let's start getting our own environment set up so we can actually start looking at our own data. So in this module, we're actually going to look at how Splunk is licensed. There's free versions out there that we can use to test and use for our local environment, but what happens when we want to use it in our enterprise? We need to know those costs before going in so that we know how plan for when those workloads get larger. Then we'll also need to know where do we get Splunk from. We want to look at their documentation and the community portal in Splunk, so we know where we can find more information when we get stuck. Once we get a good feel of the Splunk website, and where we can find information, and look through the documentation, it's time to start installing Splunk in our own local environment. We'll just install it on our Windows box to start off with. Once we have it installed, we need to put some data in it so we can make sure that it's up and running, because in the rest of this course we're going to be using that environment to analyze data and to learn how to use Splunk to get the most out of it. Now let's learn more about how Splunk's licensed.

Basic Searching Techniques
Welcome back. In the previous module, we installed Splunk and loaded our application logs files. Now we're ready to get started learning the basic Splunking techniques. Just like if you were going into a cave for the first time for a spelunking adventure, we'd want to make sure you had the basics down. You wouldn't just run down and be like, hey I'm ready to go into a cave even though I've never done it. So get ready to learn the essentials, because you're going to need them. In this module, we're going to drill down into search and learn how to create reports and alerts based on our search results. The first thing we're going to cover is how to add more data, we can always use more data. Don't worry, it's going to be just as easy as the application data we loaded in module two. After we get more data, we'll begin our decent into Splunk search. We're going to cover a lot here to get you comfortable with Splunk search, and you'll even write your own queries using SPL. Once we get a grip on how to search in Splunk, we can start to talk about how to set up these reports in our Splunk environment. And if we're going to do reporting, we're also going to want to do alerts. So we're going to develop alerts based on reports that we have set up. Alerts are very powerful because it allow for you to monitor your data without having to stare at the screen all day. And even if your job revolves around using Splunk all day, you're going to have different work streams that you're working in, so you want to be able to monitor different parts of your data even if you're not directly looking at it. So now, let's get ready to add some more data.

Enterprise Splunk Architecture
Hi, welcome back to Analyzing Machine Data with Splunk. In this module, we're going to cover Enterprise features in Splunk. During this course we've been working through data on our local machine, but what happens when we want to analyze data that's not on our local machine? Chances are your organization has data that's located in many different locations, in different datacenters, maybe even different countries, and all of that data's not going to be on the same centralized local machine that you've been using. Now we've hinted about some of these features that will allow for Splunk to send machine data to a centralized Splunk instance. In this module, we're going to go through some of those features. Most of these features are going to be under the Enterprise license of Splunk. So this is not really a target if you're planning on using it for your Splunk home environment, but these are things you're going to want to know if you're implementing in your organization. The first thing we're going to cover is how to move these log files from a remote or distributed location. Next we're going to talk about the different forwarding options in Splunk, this is how we're going to be able to move data from one machine to another. After that, we're going to walk through some of the forwarding architecture in Splunk. Finally, we'll go through a demo where we're going to be installing a forwarder on a VM, and then forwarding those log files to our Splunk development environment we've been using. Now let's talk about how to move log files in Splunk.

Splunking for DevOps and Security
Welcome back to Analyzing Machine Data with Splunk. So far we've discussed Splunk from an IT operations perspective, but even though Splunk originally started out as an IT operations tool, its turned into a powerful analytics tool outside of the datacenter. With Splunk you're only limited by the data you can find or the questions that you dare to ask. In this module, we're going to shift away from IT operations to show some of the other popular use cases in Splunk. Let's look at how this module's going to be structured. First thing we're going to cover is DevOps. We're going to talk about what DevOps is and how Splunk can help you with DevOps in your organization. Interested in rapid development release cycles? Splunk can help. Next we're going to jump back into the datacenter, well sort of, we're going to talk about the strength of Splunk and security. We're going to touch on items inside the datacenter, but also security use cases outside of the datacenter. And finally, we're going to talk about other use cases in the enterprise. We're going to talk about how your chief financial officer or your chief marketing officer can actually use Splunk to solve some problems in their organization.

Application Development in Splunkbase
Hi, and welcome back to Analyzing Machine Data with Splunk. In this module, we're going to talk about application development in Splunkbase. Everything we've covered so far has been about how to use Splunk, and the use cases around Splunk, what happens when we want to do those use cases that are outside of normal Splunk use cases? Remember, in the last module we talked about a CFO using Splunk for cash flow analysis, or other outside use cases. When you heard that you might have been thinking, wait a minute I thought Splunk was all about just pulling in machine data and making it useful. Now are you telling me that's is framework, or it's almost like a dashboard? Well, kind of. In this module, we're going to outline how you can actually get those outside use cases, or create new ones and give back to the community. Let's talk about what we're going to walk through in this module. First thing we're going to do is we're going to talk about Splunkbase, and we're going to show you what it is. We're also going to talk about how other admins, developers, and Splunk users are using Splunkbase for those outside use cases. Next we'll walk through how to use Splunkbase and what that universe looks like. There's a couple different approaches that you can use to get those applications, and we'll walk through both of those. Next, we'll show you how you can create a simple application in Splunkbase. There are many different ways you can create applications in Splunkbase, it's all about what you're trying to do and how deep you want to go. We'll talk about setting up a Splunkbase environment, and the tools you need to create Splunk applications. Now let's talk about what Splunkbase is.

Splunking on Hadoop with Hunk
Hello, and welcome back to Analyzing Machine Data with Splunk. We're entering the last module in this course, and I'll be honest, this is the most exciting module for myself. There are two reasons for my excitement, first is my background in the Hadoop ecosystem, and second is Splunk's application Hunk. This allows developers the ability to analyze machine data from Splunk environments in Hadoop. Talk about integrating two worlds that are all about data, that's pretty awesome. You've probably heard a little bit about Hadoop, but if you haven't, or you just have an interest in learning more about it, this is going to be a great module for you. One of the things that we can do with Splunk and Hunk is imagine taking that machine data and overlaying it with non-machine generated data. For example, what if we wanted to track your server outage or power spikes to local weather data? Wouldn't that be cool? Why not? Not let's look and see what we're going to cover in this module. First, we're going to get a good understanding of Hadoop, and its two major components MapReduce, and the Hadoop Distributed File System, or HDFS. Now you won't have to be an expert in Hadoop, but you will want to have a solid base of MapReduce and HDFS, that's what we're going to cover in our first part. Next we're going to get hands on in the Hadoop Sandbox. I'll show you where to go get a sandbox, and I'll also show you how to navigate the HDFS system using the hdfs dfs commands. After becoming comfortable in Hadoop, I want to walk through some of the basics Splunk in Hadoop with Splunk's Hunk platform. Hunk is different from the application we've been using to analyze data in our current environment. Once we understand what Hunk is and how it differs from the traditional Splunk environment we've been using, I want to walk through setting up our own Hunk instance. It's a different download, there are some caveats around it, but we'll walk through that. And then finally, we're going to start analyzing data in HDFS with Hunk. We'll walk through a quick analysis of the data with Hunk and HDFS in a real world example. Now let's learn some more about Hadoop.