Author avatar

Aaron Rosenmund

Setup NetFlow Monitoring with Elasticsearch SIEM

Aaron Rosenmund

  • Jan 21, 2020
  • 11 Min read
  • 4,181 Views
  • Jan 21, 2020
  • 11 Min read
  • 4,181 Views
Security
Elasticsearch

Introduction

In modern data centers, many of the services you operate your businesses with are virtualized, and taping or spanning ports on physical infrastructure is not always an option. It is, in fact, not an option at all in cloud environments. However, NetFlow 1 through IPFIX(v10) is a standard format of session data from virtual and non-virtual switches located in the datacenter, vSphere, or cloud environments. Leveraging Netflow as a data source for security provides you the opportunity to have the least impact on the operations of the company while gaining visibility into the virtual network layer that is otherwise obscured.

Technology

This guide uses the 7.4 version of Filebeats; you should, of course, use whatever version you Elastic SIEM is currently operating. Not all of the dashboards and fields are backwards compatible, and mix-matching Filebeat versions cause issues with the indexes used for visualizations.

Verify Elastic SIEM Configuration

If you followed along with the Setting Up Elasticsearch for the Elastic SIEM Guide and the subsequent Kibana installation and configuration, you have specific IP addresses that are exposed in your environment, waiting to receive information. Before you get started, make sure you collect and record that information for further use. For this guide, the IP:Port endpoint for the Elasticsearch node is 192.168.218.139:9200, and for Kibana is 192.168.218.139:5601.

Filebeat for Netflow Collection

Filebeat is one of the most versatile of the beat family, with a long list of modules supporting the shipping of data to an Elastic stack.

Filebeat acts as a collector rather than a shipper for NetFlow logs, so you are setting it up to receive the NetFlow logs from your various sources. That being so, you can install Filebeat on whatever platform you wish as long as it is configured to send the data it collects and parses to the appropriate Kibana and Elastic nodes.

Download and Install Filebeat

Download the correct architecture installation file for Filebeat from Elastic.

For this guide, follow along installing the rpm package for CentOS 7.

From the command line, use curl to download the rpm package and install with the rpm command.

1
2
[user]$ curl -L -O https://artifacts.Elastic.co/downloads/beats/Filebeat/Filebeat-7.4.0-x86_64.rpm
[user]$ sudo rpm -vi Filebeat-7.4.0-x86_64.rpm

Filebeat-install

Filebeat installs in the /etc/filbeat folder and, just like the other Elasticsearch products, requires some configuration and file modification to get going.

Filebeat-install

Configure Filebeat

Edit the config file to point to the previously configured Elasticsearch and Kibana instances with nano.

1
[user]$ sudo nano /etc/Filebeat/filbeat.yml

In the Kibana section, find the host variable.

Filebeat-Kibanabefore

Change it to the exposed external IP address configured for the Kibana service. If you install Filebeats on the same device as the Kibana node, it is not listening on localhost, but instead on the external IP configured with 0.0.0.0.

Filebeat-Kibana-local

Then find Elasticsearch in the outputs section and change the hosts variable.

Filebeat-Elasticbefore

Set it to the external IP address configured for the Elasticsearch service. In a similar fashion, if the Elastic node is on the same device interface that Filebeat is installed on, configure it with 0.0.0.0

Filebeat-Elastic-local

Exit nano, saving the config with ctrl+x, y to save changes, and enter to write to the existing filename "Filebeat.yml.

Configure Netflow Module

Now it is time to enable and configure the Netflow module and run the Filebeat setup to connect to the Elasticsearch stack and upload index patterns and dashboards.

Note: This input supports NetFlow versions 1, 5, 6, 7, 8 and 9, as well as IPFIX. For NetFlow versions older than 9, fields are mapped automatically to NetFlow v9.

First, enable the NetFlow module.

1
[user]$ sudo Filebeat modules enable netflow

Find the netflow.yml configuration located in the modules.d directory inside the /etc/Filebeat install location. Notice that it is the only file without the appending .disabled designator.

Filebeat-modules-directory

Edit this configuration file with nano.

1
[user]$ sudo nano /etc/Filebeat/modules.d/netflow.yml

Filebeat-netflow-config-before

To ensure the Filebeat service is listening on an externally available port, change the localhost to 0.0.0.0 and leave the default 2055 port.

Filebeat-netflow-config-after

Exit nano, saving the config with ctrl+x, y to save changes, and enter to write to the existing filename "netflow.yml.

Test the configuration files.

1
[user]$ sudo Filebeat test config

Filebeat-test-config

If your configuration checks out, run Filebeat initial setup to load dashboards, templates, indexes, and further validate configuration settings.

Note: If you are running Filebeat 7.4 with a lower version Elasticsearch, the dashboards are not all compatible, notably the Filebeat-aws-s3access-overview.json dashboard, and it throws an error. Either upgrade Elastic SIEM or navigate to the /usr/share/Filebeat/Kibana/7/dashboard directory and move that file to a different location. Then re-run Filebeat setup.

1
[user]$ sudo Filebeat setup -e

Filebeat-setup-success

Configure Firewall

To allow the UDP traffic from the NetFlow sources into the device running Filebeats, you have to create a firewall rule for that port and protocol by running the following commands.

1
2
3
[user]$ firewall-cmd --permanent --add-port 2055/udp
[user]$ firewall-cmd --reload
[user]$ firewall-cmd --list-all

firewall-config

Run File Beats

Then run Filebeats using the -e flag to output the activity to the console. Again, just to make sure everything is working properly.

1
[user]$ Filebeat -e

Filebeat now sits and listen on the 2055 UDP port for a NetFlow source to send it data.

If you are comfortable that everything is working properly, you can run the Filebeats service, and the configurations still apply.

1
2
[user]$ sudo systemctl start Filebeat -e
``

Configure Netflow Source

Now you need to configure your Netflow source. Many products have NetFlow output that you can configure to point towards the Filebeat NetFlow collector. For this guide, you are going to walk through setting up a virtual distributed switch on vSphere vCenter and providing a window into the virtual traffic management without impact on business resources.

Other potential sources include Cisco devices (it is a proprietary Cisco standard after all), Juniper devices, VMware Virtual Networking, AWS VPC, and more!

First, login to vCenter and navigate to the networking section. Select the virtual distributed switch you want to configure and choose the Netflow section, and then click edit configuration.

distributed-switch-config-zero

Type the Collector IP address and Collector port of the NetFlow collector. You can contact the NetFlow collector by IPv4 or IPv6 address. If you are going to set up more than one change, the identifier accordingly, and leave the switch IP blank. Leave all other settings default.

confifured-netflow-switch-after

Next, select the uplinks for the VDS, properties, and click edit. Select the monitoring section and use the drop-down to change the NetFlow setting to enabled.

configure-netflow-uplinks

Next, select the port group for the VDS, properties, and click edit. Select the monitoring section and use the drop-down to change the NetFlow setting to enabled.

netflow-configure-portgroup

Netflow in Elastic SIEM

Now it is time to check back in the Elastic SIEM by browsing to the http://x.x.x.x:5601 address and clicking the lock icon that represents the SIEM overview section. Now you should see events posting in NetFlow, as long as there are VM's using the distributed switch.

siem-netflow-events

Click on view network, and you get much of the same data that is populated form packet flow except directly from a NetFlow source.

siem-netflow-data

Pivot to the dashboards. In the search bar type "Netflow". You see a list of ten available dashboards for the Filebeat NetFlow modules.

siem-netflow-dashboards

Happy hunting!

11