Work with AWS VPC Flow Logs for Network Monitoring

1. Create a VPC flow log to S3, setting the Maximum aggregation interval to 1 minute and the S3 bucket ARN to the VPC Flow Logs S3 Bucket ARN found in the lab credentials or obtained via S3.
2. Create a CloudWatch log group. Name it VPCFlowLogs.
3. Create a VPC flow log to CloudWatch using the IAM role provided for the lab (the one with DeliverVPCFlowLogsRole in the name) and the CloudWatch log group you created.
4. Generate some traffic for the flow logs.

1. Create a CloudWatch log metric filter named dest-port-22-rejects. For the Metric namespace, use VPC Flow Logs; for Metric name, use SSH Rejects; and for Metric value, enter 1.

   • Use the below filter pattern:

     [version, account, eni, source, destination, srcport, destport="22", protocol="6", packets, bytes, windowstart, windowend, action="REJECT", flowlogstatus]
     

   • Use the following for the log data:

     2 086112738802 eni-0d5d75b41f9befe9e 61.177.172.128 172.31.83.158 39611 22 6 1 40 1563108188 1563108227 REJECT OK
     2 086112738802 eni-0d5d75b41f9befe9e 182.68.238.8 172.31.83.158 42227 22 6 1 44 1563109030 1563109067 REJECT OK
     2 086112738802 eni-0d5d75b41f9befe9e 42.171.23.181 172.31.83.158 52417 22 6 24 4065 1563191069 1563191121 ACCEPT OK
     2 086112738802 eni-0d5d75b41f9befe9e 61.177.172.128 172.31.83.158 39611 80 6 1 40 1563108188 1563108227 REJECT OK
     

2. Create an alarm called SSH-rejects based on the metric filter.

Note: There may be a New Console Experience button in the top left of the CloudWatch console. These instructions are designed for this new layout, so ensure this is toggled on if it is present.

Apply the Top 20 source IP addresses with highest number of rejected requests sample query.

1. Create the Athena table:

   CREATE EXTERNAL TABLE IF NOT EXISTS default.vpc_flow_logs (
     version int,
     account string,
     interfaceid string,
     sourceaddress string,
     destinationaddress string,
     sourceport int,
     destinationport int,
     protocol int,
     numpackets int,
     numbytes bigint,
     starttime int,
     endtime int,
     action string,
     logstatus string
   )  
   PARTITIONED BY (dt string)
   ROW FORMAT DELIMITED
   FIELDS TERMINATED BY ' '
   LOCATION 's3://{your_log_bucket}/AWSLogs/{account_id}/vpcflowlogs/us-east-1/'
   TBLPROPERTIES ("skip.header.line.count"="1");
   

2. Create partitions to be able to read the data:

   ALTER TABLE default.vpc_flow_logs
   ADD PARTITION (dt='YYYY-MM-DD')
   location 's3://{your_log_bucket}/AWSLogs/{account_id}/vpcflowlogs/us-east-1/YYYY/MM/DD';
   

3. Run the following query in a new query window:

   SELECT day_of_week(from_iso8601_timestamp(dt)) AS
     day,
     dt,
     interfaceid,
     sourceaddress,
     destinationport,
     action,
     protocol
   FROM vpc_flow_logs
   WHERE action = 'REJECT' AND protocol = 6
   order by sourceaddress
   LIMIT 100;