Configuring Amazon Web Services FlowLogs

Overview

The integration between Amazon Web Services (AWS) and Scrutinizer provides insight into network traffic destined for AWS such as:

  • Top AWS users
  • Top AWS applications
  • Overall traffic load of AWS hosted applications

After configuring Amazon Web Services Flow Log integration with Scrutinizer, the following AWS specific flow reports are available in Scrutinizer.

  • Action
  • Action with Interface
  • Action with Interface and Dst
  • Action with Interface and Src
  • Interface
  • Pair Interface
  • Pair Interface Action

Prerequisites

A minimum of Scrutinizer v18.16 is required for the configuration to successfully complete and for the AWS reports to be available.

Note

Version 18.16 is the last version of Scrutinizer that will support collecting logs via Kinesis

Required Information

Before starting the AWS configuration in Scrutinizer, the following in information needs to be collected.

  1. AWS S3 Bucket containing Flow Logs

    • The VPC(s) you want to monitor need to be configured to send Flow Logs to this S3 bucket.
    • You can ingest from multiple S3 buckets
    • Flow Logs will be deleted from the S3 bucket as Scrutinizer ingests them
  2. AWS ID and secret with full access permission to the S3 bucket containing your logs

  3. The region the S3 bucket is in?

AWS Configuration in Scrutinizer

  1. Navigate to Admin > Settings > AWS Flow Logs S3

  2. Click “Add” to create a new Flow Log source in Scrutinizer

    • Each row in the interface will appear as a separate flow “exporter” in the Scrutinizer reporting interface.
    • It will appear in the interface with the Name provided here
  3. Provide a unique name for this Flow Log source

  4. Select the collector that will communicate with AWS for this bucket

  5. Enter the bucket name, region, ID, and Secret

  6. Save the entry

Frequently Asked Questions

Q: Why are there gaps in the data in 1m and 5m intervals?
A: Amazon flow logs are updated every 10 minutes.
Q: It isn’t working, how can I see what is going on?
A: The AWS collection process runs under the umbrella of the plixer_flow_collector daemon.
- Check the collector log for errors.
- Go to the AWS interface and make sure you see flow logs in the configured bucket.
- Edit the S3 profile in Scrutinizer and use the “Test” button to make sure the configuration is correct.
Q: All commands ran successfully but I don’t see the exporter?
A: If no exporter appears after 10 minutes, verify that
the exporter is not disabled in Admin > Definitions > Manage Exporters.
Q: How do I add resources to the AWS instance?
A: Visit the page on AWS Adding Resources .
Q: Can I keep using Kinesis streaming for log collection?
A: This is the last version of Scrutinizer that will support collecting logs via Kinesis. The greater
flexibility in the new S3 configuration options along with cost savings to customers mean we no longer see
value in Kinesis integration. That was done before sending Flow Logs to S3 was an option.