Configuring Amazon Web Services Flow Logs

The integration between Amazon Web Services (AWS) and Scrutinizer provides insight into network traffic destined for AWS, such as top AWS users,top AWS applications, as well as overall traffic load of AWS hosted applications.

After configuring Amazon Web Services Flow Log integration with Scrutinizer, the following AWS specific flow reports are available in Scrutinizer.

  • Action
  • Action with Interface
  • Action with Interface and Dst
  • Action with Interface and Src
  • Interface
  • Pair Interface
  • Pair Interface Action

Prerequisites

A minimum of Scrutinizer v18.16 is required for the configuration to successfully complete and for the AWS reports to become available.

The following information is required to configure AWS Flow Logs integration:

  1. AWS S3 Bucket containing Flow Logs
  • The VPC(s) you want to monitor need to be configured to send Flow Logs to this S3 bucket;
  • You can ingest from multiple S3 buckets;
  • Flow Logs will be deleted from the S3 bucket as Scrutinizer ingests them.
  1. AWS ID and secret with full access permission to the S3 bucket containing your logs.
  2. The region the S3 bucket is in.

AWS configuration in Scrutinizer

  1. Navigate to Admin > Settings > AWS Flow Logs S3
  2. Click “Add” to create a new Flow Log source in Scrutinizer. Each row in the interface will appear as a separate exporter in the Scrutinizer reporting interface.The exporter will appear in the interface with the name provided here.
  3. Provide a unique name for the Flow Log source.
  4. Select the collector that will communicate with AWS to receive data for this bucket.
  5. Enter the bucket name, region, ID, and Secret.
  6. Save the entry

Frequently Asked Questions

Q: Why are there gaps in the data in 1m and 5m intervals?
A: Amazon flow logs are updated every 10 minutes.
Q: It isn’t working, how can I see what is going on?
A: The AWS collection process runs under the umbrella of the plixer_flow_collector daemon.
- Check the collector log for errors.
- Go to the AWS interface and make sure you see flow logs in the configured bucket.
- Edit the S3 profile in Scrutinizer and use the “Test” button to make sure the configuration is correct.
Q: All commands ran successfully but I don’t see the exporter?
A: If no exporter appears after 10 minutes, verify that
the exporter is not disabled in Admin > Definitions > Manage Exporters.
Q: Can I keep using Kinesis streaming for log collection?
A: This is the last version of Scrutinizer that will support collecting logs via Kinesis. The greater
flexibility in the new S3 configuration options along with cost savings to customers mean we no longer see
value in Kinesis integration. That was done before sending Flow Logs to S3 was an option.