AWS VPC flow logsΒΆ

With AWS VPC flow log ingestion enabled, Plixer Scrutinizer is able to report additional insights for network traffic destined for AWS, including top AWS users and applications, as well as traffic load generated by AWS-hosted applications.

The following AWS-flow-log-based Reports also become available under the Reports section:

  • Action

  • Action with Interface

  • Action with Interface and Dst

  • Action with Interface and Src

  • Availability Zones

  • Dst Service

  • Interface

  • Pair Interface

  • Pair Interface Action

  • Src Service

  • Src Service-Dst Service

  • Traffic Path

  • VPCs

Setting up S3 storage

Before configuring AWS flow log ingestion in Plixer Scrutinizer, one or more Amazon S3 storage buckets must be configured as follows:

  • The bucket(s) should have versioning disabled and be reserved for exclusive use by Plixer Scrutinizer.

  • The VPC(s) to be monitored should be set to send flow logs to the bucket(s) to be used.

    Hint

    Setting Maximum Aggregation Interval for VPC flow to 10 minutes reduces the processing load on the Plixer Scrutinizer Collector at the cost of longer update times and data spikes. For more granular reporting, choose 1-minute updates instead.

  • VPC flow logs must include the following fields:

    • log-status

    • vpc-id

    • interface-id

    • flow-direction

    Note

    When upgrading from older versions of Plixer Scrutinizer, it may be necessary to delete the old flow log configuration and create a new one that includes the interface-id and flow-direction fields.

  • To save time, buckets with a large volume of historical data can be cleared before they are added to Plixer Scrutinizer. This can be skipped to preserve the most recent 15 minutes of flow logs in the bucket(s).

Configuring AWS VPC flow log ingestion

To add an S3 bucket as a flow log ingestion source in Plixer Scrutinizer, follow these steps:

  1. Navigate to Admin > Integrations > Flow Log Ingestion in the web interface.

  2. Click the + button and select AWS VPC FlowLogs in the tray.

  3. In the secondary tray, fill in the fields with the following details:

    • A name to identify the bucket/source

    Hint

    The Amazon bucket name can also be used in the Name field to make it easier to distinguish between flow log sources.

    • The Log Downloader to assign to the bucket (dropdown)

    • The Collector to assign to the bucket (dropdown)

    • Name of bucket to be added

    • AWS region where the bucket is hosted

    • AWS IDs and Secrets with permissions granting full access to the bucket

  4. Click the Test button to verify that Plixer Scrutinizer is able to collect flow logs from the bucket.

  5. Click the Save button to add the S3 bucket with the current settings.

Once added, the bucket will be listed in the main Admin > Integrations > Flow Log Ingestion view under the configured name. Clicking a source name in this view will open a configuration tray, where its settings can be edited.

Plixer Scrutinizer will continuously monitor the bucket to collect new logs and delete files that have been ingested.

Note

The Log Downloader setting allows to set one collector to download logs from the S3 bucket, and export the logs within itself or send the logs to another collector. On the other hand, the Collector receives flows from the exporter.

Hint

To access bulk actions/operations in the main view, select one or more sources using the checkboxes and click the Bulk Actions button.

Note

After a bucket is first added, the most recent 15 minutes of flow logs are collected, and all older logs are deleted. Plixer Scrutinizer will then continue to collect and delete flow logs as normal.

Enabling role-based IAM for AWS deployments

Role-based IAM can be enabled for Plixer Scrutinizer AMI instances by ticking the checkbox in the configuration tray. The role assigned to the EC2 instance should be provisioned with the following permissions:

{ "Version": "2012-10-17",
  "Statement": \[
             { "Sid": "VisualEditor0",
               "Effect": "Allow",
               "Action": \[ "s3:GetObject", "s3:DeleteObject" \],
               "Resource": \[ "arn:aws:s3:::<S3BUCKET>/\*" \]
             },
             { "Sid": "VisualEditor1",
               "Effect": "Allow",
               "Action": "s3:\*",
               "Resource": "arn:aws:s3:::<S3_BUCKET_NAME>"
             }
  \]
}

Note

Role based authentication is only available when all Log Downloaders are hosted in AWS.

Importing AWS entity descriptions

To allow description reporting and filtering by AWS entity identifiers (interface-id, vpc-id, etc.) directly in the Plixer Scrutinizer UI, follow these steps:

  1. Provision the user or IAM role with the following additional permissions:

    ec2:DescribeInstances
ec2:DescribeSubnets
ec2:DescribeVpcs
ec2:DescribeNetworkInterfaces

  2. Start an SSH session with the Plixer Scrutinizer server (or the primary Reporter in distributed deployments), and run the following command via the scrut_util interactive CLI:

    SCRUTINIZER> awssync
      AWS entities synced!

Once entity descriptions have been synced, AWS entity identifiers will automatically be replaced with their descriptions whenever an AWS-specific Report is run.

Note

The awssync task is also automatically run hourly.

For assistance with any issues, consult the troubleshooting guide or contact Plixer Technical Support.