Google Cloud Platform VPC flow logsΒΆ

With GCP Virtual Private Cloud (VPC) flow log ingestion enabled, Plixer Scrutinizer is able to monitor and report on traffic data associated with GCP VPC assets.

This section covers the prerequisites and setup/configuration steps for GCP VPC flow log ingestion.

Setting up the Google Cloud Pub/Sub topic and subscription

Plixer Scrutinizer uses the GCP Pub/Sub messaging service as an ingestion source for VPC flow logs.

To set up the Pub/Sub topic that will receive the log entries to be ingested, follow these steps:

Note

To ensure seamless access between components/services, it is highly recommended to set everything up under the project where flow logs will originate.

  1. Enable and configure VPC Flow Logs for the target resources.

  2. Next, navigate to the Pub/Sub Topics page and create a new topic with message retention enabled and set to at least one hour (other topic settings can be configured as desired).

  3. After the topic has been created, go to the Subscriptions page and create a pull subscription for the new topic (note the Subscription ID for later use).

  4. Next, go to the Log Router page and create a sink to route the log entries to the newly created topic and configure any inclusion/exclusion filters necessary.

  5. After adding the Pub/Sub topic as a sink, navigate to the Service Accounts page and select a service account associated with the sink/topic.

  6. Under the Keys tab, click the Add Key button and select JSON to download a file containing the credentials required to subscribe to the Pub/Sub topic.

Once the above steps have been completed, verify that log entries are being correctly routed to the Pub/Sub topic, and then proceed to configuring ingestion in Plixer Scrutinizer.

Configuring GCP VPC flow log ingestion in Plixer Scrutinizer

Once the Pub/Sub topic is receiving log entries and the subscription has been set up, it can be added to Plixer Scrutinizer as follows:

  1. Navigate to Admin > Integrations > Flow Log Ingestion in the web interface.

  2. Click the + icon, and then select Google Cloud Platform in the tray.

  3. In the secondary tray, configure the subscription details as follows:

    • Enter a name to identify the source by.

    • Select the Plixer Scrutinizer servers to use as the log downloader(s) and collector(s) (the primary reporter of a distributed cluster is not recommended for either role).

    • Enter the GCP project ID associated with the topic subscription.

    • Enter the subscription name/ID used.

    • Enter/paste the contents of the service account key JSON file.

  4. Click the Save button to add the subscription with the current settings.

Note

  • After a subscription configuration has been saved, click on the name assigned to it in the main view to open the settings tray, and use the Test button to confirm that Plixer Scrutinizer is able to establish a connection with the credentials entered.

  • To verify that an GCP VPC flow log source has been successfully added, look for an exporter whose hostname matches the GCP VPC in the Explore > Exporters > By Exporters view or the Admin > Resources > Manage Exporters page (after ~1 hour).

  • Flow log ingestion processes are divided between the log downloader (downloads the flow logs through the topic subscription) and the flow collector (collects and processes the downloaded logs). A different Plixer Scrutinizer server can be used for each role, and a single subscription can have multiple downloaders and collectors.