Oracle Cloud VCN logs

Oracle Cloud VCN logs#

When OCI Virtual Cloud Network (VCN) flow log ingestion is configured and enabled, Scrutinizer can monitor and report on traffic associated with specified Oracle Virtual Network Interface Cards (VNICs).

This section covers the prerequisites and setup/configuration steps for OCI VCN flow log ingestion.

Setting up the OCI flow log stream#

VCN flow log ingestion in Scrutinizer uses the OCI streaming service as the log data source. After being downloaded from a stream, the log data is forwarded to one or more specified collectors as regular flows.

To set up the flow log stream, follow these steps:

  1. Create a new stream in any stream pool to publish the flow logs to.

  2. Enable flow logs for the VCN, subnet, or VNICs.

  3. Configure a new service connector as follows:

    • Source: Compartment, log group, and name associated with the logs enabled in step 2.

    • Target: Compartment and name associated with the stream created in step 1.

  4. Create/provision an IAM group with the use stream-pull permission and add a user to the group (or select an existing user).

  5. Generate an API signing key pair for the user and download the private key as described here.

  6. Get the private key fingerprint using this command.

Verify that the flow logs are correctly being published to the stream, and then proceed to configuring Scrutinizer to download/ingest the log data.

Note

If the key pair was not generated via the OCI console, the public key will need to be uploaded for the user.

Configuring OCI VCN flow log ingestion in Scrutinizer#

Once the OCI stream has been successfully configured, it can be added to Scrutinizer as a flow log source as follows:

  1. In the Scrutinizer web interface, navigate to Admin > Integrations > Flow Log Ingestion.

  2. Click the + icon, and then select Oracle Cloud Streams in the tray.

  3. Enter the following details in the secondary tray:

    • Enter a name to identify the stream/source by.

    • Select the Scrutinizer servers to use as log downloader(s) and collector(s) for the stream (the primary reporter of a distributed cluster is not recommended for either role).

    • Enter the URL for the stream pool containing the flow log stream.

    • Enter the OCID of the stream receiving the VCN flow logs.

    • Enter the OCID of the OCI tenancy.

    • Enter the OCID of the user to be used to access the streams (must have the required permissions).

    • Enter the fingerprint of the private API signing key generated for the user.

    • Enter the passphrase associated with the private key (leave blank if no passphrase was used when the key was generated)

    • Enter the private key in PEM format.

    • Enter the name of the home region of the tenancy.

  4. Click the Save button to add the stream with the current settings.

Once added, the stream will be listed in the main Admin > Integrations > Flow Log Ingestion view under the configured name. An exporter associated with VCN will also be added to the device lists for Scrutinizer’s various functions (reports, network maps, etc.).

Note

  • After a stream configuration has been saved, click on the name assigned to it in the main view to open the settings tray, and use the Test button to confirm that Scrutinizer is able to establish a connection to the stream with the credentials entered.

  • To verify that an OCI VCN flow log source has been successfully added, look for an exporter whose hostname matches the VCN in the Explore > Exporters > By Exporters view or the Admin > Resources > Manage Exporters page (after ~1 hour).

  • Flow log ingestion processes are divided between the log downloader (downloads the flow logs from the stream) and the flow collector (collects and processes the downloaded logs). A different Scrutinizer server can be used for each role, and a single stream can have multiple downloaders and collectors.

Troubleshooting#

MFSNs and a buildup of log files in flow log source containers are indications that the rate of flow and/or log generation exceeds the capacity of the collector assigned to the flow log source.

The following are potential solutions for an overloaded collector:

  • If the collector is a VM, allocate additional resources (starting with CPU cores) to it.

  • If the collector is ingesting flow logs from only one source (bucket or container), distribute the logs across multiple sources, which can then be assigned to different collectors.

  • If the collector is ingesting flow logs from multiple sources, reassign sources across multiple collectors.

  • If the collector license has a flow rate limit, the license may need to be upgraded.

Note

  • In distributed deployments, it is recommended to start with a 1:1 pairing of sources and collectors.

  • The Unresourced - Enabled status in the Admin > Resources > Exporters view is another indication that flow log sources are being temporarily disabled/paused due to insufficient resources.

If the Admin > Resources > Exporters view does not list exporters that are associated with the virtual network(s) set up for flow ingestion, do the following:

  1. Navigate to Admin > Integrations > Flow Ingestion, open the configuration tray for the collector it was assigned to, and then use the Test button to verify that the correct details were entered.

    Note

    The Test button only checks if the communication with the data source works.

  2. Verify that flow logs are correctly being sent to the bucket or container.

  3. Check the collector log file in /home/plixer/scrutinizer/files/logs/ for errors.

  4. Check awss3_log.json (AWS), azure_log.json (Azure), or ocist_log.json for possible source-side issues.

Note

The Admin > Resources > Exporters view also displays exporters that have been disabled. Because each AWS, Azure, or OCI flow log source counts as an exporter, one or more sources may be disabled automatically (in last-in/first-out order) if the exporter count limit of the current license is reached.

Contents