Flow Analytics

Scrutinizer includes a library of flow analytics (FA) algorithms, which are applied to all incoming flow data. This allows the system to provide additional traffic-based insights and report activity typically associated with threats to a network.

Note

To learn more about individual algorithms, see this appendix section.

Configuring Flow Analytics

To enable FA-based functions, several configuration steps must be completed after Scrutinizer has been deployed and set up.

This process helps ensure that Scrutinizer is fully adapted to an organization’s NDR requirements.

Enabling/disabling algorithms

Because Scrutinizer is designed to support the full spectrum of enterprise applications, it may include FA algorithms that may not apply to certain network configurations. This will be based on the devices and elements present on the network, the types of flow data available, and/or organizational IT policies.

As part of optimizing the system’s monitoring and reporting functions, all unnecessary FA algorithms should be disabled. This includes algorithms that:

• Only benefit devices or elements that are not present on the network

• Require flow data that is not being sent by devices on the network

• Target traffic or patterns that are made irrelevant by the organization’s IT policies

The Admin > Alarm Monitor > Flow Analytics Algorithms page lists the current state of all FA algorithms (default: enabled).

Note

Most FA algorithms can also be tuned through additional settings, allowing them to be adapted to specific monitoring and detection requirements.

Disabling FA algorithms

To disable an algorithm, click on it to open the configuration tray and use the toggle. The algorithm can also be re-enabled this way at any time.

Multiple algorithms can also be disabled or enabled as a bulk action when one or more algorithms are selected.

Adding exporters

Scrutinizer selectively applies Flow Analytics to incoming flow data, based on the exporters defined for each algorithm.

To activate the system’s FA-based functions, exporters must first be added to the enabled algorithms.

Security groups

Scrutinizer security groups are user-defined groups of exporters to which the same set of FA algorithms are applied. Security groups allow the exporter lists for all FA algorithms to be fully populated without the need to manually configure individual algorithms. Exporters can be added to security groups via the Admin > Alarm Monitor > Security Groups page.

If Flow Analytics is being configured for the first time, exporters should be added to the Core Exporters and Edge Exporters a few at a time. This will limit the volume of alarms that may need to be checked when testing Flow Analytics settings via the Alarm Monitor page.

The Security Groups view also allows new groups to be added and the settings for existing groups to be modified.

Hint

The default Firewalls, Core Exporters, Edge Exporters, and Defender Probes security groups are configured with FA algorithms based on the recommended exporter assignments.

Adding exporters individually

For more granular control over exporter-to-algorithm assignment, exporters can also be added to FA algorithms via the configuration tray of the Admin > Alarm Monitor> Flow Analytics Algorithms page.

Because alarm-triggering algorithms will only be triggered when the target is an internal address, public IP addresses must be defined as part of an IP group for them to be considered part of the protected network. For internal-to-internal and internal-to-external monitoring, core routers should be added to the relevant algorithms. For monitoring public assets, the edge routers of the relevant IP groups should be added to the algorithms.

Defining exclusions

To avoid unnecessary alarms and excessive processing load on the system, certain devices or traffic should be excluded from monitoring by specific FA algorithms.

Scrutinizer’s factory configuration includes four IP groups that are defined as exclusions under the appropriate algorithms:

• DNS servers

• Public WiFi

• Network Scanners

• SNMP Pollers

These IP groups should be populated with the correct exporters to optimize Flow Analytics monitoring and reporting.

Adding exclusions to an FA algorithm

FA algorithms can also be configured with additional exclusions beyond those defined under the above-mentioned IP groups. This is done via the algorithm’s configuration tray from the Admin > Alarm Monitor> Flow Analytics Algorithms page.

Exclusions can be defined by IP address, IP range, subnet, domain (via reverse DNS), or IP group.

Hint

The default IP group exclusions for an algorithm are also displayed under the Exclusions section of the configuration tray.

Additional settings

Scrutinizer’s flow analytics functions can be further adapted to more unique network and/or security requirements through the configuration options below.

Global settings

The following global settings (Admin > Settings > Flow Analytics Settings) can be used to enable or configure additional FA-based features:

Setting	Description
Auto-Enable Defender	When checked, FlowPro Defender is automatically enabled for algorithms that support it.
Jitter by Interface	Sets the variation in packet delay due to queueing, contention, and/or serialization (Default: 80 ms); Also used for record highlighting in Status reports
Latency	Sets the latency value used for record highlighting in Status reports (Default: 75 ms)
Share Violations	When checked, allows the system to share details of cyber attacks coming from Internet IP addresses with the Plixer Security Team (May require firewall permissions); This information is used to further improve the global host reputation list. No internal addresses will be shared.
Top Algorithm Devices	Controls whether Top X FA algorithms are applied to all exporters or need to be configured individually

Algorithm settings

In addition to inclusions and exclusions, most FA algorithms have additional settings that control how they are applied to collected flow data. These settings include thresholds for adjusting detection sensitivity and traffic directionality inclusion/exclusion options.

For a full list of algorithm settings, see this table.

Custom reputation lists

The Host Reputation FA algorithm is capable of using custom lists in conjunction with Scrutinizer’s default host reputation lists. When a host in any reputation list becomes the target of traffic, its address is reported in event is reported in event artifacts under the Host Reputation alarm policy.

To import a list of IP addresses as a custom host reputation list, follow these steps:

1. Add the hosts to a file, using one line for each IP address.

   Example:

   10.1.1.1
   10.1.1.2
   10.1.1.3
   

2. Save the file with a .import extension. (e.g., custom_threats.import)

3. Move the file to the \scrutinizer\files\threats\ directory.

The file is imported hourly, at the same time that threat lists are updated.

Hint

To manually run the file import operation, run the following:

scrut_util --downloadhostreputationlists

Reporting options

Each alarm-triggering FA algorithm is associated with one or more alarm policies, under which anomalies and other insights are reported via the Scrutinizer alarm monitor. The settings for these alarm policies can also be modified to change the reporting behavior for the individual algorithms.

To learn more about alarm policies and the Scrutinizer alarm monitor, see the alarms and events section of this manual.

Notification profiles

To forward the details of alarms and events reported by an FA algorithm to one or more users or external systems, at least one notification profile must be created and assigned to the corresponding alarm policy.

To learn more about notification profiles, see the alarm notifications section.

FA dashboard gadgets

Certain gadgets that can be added to Scrutinizer dashboards rely on one or more FA algorithms for the data they report.

These gadgets require no further configuration and can be added to any dashboard as long as the corresponding algorithms have been enabled and correctly configured.

Hint

The Flow Analytics Summary gadget can be used to troubleshoot algorithm configurations. If there are algorithms that are taking longer than 5 minutes to run, check that the correct exporters have been added.

To learn more about dashboards and gadgets, see the dashboards topic of this documentation.

Testing and tuning

To ensure that flow analytics is properly configured, testing the various definitions, settings, and enabled features is strongly recommended. This can be accomplished by checking what alarms and events are being reported in the Alarm Monitor views.

When setting up flow analytics for the first time, the following process is recommended:

1. Navigate to Admin > Definitions > IP Groups and populate the DNS Servers, Public WiFi, Network Scanners, and SNMP Pollers groups to define basic exclusions for FA algorithms.

2. Review the list of FA algorithms in the Admin > Alarm Monitor > Flow Analytics Configuration and disable any algorithms that are irrelevant.

3. Define additional exclusions for individual algorithms in their configuration trays as needed.

4. Navigate to Admin > Alarm Monitor > Security Groups and add several exporters each to the Core exporters and Edge exporters security groups.

Once the first batch of exporters has been added, review the Alarm Monitor views to verify that alarms and events are being reported correctly. Afterwards, repeat Step 4 of the process and continue checking alarms and events until all exporters have been added to security groups.

Note

• If there are continuous or unnecessary alarms or events being reported, it may also be necessary to define additional exclusions for certain algorithms.

• To enhance response/resolution workflows, create one or more notification profiles and associate them with the appropriate alarm policies.

Further tuning

After the initial setup and testing have been completed, flow analytics functions can be further adapted to an environments monitoring and detection requirements through global and individual algorithm settings.