Machine learning#
Through the Plixer ML Engine, Scrutinizer is able to leverage advanced AI, machine learning, and deep learning technologies to provide real-time anomaly detection and reporting.
Note
To learn more about ML Engine licensing options, contact Plixer Technical Support.
Once set up, the engine enables the following functions in Scrutinizer:
Anomaly recognition#
As it ingests data through Scrutinizer, the Plixer ML Engine builds behavior models based on the current inclusion/exclusion rules and dimensions configured. These models encompass all network activity, including applications and communications to/from external hosts.
When a sufficient volume of data has been ingested, the ML Engine is able to use models that represent typical, legitimate activity patterns as a baseline and recognize deviations that may indicate threats and other anomalies. Deviations that exceed the specified thresholds are then reported as alarms and events via the Scrutinizer web interface.
The ML Engine’s detection and reporting functions can be adapted to any type of enterprise network by defining the inclusions, dimensions, and sensitivity/threshold values that best suit an organization’s environment.
Malware detection#
Because irregular behavior by itself is only indicative of a possible threat and may or may not need remediation, the Plixer ML Engine utilizes additional pre-trained ML models to classify the anomalies it observes through Scrutinizer and report whether the anomaly actually constitutes malicious activity.
Note
The pre-trained models packaged with the ML Engine are IP-agnostic and allow Scrutinizer to alert users to potential threats without needing previously known domain or IP-based signatures.
This classification process is divided into four steps:
The engine ingests flow data containing anomalous traffic streamed from Scrutinizer.
The data is preprocessed by the ML Engine into feature vectors that can be used by the pre-trained ML models.
The resulting data is used as the input for the different pre-trained ML models.
Each ML model outputs a probability score, which represents the likelihood that the anomaly observed constitutes malicious behavior.
Once probability scores have been obtained, Scrutinizer compares them to a user-configurable threshold to determine whether or not an alarm should be generated for the host.
Note
The ML Engine regularly checks for updates that may include newer versions of the pre-trained ML models it uses.
Continuous learning#
To combat the growing sophistication of modern threats, the Plixer ML Engine is also equipped with deep learning capabilities that take advantage of the large quantities of flow data collected by Scrutinizer to identify complex behavioral patterns and enable advanced features, such as link prediction.
The ML Engine’s deep learning-based threat detection processes can be summarized in the following steps:
Flow data collected by Scrutinizer is forwarded to a datastore module for preprocessing.
Once preprocessed, the data is forwarded to the engine, which runs it through a multi-layered neural network designed to discover behavioral patterns in the data.
The neural network uses the patterns to learn how devices on the network typically interact with each other.
After an anomaly has been detected and classified, the system uses link detection to analyze the device’s interactions with other devices on the network.
If the deviation from what the ML Engine has learned as typical behavior exceeds a set threshold, the device involved is added to an endpoint monitoring protocol.
Devices that have been flagged for further monitoring will trigger alarms under Scrutinizer’s alarm monitor, allowing security teams to decide whether immediate action is necessary.