Baselining

Overview

Baselining is analyzing the performance of the network by comparing current performance to historical data (also known as the “baseline”). For example, when measuring the current traffic from an interface, alerts will be sent if any traffic exceeds the baseline. Plixer Scrutinizer can be configured to baseline any NetFlow or IPFIX element and alert. By default, there are several baselines enabled and configured to collect data whenever a new exporter starts exporting flows.

  • ingressInterface and octetDeltaCount
  • ingressInterface and packetDeltaCount
  • egressInterface and octetDeltaCount
  • egressInterface and packetDeltaCount
  • applicationTag (NBAR) and octetDeltaCount
  • applicationTag (NBAR) and packetDeltaCount

Baselining can be globally enabled or disabled using the Flow Analytics Configuration Manager (Admin Tab -> Settings -> FA Configuration).

Note

Baselining is disabled by default.

Using the Interactive scrut_util command, baselining behavior can be modified whenever that baseline exceeds a higher than normal traffic pattern.

Baseline reports

You can monitor the progress of the baselining function by ruiing baseline reports:

  1. In the Status tab, under Device Explorer, filter on Scrutinizer.
  2. Click on Scrutinizer.
  3. Navigate to Reports > Baselines.
  4. This will provide a list of available auto-created baseline reports on your server so that you can create a report to view.

How baselining works

Once an hour, Plixer Scrutinizer will analyze historical data and tally network performance of the configured elements for baselining. It will skip exporters that do not have the specified baseline elements in any of the templates exported. Once the data for recent data is calculated, it is compared against the historical baseline. Alerts will be sent if one of the following occur:

  • (BLACK) There is no historical baseline
  • (RED) recent data exceeds the maximum value of any historical baseline value
  • (ORANGE) recent data exceeds two standard deviations over the average baseline
  • (YELLOW) recent data exceeds one standard deviation over the average baseline

No alert is generated if the recent data exceeds the average baseline but is less than one standard deviation over the average baseline. Baselining will collect data for two weeks (by default) before alerts are generated.

Lastly, a weighted average (by default 100% of the value) is applied to the most recent baselining data and stored to compare against the next baseline collection cycle. The weighted average can be modified to allow new data to have a more or less significant role when comparing future baseline values.

Configuring baselines via the interactive scrut_util

To enable the interactive scrut_util utility, run:

/home/plixer/scrutinizer/bin/scrut_util

This will open the Plixer Scrutinizer prompt:

SCRUTINIZER>

For more information on using this utility, reference the interactive scrut_util section.

Adding/removing default exporter baselines

Run the following interactive scrut_util command, default baselines to add/remove baselines from a specified exporter.

Warning

These commands will alter the behavior of Plixer Scrutinizer baseline functionality. Please use with caution.

Adding
enable baseline <exporter_ip> default
Removing
disable baseline <exporter_ip>

Adding custom baselines

Every install is unique. Before deploying additional baselines, contact Plixer support to assist in planning, sizing, and deploying baselines in Plixer Scrutinizer.

The following interactive scrut_util command can be used to add custom (manual) baselines to a specified exporter.

Warning

This command will alter the behavior of Plixer Scrutinizer baseline functionality. Please use with caution.

SCRUTINIZER> enable baseline <exporter_ip> manual <pri_element[,sec_element]> <element> <AVG|COUNT|MIN|MAX|STD|SUM> <dailyhr|busday|sameday>

This ‘enable baseline’ command enables custom baselines (manual) based on elements from NetFlow and IPFIX templates.

Baselining has several parameters available to customize the specific baseline data to collect with the ‘manual’ option.

  • Replace <exporter_ip> with the exporter to collect the specified baseline data.

  • <pri_element> and (optionally <sec_element>) specify which IPFIX elements will be part of this baseline. This is how the data is grouped and calculated.

  • <element> must be an IPFIX element with a numeric value that the <pri_element> and/or <sec_element> will be baselined on.

    • For example: to collect data on source addresses and bytes, the <pri_element> would be sourceipv4address and the <element> would be octetdeltacount.
    • Another example: To collect data off multiple elements such as source address and applicationtag based on packets; the <pri_element> would be sourceipv4address, the <sec_element> would be applicationtag, and the <element> would be packetdeltacount.
    • To find the numeric values for IPFIX elements in Plixer Scrutinizer, go to Status > System > Templates. Drill in on the Plixer ID to see all details regarding the elements in that template.

  • <AVG|COUNT|MIN|MAX|STD|SUM> are options that are used to calculate how to measure the <element>

    • AVG = Average
    • COUNT = Flow Count
    • MIN = Minimum Value
    • MAX = Maximum value
    • STD = Standard Deviation
    • SUM = Sum

  • Lastly, there are several ways baselines can be compared:

    • dailyhr = same time frame (e.g. 1a - 2a) for each day
    • busday = same time frame (e.g. 1a - 2a) for each business day, skipping weekends
    • sameday = same day and time each week (e.g. 1a - 2a Mondays)

When baselining IP addresses, IP Groups must be configured with the ranges and subnets of addresses to be included in the baseline. This reduces a number of false positives as by excluding addresses that may talk once.

Monitoring baseline processing

To get the task ID for the baseline task (to be used in the ‘check task’ command below), run the scrut_util <configuring_baselines>`command:

show task baseline

Displays information regarding the baseline task.

Example:

SCRUTINIZER> show task baseline


+-----------+---------+----------------+--------------------------+
| TASK_NAME | TASK_ID | EXECUTABLE     | ARGUMENTS                |
+-----------+---------+----------------+--------------------------+
| baseline  | 224     | scrut_util     | ["--collect","baseline"] |
+-----------+---------+----------------+--------------------------+
1 Result(s) Found

Done (0.010545 seconds)

The ‘check task’ command provides information on the baseline task processing.

check task <id>

Checks the execution times and error codes for the baseline task. The task id is available by using the show task baseline command.

Example:

SCRUTINIZER> check task 224


+---------------------+----------+------------+----------------------------------------------+
| START_TIME          | RUN_TIME | RETURN_VAL | INFO_STR                                     |
+---------------------+----------+------------+----------------------------------------------+
| 2019-01-23 13:10:00 | 764      | 0          | baseline : scrut_util     --collect baseline |
| 2019-01-23 12:10:00 | 875      | 0          | baseline : scrut_util     --collect baseline |
| 2019-01-23 11:10:00 | 2018     | 0          | baseline : scrut_util     --collect baseline |
| 2019-01-23 10:10:00 | 691      | 0          | baseline : scrut_util     --collect baseline |
| 2019-01-23 09:10:00 | 752      | 0          | baseline : scrut_util     --collect baseline |
| 2019-01-23 08:10:00 | 637      | 0          | baseline : scrut_util     --collect baseline |
  • RUN_TIME indicates how long the baselining task has run. It should complete in a few seconds.
  • RETURN_VAL indicates if any error code was returned. It should be 0. If it is any other value,contact Plixer technical support for assistance.

Resetting baselines to defaults

With the following interactive scrut_util command, custom (manual) baselines can be reset to the default baselines for each exporter. Historical data will not be deleted. However, it will expire off based on Plixer Scrutinizer’s historical settings.

clean baseline

Warning

This command will purge data from Plixer Scrutinizer. Please use with caution.