Baselining Overview

Baselining is analyzing the performance of the network by comparing current performance to historical data (also known as the “baseline”). For example, when measuring the current traffic from an interface, alerts will be sent if any traffic exceeds the baseline. Scrutinizer can be configured to baseline any NetFlow or IPFIX element, and alert. By default, there are several baselines enabled and configured to collect data whenever a new exporter is added to Scrutinizer.

  • ingressInterface and octetDeltaCount
  • ingressInterface and packetDeltaCount
  • egressInterface and octetDeltaCount
  • egressInterface and packetDeltaCount
  • applicationTag (NBAR) and octetDeltaCount
  • applicationTag (NBAR) and packetDeltaCount

Using the Interactive scrut_util.exe command, baselining behavior can be modified whenever that baseline exceeds a higher than normal traffic pattern.

How Baselining Works

Once an hour, Scrutinizer will analyze historical data and tally network performance of the configured elements for baselining. Exporters that do not have the specified baseline elements in any of the templates exported are skipped. Once the data for recent data is calculated, it is compared against the historical baseline. Alerts will be sent if one of the following occur:

  • (BLACK) There is no historical baseline
  • (RED) recent data exceeds the maximum value of any historical baseline value
  • (ORANGE) recent data exceeds two standard deviations over the average baseline
  • (YELLOW) recent data exceeds one standard deviation over the average baseline

No alert is generated if the recent data exceeds the average baseline but is less than one standard deviation over the average baseline. Baselining will collect data for two weeks (by default) before alerts are generated.

Lastly, a weighted average (by default 100% of the value) is applied to the most recent baselining data and stored to compare against the next baseline collection cycle. The weighted average can be modified to allow new data to have a more or less significant role when comparing future baseline values.

Baselining Hosts

Scrutinizer has the ability to baseline hosts that are specified in IP Groups (e.g. 192.168.1.0/24). More than one IP range can be specified. For more information, see the section in the manual on how to configure IP Groups.

Enabling/Disabling Baseline Globally

Using the Flow Analytics Configuration Manager (Admin Tab -> Settings -> FA Configuration), Baselining can be globally enabled or disabled.

Note

Baselining is disabled by default.

Using the Interactive scrut_util.exe utility for Configuring Baselines

To enable the interactive scrut_util.exe utility, run:

/home/plixer/scrutinizer/bin/scrut_util.exe

Which will open this Scrutinizer prompt:

SCRUTINIZER>

For more information on using this utility, reference the Interactive Scrut_util section.

Adding/Removing default Baselines to an Exporter

Using the following Interactive scrut_util.exe command, default baselines can be added or removed from a specified exporter.

Warning

These commands will alter the behavior of Scrutinizer baseline functionality. Please use with caution.

Adding
enable baseline <exporter_ip> default
Removing
disable baseline <exporter_ip>

Adding Custom Baselines

Every install is unique. Before deploying additional baselines, contact Plixer Technical Support to assist in planning, sizing, and deploying baselines in Scrutinizer.

Using the following Interactive scrut_util.exe command, custom (manual) baselines can be added to a specified exporter.

Warning

This command will alter the behavior of Scrutinizer baseline functionality. Please use with caution.

SCRUTINIZER> enable baseline <exporter_ip> manual <pri_element[,sec_element]> <element> <AVG|COUNT|MIN|MAX|STD|SUM> <dailyhr|busday|sameday>

This ‘enable baseline’ command enables custom baselines (manual) based on elements from NetFlow and IPFIX templates.

Baselining has several parameters available to customize the specific baseline data to collect with the ‘manual’ option.

  • Replace <exporter_ip> with the exporter to collect the specified baseline data.

  • <pri_element> and (optionally <sec_element>) specify which IPFIX elements will be part of this baseline. This is how the data is grouped and calculated.

  • <element> must be an IPFIX element with a numeric value that the <pri_element> and/or <sec_element> will be baselined on.

    • For example: To collect data on source addresses and bytes, the <pri_element> would be sourceipv4address and the <element> would be octetdeltacount.
    • Another example: To collect data off multiple elements such as source address and applicationtag based on packets; the <pri_element> would be sourceipv4address, the <sec_element> would be applicationtag, and the <element> would be packetdeltacount.
    • To find the numeric values for IPFIX elements in Scrutinizer, go to Status > System > Templates. Drill in on the Plixer ID to see all details regarding the elements in that template.
  • <AVG|COUNT|MIN|MAX|STD|SUM> are options that are used to calculate how to measure the <element>

    • AVG = Average
    • COUNT = Flow Count
    • MIN = Minimum Value
    • MAX = Maximum value
    • STD = Standard Deviation
    • SUM = Sum
  • Lastly, there are several ways baselines can be compared:

    • dailyhr = same time frame (e.g. 1a - 2a) for each day
    • busday = same time frame (e.g. 1a - 2a) for each business day, skipping weekends
    • sameday = same day and time each week (e.g. 1a - 2a Mondays)

When baselining ip addresses, IP Groups must be configured with the ranges and subnets of addresses to include in the baseline. This protects the user from baselining addresses that may talk once and therefore eventually alarm as a false positive.

The baseline learning process takes 7 days by default.

Reset Baselines to Defaults

Using the following Interactive scrut_util.exe command, custom (manual) baselines can be reset to the default baselines for each exporter. Historical data will not be deleted. However, it will expire off based on Scrutinizer’s historical settings.

clean baseline

Warning

This command will purge data from Scrutinizer. Please use with caution.

Monitoring Baseline Processing

Use the following Interactive scrut_util.exe command to get the Task ID for the baseline task (to be used in the ‘check task’ command below)

show task baseline

Displays information regarding the baseline task.

Example:

SCRUTINIZER> show task baseline


+-----------+---------+----------------+--------------------------+
| TASK_NAME | TASK_ID | EXECUTABLE     | ARGUMENTS                |
+-----------+---------+----------------+--------------------------+
| baseline  | 224     | scrut_util.exe | ["--collect","baseline"] |
+-----------+---------+----------------+--------------------------+
1 Result(s) Found

Done (0.010545 seconds)

The ‘check task’ command provides information on the baseline task processing.

check task <id>

Checks the execution times and error codes for the baseline task. The task id is available by using the show task baseline command.

Example:

SCRUTINIZER> check task 224


+---------------------+----------+------------+----------------------------------------------+
| START_TIME          | RUN_TIME | RETURN_VAL | INFO_STR                                     |
+---------------------+----------+------------+----------------------------------------------+
| 2018-01-23 13:10:00 | 764      | 0          | baseline : scrut_util.exe --collect baseline |
| 2018-01-23 12:10:00 | 875      | 0          | baseline : scrut_util.exe --collect baseline |
| 2018-01-23 11:10:00 | 2018     | 0          | baseline : scrut_util.exe --collect baseline |
| 2018-01-23 10:10:00 | 691      | 0          | baseline : scrut_util.exe --collect baseline |
| 2018-01-23 09:10:00 | 752      | 0          | baseline : scrut_util.exe --collect baseline |
| 2018-01-23 08:10:00 | 637      | 0          | baseline : scrut_util.exe --collect baseline |


....
  • RUN_TIME indicates how long the baselining task has run. It should complete in a few seconds.
  • RETURN_VAL indicates if any error code was returned. It should be 0. Anything else, something is wrong and Plixer Technical Support should be contacted for assistance.

Baseline Reports

To monitor the progress of the baselining function, baseline reports are available on the collector’s exporter. These reports are accessible by following these steps:

  1. In the Status tab, under Device Explorer, filter on ‘Scrutinizer’

  2. Click on ‘Scrutinizer’

  3. Then click on ‘Reports’

  4. Then click on ‘Baselines’

  5. This will provide a list of available auto-created Baseline reports on your server.

    • Select a report to view
    • These reports will show if Baselining is working as configured.