Using the Reporting API

Overview

The Reporting API (Application Program Interface) is a simple way to access the report data via HTTP or command line.

Getting Started

Authentication

The Reporting API uses authentication tokens. They can be configured under Admin > Security > authentication tokens.

The auth tokens give the API privileges to access resources in the product which are based off of security groups.

Requesting a Report via HTTP

To request a report via HTTP using the API, a POST or GET request must be made to this base URI: “http://%[SCRUTINIZER_SERVER]/fcgi/scrut_fcgi.fcgi”, passing the following four (mandatory) fields:

  • rm=report_api
  • action=get
  • rpt_json={}
  • data_requested={}

rm=report_api is always the same when accessing the Report API.

action tells the API what we want to do. Currently, the only action supported is ‘get’.

rpt_json takes the JSON string containing the details for the report (See Reference Guide > rpt_json for more information).

data_requested is used to indicate what data from the report we need. It also takes a JSON string.

Requesting a Report via the Command Line

Using the Report API in the command line doesn’t require authentication since the user will need console access to the Scrutinizer server. To request a report:

Run “/home/plixer/scrutinizer/cgi-bin/reporting.cgi -getReport -rpt_json=
‘[JSON STRING WITH REPORT DETAILS]’ -data_requested=
‘[JSON STRING WITH DATA REQUESTED DETAILS]’”

Examples

HTTP

https://SCRUTINIZER_SERVER/#tab=tab3&subCat=report&rpt_json={"reportTypeLang":"conversations","reportDirections":{"selected":"inbound"},"times":{"dateRange":"LastFiveMinutes","start":"","end":""},"filters":{"sdfDips_0":"in_0A0101FB_ALL"}}

JavaScript

This example illustrates how to make an AJAX request to the reporting API using Javascript & jQuery.

var url = "http://[SCRUTINIZER SERVER]/fcgi/scrut_fcgi.fcgi";
var report_details = {
    "reportTypeLang" : "conversations",
    "reportDirections" : {
    "selected" : "both"
    },
        "times" : {
                "start" : 1426001640,
                "end" : 1426088100
        },
        "filters" : {
                "sdfDips_0" : "in_0A040101_0A040101-0"
        },
        "dataGranularity" : {
                "selected" : "auto"
        }
};
var data_i_need = {
        "inbound" : {
                    "graph" : "all",
                    "table" : {
                            "query_limit" : {
                                            "offset" : 0,
                                            "max_num_rows" : 1000
                                }
                    }
        },
        "outbound" : {
                    "table" : {
                                "query_limit" : {
                                            "offset" : 0,
                                            "max_num_rows" : 1000
                                }
                    }
        }
};
$.ajax({
        type: "POST",
        url: url,
        data: {
                rm : "report_api",
                action : "get",
                rpt_json : JSON.stringify( report_details ),
                data_requested : JSON.stringify( data_i_need )
        },
        success: function( response ) {
                    if ( response.err ) {
                                alert(response.msg);
                    } else {
                                console.log(response);
                    }
                    return;
                    }
});

Perl

There are example scripts that are included in Scrutinizer in the /home/plixer/scrutinizer/files/api_examples/ directory. Examples of files in this directory:

  • api_report_example.pl

    • This script will run a report and return the results. It consists of three subroutines:

      • host_search_request: Builds the request JSON
      • get_scrutinizer_data: Makes the request to Scrutinizer
      • format_report_data: Formats the returned data
  • api_host_example.pl

    • This script uses the host index API to perform a quick search on an IP address. It will determine if a host has been seen anywhere on the network. It consists of two subroutines:

      • get_scrutinizer_data: Makes a host search request to Scrutinizer
      • format_search_data: Formats the returned data

Note

To use HTTPS, make sure this perl module is installed: cpan -i LWP::Protocol::https

Python

  • Report Type: flowView
  • Time Frame: LastFiveMinutes
  • Exporter/Interface: IP is 10.1.1.251 which converts to 0A0101FB. in_0A0101FB_ALL means that we are going to include all interfaces for the exporter 10.1.1.251
  • Max Num Rows: 11000

This python script makes the same HTTP request but saves the json to data.txt.

import urllib
import urllib2
import json
url = 'http://SCRUTINIZER_SERVER/fcgi/scrut_fcgi.fcgi'
report_details = {
       'reportTypeLang' : 'flowView',
       'reportDirections' : {
               'selected' : 'inbound'
       },
       'times' : {
               'dateRange' : 'LastFiveMinutes'
       },
       'filters' : {
               'sdfDips_0' : 'in_0A0101FB_ALL',
       },
       'dataGranularity' : {
               'selected' : '1m'
       }
}
data_i_need = {
       'inbound' : {
              'table' : {
                       'query_limit' : {
                               'offset' : 0,
                               'max_num_rows' : 11000
                       }
               }
       },
 }
data = {
       'rm' : 'report_api',
       'action' : 'get',
       'rpt_json' : json.dumps( report_details ),
       'data_requested' : json.dumps( data_i_need )
}
data = urllib.urlencode( data )
req = urllib2.Request( url, data )
response = urllib2.urlopen( req )
report = response.read()
report_obj = json.loads( report )
with open('./data.txt', 'w') as outfile:
   json.dump(report_obj, outfile)

Reference Guide

rm=report_api (required HTTP/ static)

This URL field is required when making an HTTP request. The value for this field will always be ‘report_api’

action (required HTTP)

Indicates what we want to do. Right now the only action supported is ‘get’ to get a report.

getReport (required CMD)

This field is required when using the CMD to access the API.

rpt_json

  • reportTypeLang (required)

    This is the lang id of the report type used for the report. Available reports in the install can be found by looking at Scrutinizer > Status > System > Available Reports

    • conversations
    • host2host
    • ipGroupGroup
    • applications
    • trafficTrend
  • reportDirection (required)

    It takes an object with a single attribute ‘selected’. The values for ‘selected’ are:

    • inbound
    • outbound
    • both
  • times (required)

    • dateRange (required if no start and end)

      Indicates the time for which we want to run the report using one of the predefined date ranges. Available dataRange values:

      • LastFiveMinutes
      • LastTenMinutes
      • LastFifteenMinutes
      • LastTwentyMinutes
      • LastThirtyMinutes
      • LastFortyfiveMinutes
      • LastHour
      • LastFullHour
      • LastThreeDays
      • LastSevenDays
      • LastThirtyDays
      • Today
      • Yesterday
      • Last24Hours
      • ThisWeek
      • LastWeek
      • ThisMonth
      • LastMonth
      • ThisYear
      • Last year
    • start

      Epoch time that we want to use for the start time of the report.

    • end

      Epoch time that we want to use for the end time of the report.

    • clientTimezone

      If we want to base the report times on the time zone where the client is located, we need pass the time zone using clientTimezone, otherwise the server time zone will be used. It takes the time zone names as they are defined in the IANA time zone database, such as “Asia/Shanghai”, “Asia/Kolkata”, “America/New_York”.

  • filters (required)

    It takes an object that defines the different filters that can be applied to the report. We have many filter types that can be applied. Here are a couple examples:

    • There has to be at least an sdfDips filter. sdfDips filters are used to indicate which exporter/interface we want to use for the report. sdfDips are defined like this: “in_[IP_OF_EXPORTER_IN_HEX]_[INTERFACE_ID]” Example: - Exporter all interfaces filter : “sdfDips_0”: “in_408CF386_ALL”
      • Here are some other possible filter examples:
        • Src/Dst host filter : “sdfIps_0”: “in_10.1.1.1_Both”,
        • Well Known Port filter : “sdfPorts_0”: “in_80-6”
    • Filters are defined in the filters object like this:
{
"[filter_type]_[index]" = "filter_definition",
}

  • index is used to differentiate filters of the same type and to indicate in which order they should be applied.
  • dataGranularity

    It indicates which data interval (in minutes) we want to use. It takes an object with one key, ‘selected’. Available values for ‘selected’ are:

    • auto (lets the API decide which interval to use)
    • 1
    • 5
    • 30
    • 120
    • 720
    • 1440
    • 10080
  • rateTotal

    Indicates if we want rates or totals. Totals or rates are not available for all reports. Default depends on the report type.

{
"selected" : "rate" //"total
}
  • byInt

    Indicates whether we want to include the interface details in the report table. Values are 0 or 1. Only applicable to reports with interface information available. Defaults to 0.

{
"selected" : 1
}

data_requested

It is used to indicate to the API what part of the report we need. Reports return two data sets, one for rendering the graph and the other for rendering the table. Each one is divided into outbound and inbound.

While rpt_json tells Scrutinizer how to prepare the data (timeframe, filters, aggregation, etc.), data_requested tells Scrutinizer what data should be returned with the request (inbound, outbound, table, graph, or just a table name). See examples for details.

If “table_name: 1” is passed in data_requested, no data will be returned. Only the name of a temporary database table will be returned. Example:

"inbound" : {
            "graph" : "all",
            "table" : {
                    "query_limit" : {
                                    "offset" : 0,
                                    "max_num_rows" : 1000
                                    }
                        }
            }

Return Data

Example
{
  "report": {
    "table": {
      "inbound": {
        "totalRowCount": "50",
        "columns": [
          {
            "title": "Rank",
            "label": "",
            "klassth": "rankWidth"
          },
          {
            "colHasBbp": 0,
            "defaultSort": null,
            "klassth": "",
            "klassLabel": "",
            "format": {},
            "elementName": "sourceipaddress",
            "canBeRate": null,
            "label": "Source",
            "title": "sourceipaddress"
          },
          {
            "colHasBbp": 0,
            "defaultSort": null,
            "klassth": "commPortWidth",
            "klassLabel": "",
            "format": {},
            "elementName": "commonport",
            "canBeRate": null,
            "label": "Well Known",
            "title": "commonport"
          },
          {
            "colHasBbp": 0,
            "defaultSort": null,
            "klassth": "",
            "klassLabel": "",
            "format": {},
            "elementName": "destinationipaddress",
            "canBeRate": null,
            "label": "Destination",
            "title": "destinationipaddress"
          },
          {
            "colHasBbp": 0,
            "defaultSort": null,
            "klassth": "alignRight dataWidth sortable",
            "klassLabel": "",
            "format": {
              "niceLtr": "p",
              "perSecMode": 0,
              "bitOverRide": 0
          },
          "elementName": "sum_packetdeltacount",
          "canBeRate": 1,
          "label": "Packets",
          "title": "sum_packetdeltacount"
        },
        {
          "colHasBbp": 0,
          "defaultSort": null,
          "klassth": "alignRight dataWidth",
          "klassLabel": "",
          "format": {},
          "elementName": "percenttotal",
          "canBeRate": null,
          "label": "Percent",
          "title": "percenttotal"
        },
        {
          "colHasBbp": 1,
          "defaultSort": "DESC",
          "klassth": "alignRight dataWidth sortable hasBbp ",
          "klassLabel": " sortdesc ",
          "format": {
            "niceLtr": "b",
            "perSecMode": 0,
            "bitOverRide": 0
          },
          "elementName": "sum_octetdeltacount",
          "canBeRate": 1,
          "label": "sum_octetdeltacount",
          "title": "sum_octetdeltacount"
        }
      ],
      "rows": [
        [
          {
            "klasstd": "rank1",
            "title": "Rank",
            "label": "1"
          },
          {
            "rawValue": "10.1.4.105",
            "dataJson": "{\"column\":\"sourceipaddress\"}",
            "klasstd": "alignLeft",
            "klassLabel": "ipDns",
            "title": "10.1.4.105",
            "label": "10.1.4.105"
          },
          {
            "rawValue": "av-emb-config (2050 - TCP)",
            "dataJson": "{\"column\":\"commonport\"}",
            "klasstd": "alignLeft",
            "klassLabel": "",
            "title": "()",
            "label": "av-emb-config (2050 - TCP)"
          },
          {
            "rawValue": "10.1.11.51",
            "dataJson": "{\"column\":\"destinationipaddress\"}",
            "klasstd": "alignLeft",
            "klassLabel": "ipDns", "title": "10.1.11.51",
            "label": "10.1.11.51"
          },
          {
            "rawValue": "20132651",
            "dataJson": "{\"column\":\"sum_packetdeltacount\"}",
            "klasstd": "alignRight",
            "klassLabel": "",
            "title": "20132651",
            "label": "20.13 Mp"
          },
          {
            "rawValue": null,
            "dataJson": "{\"column\":\"percenttotal\"}",
            "klasstd": "alignRight",
            "klassLabel": "",
            "title": "19.53 %",
            "label": "19.53 %"
          },
          {
            "rawValue": "26274904910",
            "dataJson": "{\"column\":\"sum_octetdeltacount\"}",
            "klasstd": "alignRight",
            "klassLabel": "",
            "title": "26274904910",
            "label": "26.27 Gb"
          }
        ]
      ],
      "footer": [
        [
          {
            "title": "Other",
            "label": "Other"
          },
          {
            "klasstd": "alignLeft",
            "translations": "",
            "elementName": "sourceipaddress",
            "title": "Group by columns are not summarized",
            "label": " - ",
            "menuType": "",
            "filterType": ""
          },
          {
            "klasstd": "alignLeft",
            "translations": "",
            "elementName": "commonport",
            "title": "Group by columns are not summarized",
            "label": " - ",
            "menuType": "",
            "filterType": ""
          },
          {
            "klasstd": "alignLeft",
            "translations": "",
            "elementName": "destinationipaddress",
            "title": "Group by columns are not summarized",
            "label": " - ",
            "menuType": "",
            "filterType": ""
          },
          {
            "klasstd": "alignRight",
            "translations": "",
            "elementName": "sum_packetdeltacount",
            "title": 83442022,
            "label": "83.44 Mp",
            "menuType": "",
            "filterType": ""
          },
          {
            "klasstd": "alignRight",
            "translations": "",
            "elementName": "percenttotal",
            "title": "Group by columns are not summarized",
            "label": " - ",
            "menuType": "",
            "filterType": ""
          },
          {
            "klasstd": "alignRight",
            "translations": "",
            "elementName": "sum_octetdeltacount",
            "title": 108236252030,
            "label": "108.24 Gb",
            "menuType": "",
            "filterType": ""
          }
        ],
        [
          {
            "title": "(from conv tables)",
            "label": "Total*"
          },
          {
            "klasstd": "alignLeft",
            "translations": "",
            "elementName": "sourceipaddress",
            "title": "Group by columns are not summarized",
            "label": " - ",
            "menuType": "",
            "filterType": ""
         },
         {
            "klasstd": "alignLeft",
            "translations": "",
            "elementName": "commonport",
            "title": "Group by columns are not summarized",
            "label": " - ",
            "menuType": "",
            "filterType": ""
          },
          {
            "klasstd": "alignLeft",
            "translations": "",
            "elementName": "destinationipaddress",
            "title": "Group by columns are not summarized",
            "label": " - ",
            "menuType": "",
            "filterType": ""
          },
          {
            "klasstd": "alignRight",
            "translations": "",
            "elementName": "sum_packetdeltacount",
            "title": "103574673",
            "label": "103.57 Mp",
            "menuType": "",
            "filterType": ""
          },
          {
            "klasstd": "alignRight",
            "translations": "",
            "elementName": "percenttotal",
            "title": "Group by columns are not summarized",
            "label": " - ",
            "menuType": "",
            "filterType": ""
          },
          {
            "klasstd": "alignRight",
            "translations": "",
            "elementName": "sum_octetdeltacount",
            "title": "134511156940",
            "label": "134.51 Gb",
            "menuType": "",
            "filterType": ""
          }
        ]
      ]
    }
   }
  }
 }

report (Object)

Is the object that contains the data returned by the API. The data is divided into “table” and “graph”.

table (Object)

The table data is divided into inbound or outbound. For each direction the following are provided:

  • totalRowCount: integer representing the total number of rows available.
  • columns: See columns section below.
  • rows: See rows section below.
  • footer: See the footer section below.

columns (Collection)

It is an array of objects that represent the definition of each column in the table. Most of the data is used to create the html table in the browser. Here is a list of the most relevant information included in each object:

  • elementName: Name of the element that the data in the column is for.
  • format: Details about how the data in the column needs to be formatted.
  • label: Label used in the table header.

rows (Array)

Array of collection. Each collection in the array represents a row and each object in the collection represents a table cell for that row. Here are some details about the keys for each object representing a cell:

  • rawValue: The unformatted value as it is return from the database
  • label: formatted value