Virtual Appliance Deployment Guide

What you need to know about deploying a Scrutinizer virtual appliance

To help improve the deployment process, this section will outline the Scrutinizer Virtual Appliance (VA) deployment process and give you the understanding of what is required to successfully complete this deployment.

The Scrutinizer Virtual Appliance can be obtained from Plixer or your local reseller; and is downloaded as an all-in-one virtual appliance which can be deployed on an ESXi v5.5 and above or Hyper-V 2012 hypervisor.

  • You will need to obtain an appliance license or evaluation license from Plixer or your local reseller in order for the Scrutinizer Virtual Appliance to function properly.
  • It is recommended to give the Scrutinizer virtual machine NIC a static MAC address to prevent the machine ID from changing. This is especially important in clustered virtual environments where the VM can change hosts and MAC addresses. If the MAC address changes, the VM will need a new license key.
  • The Scrutinizer Virtual Appliance is deployed on a hypervisor server; it will use 100GB of disk space, 16GB of RAM, and 1 CPU with 4 cores.
  • The performance you get out of a Scrutinizer Virtual Appliance will be directly dependent on the hardware in which it’s deployed on. It’s recommended to dedicate, not share, all the resources that are allocated to the Scrutinizer virtual machine. This is especially important for the Scrutinizer datastores. In environments with high volumes of NetFlow data, Scrutinizer will require dedicated datastores which are discussed in further detail later in this document. Scrutinizer hardware appliances are recommended for deployments of exceedingly high volume of flow as they are designed to handle the highest flow rates.
  • With the default of 100GB of disk space, you can store up to 1 month of NetFlow v5 data from 25 devices at 1,500 flows a second. If you’re planning on exceeding this volume of flow data, or if you need to store data for longer than 30 days, there are detailed steps indicated below that will show you how to expand the amount of disk space allocated to the appliance.
  • To enable the ability to shut down the Scrutinizer Virtual Appliance through vSphere, install VMware Tools using the instructions in this document. Using the “Power -> Off” method will result in database corruption.

Here at Plixer, we don’t like our customers to encounter difficulty, so if you have any questions please do not hesitate to contact our support team .

System Requirements

The Scrutinizer Virtual Appliance has the following requirements:

Component Minimum Specifications
(for trial installations)
Recommended Specifications
(for production environments)
RAM 16GB 64GB
Disks 100GB 1+ TB 15K RAID 0 or 10 configuration
Processor 1 CPU 4 cores 2GHz+ 2 CPUs 8 Cores 2GHz+
Operating System ESXi 5.5+, Hyper-V 2012, KVM 14 ESXi 6+, Hyper-V 2012, KVM 16

Scrutinizer OVF Deployment on ESX

  1. Download the latest Scrutinizer Virtual Appliance

  2. Using VMware vSphere, or vCenter, connect to the ESX host where you will deploy the Scrutinizer Virtual Appliance

    images/chapter3/2.png
  3. Go to File > Deploy OVF Template

    images/chapter3/3.png
  4. Select “Local file” and browse to the downloaded Scrutinizer OVF file and the Scrutinizer VMDK file, then click “Next”.

    images/chapter3/4.png
  5. Give your Scrutinizer VA a name and press “Next”.

    images/chapter3/5.png
  6. Select an ESX to deploy the machine on if your host is not already selected and press “Next”.

    images/chapter3/6.png
  7. Review the details of the virtual machine and press “Next”.

    images/chapter3/7.png
  8. Select your datastore, set your disk format to “Thin Provision” and press “Next”.

    Note

    Be sure to read the Optimizing Scrutinizer Datastores section to obtain the best performance and collection rates.

    images/chapter3/8.png
  9. Select the network to be used by the Scrutinizer Virtual Appliance.

    images/chapter3/9.png
  10. A summary of the options you chose will appear. Click “Finish” and it will import the Scrutinizer Virtual Appliance. This can take a few moments.

images/chapter3/10.png
  1. Before powering on the Scrutinizer virtual machine, it’s important to set a static MAC address for licensing purposes. Right-click on the Scrutinizer VM and select “Edit Settings…”

    images/chapter3/11.png
  2. Select the Network adapter, set the MAC Address to Manual, enter in a unique MAC Address, and then proceed to the next step.

    images/chapter3/12.png
  3. The next step is to allocate and dedicate resources to the Scrutinizer virtual machine. For evaluation purposes, the Scrutinizer OVF grabs 1 CPU with 4 cores, 16GB of RAM, and 100GB of disk space.

    When deploying the Scrutinizer Virtual Appliance it’s recommended to increase the resources to meet the recommended system requirements listed earlier in this document. Since all installs will vary, more resources may be required.

    Start on the “Virtual Hardware” tab and increase the Memory, CPUs, and Hard disk as necessary (see system requirements section for more detail).

    images/chapter3/13.png
  4. Next, navigate to the “Resources” tab. Under CPU and Memory, set the “Shares” value to High and set the “Reservation” maximum value to the amount of resources dedicated to the virtual machine. Now press “OK”.

    Note

    The amount of RAM in the screenshot below is on a small test ESX server, so it won’t match a production install.

    images/chapter3/14.png
  5. Right-click on the Scrutinizer virtual machine and power it on.

    images/chapter3/15.png
  6. Click the console preview window and select “Open Remote Console”. A new window will open and you can then login to the Scrutinizer Virtual Appliance using root/scrutinizer.

    Note

    The server will perform a quick setup and immediately reboot.

    images/chapter3/16.png
  7. Log in to the server again and answer the provided questions. Press “Enter” and the server will reboot to apply the necessary settings.

    images/chapter3/17.png
  8. Now log in to the Scrutinizer web interface in your web browser and apply the necessary license key.

    images/chapter3/18.png

Upgrading the Virtual Machine Hardware Version for ESXi

The Scrutinizer Virtual Appliance is built on Virtual Machine Hardware Version 11 to maintain backwards compatibility with older ESX hypervisors. If you’re running vSphere 6.0 or 6.5 you can take advantage of the newer feature sets by upgrading the Virtual Machine Hardware Version as indicated below.

  1. While the virtual machine is powered off, in vSphere (or vCenter), right-click on the virtual machine and under the “Compatibility” menu, select “Upgrade VM Compatibility”.

    images/chapter4/1.png
  2. Next, power on the virtual machine

Installing VMware Tools for ESXi

After you have powered on and gone through the initial Scrutinizer configuration, optionally, you can install VMware Tools on the appliance. VMware Tools doesn’t come installed by default because each version of ESX installs a different VMware Tools package. Instead, there’s a command you can run from the interactive prompt:

  1. Log in to the appliance as the “plixer” user

    • Use the password you set in the initial deployment
  2. In the Scrutinizer interactive prompt, type the following command:

    images/chapter5/2.png
  3. The tool will then output the necessary files to the terminal. You will return to the prompt afterward.

    images/chapter5/3.png
  4. Celebrate. You will be able to see details of the appliance in vSphere under the summary tab.

    Note

    Installing VMware Tools will allow you to properly shut down the Scrutinizer virtual machine from within vSphere by going to Power > Shut Down Guest.

    When shutting down the Scrutinizer virtual machine, DO NOT select Power > Power Off, as it will result in database corruption. Powering off a virtual machine is equivalent to unplugging a physical computer.

Expanding the database size for ESXi

Depending on the volume of NetFlow data that will be sent to the Scrutinizer appliance, you may need to expand the size of the database. Expanding the size of the database is a multi-stage process. If you have any questions, please contact your support representative .

  1. Power off the Scrutinizer virtual machine by logging in and issuing the “shutdown -h now” command.

    images/chapter6/1.png
  2. Add an additional hard drive to your Scrutinizer Virtual Appliance by right-clicking on the Scrutinizer virtual machine and going to “Edit Settings…”

    images/chapter6/2.png
  3. On the “Virtual Hardware” tab, click the “New Device” dropdown and select “New Hard Disk” and then press “OK”.

    images/chapter6/3.png
  4. Choose the type of Disk Provisioning and alter the Capacity of the disk size. Press “OK”.

    images/chapter6/4.png
  5. Power on the virtual machine by right-clicking on the Scrutinizer virtual machine in vSphere. Mouse over to “Power” -> “Power On”.

    images/chapter6/5.png
  1. Now that the new hard drive is added, we have to resize the volume group, the partition volume, and the file system so that Scrutinizer can use the newly allocated space.

    • Start by logging in to the Scrutinizer Virtual Appliance as the ‘plixer’ user
    • Type ‘show diskspace’ to view the current size of the database, which is mounted on /var/db. This is the current size of disk before we add the new space.
    • Type ‘show partitions’ and make note of the disk in use for the newly added space.
    images/chapter6/6.png
  2. Now that we know the disk to use, we can run a command to use the newly added space. There will be an interactive prompt to follow. One of the questions asked is if you have taken a backup of your data before proceeding.

    • Type ‘set partitions /dev/sd[from above]’
    • In the example in this guide, /dev/sdb is the correct partition.
    images/chapter6/7a.png
    • The interactive prompt will ask if the command completed successfully; verify that the disk matches that output. If the values are correct, type ‘y’ to continue.
    images/chapter6/7b.png
    • Confirm that the new diskspace was added to the volume group.
    images/chapter6/7c.png
    • The next step will be automatic, please be patient. When it’s finished, compare the output to the ‘show partitions’ command we ran in step 6.
    images/chapter6/7d.png
  3. Celebrate!

Scrutinizer Deployment on Hyper-V

  1. Download the latest Scrutinizer Virtual Appliance

  2. Unzip the file on your Hyper-V server

  3. Open Hyper-V Manager and select Import Virtual Machine

    images/chapter7/3.png
  4. Specify the Scrutinizer Incident Response System folder

    images/chapter7/4.png
  5. Select the Virtual Machine

    images/chapter7/5.png
  6. Choose Import Type

    images/chapter7/6.png
  7. Go to Settings

    images/chapter7/7.png
  8. Make sure the memory is set to 16GB.

    images/chapter7/8.png
  9. Select your Network Adapter and assign it to the appropriate Virtual Switch.

    images/chapter7/9.png
  10. Expand the Network Adapter section, select Advanced Features, set the MAC Address to Static, enter a unique MAC Address, and then press “OK”.

    images/chapter7/10.png
  11. Start the Virtual Machine.

    images/chapter7/11.png
  12. Right-click on the Virtual Machine and click Connect to log in to the Scrutinizer Virtual Appliance using root/scrutinizer. The server will perform a quick setup and immediately reboot.

    images/chapter7/12.png
  13. Log in to the server again and answer the provided questions. Press “Enter” and the server will reboot to apply the necessary settings.

    images/chapter7/13.png
  14. Now log in to the Scrutinizer web interface in your web browser and apply the necessary license key.

    images/chapter7/14.png

Expanding the database size for Hyper-V

Depending on the volume of NetFlow data that will be sent to the Scrutinizer appliance, you may need to expand the size of the database. Expanding the size of the database is a multi-stage process. If you have any questions, please contact your support representative.

  1. Power off the Scrutinizer virtual machine by logging in and issuing the “shutdown -h now” command.

  2. In the Hyper-V Manager, right-click on the Scrutinizer virtual machine and select “Settings”.

  3. Next, select the IDE Controller and click “Add” to a hard drive.

    images/chapter8/3.png
  4. Under Virtual hard disk, select “New”.

    images/chapter8/4.png
  5. On the New Virtual Hard Disk Wizard, select “Next”.

    images/chapter8/5.png
  6. On the Choose Disk Format page, select VHDX. It’s common for Scrutinizer VMs to expand past 2TB of disk space, so VHD is not recommended.

    images/chapter8/6.png
  7. On the Choose Disk Type page, select your preferred disk type and then press “Next”.

    images/chapter8/7.png
  8. On the Specify Name and Location page, give your VHDX a name and then select the location for the virtual disk.

    images/chapter8/8.png
  9. Set the size of the new virtual disk and then press “Next”.

    images/chapter8/9.png
  10. Review the new disk settings and then click “Finish”.

    images/chapter8/10.png
  11. Power on the Virtual Machine

  12. Follow from step 6 onward under the “Expanding the database size for ESX” section of this manual.

Scrutinizer Deployment on KVM

  1. Create a directory for your install

    mkdir kvm/Scrut_VM_Guide/

  2. Download the latest Scrutinizer Virtual Appliance to your KVM install

    Command line example:

    wget https://files.plixer.com/Scrutinizer_KVM_Image.tar.gz

    images/chapter9/2.png

Note

Contact support for latest image if the URL above does not work.

  1. Unzip the file on your KVM server to your new folder.

    sudo tar xvzf Scrutinizer_KVM_Image.tar.gz

    images/chapter9/3.png
  2. Run your script to install Scrutinizer

    sudo ./install-kvm-scrut.sh

    images/chapter9/4a.png

At this point, you should see that your machine has been created from the image we deployed:

images/chapter9/4b.png
  1. Lastly, we just need to log in to the machine now that it is deployed. Run this command to get to the console:

    virsh console Scrutinizer

You will be prompted to log in; the default credentials are root/scrutinizer. The machine will reboot and you will be asked to log in again. This time you will be presented with a shell script asking for networking information. Follow the on-screen instructions and celebrate!

images/chapter9/5.png

Optimizing Scrutinizer datastores

Due to the nature of NetFlow, large deployments require a very high volume of disk I/O. For the best performance, the Scrutinizer Virtual Appliance should be deployed on a dedicated 15,000RPM RAID 10 datastore, with the amount of disk space that is required to meet your history setting requirements; 1.8 TB of disk space in RAID 10 is the recommended datastore deployment size.

If Scrutinizer is deployed on shared drives, such as a storage area network (SAN) or network-attached storage (NAS), then collection rates cannot be guaranteed as the collection rates will directly depend on what other applications are also using the same disk I/O.

In high flow volume environments, if you cannot get dedicated datastores, it’s recommended to use a Scrutinizer Hardware Appliance for the dedicated resources and higher collection rates.

FAQ

Q: I got an UNEXPECTED INCONSISTENCY error when trying to power on the Scrutinizer Virtual Appliance. What do I do now?
A: This error indicates that the clock on the ESX server is not set correctly and is in the past. As a result, the disk checks fail which does not allow the virtual machine to start. To resolve this, set your ESX host to sync with an NTP server and then redeploy the Scrutinizer OVF.

Q: How do I stop/start the services?
A: Run the following commands (stop|start means type one OR the other): | service plixer_flow_collector stop|start | service plixer_syslogd stop|start | service httpd stop|start | service plixer_db stop|start

Q: I have a German ‘QWERTZ’ keyboard layout, how come I keep getting password failures when logging into the appliance for the first time?
A: On the German ‘QWERTZ’ keyboard layout, the Z and Y keys are switched. You’ll need to login with the password ‘scrutiniyer’.