Version upgrades#
Replicator will continuously be updated to add new features, enhance existing functionality, and/or apply patches for security threats. All update packages will have been applied to Plixer’s QA servers and thoroughly tested before being made available to the public.
Note
Replicator update packages can be installed without assistance, but it is strongly recommended to contact Plixer Technical Support to allow our support engineers to guide you through the process.
On this page:
Pre-upgrade checks/preparations#
The checks below are meant to identify and resolve any underlying issues with the current Replicator install and should be completed before the scheduled upgrade window to help ensure that the upgrade will be successfully completed.
All install logs are saved to /var/log/Replicator-Install.log.
All upgrades#
[ ] A VM snapshot of the standalone/configuration instance (v20.0.0+) or the appliance to be upgraded (pre-v20.0.0 installs) is available.
[ ] Web interface and SSH (
replicatoruser for v19.1.1 orplixeruser for v20.0.0+) credentials are available for each Replicator instance to be upgraded.[ ] Vitals LEDs in the web interface Admin section are green.
[ ] The root disk has at least 5 GB of free space:
df -h /[ ] The upgrade path and estimated duration have been confirmed against the upgrade path table.
Offline upgrades#
[ ] The offline packages have been downloaded, moved to
/var/db/big/offline/(requires an additional 84 GB), and checksum-verified. Allow extra time to download/transfer the offline bundle due to its size. If necessary, add more storage or use/deploy a Scrutinizer server as a temporary offline repository host.[ ] If using an active Scrutinizer primary reporter for the offline repository, 10% of its total disk space must remain available after the offline packages are downloaded and extracted. This will prevent automatic history trimming. The offline packages should not be staged on the root filesystem as they can fill the root disk and stop flow collection.
Situational checks#
Each check below leads with the situation it applies to. Skim the bold labels and complete only the checks that match the deployment being upgraded.
[ ] Version upgrades - Verify that the Replicator appliance meets the target version’s minimum resource requirements.
Important
Do not proceed with the upgrade if the server is underprovisioned. Contact Plixer Technical Support for assistance with tuning system parameters or allocating additional resources to the server.
[ ] Headless conversion (upgrades from v19.1.1 only) - Log in to the configuration instance and register/add a new headless instance using the IP address that will be assigned to the v19.1.1 appliance after it is upgraded and converted to a headless instance.
[ ] v19.1.1 configuration migration - Back up the v19.1.1 configuration data (to be restored on the v20.0.0+ configuration instance after the upgrade and conversion to headless)
[ ] Internal NTP server - If the server being upgraded to Replicator 20.0.0+ cannot access the Internet, an internal NTP server can be configured by running the following:
sed -i -e '/^pool/aserver NTP_ADDRESS' -e 's/^pool/#&/' /etc/chrony.conf[ ] Online upgrades - Confirm that the Replicator appliances to be upgraded can reach
https://files.plixer.comby downloading the checksum file:curl -O https://files.plixer.com/plixer-repo/scrutinizer/19.8.1/replicator-install.run.sha256
For deployments without Internet access, download the file from the IP address (
REPO_HOST_IP) of the offline yum/dnf repository host instead.[ ] High-availability pairs - Stop the
keepalivedservice on each node before upgrading:sudo systemctl stop keepalivedWhen upgrading v20.0.x HA pairs, the secondary instance must be upgraded before the primary. V20.1.x pairs can be upgraded in any order.
[ ] Redeploying from older versions - Obtain a valid license key for the target version if one has not already been acquired.
[ ] Previously upgraded servers - Delete any older versions of
replicator-install.runon the Replicator instance. This prevents them from being used instead of the latest/correct installer.[ ] Hardware appliances - Log in to iDRAC and perform a hardware health check. Escalate any issues found to Dell for resolution. A reboot is also recommended as an additional check for underlying hardware issues.
Once all applicable checks are complete, follow the instructions in the next section to upgrade Replicator to the latest version.
Changed in version 20.0.0: plixer is the recommended OS user for command line access. The replicator and root users are no longer required (the replicator user will be retained on instances that were upgraded from v19.1.1).
Replicator 20.1.1#
The standard procedure upgrades a compatible Replicator instance (see below) to the latest release.
Important
Replicator 19.1.1 appliances that are upgraded to v20.0.0+ default to standalone/configuration instance mode.
If an appliance needs to be converted to a headless instance, refer to these instructions before starting the upgrade process.
Upgrade paths#
The Replicator 20.1.1 upgrade requires the target VM or hardware appliance to be running v19.1.1 or higher and have at least 2 CPU cores, 8 GB of RAM, and 16 GB free on /.
Instances on older versions must be upgraded in stages, as indicated below.
Starting version |
Upgrade path |
Estimated time |
|---|---|---|
20.0.x |
20.1.1 |
30 - 45 minutes |
19.1.1 |
20.1.1 |
60 - 90 minutes |
19.1.0 |
19.1.1 -> 20.1.1 |
~1 hour |
19.0.x |
OS update + 19.1.1 -> 20.1.1 |
~3 hours |
AWS 19.0.x |
Must be redeployed as a 20.1.1 instance |
30 minutes + setup and configuration |
18.x and older |
Must be redeployed as a 20.1.1 instance |
30 minutes + setup and configuration |
All upgrades from Replicator 19.1.1+ to the latest version are installed following the same procedure. For upgrades from older versions to v19.1.1, follow the links in the table above.
Backing up configuration data (upgrade from v19.1.1 -> convert to headless only)#
If a v19.1.1 instance is going to be converted to headless mode post-upgrade, its existing configuration data (profiles, collectors, and exporters) can be backed up before the upgrade. The backup data can then be restored on the v20.0.0+ configuration instance after the upgrade and conversion have been completed.
Click to view instructions
Note
If the v19.1.1 instance has no configuration data that needs to be migrated, skip these steps and proceed with upgrading and converting the appliance.
Log in to the configuration instance and register/add a new headless instance.
SSH to the source host as the
rootuser and create a directory for the backup files:mkdir /tmp/migrationDownload the migration utility and apply the necessary permissions:
curl -o /tmp/migration/Replicator-19-conf-tool.sh https://files.plixer.com/plixer-repo/scrutinizer/19.8.1/util/Replicator-19-conf-tool.sh chmod 755 /tmp/migration/Replicator-19-conf-tool.sh
Run the utility with the address that will be assigned to the headless instance after the upgrade (
HEADLESS_IP):cd /tmp/migration ./Replicator-19-conf-tool.sh backup <BACKUP_NAME> <HEADLESS_IP>
Note
HEADLESS_IPdoes not need to be the same as the pre-upgrade IP address, but it must match the IP address of the post-upgrade headless instance for the backup data to be restored successfully.Verify the contents of the backup file:
tar -vztf BACKUP_NAME.tar.gz ls -l /home/replicator/v19_backup
The files should be the same size and not be empty.
Copy the backup file from the v19.1.1 hardware appliance to the configuration instance:
scp BACKUP_NAME.tar.gz plixer@CONFIGURATION_HOST_IP:/home/plixer/scrutinizer/files/BACKUP_NAME.tar.gz
After the backup file has been created and copied to the configuration instance, follow the instructions in the next section to upgrade and convert the Replicator 19.1.1 appliance.
Installing the upgrade#
Once all preparation steps have been completed, the Replicator 20.1.1 installer can be downloaded and installed as follows:
Click to view instructions
SSH to the Replicator server to be upgraded as the
plixeruser (use thereplicatoruser if upgrading from v19.1.1):ssh plixer@REPLICATOR_IPStart a new tmux session (to maintain the upgrade session if the SSH connection is lost):
tmux new -s upgradeVerify that the current working directory is correct (
replicator):cd /home/replicator/Download the installer and the checksum file for the latest version:
curl -O https://files.plixer.com/plixer-repo/scrutinizer/19.8.1/replicator-install.run curl -O https://files.plixer.com/plixer-repo/scrutinizer/19.8.1/replicator-install.run.sha256
Verify the checksum:
sha256sum -c replicator-install.run.sha256Set the correct permissions for the installer:
chmod 755 replicator-install.runRun the installer as the
plixeruser:./replicator-install.runAfter the installation script finishes running, reboot the appliance:
sudo shutdown -r now
The Replicator appliance will be on v20.1.1 after the reboot.
[Optional] Converting the upgraded instance to headless mode (upgrades from v19.1.1 only)
After the reboot, log in to the configuration instance and register/add a new headless instance with the IP address of the upgraded appliance.
SSH to the upgraded instance as the
plixeruser:ssh plixer@REPLICATOR_IPRun the setup script on the upgraded instance to demote it to headless mode:
MAKE_HEADLESS=1 /usr/share/replicator/util/setup.sh
When prompted, provide the configuration instance IP address as well as the name entered and authentication token generated when the headless instance was registered.
In the configuration instance web interface, navigate to Admin > Resources > Replicators again and verify that the upgraded instance has self-registered.
The new headless instance can then be further configured from the Replicator 20.0.0+ configuration instance. If a backup of the v19.1.1 configuration was saved, it can be restored on the configuration host (see below) after the conversion is complete.
Note
V19.1.1 appliances (including hardware appliances) will lose their web interface after being upgraded and converted and must be managed from the remote configuration instance they were registered on.
[Optional] Restoring v19.1.1 configuration data
SSH to the configuration instance as the
plixeruser and apply the necessary permissions to the bundled utility:chmod 755 /home/plixer/scrutinizer/files/Replicator-19-conf-tool.shRun the utility to restore the backup data for the headless instance:
cd /home/plixer/scrutinizer/files/ ./Replicator-19-conf-tool.sh restore BACKUP_NAME.tar.gz
Navigate to the Replicator UI/page on the configuration instance web interface and verify that the configuration data was successfully restored and associated with the headless instance.
After the backed up data has been restored, the Replicator service should be updated with the new profile configuration data and start replication within one minute.
Offline upgrades#
Replicator servers that cannot access the default yum/dnf repository on https://files.plixer.com can be upgraded by setting up an offline repository on the local network. This repository can be hosted on the Replicator standalone/configuration instance or a Plixer One/Scrutinizer server.
Warning
When using an active Scrutinizer primary reporter as the offline repository host, always download and stage the offline bundle under /var/db/big/offline/. Downloading it to the root filesystem (e.g., to /home/plixer) can fill the root disk and stop flow collection.
Before starting, verify that /var/db/big/ has at least 84 GB of free disk space:
df -h --output='avail' /var/db/big/
If necessary, add more storage or deploy a temporary Scrutinizer server to host the offline repository.
Click to view instructions
Setting up the offline repository
Follow these steps to set up the offline repository on a Replicator configuration instance (with IP address REPO_HOST_IP):
Download the offline repo package and checksum file on a host with Internet access:
curl -O https://files.plixer.com/plixer-repo/scrutinizer/19.8.1_offline.tgz curl -O https://files.plixer.com/plixer-repo/scrutinizer/19.8.1_offline.tgz.sha256
Start an SSH session with the offline repo host as the
plixeruser:ssh plixer@REPO_HOST_IPCreate a new directory for the offline installation files and set the correct permissions to give the plixer user access to it:
sudo mkdir -p /var/db/big/offline sudo chown plixer:plixer /var/db/big/offline
If the directory already exists, delete all its contents (older offline installation files) before continuing.
On the Internet-connected host, copy the files downloaded in step 1 to the staging directory on the repo host:
scp 19.8.1_offline.tgz* plixer@REPO_HOST_IP:/var/db/big/offline/
On the repo host, validate the checksum:
(cd /var/db/big/offline/ ; sha256sum -c 19.8.1_offline.tgz.sha256)Extract the repository:
tar -zxvf /var/db/big/offline/19.8.1_offline.tgz -C /var/db/big/offlineCreate a link to the offline repository in a directory accessible to the web server:
sudo -u webapp ln -sf /var/db/big/offline/plixer-repo /home/webapp/html/
Installing the upgrade
After the offline repo host has been set up, SSH to the Replicator server to be upgraded as the
plixeruser (use thereplicatoruser if upgrading from v19.1.1):ssh plixer@REPLICATOR_IPStart a new tmux session (to maintain the upgrade session if the SSH connection is lost):
tmux new -s upgradeVerify that the current working directory is correct (
replicator):cd /home/replicator/Download the installer and the checksum file for the latest version:
curl -O https://REPO_HOST_IP/plixer-repo/scrutinizer/19.8.1/replicator-install.run curl -O https://REPO_HOST_IP/plixer-repo/scrutinizer/19.8.1/replicator-install.run.sha256
Verify the checksum:
sha256sum -c replicator-install.run.sha256Set the correct permissions for the installer:
chmod 755 replicator-install.runRun the installer:
If the offline repo is hosted on the same Replicator standalone/configuration instance, run:
REPO_HOST=local LOCAL_REPO_BASEDIR=/var/db/big/offline ./replicator-install.run
If the offline repo is hosted on a different standalone Replicator instance or a Scrutinizer server, run:
REPO_HOST=REPO_HOST_IP ./replicator-install.run -- -k
After the installation script finishes running, reboot the appliance:
sudo shutdown -r now[Optional] Delete the offline bundle to reclaim disk space (repository has already been extracted):
rm /var/db/big/offline/19.8.1_offline.tgz
The Replicator appliance will be on v20.1.1 after the reboot.
[Optional] Converting the upgraded instance to headless mode (upgrades from v19.1.1 only)
After the reboot, log in to the configuration instance and register/add a new headless instance with the IP address of the upgraded appliance.
SSH to the upgraded instance as the
plixeruser:ssh plixer@REPLICATOR_IPRun the setup script on the upgraded instance to demote it to headless mode:
MAKE_HEADLESS=1 /usr/share/replicator/util/setup.sh
When prompted, provide the configuration instance IP address as well as the name entered and authentication token generated when the headless instance was registered.
In the configuration instance web interface, navigate to Admin > Resources > Replicators again and verify that the upgraded instance has self-registered.
The new headless instance can then be further configured from the Replicator 20.0.0+ configuration instance. If a backup of the v19.1.1 configuration was saved, it can be restored on the configuration host (see below) after the conversion is complete.
Note
V19.1.1 appliances (including hardware appliances) will lose their web interface after being upgraded and converted and must be managed from the remote configuration instance they were registered on.
[Optional] Restoring v19.1.1 configuration data
SSH to the configuration instance as the
plixeruser and apply the necessary permissions to the bundled utility:chmod 755 /home/plixer/scrutinizer/files/Replicator-19-conf-tool.shRun the utility to restore the backup data for the headless instance:
cd /home/plixer/scrutinizer/files/ ./Replicator-19-conf-tool.sh restore BACKUP_NAME.tar.gz
Navigate to the Replicator UI/page on the configuration instance web interface and verify that the configuration data was successfully restored and associated with the headless instance.
After the backed up data has been restored, the Replicator service should be updated with the new profile configuration data and start replication within one minute.
Upgrading v19.1.1 HA pairs#
To upgrade a Replicator 19.1.1 high-availability (HA) pair to v20.1.1, follow these steps:
Note
To set up a new v20.0.0+ HA pair, follow this guide instead.
Click to view instructions
Important
If the HA pair to be upgraded is using a virtual IP address (VIP), note the address being used (can be found in /etc/keepalived/keepalived.conf or by running ip addr show) before starting the upgrade process. The VIP will be required to complete several of the steps below.
Stop and disable the
keepalivedservice.sudo systemctl stop keepalived sudo systemctl disable keepalived
Verify that the VIP was released by the primary and has been taken over by the secondary instance (run on both):
ip aUpgrade the primary instance following this guide.
After the upgrade, log into the web interface and verify that profiles were successfully migrated to the upgraded primary instance.
Apply a new license (must support at least two Replicator instances).
Register a new instance under Admin > Resources > Replicators and note the API authentication token.
Reboot the primary instance, and then upgrade the secondary instance following the same steps as before.
After the upgrade, run the following on the secondary instance to set it up as a headless instance:
MAKE_HEADLESS=1 /usr/share/replicator/util/setup.sh
Enter the following details when prompted:
Configuration instance (primary instance) IP address
API authentication token created when the new instance was created (step 6)
Name assigned to the new instance
Restart the secondary instance, and then return to Admin > Resources > Replicators in the web interface to verify that the secondary instance has successfully registered itself with the primary (IP address should be displayed).
On the same page, click on the primary instance name (usually ‘Local’), and configure high availability using the primary and secondary instances (enter the original VIP if necessary).
Click the Save button, and then wait two minutes.
Verify that
/etc/keepalived/keepalived_replicator.confcontains the following (on both devices):# this file is automatically generated do not edit global_defs { script_user plixer enable_script_security } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 priority 90 advert_int 1 preempt_delay 5 notify /usr/share/replicator/bin/notify.sh authentication { auth_type PASS auth_pass zhO2brnVF2byl4+zL/ssTaCPmHFr5IMQJEIVDkJiFCCvBIttqjBptwm8c8PWbjqqN4BVieKunkVqPWIRp4fG0Q== } virtual_ipaddress { 10.42.150.17 } }
Modify
/etc/keepalived/keepalived.confon both instances:sudo bash -c 'echo "include /etc/keepalived/keepalived_replicator.conf" > /etc/keepalived/keepalived.conf'Restart the
keepalivedservice on both instances:sudo systemctl restart keepalived
Both Replicator instances should now be running v20.1.1 and set up as a high-availability pair using the same (pre-upgrade) configuration.
Upgrading to v19.1.1#
The Replicator v19.1.1 upgrade requires a minimum of v19.0.1 due to an OS update:
V19.0.1 appliances - Update the OS before installing the upgrade as described in the guide below
V19.1.0 appliances - Skip the OS update and proceed to installing the v19.1.1 upgrade.
For assistance or clarifications, contact Plixer Technical Support.
Pre-upgrade preparation#
[Hardware appliances] Create a backup of the current Replicator appliance using the backup interactive mode command (will be saved in
/home/replicator/backups) and store it on an external system/drive.[Virtual appliances] Backup the current Replicator install by taking a VM snapshot.
Confirm the current password for the
replicatorSSH user (runpasswd replicator)Verify that root login is disabled by running:
sudo sed -i 's/^#PermitRootLogin.*/PermitRootLogin no/g' /etc/ssh/sshd_config[Offline upgrades] If the Replicator appliance does not have access to
files.plixer.com, set up an offline repository for this upgrade.[High-availability pair] Stop the
keepalivedservice before upgrading each instance (primary before secondary):sudo systemctl stop keepalived
Upgrade process#
The process of upgrading a v19.0.1 Replicator appliance to v19.1.1 involves the following steps:
Backing up the v19.0.1 database and server-specific files using the
backupinteractive mode command.Downloading the operating system upgrade script,
olmigrate.run, and running it a total of four times (with a reboot between runs)Downloading and running the Replicator v19.1.1 installation script (
replicator-install.run)Verifying that the v19.0.1 data has been successfully migrated after v19.1.1 is installed
Important notes#
The upgrade will take at least one hour to complete.
If the Replicator appliance is able to access files.plixer.com, the
REPO_HOSTvariable should be set tofiles.plixer.comfor the steps outlined below. For offline upgrades, the IP address of the offline repo should be used instead.Due to an increase in minimum specs, older VMs with 2 GB of RAM should be provisioned with 4 GB RAM before the upgrade.
The upgrade requires a minimum of 16 GB free space on root (
/). There may be older logs (sudo rm /var/log/messages-*) that can be deleted to free up space.
Updating the OS#
Once all preparation steps have been completed, follow these steps to update the OS to Oracle Linux 9:
Click to view instructions
Note
To verify the current progress of the OS update at any time:
cat /etc/motd
or check versions between runs (NAME= and VERSION= lines):
cat /etc/os-release
SSH to the v19.0.1 appliance to be upgraded as the
replicatoruser.Verify that the current working directory is correct (
replicator):cd /home/replicator/Download the OS upgrade script and its checksum file:
REPO_HOST=files.plixer.com curl -k -o olmigrate.run https://$REPO_HOST/plixer-repo/replicator/19.1.1/olmigrate.run curl -k -o olmigrate.run.sha256 https://$REPO_HOST/plixer-repo/replicator/19.1.1/olmigrate.run.sha256
Validate the integrity of
olmigrate.run:cat olmigrate.run.sha256 sha256sum olmigrate.run
Set the correct permissions for the OS upgrade script:
chmod a+x olmigrate.runRun the
olmigrate.runscript a total of four times:REPO_HOST=files.plixer.com ./olmigrate.run -- -k
Important
Reboots between runs of the OS upgrade script (
olmigrate.run) can take a long time. Before trying to reconnect to the appliance, start aPINGto the Replicator IP address and wait for it to become available again. Do NOT manually reboot the server.
After the fourth olmigrate.run run (there will be no reboot), the update to OL9 OS will be complete, and the v19.1.1 upgrade can be installed.
Installing the upgrade
Change directories to
/tmp:cd /tmp/Download the Replicator v19.1.1 installation script and its checksum file:
REPO_HOST=files.plixer.com curl -k -o replicator-install.run https://$REPO_HOST/plixer-repo/replicator/19.1.1/replicator-install.run curl -k -o replicator-install.run.sha256 https://$REPO_HOST/plixer-repo/replicator/19.1.1/replicator-install.run.sha256
Validate the integrity of
replicator-install.run:cat replicator-install.run.sha256 sha256sum replicator-install.run
Update permissions for the
replicator-install.runscript:chmod a+x replicator-install.runRun
replicator-install.runto install Replicator v19.1.1:REPO_HOST=files.plixer.com ./replicator-install.run -- -k
After the installation script finishes running, reboot the appliance:
sudo shutdown -r nowAfter the reboot, SSH to the appliance again to reset the password for the
adminUI user:manage --cli password webui
After completing the above steps, the Replicator appliance will be on v19.1.1.
Offline upgrades to v19.1.1#
The following instructions for setting up an offline repo are intended for upgrading to Replicator v19.1.1 only:
Click to view instructions
Setting up the offline repository
SSH to the VM as the
plixeruser:ssh plixer@SCRUTINIZER_VM_IPCreate the offline repo directory and assign it the correct permissions:
sudo mkdir /var/db/big/offline sudo chown plixer:plixer /var/db/big/offline
Download the offline tar file for Replicator 19.1.1 and its checksum file:
curl -o /var/db/big/offline/19.1.1_offline.tgz https://files.plixer.com/plixer-repo/replicator/19.1.1_offline.tgz curl -o /var/db/big/offline/19.1.1_offline.tgz.sha256 https://files.plixer.com/plixer-repo/replicator/19.1.1_offline.tgz.sha256
Validate the integrity of
19.1.1_offline.tgz:cat /var/db/big/offline/19.1.1_offline.tgz.sha256 sha256sum /var/db/big/offline/19.1.1_offline.tgz
Extract the offline tar file:
cd /var/db/big/offline tar xvf 19.1.1_offline.tgz
Create a symlink in the html directory to the offline repo:
ln -s /var/db/big/offline/plixer-repo /home/plixer/scrutinizer/html/plixer-repo
Installing the upgrade
SSH as the
plixeruser to the appliance to be upgraded and switch to the/tmpdirectory:SSH plixer@REPLICATOR_IP cd /tmp/
Download the Replicator v19.1.1 installation script and its checksum file:
REPO_HOST=SCRUTINIZER_VM_IP curl -k -o replicator-install.run https://$REPO_HOST/plixer-repo/replicator/19.1.1/replicator-install.run curl -k -o replicator-install.run.sha256 https://$REPO_HOST/plixer-repo/replicator/19.1.1/replicator-install.run.sha256
Validate the integrity of
replicator-install.run:cat replicator-install.run.sha256 sha256sum replicator-install.run
Update permissions for the
replicator-install.runscript:chmod a+x replicator-install.runRun
replicator-install.runto install Replicator v19.1.1:REPO_HOST=SCRUTINIZER_VM_IP ./replicator-install.run -- -k
After the installation script finishes running, reboot the appliance:
sudo shutdown -r nowAfter the reboot, SSH to the appliance again to reset the password for the
adminUI user:manage --cli password webui
After completing the above steps, the Replicator appliance will be on v19.1.1.