Data migration

Data migration#

The scrut_util --migrate command can be used to migrate configuration and/or historical data from source Scrutinizer server to a destination server.

Note

Because the steps outlined here can potentially result in an irrecoverable state for the source and/or destination Scrutinizer server, it is highly recommended to contact Plixer Technical Support for assistance. In case of issues, the migration utility logs (/var/log/migrate.log) can be provided to help with troubleshooting.

Source and destination server support#

The migration utility includes separate runmodes for configuration data migration and historical data migration.

The following table shows supported migration modes between Scrutinizer versions:

19.5.x destination

19.6.x destination

18.20 source

Data

Data

19.4.0 source

Configuration and data

Data

19.5.x source

Configuration and data

Configuration and data

19.6.x source

Configuration and data

Note

  • If a Scrutinizer 18.20 server is upgraded v19.4.0, its configuration data can be migrated to a Scrutinizer 19.5.x server.

  • The utility may report pg_restore errors after cross-server configuration migrations. These errors can safely be ignored, and any issues are addressed by re-running the installer as described in these configuration migration instructions.

Source and destination server setup#

Follow the steps below to prepare source and destination Scrutinizer servers for migration.

Source server#

Run the following on the source server to create a temporary migration role and allow connections from the destination server using the role:

DESTINATION=DESTINATION_IP
psql -c "CREATE USER migrator WITH SUPERUSER ENCRYPTED PASSWORD 'ROLE_PASSWORD'"
sudo sed -i -e "1ihost plixer migrator $DESTINATION/32 md5" /var/db/big/pgsql/data/pg_hba.conf
psql -c "SELECT pg_reload_conf()"

Note

When migrating historical data from a Scrutinizer 18.20 server, modify the psql commands above to use the root user instead:

DESTINATION=x.x.x.x
psql -Uroot plixer -c "CREATE USER migrator WITH SUPERUSER ENCRYPTED PASSWORD 'replace_me'"
sudo sed -i -e "1ihost plixer migrator $DESTINATION/32 md5" /var/db/big/pgsql/data/pg_hba.conf
psql -Uroot plixer  -c "SELECT pg_reload_conf()"

Destination server#

Deploy and license the destination server, and then configure its data sources. The destination should be collecting data from the same exporters as the source to allow for reporting continuity.

After verifying that the destination is collecting data, update /etc/migrate.ini with the details of the source server and the temporary migration role.

Configuration migrations#

Configuration migrations transfer application configuration and user preference data from the source Scrutinizer to the destination.

After the source and destination servers have been set up, follow these steps to perform a configuration (and optional historical data) migration):

View instructions

  1. Run the migration utility on the destination server:

    SOURCE=SOURCE_IP
scp $SOURCE:/etc/plixer.key /etc/plixer.key
sudo systemctl stop plixer_flow_collector
scrut_util --migrate config

  2. Once the migration is complete, reinstall the current Scrutinizer package on the destination and restart the flow collection service

    PKG=$(rpm -qa plixer-scrutinizer_pg\*)
sudo yum reinstall -y $PKG
sudo systemctl start plixer_flow_collector

  3. Log in to the web interface and verify that all configuration data and user preferences have been successfully migrated.

  4. [Optional] Run the command to also migrate historical data on the destination server:

    scrut_util --migrate data

  5. Return to the web interface and run any report to verify that the historical data migration was successful.

  6. Remove the hba rule and temporary migrator role on the source server:

    sudo sed -i '/migrator/d' /var/db/big/pgsql/data/pg_hba.conf
psql -c "SELECT pg_reload_conf()"
psql -c "DROP ROLE migrator"

Note

  • Configuration migrations only include Scrutinizer application data and not system configurations. For example, a custom listening port defined in the admin menu will be migrated, but the corresponding port will not be opened on the system. In such cases, undo and re-apply the setting in Scrutinizer after the migration.

  • Distributed cluster settings on the destination server are retained by default. These settings can also be migrated (while still preserving the destination server’s IP address and machine ID) using the --force_dist option as:

    scrut_util --migrate config --force_dist

  • To extend the utility to also migrate contributions to the Scrutinizer SQL codebase, update /home/plixer/scrutinizer/database/utils/scrut_conf_dump.sh to include them.

Historical data migration#

If only historical data needs to be migrated, follow these steps after completing the setup steps for the source and destination servers:

Note

To maintain reporting continuity after the migration, the destination server should be collecting flows from the same exporters as the source.

View instructions

  1. Run the migration utility on the destination server:

    SOURCE=SOURCE_IP
scp $SOURCE:/etc/plixer.key /etc/plixer.key
scrut_util --migrate data

  2. Log in to the web interface and run any report to verify that the historical data migration was successful.

  3. Remove the hba rule and temporary migrator role from the source server:

    sudo sed -i '/migrator/d' /var/db/big/pgsql/data/pg_hba.conf
psql -c "SELECT pg_reload_conf()"
psql -c "DROP ROLE migrator"

Reverting a historical data migration#

Run the following on the destination server to revert the previous historical data migration:

scrut_util --migrate revert
Contents