Auto Replicate#
Replicator Auto Replicate allows flow streams to all be sent to a single Replicator instance, which will then automatically distribute them across collectors in the cluster based on their available capacity.
Auto Replicate is enabled by creating a collector profile for each destination collector and associating them with a seed profile with the necessary exporter inclusion/exclusion policies. The profiles must be created on the local Replicator instance on a Scrutinizer deployment or a headless instance registered with the primary reporter. Destination collectors for autoreplication must be part of the same cluster, whose primary reporter will have access to all collector configurations and current loads.
Note
Multiple seed profiles can be created to enable autoreplication for separate collector groups. All seed profiles are automatically discovered and processed when rebalancing and assigning exporters.
Creating collector profiles#
Auto Replicate Collector profiles define the destination collectors for a seed profile. A collector profile must be created for each collector and then assigned to the seed profile for its cluster.
To create a new collector profile:
View instructions
Navigate to Replicator > Collectors, and then click the + icon to create a new collector profile.
In the Add Replicator Profile tray, enter a name for the profile.
Select Auto Replicate Collector as the profile type, and then select the Replicator instance to create the profile on.
Enter the collector’s IP address and port number to use.
Enter an exporter count limit and a flow rate limit for the collector.
[OPTIONAL] Add a description for the collector/profile.
Click Save.
Repeat the above steps to create a collector profile for each destination collector, and then proceed to create the seed profile.
Creating the seed profile#
The Auto Replicate Seed profile contains inclusion and exclusion policies that define the exporters/streams that should be autoreplicated.
Flows from matching exporters are sent to one of the collectors defined by the collector profiles associated with the seed profile. Each new exporter is always assigned to the collector with the most available capacity. If a collector becomes overloaded (based on exporter count or flow rate) as a result, the exporter will be reassigned to a collector with the required capacity available.
To create the seed profile:
View instructions
Navigate to Replicator > Profiles, and then click the + icon to create a new profile.
In the Add Replicator Profile tray, enter a name for the profile.
Select Auto Replicate Seed as the profile type in the dropdown, and then select the Replicator instance to create the profile on.
[OPTIONAL] Add a description for the collector/profile.
Click Save, and then return to the main Profiles view.
Click on the newly created profile to open the configuration tray.
Create inclusion and exclusion policies to define source exporters for autoreplication.
Select the collector profiles of all destination collectors to associate with the seed profile (only collector profiles not currently associated with a seed profile can be selected).
Click Save.
Once the seed profile has been configured, enabling it will start autoreplication.
Note
By default, new exporters/streams are assigned to collectors once every hour, and collectors are checked to verify that they are not over capacity once a day. These times can be adjusted in /home/plixer/scrutinizer/files/conf/rebalance.yaml. Rebalancing can also be manually initiated via the seed profile in the Replicator UI. Exporter reassignment is kept to a minimum to improve system performance.
Managing collector assignments#
The Collector Profiles section of the configuration tray can be used to add or remove collector profiles from the seed profile.
When adding collector profiles, only profiles that are currently unlinked (created without a seed profile association or removed from another seed profile) will be available.