System Health#
The System Health page provides administrators a single place to monitor the status of every server in a Scrutinizer cluster. From one view, administrators can check whether servers are running normally, spot resource problems before they become outages, review the health of individual services, and dig into logs without needing to log in to each machine separately.
Accessing System Health#
There are three ways to access the System Health page:
Admin Dashboard: The System Health table is displayed at the top of the Admin Dashboard. Clicking any row in the table opens the detail page for that specific server.
Stethoscope Icon: Clicking the stethoscope icon in the upper-right corner of the System Health table opens the main System Health page, which shows all servers with full resource detail.
Server Health LED: Clicking the Server Health status icon in the upper-right corner of the Admin Dashboard page takes you to the full System Health view.
Note
The System Health table in Admin Dashboard is a quick-read summary. The full System Health view (accessed via the stethoscope icon and/or Server Health LED indicator) shows complete resource bars, upload/download options, and direct access to logs and services for each server.
Clicking the Exporter Health icon opens the Admin > Exporters page.
Viewing System Health#
System Health table#
The System Health table on the Admin Dashboard provides a concise, sortable overview of every server in the cluster. Each row represents one server and shows:
Column |
Description |
|---|---|
Name |
The server’s hostname and its role icon |
Resource |
The overall health of CPU, memory, pressure, and load metrics |
Disk |
The overall health of disk usage across all monitored partitions |
Services |
Whether all monitored services are running as expected |
Health Checks |
The combined health of database connections, cluster communication, and automated checks |
CPU |
Current CPU usage percentage |
Memory |
Current memory usage percentage |
Disk % |
Current disk usage percentage (highest across monitored partitions) |
Last Seen |
When health data was last successfully collected from this server |
Click any row to go directly to that server’s detail page, or click the stethoscope icon in the upper-right corner of the System Health table to open the full System Health view.
Full System Health view#
The main System Health view shows each server as an expanded card that displays:
The server hostname and its role name
The server’s current health status (Green / Yellow / Red) and status category badges (R, D, S, SYS)
Resource usage bars showing how much of each resource is currently in use:
CPU and Memory
CPU Pressure and IO Pressure
Swap
Load average shown as three numbers representing the last 1, 5, and 15 minutes, along with the total CPU core count
Disk usage for the root filesystem, database storage, and the flow spool directory
A Services counter (for example, 10 / 10) showing how many of the server’s monitored services are healthy
The time the data was last collected
Upload and Download buttons for sharing diagnostic data with Plixer Support
Click any card to open that server’s full detail page.
Note
All data displayed in the System Health page is refreshed every 15 seconds, so the information available is always current within that window.
The System Health view can be displayed in two layouts:
List View: A compact table showing each server as a single row with color-coded status squares for each health category, plus CPU, Memory, Disk %, and Last Seen columns. Useful for environments with many servers where a quick scan is more practical than expanded cards.
Grid View: Shows each server as an expanded card with full resource bar graphs, giving you a richer at-a-glance view without having to open individual detail pages. This is the default view.
Use the view toggle buttons in the upper-right corner of the page to switch between these layouts.
Health Status#
Every server, and the cluster as a whole, is always in one of three states:
Status |
Color |
What it means |
|---|---|---|
Healthy |
Green |
Everything is operating within normal ranges |
Degraded |
Yellow |
Something is approaching a problem threshold and may need attention |
Unhealthy |
Red |
Something has exceeded a critical threshold and needs immediate attention |
Health status rolls up from the bottom. The most severe issue on any individual server determines that server’s status. The most severe status across all servers determines what the cluster-level indicator shows in the upper-right corner.
For example, if a single check on a secondary server goes critical, the top-level indicator turns red, even if all other servers are healthy.
Status Category Badges#
Each server status card (grid view) displays up to four colored letter badges, one for each health category being monitored on that server. The badge color reflects the health status of that category.
Badge |
Category |
What it covers |
|---|---|---|
R |
Resource Status |
CPU, memory, pressure, swap, and load metrics |
D |
Disk Status |
Usage across all monitored disk partitions |
S |
Service Status |
The running state of all monitored Scrutinizer services |
SYS |
Systems Status |
Database connection health and cluster communication |
Server Actions#
Each server status card (grid view) has four action buttons in the header row:
Button |
What it does |
|---|---|
View Details (→) |
Opens the full detail page for that server |
Device Performance (ⓘ) |
Opens the System Performance page for that server, showing disk utilization, CPU and memory usage relative to exporters, and feature resource consumption |
Upload to Support (↑) |
Sends a diagnostic bundle for that server directly to Plixer Support |
Download Bundle (↓) |
Downloads a diagnostic bundle for that server to your computer |
Device Performance#
Clicking the Device Performance button on a server card opens the System Performance page for that server. This page helps understand how the server’s hardware capacity relates to the amount of data it is processing.
The page displays the server’s IP address at the top and offers four chart views, selectable from the dropdown in the upper-right corner:
Chart |
Description |
|---|---|
Disk Utilization |
Current and predicted hard disk usage, data retention per collection interval, and disk utilization broken down by interval. Also shows the server’s core count, total memory, suggested disk space, and total disk space under Properties |
Cores X Exporters |
How CPU core count relates to the number of exporters being processed |
Memory X Exporters |
How available memory relates to the number of exporters being processed |
Feature Resources |
Resource consumption broken down by Scrutinizer feature |
Use the Back button in the upper-right corner to return to the previous page.
Server Detail Page#
The detail page provides a complete picture of a single server’s health. At the top of the page, the server’s operating system, agent version, IP address, and the last time health data was successfully collected are displayed.
The rest of the page is divided into the following sections:
Resource#
This section provides an overview of system resource utilization and hardware performance metrics.
Metric |
Description |
Warning / Critical |
|---|---|---|
CPU |
Percentage of CPU currently in use. High values can slow flow processing, queries, and the UI. |
70% / 90% |
Memory |
Percentage of physical memory currently in use. High values increase the risk of service instability or out-of-memory events. |
80% / 90% |
CPU Pressure (some) |
Percentage of time at least one process was waiting for CPU resources. Indicates CPU contention. |
25% / 50% |
CPU Pressure (full) |
Percentage of time all processes were waiting for CPU resources. Indicates severe CPU saturation. |
5% / 10% |
Mem Pressure (some) |
Percentage of time at least one process was waiting for memory resources. Can cause slower query and UI performance. |
25% / 50% |
Mem Pressure (full) |
Percentage of time all processes were waiting for memory resources. Indicates significant memory pressure. |
5% / 10% |
IO Pressure (some) |
Percentage of time at least one process was waiting for disk I/O operations. |
25% / 50% |
IO Pressure (full) |
Percentage of time all processes were waiting for disk I/O operations. Indicates disk I/O is a system bottleneck. |
5% / 10% |
Swap |
Percentage of configured swap space currently in use. Higher values can significantly reduce performance. |
50% / 90% |
Scrutinizer categorizes resource pressure into two distinct operational states:
Some: At least one program/process had to wait for the resource. This is an early warning sign that the server is starting to get busy and delays could happen soon.
Full: Every running program was completely stopped, waiting for the resource. This is a serious condition that usually causes noticeable slowdowns in system performance.
Disk#
This section shows how full each of the server’s key storage locations is.
Location |
Data stored in the location |
|---|---|
root (/) |
Stores operating system files, logs, journals, and temporary data. |
spools |
Stores flow records temporarily before they are written to the database. |
database |
Stores PostgreSQL data used by Scrutinizer for flow and reporting data. |
Services#
This section lists every Scrutinizer-related service on the server, along with its current state and memory usage. The services counter on the server card (for example, 10 / 10) reflects the number of healthy services shown here.
Service |
Description |
|---|---|
plixer_api |
Backend API service used by Scrutinizer components and the web interface. |
plixer_autoreplicate |
Manages automated replication-related tasks between Scrutinizer systems. |
plixer_db |
Database management service used to monitor and maintain PostgreSQL operations. |
plixer_collector |
Receives, processes, and writes flow data to the database. |
plixer_fcgi |
FastCGI service that processes API and application requests. |
plixer_host_index |
Maintains host indexing information used for searching and reporting. |
plixer_mcp |
Internal management and control service used by Scrutinizer components. |
plixer_snmp |
Collects and processes SNMP data from monitored devices. |
plixer_webapp |
Scrutinizer web application service. |
memcached |
In-memory cache used to improve application performance by reducing database queries. |
nginx |
Web server and reverse proxy that handles incoming HTTP and HTTPS requests. |
pgbouncer |
PostgreSQL connection pooler that manages and reuses database connections. |
replicator |
Transfers data between Scrutinizer systems in a replication deployment. |
Note
Some services are designed to run a task and stop cleanly. A status of active with a state of exited is expected for these services and does not indicate a problem. The state to watch for is failed in either the Status or State column.
Health Checks#
This section shows the database connection health, cluster communication metrics, and automated system checks in one place. Each item displays a color indicator and, where applicable, a current value.
Metric |
Description |
|---|---|
SSL Certificate Expiry |
Number of days remaining before the server’s SSL certificate expires. |
Time Synchronization |
Time difference between the server clock and its configured NTP source. |
DNS Resolution |
Verifies that hostnames can be resolved through the configured DNS server. |
Critical Service Ports |
Verifies that required service ports are listening and accepting connections. |
Spool Directory |
Verifies that the collector spool directory exists and is accessible. |
Meta Sync |
Verifies that metadata synchronization between cluster nodes is functioning correctly. |
Time Sync |
Clock offset from the configured time source, measured in milliseconds. |
History Maintenance |
Number of failed history maintenance jobs responsible for rollups, retention, and partition management. |
Expired Tables |
Number of expired database partitions waiting to be removed. |
Storage Artifacts |
Number of orphaned or stray storage objects detected on disk. |
api |
Response time for health checks performed against the node’s API endpoint. |
db_direct |
Response time for direct database connectivity checks. |
db_pgbouncer |
Response time for database connectivity checks through PgBouncer. |
PG Connections |
Percentage of PostgreSQL connection slots currently in use. |
PG Frozen XID Age |
PostgreSQL transaction ID age used to monitor autovacuum health and prevent transaction wraparound. |
Log Files#
This section lists all the log files Scrutinizer monitors on a given server. This makes it easy to spot which component is generating errors without having to access the server directly.
Column |
What it shows |
|---|---|
Log File |
The name of the log and where it lives on the server |
Format |
The type of log file (affects how errors and warnings are counted) |
Size |
How large the file currently is |
Lines |
The total number of log entries |
Errors |
How many error-level entries were written during the selected time window |
Warnings |
How many warning-level entries were written during the selected time window |
Modified |
When the log was last written to |
The Errors and Warnings counts reflect activity within the currently selected time window, and not lifetime totals. As the window moves forward, counts from quieter periods will naturally drop. These numbers cannot be reset manually.
If a log file shows — for size and lines with a date of 0000-12-31, it means that feature is either not configured or has not yet produced any data on this server (for example, a cloud flow source that is not in use).
Clicking any log file name opens it in the log viewer. From there you can:
Filter by level: Show only errors, warnings, info, or debug entries, or view all levels together.
Search: Type to filter the loaded entries by keyword.
Tail: Click TAIL to watch the log update live. The viewer loads the most recent entries and streams new lines as they are written, scrolling automatically to keep up. Click Stop Tail to stop the live feed and return to normal browsing.
Health Events Log#
The Health Events log is a focused record of every time something changed in System Health (e.g. status transitions, automated responses, and skipped actions). This is the first place to check to understand what happened on a server and when.
There are three types of entries:
Entry |
Description |
|---|---|
Health state changed |
A metric or check crossed a threshold and the server’s status changed (for example, from Green to Yellow) |
Action executed |
Scrutinizer automatically took a corrective action, with a note on whether it succeeded |
Action skipped |
A corrective action was triggered but not taken, along with the reason (for example, the action is still in its cooldown period) |
Each entry is tagged with a severity that matches the resulting health state: Info for a return to green, Warning for a degraded state, and Error for an unhealthy state.
The Health Events log is separate from the general System Health log, which records the internal workings of the health monitoring service itself such as startup events, collection cycles, and internal errors. The Health Events log answers “what changed and when?”, while the System Health log is for deeper troubleshooting of the monitoring system itself.
Downloading and Uploading System Health Data#
If further assistance is needed, a support bundle can be sent directly from the System Health view. A support bundle is a compressed archive that contains the server’s current status, log files, and configuration.
To download a bundle to your computer: Click the Download icon on a server card. The bundle will be saved as a file you can attach to a support case.
To send a bundle directly to Plixer: Click the Upload icon on a server card. A loading spinner appears while the transfer is in progress, then changes to a checkmark when the upload is complete. No additional confirmation is required.
There are two available options for scope:
Single server: Use the Upload or Download icon on that server’s card to send data for that server only.
Entire cluster: Use the Upload or Download icon at the upper-right corner of the main System Health page to send data for all servers at once.