In Redis Software, the Metrics UI may occasionally display 'N/A' or blank values instead of expected performance metrics. This often occurs during events like shard failovers/migrations, when system resources are constrained, or if the metrics configuration isn't properly tuned for your environment. To help you resolve these visibility issues, this article covers how to begin by Identifying the Issue, then walks through a Step-by-Step Resolution, and provides guidance for Common Troubleshooting Scenarios. With the right steps, you can restore full metrics functionality and ensure accurate monitoring of your Redis cluster.
Identifying the Issue
The issue presents as missing metrics (e.g., 'N/A' values or blank graphs) for nodes, databases, or shards. Causes may include:
- Shard migration, failover, or resharding events
- Exceeding UI display limits (128+ shards)
- Resource constraints on the cluster master
- Misconfigurations in metrics engine or exporters
- On older exporters (/metrics or /v1), UI aggregation gaps can appear in large clusters. Use the /v2 stream exporter when available to ensure complete metric visibility.
Step-by-Step Resolution
1. Refresh and Reauthenticate
- Reload the Metrics UI.
- Reauthenticate if prompted to ensure access to real-time metrics.
2. Check Cluster and Node Health
- Use the Cluster Manager UI or
rladmin status. - Confirm all nodes are reachable, healthy, and not overloaded.
Check that there are no ongoing or stuck changes.
Using the web UI, you will see this as “Active Change Pending”
In the rladmin status extra all output, you will see this under the EXEC_STATE column for the database.
- Ensure the master node has adequate CPU, memory, and disk space.
3. Validate Metrics Engine Configuration
- For Redis 7.8.2+: Verify the metrics stream engine is enabled
- Redis 8+ recommended: Verify the metrics stream engine is enabled.
- For Redis 7.8-7.22, the stream engine may be available on supported patch levels; otherwise use the legacy /metrics or /v1 path.
- If you use Prometheus or Grafana, confirm external tool integration and data flow are functioning properly.
4. Prefer the /v2 Stream Exporter on Redis 8
If you’re running Redis Software 8 or later, scrape https://<node>:8070/v2 in your external monitoring tools.
The /v2 exporter provides raw, non-aggregated time-series data and eliminates known UI aggregation gaps at larger scale.
Migrate existing dashboards using Redis v1→v2 PromQL mapping or prebuilt v2 dashboards.
5. Inspect for Shard Migration or UI Limitations
- If >128 shards exist, use Prometheus or Grafana to view metrics (UI has known limitations).
- During shard migrations, failovers, or resharding, temporary 'N/A' displays are expected. Wait for operations to complete.
6. Review System Logs
- Check logs in
/var/opt/redislabs/log/for:metrics_exporter.logevent_log.log- System alerts (e.g., disk, memory, network)
If E-mail alerts are enabled, check if you’ve received an alert from the cluster (for low disk space, memory, network, etc.)
Confirm metrics-related services are active with the command supervisorctl status all.
7. Switch Cluster Master (if needed)
-
If the master node is under resource pressure, migrate the master role to a healthier node:
rladmin cluster master set <node_id>
8. Reproduce and Report
- If unresolved:
- Document observed behavior and UI screenshots.
- Collect logs and Redis version.
- Open a support ticket and attach findings.
Common Troubleshooting Scenarios
Metrics Missing During Shard Migration
- Expected behavior. Wait for resharding or node changes to complete before metrics return.
UI Limitations with Large Clusters
- Clusters with over 128 shards may not fully populate the UI.
- Use Prometheus and Grafana for visibility at scale.
Resource Bottlenecks
- CPU or disk pressure on the master can block metrics.
- Reallocate roles or scale up node capacity.
Metrics API Errors
- A
400 Bad Requestor similar when switching nodes may indicate a software bug. - Upgrade to the latest supported Redis version.
0 comments
Please sign in to leave a comment.