Redis Software clusters rely on consistent internal node-to-node communication (control-plane services, database services, and DNS) to form the cluster and validate health.
When nodes show as UNREACHABLE shortly after provisioning, the most common causes are:
DNS not working as expected (wrong/missing delegation records, wrong node records, or cluster DNS service not reachable)
Internal network policy/firewall rules blocking required intra-cluster traffic (usually not a Redis “configuration/tuning” problem)
Quick Fix
| What you see | What to do |
|---|---|
| Nodes show UNREACHABLE right after cluster creation | Validate DNS setup and reachability, and confirm firewall/security groups allow required intra-cluster traffic using the official port matrix |
| Cluster is degraded, but all nodes are running | Check for DNS misconfiguration or blocked internal traffic between nodes (control plane and database services); validate against the port matrix |
| rladmin status shows nodes missing or not OK | Confirm each node can resolve and reach peer nodes, then re-check cluster status |
| Connectivity tests fail between nodes | Update firewall rules/security groups to allow required intra-cluster communication and DNS traffic using the official port matrix |
| Hostnames resolve to incorrect IPs | Fix DNS records or /etc/hosts so each node maps to the correct IP |
Prerequisites
SSH access to all cluster nodes
Ability to run rladmin, getent, and basic network tests (for example, ping, nc)
Visibility into your DNS setup (delegation records) and network policy (firewall/security groups)
Step-by-Step Instructions
1) Validate DNS resolution (what Redis Software expects)
By default, Redis Software deployments use DNS to communicate between nodes, and each node runs a bundled DNS server for internal cluster functions (HA, failover, migration, etc.). See Configure cluster DNS for the supported models and expectations.
A. Confirm which DNS/connection model you’re using
If you use URL-based connections, ensure your cluster’s FQDN and node records are set up correctly (NS delegation + node A records). Follow DNS Setup for Redis Cluster Access for the required record types and examples.
If you use IP-based connections, you may not rely on DNS delegation for client access, but you must still ensure node reachability and the required internal connectivity remain correct.
B. From each node, verify forward resolution of peer node names
getent hosts <peer-hostname>Must return the correct IP for each peer node.
Ensure there are no duplicate or stale mappings (especially if /etc/hosts is used).
C. Confirm DNS service reachability
The DNS setup guide includes a practical connectivity check to confirm that DNS queries can reach the nodes (DNS is typically UDP, and often TCP as well).
nc -vzu <node_ip> 53Important: Reverse DNS (PTR records) is not required for Redis Software clusters.
If reverse DNS is missing, focus first on correct delegation/forward records and network reachability.
2) Validate node-to-node connectivity (beyond “ping”)
Once name resolution is correct, the next step is to confirm that nodes can reach each other over the required Redis Software intra-cluster services (control plane, internode communication, shard traffic, database traffic, metrics/internal services as applicable).
Use Network port configurations as the source of truth for what must be allowed in your environment, rather than hardcoding a list into your firewall request.
Operational notes from that same doc (links below):
When you create a new database, you must verify that the assigned endpoint ports are open; the cluster won’t verify this for you.
If your nodes are on different VLANs/segments, you may need broader allowances between nodes (see the VLAN note in the port configuration doc).
About ping/ICMP: Ping is a quick way to validate basic routing/reachability. However, Redis Software requires the correct TCP/UDP ports between nodes; use the Network port configurations as the source of truth for firewall rules.
3) Re-check cluster status
rladmin statusSee rladmin status for command details and expected output.
- If DNS and internal connectivity are corrected, nodes should return to OK status, and cluster health should recover.
Troubleshooting
| Issue | Likely cause | Resolution |
|---|---|---|
| Node names resolve to the wrong IP | Incorrect or missing DNS records or /etc/hosts entries | Correct delegation and node records, then re-test from every node using getent hosts |
| DNS lookups time out or fail | DNS traffic is blocked or OS-level port conflict | Ensure DNS traffic is permitted and review port 53 conflicts if applicable |
| TCP/UDP connectivity fails for internal services | Firewall or security group blocking required intra-cluster traffic | Use the official Network port configurations to allow required communication |
| Stateful firewall rules drop traffic unexpectedly | Connection tracking behavior affecting stateful rules | Review iptables performance optimization guidance here and adjust firewall rules accordingly |
| firewalld (RHEL/CentOS) blocks required ports | Default firewalld zones block most ports | Configure firewalld using Redis-provided services and recommended settings. Read more at Configure CentOS/RHEL firewall |
Key takeaways
UNREACHABLE nodes right after provisioning are usually a DNS and/or internal network allowlist issue, not a Redis database configuration issue.
Validate the expected DNS setup (FQDN + delegation + node records) and that nodes can reach DNS as needed.
Validate intra-cluster connectivity using the official Network port configurations matrix; avoid guessing ports or relying on ping alone.
Reverse DNS (PTR) is not required for Redis Software clusters.
0 comments
Please sign in to leave a comment.