Enable Replication to Protect Against OOM-Related Data Loss

In Redis Software, an out-of-memory (OOM) event occurs when a node hosting one or more primary shards exhausts available memory and the operating system terminates the Redis process. If replication is disabled, the affected shard has no redundant copy, increasing recovery time and exposing the database to potential data loss.

Production databases in Redis Software should not run with replication factor 0 unless data-loss risk is explicitly accepted.

This risk increases in environments using AOF persistence or Redis modules, where memory usage during a restart or replay can temporarily exceed steady-state levels.

Configuring replication ensures that each primary shard maintains at least one replica shard on a separate node. If a primary fails due to OOM, Redis Software promotes the replica and the proxy layer continues routing traffic without endpoint changes.

This article covers Quick Risk Mitigation, High-Risk Configuration Patterns, OOM During Restart and AOF Replay, Configuration Steps, Verification, and Recovery Considerations.

This guidance applies to Redis Software clusters.

Quick Risk Mitigation

High-Risk Configuration Patterns

The following patterns significantly increase data-loss exposure during OOM events:

• Replication disabled (replication factor 0)

  • No redundant shard copy exists.

• Single-shard databases

  • All data resides on one primary shard.

• AOF enabled without replication

  • Recovery depends entirely on AOF integrity.

• Modules enabled (Search, JSON, Bloom, TimeSeries, others)

  • Modules may allocate additional memory during:

    • Index rebuilds

    • Bulk inserts

    • Large queries

    • Restore or replay operations

• Memory limit too close to node capacity

  • Insufficient headroom for:

    • Fragmentation

    • Transient allocations

    • AOF replay amplification

Combining these factors increases the likelihood of restart failure after OOM.

OOM During Restart and AOF Replay

When a node crashes due to OOM:

1. Redis Software attempts to restart the shard.

2. If AOF persistence is enabled, the dataset is reconstructed from the append-only file.

3. During replay, memory consumption can temporarily exceed steady-state usage.

4. If sufficient headroom is not available, the node may OOM again during restore.

If replication is enabled, the cluster can promote the replica instead of relying solely on AOF recovery.

If replication is disabled, recovery depends entirely on persistence integrity and available memory.

Common Root Causes of OOM

• Unbounded dataset growth

  • No eviction policy configured.

• Module-driven memory spikes

  • Search indexing, JSON expansion, time-series compaction.

• Bulk ingestion or migration jobs

  • Large write bursts without throttling.

• Memory fragmentation

  • RSS significantly exceeds logical dataset size.

• Insufficient memory headroom

  • Database memory allocation too close to physical limits.

Replication mitigates impact but does not eliminate underlying causes.

Prerequisites Before Enabling Replication

Verify cluster capacity

rladmin status nodes

Ensure sufficient free memory and CPU exist on separate nodes.

Confirm shard distribution
Replica shards must reside on different nodes than their primaries.

Review failover policy
Decide whether automatic failover should be enabled.

Enable Replication

Using the Cluster Manager UI

1. Navigate to Databases.

2. Select the database.

3. Click Edit.

4. Set Replication to at least 1.

5. Confirm replica placement on separate nodes.

6. Enable Auto Failover.

7. Save and allow rebalancing.

Using rladmin

Check current replication:

rladmin status databases

Increase replication factor:

rladmin tune db db:<db-id> replication 1

Verify shard placement:

rladmin status shards

Ensure every primary shard has a replica on a different node.

Verification

After enabling replication:

Confirm shard health

rladmin status shards

All shards should report ok.

Confirm memory headroom

rladmin status nodes

Maintain at least 20–25% memory headroom per node.

Test failover during maintenance
Validate application behavior during promotion.

Recovery Considerations After OOM

If replication is disabled:

• Restart may fail if memory pressure persists.

• Recovery depends on AOF or backup integrity.

• Data may be unrecoverable if persistence is corrupted.

If replication is enabled:

• Replica promotion restores availability.

• A new replica can be created after stabilization.

• Recovery time and RPO exposure are significantly reduced.

Best Practices

• Maintain 20–25% memory headroom.

• Enable replication for all production databases.

• Monitor fragmentation and RSS growth.

• Configure eviction for cache workloads.

• Throttle bulk ingestion.

• Use backups in addition to replication.