Troubleshooting Missing or Incomplete Indexing in Redis Open Source

In Redis Open Source 8 and later, Redis Search automatically indexes JSON and Hash documents as you write or update data.
If documents don’t appear in search results or indexes seem incomplete, the issue is usually related to asynchronous indexing, schema mismatches, or cluster configuration.

This guide helps you identify and fix incomplete or inconsistent indexing, including verification commands, cleanup steps, and best practices.

(For Redis 7.x users, the same behavior applies when using Redis Stack.)

Covered:

• How Redis indexing works

• Quick verification commands

• Common causes of missing indexes

• Step-by-step troubleshooting

• Cluster and OSS Cluster API limitations

• Best practices and prevention

How Indexing Works

Redis indexes documents automatically when keys match the prefix and field types defined in an index schema.

Index lifecycle

Quick Verification Commands

Run these quick checks to confirm index status (the first command returns a list of indexes that can be passed to the later commands):

FT._LIST
FT.INFO <index> | grep indexing
FT.INFO <index> | grep hash_indexing_failures
FT.SEARCH <index> "*" LIMIT 0 10

Common Causes of Missing or Incomplete Indexing

1. Asynchronous indexing not finished
   Index creation triggers background indexing for existing data. Searches may appear incomplete until finished.

2. Schema or data type mismatch
   Mismatched field types (for example, string in a NUMERIC field) cause indexing failures.

3. Incorrect prefix
   Keys not matching the PREFIX value in FT.CREATE are ignored.

4. OSS Cluster API limitation
   Redis Search isn't supported on sharded databases using the OSS Cluster API. Indexes may appear only on certain nodes.

5. Partial deletions
   FT.DROPINDEX <index> DD deletes only successfully indexed documents. Others remain until cleaned up manually.

6. Timeouts or load
   Large or busy datasets can delay indexing or cause it to stall.

7. Frequent updates (high churn)
   Heavy write activity can cause fragmentation and delayed index consistency.

8. Missing data after index creation
   In index created with the option SKIPINITIALSCAN during FT.CREATE will not index any existing keys in the database, only new keys added/modified after the index creation.

Step-by-Step Troubleshooting

1. Check Index Status

Confirm:

• indexing: 0 → Indexing complete

• percent_indexed: 1 → 100 % coverage

• hash_indexing_failures: 0 → No skipped records

If indexing is still active, wait until it completes or resolves any memory or CPU bottlenecks.

2. Investigate Indexing Failures

If hash_indexing_failures > 0:

• Review affected documents for data type mismatches.

• Correct or remove invalid entries.

• Rebuild the index if failures persist.

3. Validate Prefix and Schema

• Ensure the prefix in FT.CREATE matches your document keys (user: vs users:).

• Confirm the schema field types align with stored values.
  Example: NUMERIC fields must contain integers or floats, not strings.

4. Check for Leftover or Unindexed Keys

After dropping an index with DD, scan for remaining keys:

Safely delete them using non-blocking cleanup:

5. When to Rebuild an Index

Recreate the index only if:

• Indexing failures persist after fixing data

• Schema fields or types were changed

• Modules were not loaded when the index was first created

• Index was inadvertently created with SKIPINITIALSCAN

6. Cluster Limitations and OSS Cluster API Behavior

Redis Search is not supported for sharded databases under the OSS Cluster API.
To use search and query:

• Disable OSS Cluster API and use a proxy-based setup

• Limit search to single-shard environments only for testing

7. Monitor High-Churn Environments

Frequent updates can cause index bloat and slow queries.
Tune garbage-collection parameters to improve performance:

Separate frequently updated data from indexed (read-heavy) data whenever possible.

Best Practices

1. Wait for indexing to finish before deleting or rebuilding indexes.

2. Monitor hash_indexing_failures regularly to catch schema drift early.

3. Use consistent prefixes to simplify lifecycle management.

4. Use UNLINK instead of DEL for non-blocking cleanup.

5. Avoid OSS Cluster API for Redis Search in production.

6. Use aliasing or versioning for zero-downtime schema changes.

7. Monitor performance with FT.INFO, FT.PROFILE, and memory stats.

Common Issues and Solutions