Fix Redis 'ERR offset is out of range' in SETRANGE and Bitmap Operations

The Scene

It's 2 AM. Monitoring fires — a spike in Redis errors. You tail the logs and see this repeating:

ERR offset is out of range

The broken feature is a daily active user (DAU) tracker built on Redis bitmaps — one bit per user ID. It worked fine in staging. Production has user IDs in the tens of millions, pushing into the billions. That's your problem.

What Triggers This Error

Redis raises ERR offset is out of range in two distinct scenarios:

SETBIT / GETBIT / BITCOUNT / BITPOS: The bit offset exceeds 2^32 - 1 (4,294,967,295), which corresponds to a 512MB string. Negative offsets also trigger it.
SETRANGE / GETRANGE: The offset is negative, or the resulting string would exceed Redis's hard 512MB string size limit.

Put the numbers plainly: bit offset 4,294,967,295 requires exactly 512MB of storage. Redis caps all strings at 512MB — bitmaps are just strings under the hood. One step past that and you hit this wall.

Debug Process

Step 1: Reproduce in redis-cli

Open redis-cli and confirm the boundary behavior:

# Offset just at the limit — works fine
SETBIT dau:2025-01-15 4294967294 1
# (integer) 0

# One step over the limit — triggers the error
SETBIT dau:2025-01-15 4294967296 1
# (error) ERR offset is out of range

# Negative offset — always invalid
SETBIT dau:2025-01-15 -1 1
# (error) ERR offset is out of range

# SETRANGE pushing past 512MB
SETRANGE mykey 536870912 "x"
# (error) ERR offset is out of range

Step 2: Find the actual offset your code is sending

Redis's error message doesn't include the offset value — you have to capture it yourself, before the call hits Redis. Wrap the failing operation with debug logging:

# Python (redis-py)
import redis
import logging

r = redis.Redis(host='localhost', port=6379)

def track_user_active(date_key, user_id):
    logging.debug(f"SETBIT {date_key} offset={user_id}")
    r.setbit(date_key, user_id, 1)

// Node.js (ioredis)
async function trackUserActive(dateKey, userId) {
  console.debug(`SETBIT ${dateKey} offset=${userId}`);
  await redis.setbit(dateKey, userId, 1);
}

Trigger the failing request and capture the offset. If user_id is something like 5000000000, you've found the root cause — it exceeds the 4,294,967,295 bitmap limit.

Step 3: Check where the offset comes from

Out-of-range offsets almost always trace back to one of these:

Auto-incrementing database IDs that have grown past 4 billion
Unix timestamps in milliseconds (current timestamp ~1.75 trillion — way over the limit)
Arithmetic that produces negative values (e.g., subtracting a base offset from a smaller number)
64-bit integers passed directly without range checks

# Check what your ID actually looks like
python3 -c "import time; print(int(time.time() * 1000))"  # milliseconds timestamp
# 1736899200000  ← this is 1.7 trillion, massively over limit

python3 -c "print(int(time.time()))"  # seconds timestamp
# 1736899200  ← this is fine, under 4.29 billion

Solutions

Option 1: Validate offsets before calling Redis (immediate fix)

Add a guard in your application code — reject anything outside the valid range before it reaches Redis:

# Python
MAX_BIT_OFFSET = 2**32 - 1  # 4,294,967,295

def track_user_active(date_key, user_id):
    if user_id  MAX_BIT_OFFSET:
        raise ValueError(f"user_id {user_id} is out of Redis bitmap range (0-{MAX_BIT_OFFSET})")
    r.setbit(date_key, user_id, 1)

// Node.js
const MAX_BIT_OFFSET = 4294967295;

async function trackUserActive(dateKey, userId) {
  if (userId  MAX_BIT_OFFSET) {
    throw new RangeError(`userId ${userId} is out of Redis bitmap range`);
  }
  await redis.setbit(dateKey, userId, 1);
}

Option 2: Segment large bitmaps (handles user IDs beyond 4 billion)

If your user IDs genuinely exceed 4 billion, split the bitmap into segments. Each segment key covers a range of up to 4,294,967,295 IDs:

# Python — segmented bitmap
SEGMENT_SIZE = 4_294_967_295

def track_user_active_segmented(date_key, user_id):
    segment = user_id // SEGMENT_SIZE
    local_offset = user_id % SEGMENT_SIZE
    key = f"{date_key}:seg:{segment}"
    r.setbit(key, local_offset, 1)

def is_user_active_segmented(date_key, user_id):
    segment = user_id // SEGMENT_SIZE
    local_offset = user_id % SEGMENT_SIZE
    key = f"{date_key}:seg:{segment}"
    return bool(r.getbit(key, local_offset))

def count_active_users(date_key, num_segments=10):
    total = 0
    for seg in range(num_segments):
        key = f"{date_key}:seg:{seg}"
        total += r.bitcount(key)
    return total

The tradeoff: BITCOUNT now iterates across segment keys. With a known max segment count, that's a negligible extra call — acceptable for most production workloads.

Option 3: Use HyperLogLog for cardinality estimation

If you only need approximate unique counts (within ~0.81% error) and don't need per-user lookups, HyperLogLog sidesteps the offset problem entirely:

# No offset issues — just add user IDs as elements
PFADD dau:2025-01-15 "user:5000000000" "user:8000000000"
# (integer) 1

PFCOUNT dau:2025-01-15
# (integer) 2

# Uses max 12KB of memory regardless of cardinality
DEBUG OBJECT dau:2025-01-15
# serializedlength:304  ← tiny

Option 4: Map user IDs to a bounded hash space

Hash each user ID into a fixed-size bitmap range. Best when IDs are sparse and you need predictable memory usage:

# Python — hash to fixed range
import hashlib

BITMAP_SIZE = 10_000_000  # 10M slots, adjust for your DAU scale

def track_user_active_hashed(date_key, user_id):
    # Deterministic hash — same user_id always maps to same bit
    h = int(hashlib.sha256(str(user_id).encode()).hexdigest(), 16)
    offset = h % BITMAP_SIZE
    r.setbit(date_key, offset, 1)

Warning: hash collisions mean two different users can map to the same bit, so BITCOUNT gives a lower bound, not exact count. Acceptable for approximate DAU dashboards.

Option 5: SETRANGE with a safe offset cap

If the error comes from SETRANGE with an offset derived from user input or calculated positions:

# Redis max string: 512MB = 536,870,912 bytes
MAX_STRING_OFFSET = 536_870_911

def safe_setrange(key, offset, value):
    if offset  MAX_STRING_OFFSET:
        raise ValueError(f"SETRANGE offset {offset} out of range (0-{MAX_STRING_OFFSET})")
    r.setrange(key, offset, value)

Verify the Fix

Deploy the fix, then run these sanity checks:

# Test at the valid boundary
SETBIT dau:test 4294967294 1
# (integer) 0  ← success

# Confirm the key exists with expected size
STRLEN dau:test
# (integer) 536870912  ← exactly 512MB (bitmap for 4294967295 bits)

# BITCOUNT should work cleanly
BITCOUNT dau:test
# (integer) 1

# Confirm the bit is set
GETBIT dau:test 4294967294
# (integer) 1

# Clean up the test key
DEL dau:test
# (integer) 1

Pull up your error monitoring dashboard — the ERR offset is out of range entries should go quiet. Ten to fifteen minutes of real traffic is enough to confirm the fix held.

Lessons Learned

This error almost always comes from treating Redis bitmaps like infinite arrays. They cap at 512MB — no exceptions.

Never pass raw external IDs as bitmap offsets without bounds checking. User IDs, order IDs, and timestamps (especially millisecond timestamps) can easily exceed 4 billion.
Staging won't catch this if your staging ID pool is small. The bug only surfaces when IDs grow past the 4.29 billion threshold in production. Add the bounds check in code on day one.
Millisecond timestamps as offsets are a trap. Unix time in milliseconds is already ~1.75 trillion — always use seconds (~1.75 billion in 2026, with headroom for another ~80 years) or a derived smaller value.
Negative offsets are always invalid. If you compute offsets via arithmetic (e.g., subtracting a base value), add an explicit >= 0 check before every Redis bit operation.
For growing user bases, design bitmap segmentation upfront — retrofitting it after IDs exceed 4 billion means a data migration under pressure.