Redis Benchmark with redis-plus-plus

This project benchmarks various Redis operations using Google Benchmark library to compare different data access patterns.

Overview

The benchmark compares two scenarios for storing and retrieving datagram data in Redis, with parametric testing based on:

• N: Number of different datagram types
• M: Number of instances per type
• Total datagrams: N × M

Scenarios

Scenario 1: Individual Operations

Each datagram is stored as a separate key, accessed individually:

• Data structure: DGRAM-0:0, DGRAM-0:1, ..., DGRAM-1:0, ...
• Access pattern: SCAN for pattern matching + individual GET operations
• For each type: begin -> SCAN "TYPE:*" -> GET each instance -> end

Characteristics:

• High network round-trips (one per datagram)
• Scales poorly with increasing N (number of types)
• Simple but inefficient for batch operations

Scenario 2: Batch Operations (Hash-based)

Each datagram type is stored in its own Redis hash:

• Data structure: hash:DGRAM-0 = {"0": "value", "1": "value", ...}
• Access pattern: HGETALL per type (all instances retrieved at once)
• For each type: HGETALL hash:TYPE (single operation)

Characteristics:

• Minimal network round-trips (one per type)
• Excellent scaling with both N and M
• Optimal for batch retrieval scenarios

Requirements

Dependencies

• CMake 3.15+
• C++17 compatible compiler (g++, clang++)
• Redis server (running)
• Google Benchmark
• redis-plus-plus
• hiredis

Installation on Ubuntu/Debian:

# Update package list
sudo apt-get update

# Install build tools and dependencies
sudo apt-get install -y build-essential cmake git pkg-config

# Install Redis server
sudo apt-get install -y redis-server

# Install hiredis
sudo apt-get install -y libhiredis-dev

# Install Google Benchmark
sudo apt-get install -y libbenchmark-dev

# Install redis-plus-plus (build from source)
cd /tmp
git clone https://github.com/sewenew/redis-plus-plus.git
cd redis-plus-plus
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j4
sudo make install
sudo ldconfig

Building

mkdir build
cd build
cmake ..
make

Running

Ensure Redis server is running:

# Start Redis server
sudo systemctl start redis-server

# Check status
sudo systemctl status redis-server

# Or run manually
redis-server

Run the benchmark:

# Simple run (console output only)
cd build
./redis_benchmark

# Or use the automated script (saves to multiple formats)
./run_benchmark.sh

Filtering Benchmarks

# Only Scenario 1
./redis_benchmark --benchmark_filter=BM_Scenario1

# Only Scenario 2
./redis_benchmark --benchmark_filter=BM_Scenario2

# Specific N and M values (e.g., N=10, M=1000)
./redis_benchmark --benchmark_filter=BM_Scenario.*/10/1000

Save Results to Files

# Method 1: Use the automated script (recommended)
./run_benchmark.sh
# Results saved to ./results/ directory with timestamp
# Includes: TXT, JSON, and CSV formats

# Method 2: Manual export
cd build

# Save as JSON
./redis_benchmark --benchmark_out=results.json --benchmark_out_format=json

# Save as CSV
./redis_benchmark --benchmark_out=results.csv --benchmark_out_format=csv

# Save console output
./redis_benchmark | tee results.txt

# Run with multiple repetitions
./redis_benchmark --benchmark_repetitions=10

Output Metrics

Benchmark results include:

• Time: Average time per iteration
• CPU: CPU time consumed
• Iterations: Number of times the benchmark was executed
• items_per_second: Throughput (items processed per second)
• N_types: Number of datagram types
• M_instances: Number of instances per type
• total_datagrams: Total number of datagrams (N × M)

Architecture

Data Generation

• Datagram types are named DGRAM-0, DGRAM-1, DGRAM-2, etc.
• Each instance is identified by type and instance ID
• Values contain random 100-character strings (lorem ipsum style)

Scenario 1 Implementation

// Store: Each datagram as a separate key
redis.set("DGRAM-0:0", "lorem...");
redis.set("DGRAM-0:1", "lorem...");

// Retrieve: SCAN + GET for each type
for each type:
    SCAN "DGRAM-{type}:*"
    for each key:
        GET key

Scenario 2 Implementation

// Store: Each type in its own hash
redis.hset("hash:DGRAM-0", "0", "lorem...");
redis.hset("hash:DGRAM-0", "1", "lorem...");

// Retrieve: HGETALL for each type
for each type:
    HGETALL "hash:DGRAM-{type}"  // All instances at once

Important Notes

• Tests clear the Redis database before running (FLUSHDB)
• Each test performs its own cleanup
• Warning: Do not run on production Redis instances!
• Redis server must be running on localhost:6379
• Benchmark results may vary based on system performance and Redis configuration

Troubleshooting

Redis Connection Issues

# Check Redis server status
sudo systemctl status redis-server

# Check if Redis is listening
netstat -tlnp | grep redis

# Test connection
redis-cli ping
# Should return: PONG

Build Errors

# If redis-plus-plus is not found, specify library path manually
cmake .. -DREDIS_PLUS_PLUS_HEADER=/usr/local/include \
         -DREDIS_PLUS_PLUS_LIB=/usr/local/lib/libredis++.so

# Check if libraries are in the cache
ldconfig -p | grep redis++
ldconfig -p | grep hiredis

Performance Issues

• Disable CPU frequency scaling for more consistent results
• Build in Release mode (already configured in CMakeLists.txt)
• Ensure Redis is not under load from other processes