← Back to Frameworks

DragonflyDB

Modern Redis & Memcached replacement. Multi-threaded architecture achieves linear scaling. 4.5x higher throughput and scalability according to CPU cores.

cache-serverRedis-alternativemulti-threadedhigh-performancein-memorylinear-scaling

DragonflyDB

DragonflyDB is a next-generation in-memory data store developed as a modern Redis & Memcached alternative. With multi-threaded architecture achieving linear scaling, it provides 4.5x higher throughput and scalability according to CPU cores compared to traditional solutions.

Overview

DragonflyDB is positioned as the most promising next-generation cache solution in 2024. It excels in high-concurrency, CPU-intensive workloads and fundamentally solves Redis's single-thread bottleneck by achieving linear performance improvements with CPU core scaling. By adopting Redis-compatible protocol, migration from existing Redis clients is straightforward, enabling performance improvements with minimal changes.

Features

Key Capabilities

  • Multi-threaded Architecture: Linear performance improvement according to CPU core count
  • Redis Compatibility: Use existing Redis client libraries directly
  • High Throughput: 4.5x+ processing performance compared to traditional Redis
  • Memory Efficiency: Optimized memory usage patterns
  • Snapshot Consistency: Advanced snapshot functionality ensuring data integrity
  • Modern C++ Implementation: Efficient system design with modern C++

Architecture Characteristics

DragonflyDB's technical advantages:

  • Share-Nothing Architecture: Each thread manages independent data regions
  • Fiber-Based: Lightweight cooperative multitasking
  • Efficient Memory Management: Elimination of GC overhead
  • Optimized Data Structures: Internal structures specialized for CPU-intensive processing

Performance Characteristics

  • Processing capacity of 300,000+ requests/second
  • Scalability proportional to CPU core count
  • Optimized memory usage
  • Stable operation with low latency

Pros and Cons

Advantages

  • Outstanding Performance: 4.5x+ higher throughput than Redis
  • Linear Scaling: Reliable performance improvement with CPU core increase
  • Easy Migration: Simple migration process with Redis-compatible protocol
  • High Concurrency: Efficient handling of massive simultaneous client connections
  • Memory Efficiency: High capacity through optimized memory usage
  • Modern Design: Future-proof architecture with latest technology

Disadvantages

  • Emerging Technology: Limited long-term stability track record due to recent origin
  • Ecosystem: Fewer third-party tools compared to Redis
  • Learning Curve: Need to accumulate unique operational knowledge
  • Community: Smaller community compared to Redis
  • Feature Limitations: Some Redis features may not be implemented
  • Compatibility Risk: Limitations in complete Redis compatibility guarantee

References

Code Examples

Installation Using Docker

# Start DragonflyDB container (Linux)
docker run --network=host --ulimit memlock=-1 docker.dragonflydb.io/dragonflydb/dragonfly

# Port mapping for macOS
docker run -p 6379:6379 --ulimit memlock=-1 docker.dragonflydb.io/dragonflydb/dragonfly

# Start with configuration options
docker run -p 6379:6379 --ulimit memlock=-1 \
  docker.dragonflydb.io/dragonflydb/dragonfly \
  --logtostderr --requirepass=youshallnotpass --cache_mode=true \
  --maxmemory=4gb --keys_output_limit=12288

Installation from Binary

# Download binary
wget https://github.com/dragonflydb/dragonfly/releases/latest/download/dragonfly-x86_64
chmod +x dragonfly-x86_64

# Basic startup
./dragonfly-x86_64

# Start with configuration options
./dragonfly-x86_64 --logtostderr --requirepass=mypassword \
  --cache_mode=true --dbnum=1 --bind=localhost --port=6379 \
  --maxmemory=8gb --keys_output_limit=12288 --dbfilename=dump.rdb

Basic Operations with Redis-cli

# Connect to DragonflyDB
redis-cli -h localhost -p 6379

# With authentication
redis-cli -h localhost -p 6379 -a youshallnotpass

# Basic key-value operations
127.0.0.1:6379> SET hello world
OK
127.0.0.1:6379> GET hello
"world"
127.0.0.1:6379> KEYS *
1) "hello"

# Set data with expiration
127.0.0.1:6379> SETEX temp_key 300 "temporary value"
OK
127.0.0.1:6379> TTL temp_key
(integer) 296

# List operations
127.0.0.1:6379> LPUSH tasks "task1" "task2" "task3"
(integer) 3
127.0.0.1:6379> LRANGE tasks 0 -1
1) "task3"
2) "task2" 
3) "task1"

# Hash operations
127.0.0.1:6379> HSET user:1000 name "John Doe" email "[email protected]" age 30
(integer) 3
127.0.0.1:6379> HGETALL user:1000
1) "name"
2) "John Doe"
3) "email"
4) "[email protected]"
5) "age"
6) "30"

# Set operations
127.0.0.1:6379> SADD tags "cache" "database" "performance"
(integer) 3
127.0.0.1:6379> SMEMBERS tags
1) "performance"
2) "database"
3) "cache"

Using Python Client

import redis
import time

# Connect to DragonflyDB
r = redis.Redis(host='localhost', port=6379, decode_responses=True)

# With authentication if required
# r = redis.Redis(host='localhost', port=6379, password='youshallnotpass', decode_responses=True)

# Basic operations
r.set('user:session:123', 'active')
session_status = r.get('user:session:123')
print(f"Session status: {session_status}")

# Key with expiration
r.setex('temp_data', 60, 'this will expire in 60 seconds')

# Multiple key-value operations
user_data = {
    'user:1:name': 'Alice',
    'user:1:email': '[email protected]',
    'user:1:last_login': str(int(time.time()))
}
r.mset(user_data)

# User data using hash
r.hset('user:profile:1', mapping={
    'name': 'Bob Smith',
    'email': '[email protected]', 
    'department': 'Engineering',
    'join_date': '2024-01-15'
})

profile = r.hgetall('user:profile:1')
print(f"User profile: {profile}")

# Task queue using list
r.lpush('task_queue', 'process_payment', 'send_email', 'update_inventory')

# Process tasks
while True:
    task = r.brpop('task_queue', timeout=5)
    if task:
        queue_name, task_name = task
        print(f"Processing task: {task_name}")
        # Process task here
        time.sleep(1)  # Simulate processing time
    else:
        print("No tasks available")
        break

# Tag management using sets
r.sadd('article:1:tags', 'python', 'database', 'performance')
r.sadd('article:2:tags', 'python', 'web', 'api')

# Find common tags
common_tags = r.sinter('article:1:tags', 'article:2:tags')
print(f"Common tags: {common_tags}")

Using Node.js Client

const redis = require('redis');

// Create DragonflyDB client
const client = redis.createClient({
    host: 'localhost',
    port: 6379,
    // password: 'youshallnotpass', // if authentication required
});

client.on('error', (err) => {
    console.error('Redis connection error:', err);
});

// Connect
await client.connect();

// Basic operations
await client.set('app:config:version', '1.2.3');
const version = await client.get('app:config:version');
console.log(`App version: ${version}`);

// Store JSON data
const userData = {
    id: 123,
    name: 'John Doe',
    preferences: {
        theme: 'dark',
        notifications: true
    }
};

await client.set('user:123', JSON.stringify(userData));
const storedUser = JSON.parse(await client.get('user:123'));
console.log('Stored user:', storedUser);

// Event log using list operations
await client.lPush('events:log', JSON.stringify({
    timestamp: new Date().toISOString(),
    event: 'user_login',
    userId: 123
}));

await client.lPush('events:log', JSON.stringify({
    timestamp: new Date().toISOString(),
    event: 'page_view',
    userId: 123,
    page: '/dashboard'
}));

// Get recent events
const recentEvents = await client.lRange('events:log', 0, 4);
recentEvents.forEach(event => {
    console.log('Event:', JSON.parse(event));
});

// Counter operations
await client.incr('stats:page_views');
await client.incrBy('stats:api_calls', 5);

const pageViews = await client.get('stats:page_views');
const apiCalls = await client.get('stats:api_calls');
console.log(`Page views: ${pageViews}, API calls: ${apiCalls}`);

// Pipeline operations (batch processing)
const pipeline = client.multi();
pipeline.set('batch:1', 'value1');
pipeline.set('batch:2', 'value2');
pipeline.set('batch:3', 'value3');
pipeline.expire('batch:1', 3600);
pipeline.expire('batch:2', 3600);
pipeline.expire('batch:3', 3600);

const results = await pipeline.exec();
console.log('Batch operation results:', results);

// Close connection
await client.quit();

Performance Benchmarking Examples

# Performance testing using memtier_benchmark
# Basic GET/SET operations benchmark
memtier_benchmark -h localhost -p 6379 --ratio 1:1 -n 100000 \
  --threads=4 -c 20 --distinct-client-seed \
  --key-prefix="benchmark:" --hide-histogram -d 256

# Read-only workload test
memtier_benchmark -h localhost -p 6379 --ratio 0:1 -n 200000 \
  --threads=8 -c 30 --distinct-client-seed \
  --key-prefix="readonly:" --hide-histogram -d 128

# Expiry keys test
memtier_benchmark -h localhost -p 6379 --ratio 1:0 -n 300000 \
  --threads=2 -c 20 --distinct-client-seed \
  --key-prefix="expiry:" --hide-histogram \
  --expiry-range=30-30 --key-maximum=100000000 -d 256

# Memory efficiency test with large data
memtier_benchmark -h localhost -p 6379 \
  --command "sadd __key__ __data__" -n 5000000 \
  --threads=1 -c 1 --command-key-pattern=R \
  --data-size=10 --key-prefix="memory_test:" \
  --hide-histogram --random-data --key-maximum=1 \
  --randomize --pipeline 20

Configuration and Tuning

# Memory limit and cache mode
./dragonfly-x86_64 --maxmemory=16gb --cache_mode=true

# Database count and network settings
./dragonfly-x86_64 --dbnum=16 --bind=0.0.0.0 --port=6379

# Logging and performance monitoring
./dragonfly-x86_64 --logtostderr \
  --vmodule=dragonfly_connection=2 \
  --alsologtostderr

# Snapshot configuration
./dragonfly-x86_64 --dbfilename=dragonfly_snapshot.rdb \
  --dir=/data/dragonfly --save_schedule="*/10 * * * *"

# Replication setup (master)
./dragonfly-x86_64 --bind=0.0.0.0 --port=6379 \
  --requirepass=master_password

# Replication setup (replica)
./dragonfly-x86_64 --bind=0.0.0.0 --port=6380 \
  --replicaof=master_host:6379 \
  --masterauth=master_password

Monitoring and Metrics

import redis
import time

def monitor_dragonfly_stats():
    r = redis.Redis(host='localhost', port=6379, decode_responses=True)
    
    while True:
        info = r.info()
        
        print(f"=== DragonflyDB Stats ===")
        print(f"Connected clients: {info.get('connected_clients', 0)}")
        print(f"Used memory: {info.get('used_memory_human', 'N/A')}")
        print(f"Total commands processed: {info.get('total_commands_processed', 0)}")
        print(f"Instantaneous ops/sec: {info.get('instantaneous_ops_per_sec', 0)}")
        print(f"Keyspace hits: {info.get('keyspace_hits', 0)}")
        print(f"Keyspace misses: {info.get('keyspace_misses', 0)}")
        
        # Calculate hit rate
        hits = info.get('keyspace_hits', 0)
        misses = info.get('keyspace_misses', 0)
        if hits + misses > 0:
            hit_rate = (hits / (hits + misses)) * 100
            print(f"Hit rate: {hit_rate:.2f}%")
        
        print("-" * 30)
        time.sleep(5)

if __name__ == "__main__":
    monitor_dragonfly_stats()

DragonflyDB, with its innovative multi-threaded architecture, breaks through traditional Redis constraints and provides a true high-performance caching solution for modern multi-core environments.