← Back to Key-Value

Database

Riak KV

Overview

Riak KV is a distributed NoSQL key-value database designed with emphasis on high availability, fault tolerance, and operational simplicity. Strongly influenced by Amazon's Dynamo paper and CAP theorem principles, it's implemented in Erlang. It features a masterless architecture where all nodes can handle read and write requests, with automatic data distribution and replication across the cluster.

Details

Riak KV was developed by Basho Technologies in 2009 and is currently maintained and supported by Riak, Inc. It employs a distributed "Ring" architecture that logically distributes data across a 160-bit key space. Each node hosts multiple Virtual Nodes (vnodes) that are responsible for data fragments.

Key features of Riak KV:

  • Masterless architecture (all nodes are equal)
  • Automatic data distribution and replication
  • Vector Clocks for causality tracking
  • Tunable consistency levels (R/W values)
  • Multiple storage backend support
  • Gossip protocol for cluster state sharing
  • HTTP API and Protocol Buffers API
  • Network partition tolerance
  • Horizontal scalability
  • Eventually consistent

Storage backends include Bitcask (default), LevelDB, Memory, and Leveled options.

Advantages and Disadvantages

Advantages

  • High availability: Ideal for 24/7 operational environments, continues operating with multiple node failures
  • Horizontal scalability: Linear performance improvement through node addition
  • Operational simplicity: Easy management with masterless design, no single point of failure
  • Fault tolerance: Automatic failover and data recovery
  • Flexible consistency: R/W value-based consistency level tuning
  • Multi-datacenter support: Supports geographically distributed environments
  • Eventual consistency: Prioritizes CAP theorem's A (availability) and P (partition tolerance)

Disadvantages

  • No complex queries: Cannot perform SQL-style SELECT statements or JOINs
  • Overkill for small environments: Minimum 5-node configuration recommended, leading to high costs
  • Not suitable for strong consistency: Inappropriate for use cases requiring ACID properties
  • Poor fit for large objects: Optimized for data under 1MB
  • Learning curve: Requires understanding distributed system concepts
  • Development complexity: Conflict resolution must be implemented application-side

Key Links

Usage Examples

Installation & Setup

# Ubuntu/Debian
curl -s https://packagecloud.io/install/repositories/basho/riak/script.deb.sh | sudo bash
sudo apt-get install riak

# Red Hat/CentOS
curl -s https://packagecloud.io/install/repositories/basho/riak/script.rpm.sh | sudo bash
sudo yum install riak

# macOS (Homebrew)
brew tap basho/riak
brew install riak

# Docker
docker run -d --name riak-container -p 8087:8087 -p 8098:8098 basho/riak-kv

# Start service
sudo systemctl start riak
sudo systemctl enable riak

# Verify operation
curl http://127.0.0.1:8098/ping
# Expected: OK

Basic Operations (HTTP API)

# Store data
curl -X PUT -H "content-type: text/plain" \
  http://127.0.0.1:8098/riak/users/user1 \
  --data '{"name": "John Doe", "age": 30, "email": "[email protected]"}'

# Retrieve data
curl -i http://127.0.0.1:8098/riak/users/user1

# Conditional retrieval (specify consistency level)
curl http://127.0.0.1:8098/riak/users/user1?r=2

# Update data
curl -X PUT -H "content-type: application/json" \
  -H "X-Riak-Vclock: a85hYGBgzGDKBVIcypz/fgaUHjmdwZTImMfKyDt97Q==" \
  http://127.0.0.1:8098/riak/users/user1 \
  --data '{"name": "John Doe", "age": 31, "email": "[email protected]"}'

# Delete data
curl -X DELETE http://127.0.0.1:8098/riak/users/user1

# List keys (development only)
curl http://127.0.0.1:8098/riak/users?keys=true

Bucket Configuration

# Get bucket configuration
curl http://127.0.0.1:8098/riak/users

# Set replication count (N-value)
curl -X PUT -H "content-type: application/json" \
  http://127.0.0.1:8098/riak/users \
  --data '{"props":{"n_val":5}}'

# Set consistency levels
curl -X PUT -H "content-type: application/json" \
  http://127.0.0.1:8098/riak/users \
  --data '{"props":{"r":"quorum","w":"quorum","dw":"quorum"}}'

# Configure conflict resolution
curl -X PUT -H "content-type: application/json" \
  http://127.0.0.1:8098/riak/users \
  --data '{"props":{"allow_mult":true,"last_write_wins":false}}'

Node.js Client Usage

const RiakClient = require('basho-riak-client')

// Create client
const client = new RiakClient(['127.0.0.1:8087'])

// Store data
const storeValue = {
    bucket: 'users',
    key: 'user1',
    value: JSON.stringify({
        name: 'John Doe',
        age: 30,
        email: '[email protected]'
    }),
    contentType: 'application/json'
}

client.storeValue(storeValue, (err, rslt) => {
    if (err) {
        console.error('Store error:', err)
    } else {
        console.log('Data stored successfully')
        console.log('Vector Clock:', rslt.vclock)
    }
})

// Retrieve data
const fetchValue = {
    bucket: 'users',
    key: 'user1'
}

client.fetchValue(fetchValue, (err, rslt) => {
    if (err) {
        console.error('Fetch error:', err)
    } else if (rslt.values.length === 0) {
        console.log('Data not found')
    } else {
        const userData = JSON.parse(rslt.values[0].value.toString())
        console.log('User data:', userData)
    }
})

// Handle multiple versions (conflict resolution)
client.fetchValue(fetchValue, (err, rslt) => {
    if (rslt.values.length > 1) {
        console.log('Conflict detected')
        rslt.values.forEach((value, index) => {
            console.log(`Version ${index + 1}:`, 
                JSON.parse(value.value.toString()))
        })
        
        // Select latest update (example)
        const latest = rslt.values.reduce((prev, current) => {
            const prevData = JSON.parse(prev.value.toString())
            const currentData = JSON.parse(current.value.toString())
            return currentData.updatedAt > prevData.updatedAt ? current : prev
        })
        
        console.log('Selected version:', JSON.parse(latest.value.toString()))
    }
})

// Close connection
client.stop()

Cluster Management

# Check cluster status
riak-admin cluster status

# Add new node
riak-admin cluster join [email protected]

# Display cluster plan
riak-admin cluster plan

# Execute plan
riak-admin cluster commit

# Node leave
riak-admin cluster leave [email protected]

# Check ring status
riak-admin ring-status

# Manually execute handoff
riak-admin transfer-limit 4
riak-admin handoff status

Monitoring & Maintenance

# Node statistics
riak-admin stat

# Display specific statistics only
riak-admin stat | grep -E "(ring_|handoff_|coord_redirs)"

# Check vnode status
riak-admin vnode-status

# Performance diagnostics
riak-admin diag

# AAE (Anti-Entropy) status
riak-admin aae-status

# Force repair execution
riak-admin repair-2i users

# Backup (data directory)
sudo tar -czf riak-backup-$(date +%Y%m%d).tar.gz /var/lib/riak

Configuration File Example (riak.conf)

# Node configuration
nodename = [email protected]
distributed_cookie = riak

# Directory configuration
platform_data_dir = /var/lib/riak
platform_etc_dir = /etc/riak
platform_log_dir = /var/log/riak

# Network configuration
listener.http.internal = 127.0.0.1:8098
listener.protobuf.internal = 127.0.0.1:8087

# Storage configuration
storage_backend = bitcask

# Cluster configuration
ring_size = 64

# Default bucket configuration
buckets.default.n_val = 3
buckets.default.r = 2
buckets.default.w = 2
buckets.default.dw = 1

# Performance configuration
erlang.schedulers.force_wakeup_interval = 500
erlang.schedulers.compaction_of_load = false

# Log configuration
log.error.file = /var/log/riak/error.log
log.console.level = info
log.crash.file = /var/log/riak/crash.log

Best Practices

# 1. Appropriate cluster size (minimum 5 nodes)
# 3 nodes: Development environment only
# 5 nodes: Production environment recommended
# 7+ nodes: Large-scale environments

# 2. Data modeling
# - Object size: Under 1MB
# - Use natural keys: User ID, timestamp, etc.
# - Bucket design: Logical data grouping

# 3. Consistency level settings
# R + W > N: Strong consistency
# R = 1, W = 1: High availability priority
# R = quorum, W = quorum: Balanced approach

# 4. Operational considerations
# - Keep simultaneous node failures below N/2
# - Regular AAE execution for consistency verification
# - Wait for handoff completion before next node operation
# - Monitoring targets: CPU under 30%, I/O under 90%

# 5. Performance optimization
# - Regular Bitcask merge execution
# - Appropriate erlang.schedulers configuration
# - Ensure network bandwidth availability
# - Recommend SSD storage usage