PropTest

Overview

PropTest is a property-based testing framework for Rust. As a member of the QuickCheck family, it is inspired by the Hypothesis framework for Python. It tests that certain properties of your code hold for arbitrary inputs, and if a failure is found, automatically finds the minimal test case to reproduce the problem. With its strategy-based generation system and advanced shrinking capabilities, it efficiently detects edge cases that would be difficult to find with traditional unit tests.

Details

Key Features

• Strategy-Based Generation: Flexible test data generation using explicit Strategy objects
• Automatic Shrinking: Automatically reduces failing test cases to their minimal form
• Constraint-Aware Generation: Smart generation that recognizes constraints and avoids generating invalid values
• Rich Built-in Strategies: Comprehensive strategies from primitive types to collections
• Custom Strategy Creation: Flexible strategy creation for custom data types
• Stateful Testing: Support for testing complex state transitions
• Regression Testing: Persistence of once-found failure cases

Architecture

PropTest consists of the following core components:

• Strategy: Definition of random value generation and shrinking
• Test Runner: Execution and reporting of property tests
• Shrinking Engine: Minimization engine for failure cases
• Value Tree: Management of relationships between generated values and shrinking

Strategy System

Strategies define two important functions:

1. How to generate random values of a particular type from a random number generator
2. How to "shrink" such values into "simpler" forms

Pros and Cons

Pros

• Comprehensive Testing: Thorough verification with large amounts of random input
• Edge Case Discovery: Automatic discovery of boundary conditions that humans might not think of
• Efficient Debugging: Provision of minimal failure examples through automatic shrinking
• High Expressiveness: Natural description of complex properties
• Regression Prevention: Automatic regression testing of discovered issues
• Constraint Support: Generation of test data that satisfies complex constraint conditions

Cons

• Performance Cost: Up to one order of magnitude slower generation compared to QuickCheck
• Learning Curve: Need to learn property-based testing concepts
• Non-determinism: Reproducibility challenges due to random generation
• Complexity: Advanced shrinking model requiring state management

References

• PropTest Official Documentation
• GitHub - proptest-rs/proptest
• Rust docs.rs - proptest
• Property Testing Guide

Code Examples

Basic Property Testing

use proptest::prelude::*;

// Test commutativity of addition
proptest! {
    #[test]
    fn test_addition_commutative(a: i32, b: i32) {
        prop_assert_eq!(a + b, b + a);
    }
}

// Test associativity of multiplication
proptest! {
    #[test]
    fn test_multiplication_associative(a: i32, b: i32, c: i32) {
        prop_assert_eq!((a * b) * c, a * (b * c));
    }
}

// String operation properties
proptest! {
    #[test]
    fn test_string_reverse_involution(s: String) {
        let reversed_twice: String = s.chars().rev().collect::<String>()
            .chars().rev().collect();
        prop_assert_eq!(s, reversed_twice);
    }
}

Custom Strategies

use proptest::prelude::*;

// Custom struct
#[derive(Debug, Clone, PartialEq)]
struct Person {
    name: String,
    age: u8,
    email: String,
}

// Strategy for Person
fn person_strategy() -> impl Strategy<Value = Person> {
    (
        "[a-zA-Z]{1,20}",  // Name: 1-20 alphabetic characters
        0u8..150u8,        // Age: 0-149 years
        r"[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,}" // Email address
    ).prop_map(|(name, age, email)| Person {
        name,
        age,
        email,
    })
}

proptest! {
    #[test]
    fn test_person_serialization(person in person_strategy()) {
        let json = serde_json::to_string(&person).unwrap();
        let deserialized: Person = serde_json::from_str(&json).unwrap();
        prop_assert_eq!(person, deserialized);
    }
}

Collection Strategies

use proptest::collection::{vec, hash_set};

proptest! {
    #[test]
    fn test_vec_push_length(
        mut v in vec(any::<i32>(), 0..100),
        item: i32
    ) {
        let original_len = v.len();
        v.push(item);
        prop_assert_eq!(v.len(), original_len + 1);
    }
    
    #[test]
    fn test_set_insert_idempotent(
        mut set in hash_set(any::<i32>(), 0..100),
        item: i32
    ) {
        set.insert(item.clone());
        let size_after_first = set.len();
        
        set.insert(item);
        let size_after_second = set.len();
        
        prop_assert_eq!(size_after_first, size_after_second);
    }
}

Constrained Generation

// Generate only positive integers
proptest! {
    #[test]
    fn test_positive_square_root(n in 1u32..1000000u32) {
        let sqrt = (n as f64).sqrt() as u32;
        prop_assert!(sqrt * sqrt <= n);
        prop_assert!((sqrt + 1) * (sqrt + 1) > n);
    }
}

// Strings satisfying specific conditions
proptest! {
    #[test]
    fn test_valid_identifier(
        name in r"[a-zA-Z_][a-zA-Z0-9_]*"
    ) {
        prop_assert!(is_valid_identifier(&name));
        prop_assert!(!name.is_empty());
        prop_assert!(name.chars().next().unwrap().is_alphabetic() || 
                    name.chars().next().unwrap() == '_');
    }
}

// Range constraints
proptest! {
    #[test]
    fn test_percentage_calculation(
        total in 1u32..1000000u32,
        part in any::<u32>().prop_filter("part <= total", |&part| part <= total)
    ) {
        let percentage = (part as f64 / total as f64) * 100.0;
        prop_assert!(percentage >= 0.0);
        prop_assert!(percentage <= 100.0);
    }
}

Stateful Testing

use proptest::test_runner::{TestCaseResult, TestRunner};
use proptest::state_machine::{StateMachineTest, ReferenceStateMachine};

#[derive(Debug, Clone)]
struct Stack<T> {
    items: Vec<T>,
}

impl<T> Stack<T> {
    fn new() -> Self {
        Stack { items: Vec::new() }
    }
    
    fn push(&mut self, item: T) {
        self.items.push(item);
    }
    
    fn pop(&mut self) -> Option<T> {
        self.items.pop()
    }
    
    fn is_empty(&self) -> bool {
        self.items.is_empty()
    }
    
    fn len(&self) -> usize {
        self.items.len()
    }
}

// Stack state transition testing
#[derive(Debug, Clone)]
enum StackCommand {
    Push(i32),
    Pop,
}

proptest! {
    #[test]
    fn test_stack_operations(
        commands in prop::collection::vec(
            prop_oneof![
                any::<i32>().prop_map(StackCommand::Push),
                Just(StackCommand::Pop),
            ],
            0..100
        )
    ) {
        let mut stack = Stack::new();
        let mut reference_vec = Vec::new();
        
        for command in commands {
            match command {
                StackCommand::Push(value) => {
                    stack.push(value);
                    reference_vec.push(value);
                    prop_assert_eq!(stack.len(), reference_vec.len());
                }
                StackCommand::Pop => {
                    let stack_result = stack.pop();
                    let reference_result = reference_vec.pop();
                    prop_assert_eq!(stack_result, reference_result);
                    prop_assert_eq!(stack.len(), reference_vec.len());
                }
            }
        }
    }
}

Advanced Strategy Combinations

// Generate dependent values
fn ordered_pair() -> impl Strategy<Value = (i32, i32)> {
    any::<i32>().prop_flat_map(|first| {
        (Just(first), first..i32::MAX)
    })
}

// Conditional strategies
fn conditional_strategy() -> impl Strategy<Value = String> {
    any::<bool>().prop_flat_map(|use_numbers| {
        if use_numbers {
            r"[0-9]{1,10}".boxed()
        } else {
            r"[a-zA-Z]{1,10}".boxed()
        }
    })
}

// Complex nested structures
#[derive(Debug, Clone, PartialEq)]
struct Config {
    database_url: String,
    port: u16,
    features: Vec<String>,
    cache_settings: Option<CacheConfig>,
}

#[derive(Debug, Clone, PartialEq)]
struct CacheConfig {
    max_size: usize,
    ttl_seconds: u64,
}

fn cache_config_strategy() -> impl Strategy<Value = CacheConfig> {
    (1usize..1000000, 1u64..86400).prop_map(|(max_size, ttl_seconds)| {
        CacheConfig { max_size, ttl_seconds }
    })
}

fn config_strategy() -> impl Strategy<Value = Config> {
    (
        r"postgresql://[a-zA-Z0-9]+:[a-zA-Z0-9]+@[a-zA-Z0-9.-]+:[0-9]+/[a-zA-Z0-9_]+",
        1024u16..65535u16,
        prop::collection::vec(r"[a-zA-Z_][a-zA-Z0-9_]*", 0..10),
        prop::option::of(cache_config_strategy()),
    ).prop_map(|(database_url, port, features, cache_settings)| {
        Config { database_url, port, features, cache_settings }
    })
}

proptest! {
    #[test]
    fn test_config_validation(config in config_strategy()) {
        prop_assert!(config.port >= 1024);
        prop_assert!(!config.database_url.is_empty());
        
        if let Some(cache) = &config.cache_settings {
            prop_assert!(cache.max_size > 0);
            prop_assert!(cache.ttl_seconds > 0);
        }
    }
}

Error Handling and Regression Testing

use proptest::test_runner::TestCaseError;

// Custom assertions
fn prop_assert_nearly_equal(a: f64, b: f64, epsilon: f64) -> Result<(), TestCaseError> {
    if (a - b).abs() <= epsilon {
        Ok(())
    } else {
        Err(TestCaseError::Fail(format!(
            "assertion failed: `{} ≈ {}` (epsilon: {})", a, b, epsilon
        ).into()))
    }
}

proptest! {
    #[test]
    fn test_floating_point_arithmetic(
        a in -1000.0..1000.0f64,
        b in -1000.0..1000.0f64
    ) {
        // Division testing (avoiding division by zero)
        if b.abs() > 1e-10 {
            let result = a / b;
            prop_assert_nearly_equal(result * b, a, 1e-10)?;
        }
    }
}

// Regression test configuration
proptest! {
    #![proptest_config(ProptestConfig {
        cases: 1000,  // Number of test cases
        max_shrink_iters: 10000,  // Maximum shrinking iterations
        failure_persistence: Some(Box::new(
            proptest::test_runner::FileFailurePersistence::
                WithSource("regressions")
        )),
        .. ProptestConfig::default()
    })]
    
    #[test]
    fn test_complex_algorithm(input in complex_input_strategy()) {
        let result = complex_algorithm(&input);
        prop_assert!(validate_result(&result));
    }
}