QuickCheck

Overview

QuickCheck is known as the grandfather of property-based testing libraries for Haskell. Developed in 1999 by Koen Claessen and John Hughes, it verifies program properties using randomly generated test cases. Despite being ported to over 30 languages, the original implementation remains preeminent due to Haskell's type system and consistent logic. It provides an innovative testing approach by describing abstract invariants that functions should satisfy and automatically generating and executing thousands of test cases.

Details

Key Features

• Property-Based Testing: Description and verification of properties that functions should universally satisfy
• Automatic Test Data Generation: Type-based automatic data generation using the Arbitrary type class
• Shrinking: Automatic minimal counterexample discovery on failure
• High-Level Approach: Focus on abstract invariants rather than concrete test cases
• Type Safety: Complete integration with Haskell's powerful type system
• Monadic Testing: Support for testing functions with state changes and I/O
• Custom Generators: Capability to create custom test data generation logic

Architecture

QuickCheck consists of the following core components:

• Arbitrary Type Class: Definition of test data generators for each type
• Gen Type: Monadic random value generator
• Property Type: Representation of testable properties
• Test Runner: Property execution and result reporting
• Shrinking Engine: Minimization functionality for failure cases

Generator System

Generators have the following characteristics:

• Start with small values and progressively generate larger ones
• Automatic generation based on type information
• Support for custom distributions and filtering

Pros and Cons

Pros

• Comprehensive Testing: Automatic execution of massive test cases impossible to write by hand
• Edge Case Discovery: Detection of subtle boundary conditions easily missed by humans
• Type Safety: Strong safety guarantees through Haskell's type system
• High-Level Description: Improved maintainability by describing abstract properties rather than concrete examples
• Efficient Debugging: Automatic discovery of minimal counterexamples through shrinking
• Functional Integration: Natural integration with pure functional programming

Cons

• Learning Curve: Need to learn property-based testing concepts
• Haskell Dependency: Language-specific features make porting to other languages difficult
• Property Design: Designing good properties requires experience and insight
• Non-determinism: Reproducibility challenges due to random generation
• Execution Time: Time cost due to generation and execution of large numbers of test cases

References

• QuickCheck Official Documentation
• QuickCheck Manual
• Test.QuickCheck
• Real World Haskell - Testing

Code Examples

Basic Property Testing

import Test.QuickCheck

-- Property about list length
prop_lengthAppend :: [Int] -> [Int] -> Bool
prop_lengthAppend xs ys = length (xs ++ ys) == length xs + length ys

-- Reverse involution
prop_reverseInvolution :: [Int] -> Bool
prop_reverseInvolution xs = reverse (reverse xs) == xs

-- Commutativity of addition
prop_addCommutative :: Int -> Int -> Bool
prop_addCommutative x y = x + y == y + x

-- Test execution
main = do
    quickCheck prop_lengthAppend
    quickCheck prop_reverseInvolution
    quickCheck prop_addCommutative

Custom Data Types and Arbitrary Instances

import Test.QuickCheck
import Control.Monad (liftM2)

-- Custom data type
data Tree a = Leaf a | Node (Tree a) (Tree a)
    deriving (Show, Eq)

-- Arbitrary instance definition
instance Arbitrary a => Arbitrary (Tree a) where
    arbitrary = sized tree'
      where
        tree' 0 = liftM Leaf arbitrary
        tree' n | n > 0 = 
            oneof [ liftM Leaf arbitrary
                  , liftM2 Node subtree subtree
                  ]
          where 
            subtree = tree' (n `div` 2)

    shrink (Leaf x) = [Leaf x' | x' <- shrink x]
    shrink (Node l r) = [l, r] ++ 
                        [Node l' r | l' <- shrink l] ++
                        [Node l r' | r' <- shrink r]

-- Tree properties
treeSize :: Tree a -> Int
treeSize (Leaf _) = 1
treeSize (Node l r) = treeSize l + treeSize r

flattenTree :: Tree a -> [a]
flattenTree (Leaf x) = [x]
flattenTree (Node l r) = flattenTree l ++ flattenTree r

-- Relationship between tree size and flattening
prop_treeSizeConsistent :: Tree Int -> Bool
prop_treeSizeConsistent t = treeSize t == length (flattenTree t)

Generator Combinations

import Test.QuickCheck

-- Generator for integers in specific range
smallInt :: Gen Int
smallInt = choose (-100, 100)

-- Generator for even numbers
evenInt :: Gen Int
evenInt = (*2) <$> arbitrary

-- Generator for positive integers
positiveInt :: Gen Int
positiveInt = abs <$> arbitrary `suchThat` (> 0)

-- Generator for non-empty lists
nonEmptyList :: Arbitrary a => Gen [a]
nonEmptyList = listOf1 arbitrary

-- Generator for strings
identifierString :: Gen String
identifierString = do
    first <- elements ['a'..'z']
    rest <- listOf (elements (['a'..'z'] ++ ['A'..'Z'] ++ ['0'..'9']))
    return (first : rest)

-- Properties using custom generators
prop_customGenerators :: Property
prop_customGenerators = forAll positiveInt $ \n ->
    n > 0

prop_identifierProperty :: Property
prop_identifierProperty = forAll identifierString $ \s ->
    not (null s) && head s `elem` ['a'..'z']

Distribution Control with Frequency

-- Control frequency of different data types
data Shape = Circle Double | Rectangle Double Double | Triangle Double Double Double
    deriving (Show, Eq)

instance Arbitrary Shape where
    arbitrary = frequency 
        [ (3, Circle <$> positiveFloat)      -- 30% probability
        , (5, Rectangle <$> positiveFloat <*> positiveFloat)  -- 50% probability
        , (2, Triangle <$> positiveFloat <*> positiveFloat <*> positiveFloat)  -- 20% probability
        ]
      where
        positiveFloat = abs <$> arbitrary `suchThat` (> 0)

-- Area calculation properties
area :: Shape -> Double
area (Circle r) = pi * r * r
area (Rectangle w h) = w * h
area (Triangle a b c) = 
    let s = (a + b + c) / 2
    in sqrt (s * (s - a) * (s - b) * (s - c))

prop_positiveArea :: Shape -> Bool
prop_positiveArea shape = area shape > 0

Conditional Properties

-- Testing with preconditions
prop_divisionProperty :: Int -> Int -> Property
prop_divisionProperty x y = y /= 0 ==> (x `div` y) * y + (x `mod` y) == x

-- Complex preconditions
prop_sortedListProperty :: [Int] -> Property
prop_sortedListProperty xs = 
    not (null xs) ==> 
    let sorted = sort xs
    in head sorted <= last sorted

-- Multiple preconditions
prop_listIndexing :: [Int] -> Int -> Property
prop_listIndexing xs i = 
    not (null xs) && i >= 0 && i < length xs ==>
    (xs !! i) `elem` xs

Monadic Testing

import Test.QuickCheck
import Test.QuickCheck.Monadic
import qualified Data.Map as Map
import Control.Monad.State

-- Testing stateful operations
type Counter = State Int

increment :: Counter ()
increment = modify (+1)

decrement :: Counter ()
decrement = modify (\x -> x - 1)

getCount :: Counter Int
getCount = get

-- Monadic properties
prop_counterProperty :: [Bool] -> Property
prop_counterProperty operations = monadicST $ do
    let actions = map (\b -> if b then increment else decrement) operations
    result <- run $ execState (sequence_ actions) 0
    let expected = length (filter id operations) - length (filter not operations)
    assert (result == expected)

-- Testing with I/O
prop_fileOperations :: String -> Property
prop_fileOperations content = monadicIO $ do
    let filename = "test.tmp"
    run $ writeFile filename content
    readContent <- run $ readFile filename
    run $ removeFile filename
    assert (content == readContent)

Labeled Tests and Classification

-- Test case classification
prop_listSortProperties :: [Int] -> Property
prop_listSortProperties xs = 
    classify (null xs) "empty list" $
    classify (length xs == 1) "singleton" $
    classify (length xs >= 2 && length xs <= 10) "small list" $
    classify (length xs > 10) "large list" $
    let sorted = sort xs
    in collect (length xs) $
       sorted == sort sorted

-- Coverage verification
prop_mathOperations :: Int -> Int -> Property
prop_mathOperations x y =
    label ("x positive: " ++ show (x > 0)) $
    label ("y positive: " ++ show (y > 0)) $
    label ("same sign: " ++ show (signum x == signum y)) $
    x + y - y == x

Advanced Testing Features

-- Testing expected exceptions
prop_divisionByZero :: Int -> Property
prop_divisionByZero x = expectFailure $ 
    within 1000000 $ -- Must complete within 1 second
    return (x `div` 0 == 0)

-- Performance testing
prop_sortPerformance :: [Int] -> Property
prop_sortPerformance xs = 
    length xs <= 1000 ==> 
    within 1000000 $ -- Within 1 second
    return (length (sort xs) == length xs)

-- Custom test configuration
customCheck :: Testable prop => prop -> IO ()
customCheck = quickCheckWith stdArgs 
    { maxSuccess = 1000    -- Number of test cases
    , maxSize = 100        -- Maximum size
    , chatty = True        -- Verbose output
    }

-- Automatic discovery with Template Haskell
{-# LANGUAGE TemplateHaskell #-}
import Test.QuickCheck.All

prop_automaticTest1 :: [Int] -> Bool
prop_automaticTest1 xs = length xs >= 0

prop_automaticTest2 :: Int -> Int -> Bool  
prop_automaticTest2 x y = x + y == y + x

-- Automatically run all functions starting with prop_
return []
runTests = $quickCheckAll

Real-World Property Examples

-- JSON serialization/deserialization
prop_jsonRoundTrip :: Person -> Bool
prop_jsonRoundTrip person = 
    case decode (encode person) of
        Just decoded -> decoded == person
        Nothing -> False

-- Encryption/decryption
prop_encryptDecrypt :: String -> String -> Bool
prop_encryptDecrypt key plaintext = 
    decrypt key (encrypt key plaintext) == plaintext

-- Database operations
prop_databaseConsistency :: UserId -> User -> Property
prop_databaseConsistency uid user = monadicIO $ do
    run $ insertUser uid user
    retrieved <- run $ getUser uid
    run $ deleteUser uid
    assert (retrieved == Just user)