Library

Utf8Json - Fastest and Zero Allocation JSON Serializer for C#

Overview

Utf8Json is an ultra-high-performance JSON serializer developed for C# (.NET, .NET Core, Unity, Xamarin). By directly reading and writing UTF-8 binary data, it completely eliminates string conversion overhead and achieves zero allocation. Through thorough performance optimizations including dynamic IL code generation, automata-based property matching, and custom numeric conversion algorithms, it achieves performance that significantly exceeds existing JSON libraries.

Details

Utf8Json is a performance-focused JSON serializer developed by neuecc (Yoshifumi Kawai), also known for creating MessagePackSharp and ZeroFormatter. While traditional JSON libraries require conversion to UTF-16 strings, Utf8Json processes UTF-8 byte arrays directly, achieving dramatic performance improvements.

The library's core feature is its zero-allocation design. Buffers under 64KB are obtained from a thread-local memory pool, minimizing garbage collection overhead. It dynamically generates IL code at runtime to create optimized formatters for custom types, with pre-code generation support for AOT environments.

Property matching employs an automata-based approach, matching property names directly from raw UTF-8 bytes. For numeric conversions, it implements custom itoa/atoi and dtoa/atod algorithms, enabling direct conversion without string intermediates. These optimizations achieve 2-4x performance improvements compared to existing JSON libraries.

Note that the original repository has been archived, and community forks are now recommended for use.

Advantages and Disadvantages

Advantages

• Exceptional Speed: 2-4x performance compared to existing JSON libraries
• Zero Allocation: Minimizes GC pressure through memory pool usage
• Direct UTF-8 Processing: Completely eliminates string conversion overhead
• Unity Support: Native support for Unity-specific types like Vector3, Quaternion
• Flexible Customization: Advanced control through Resolver/Formatter system
• AOT Environment Support: Works on Xamarin, Unity IL2CPP through pre-code generation

Disadvantages

• Maintenance Stopped: Original repository is archived
• Complexity: Complex internal implementation due to advanced optimizations
• Feature Limitations: Some advanced features (circular references, etc.) unsupported due to performance priority
• Learning Curve: Time needed to understand Resolver/Formatter system
• Debugging Difficulty: Dynamically generated IL code is hard to debug
• Lack of Standardization: Gap with standard library after System.Text.Json introduction

References

• Utf8Json Official Repository (Archived)
• ZCS.Utf8Json - Community Fork
• Utf8Json Performance Benchmarks
• The Battle of C# JSON Serializers

Code Examples

1. Basic Serialization

using Utf8Json;

public class Person
{
    public string Name { get; set; }
    public int Age { get; set; }
    public DateTime BirthDate { get; set; }
}

class BasicExample
{
    static void Main()
    {
        var person = new Person 
        { 
            Name = "John Doe",
            Age = 30,
            BirthDate = new DateTime(1993, 5, 15)
        };

        // Object → byte[] (UTF-8)
        byte[] utf8Json = JsonSerializer.Serialize(person);
        
        // byte[] → Object
        var deserialized = JsonSerializer.Deserialize<Person>(utf8Json);
        
        // Object → String
        string jsonString = JsonSerializer.ToJsonString(person);
        Console.WriteLine(jsonString);
        
        // Write to Stream
        using (var stream = new FileStream("person.json", FileMode.Create))
        {
            JsonSerializer.Serialize(stream, person);
        }
    }
}

2. Private Member Serialization

public class Employee
{
    private string employeeId = "EMP-001";
    private double salary = 50000;
    
    public string Name { get; set; }
    public string Department { get; set; }
    
    // Accessors for private fields
    public string GetEmployeeId() => employeeId;
    public double GetSalary() => salary;
}

class PrivateMemberExample
{
    static void SerializePrivateMembers()
    {
        var employee = new Employee 
        { 
            Name = "Jane Smith",
            Department = "Engineering"
        };
        
        // Serialize including private members
        var json = JsonSerializer.ToJsonString(employee, StandardResolver.AllowPrivate);
        Console.WriteLine(json);
        // Output: {"employeeId":"EMP-001","salary":50000,"Name":"Jane Smith","Department":"Engineering"}
        
        // Deserialize with private member restoration
        var deserialized = JsonSerializer.Deserialize<Employee>(json, StandardResolver.AllowPrivate);
        Console.WriteLine($"ID: {deserialized.GetEmployeeId()}, Salary: {deserialized.GetSalary()}");
    }
}

3. Custom Resolvers and Formatters

using Utf8Json.Formatters;
using Utf8Json.Resolvers;

// Custom formatter
public class DateOnlyFormatter : IJsonFormatter<DateTime>
{
    public void Serialize(ref JsonWriter writer, DateTime value, IJsonFormatterResolver formatterResolver)
    {
        writer.WriteString(value.ToString("yyyy-MM-dd"));
    }
    
    public DateTime Deserialize(ref JsonReader reader, IJsonFormatterResolver formatterResolver)
    {
        var dateString = reader.ReadString();
        return DateTime.ParseExact(dateString, "yyyy-MM-dd", null);
    }
}

// Custom resolver
public class CustomResolver : IJsonFormatterResolver
{
    public static IJsonFormatterResolver Instance = new CustomResolver();
    
    private CustomResolver() { }
    
    public IJsonFormatter<T> GetFormatter<T>()
    {
        if (typeof(T) == typeof(DateTime))
        {
            return (IJsonFormatter<T>)new DateOnlyFormatter();
        }
        
        return StandardResolver.Default.GetFormatter<T>();
    }
}

// Usage example
class CustomFormatterExample
{
    static void UseCustomFormatter()
    {
        var data = new { Date = DateTime.Now, Name = "Test" };
        
        var json = JsonSerializer.ToJsonString(data, CustomResolver.Instance);
        Console.WriteLine(json); // {"Date":"2023-12-15","Name":"Test"}
    }
}

4. Unity Usage Example

using UnityEngine;
using Utf8Json;
using Utf8Json.Resolvers;

[System.Serializable]
public class GameData
{
    public string PlayerName { get; set; }
    public int Score { get; set; }
    public Vector3 Position { get; set; }
    public Quaternion Rotation { get; set; }
    public Color PlayerColor { get; set; }
}

public class UnityExample : MonoBehaviour
{
    void Start()
    {
        // Set Unity resolver as default
        JsonSerializer.SetDefaultResolver(StandardResolver.Default);
        
        var gameData = new GameData
        {
            PlayerName = "Player1",
            Score = 1000,
            Position = new Vector3(10f, 5f, 3f),
            Rotation = Quaternion.Euler(0, 45, 0),
            PlayerColor = Color.red
        };
        
        // Serialize object containing Unity types
        byte[] jsonBytes = JsonSerializer.Serialize(gameData);
        
        // Save to PlayerPrefs
        string base64 = System.Convert.ToBase64String(jsonBytes);
        PlayerPrefs.SetString("GameData", base64);
        
        // Load
        string loadedBase64 = PlayerPrefs.GetString("GameData");
        byte[] loadedBytes = System.Convert.FromBase64String(loadedBase64);
        var loadedData = JsonSerializer.Deserialize<GameData>(loadedBytes);
        
        Debug.Log($"Loaded: {loadedData.PlayerName}, Score: {loadedData.Score}");
    }
}

5. Dynamic JSON Processing

class DynamicJsonExample
{
    static void ProcessDynamicJson()
    {
        // Dynamic JSON deserialization
        var json = @"{
            ""name"": ""Product A"",
            ""price"": 29.99,
            ""tags"": [""new"", ""featured""],
            ""details"": {
                ""weight"": 500,
                ""dimensions"": {
                    ""width"": 10,
                    ""height"": 20,
                    ""depth"": 5
                }
            }
        }";
        
        dynamic obj = JsonSerializer.Deserialize<dynamic>(json);
        
        // Dynamic access
        string name = obj["name"];                    // "Product A"
        double price = obj["price"];                  // 29.99
        var firstTag = obj["tags"][0];               // "new"
        var weight = obj["details"]["weight"];       // 500
        var width = obj["details"]["dimensions"]["width"]; // 10
        
        Console.WriteLine($"Product: {name}, Price: ${price}");
        
        // Building dynamic objects
        var newObj = new Dictionary<string, object>
        {
            ["id"] = 123,
            ["items"] = new[] { "item1", "item2" },
            ["metadata"] = new Dictionary<string, object>
            {
                ["created"] = DateTime.Now,
                ["version"] = 1.0
            }
        };
        
        string newJson = JsonSerializer.ToJsonString(newObj);
        Console.WriteLine(newJson);
    }
}

6. Advanced Performance Optimization

using Utf8Json;
using System.Buffers;

class PerformanceOptimization
{
    // Zero-copy processing using ArraySegment
    static void ZeroCopyProcessing()
    {
        var largeObject = GenerateLargeObject();
        
        // Use internal buffer pool with ArraySegment
        ArraySegment<byte> segment = JsonSerializer.SerializeUnsafe(largeObject);
        
        try
        {
            // Process segment directly (no copy)
            ProcessJsonBytes(segment.Array, segment.Offset, segment.Count);
        }
        finally
        {
            // Important: Return ArraySegment immediately after use
            JsonSerializer.ReturnArrayToPool(segment.Array);
        }
    }
    
    // Streaming processing
    static async Task StreamingSerializationAsync()
    {
        var items = GetLargeDataset(); // Large amount of data
        
        using (var stream = new FileStream("large.json", FileMode.Create))
        using (var writer = new StreamWriter(stream))
        {
            await writer.WriteAsync("[");
            
            bool first = true;
            foreach (var item in items)
            {
                if (!first) await writer.WriteAsync(",");
                first = false;
                
                // Serialize each item individually
                var json = JsonSerializer.ToJsonString(item);
                await writer.WriteAsync(json);
            }
            
            await writer.WriteAsync("]");
        }
    }
    
    // Using pre-generated formatters (for AOT environments)
    static void UsePreGeneratedFormatter()
    {
        // Pre-generation required for AOT environments
        // Generate using Utf8Json.UniversalCodeGenerator.exe
        
        Utf8Json.Resolvers.GeneratedResolver.Instance.Register();
        
        var obj = new MyGeneratedType { Value = 42 };
        byte[] json = JsonSerializer.Serialize(obj);
    }
}