Serialization Library

protobuf-net

Overview

protobuf-net is a .NET implementation of Protocol Buffers developed by Marc Gravell. While using Google's Protocol Buffers binary format, it provides an easy-to-use API that follows .NET conventions. It supports both attribute-based code-first approaches and contract-first approaches using .proto files, achieving high-performance serialization.

Details

protobuf-net is a Protocol Buffers implementation optimized for the .NET environment with the following characteristics:

Key Features

1. .NET-Idiomatic Design

   • Similar usage experience to XmlSerializer and DataContractSerializer
   • Simple attribute-based configuration
   • Intuitive API for .NET developers

2. High Speed and Efficiency

   • Optimization through runtime code generation
   • Minimal memory usage
   • Approximately 20x faster than BinaryFormatter

3. Flexible Development Approaches

   • Code-first: Start with C# classes
   • Contract-first: Start with .proto files
   • Conversion between both approaches is possible

4. Rich .NET Feature Support

   • Inheritance support (protobuf-net proprietary extension)
   • Full nullable type support
   • Collections and generics support
   • Serialization callbacks

5. Build Tools Integration

   • Complete integration with MSBuild
   • Automatic code generation from .proto files
   • Easy installation via NuGet packages

Performance Characteristics

• Serialization Speed: Approximately 8-10x faster than BinaryFormatter
• Deserialization Speed: Approximately 11-12x faster than BinaryFormatter
• Data Size: About 1/10 of XML, 1/3-1/5 of JSON
• Memory Efficiency: Low memory usage through streaming APIs

Pros and Cons

Pros

• High Performance: Fastest-level serialization in .NET environment
• Small Data Size: Efficient binary format
• Ease of Use: Natural attribute-based API for .NET developers
• Flexibility: Supports both code-first and contract-first
• Compatibility: Uses standard Protocol Buffers format
• Streaming Support: Efficient processing of large data
• Inheritance Support: Supports .NET-specific inheritance relationships

Cons

• No Human Readability: Cannot be read directly due to binary format
• Schema Evolution Limitations: Field number management required
• Learning Cost: Need to understand Protocol Buffers concepts
• Debugging Difficulty: Binary data debugging is challenging
• Cross-platform Constraints: Code-first is .NET-only

Reference Pages

• Official GitHub
• NuGet Package
• Official Documentation
• Protocol Buffers Official
• Marc Gravell's Blog

Code Examples

Basic Usage with Attributes

using ProtoBuf;
using System;
using System.IO;

// Define data models
[ProtoContract]
public class Person
{
    [ProtoMember(1)]
    public int Id { get; set; }
    
    [ProtoMember(2)]
    public string Name { get; set; }
    
    [ProtoMember(3)]
    public DateTime BirthDate { get; set; }
    
    [ProtoMember(4)]
    public Address Address { get; set; }
}

[ProtoContract]
public class Address
{
    [ProtoMember(1)]
    public string Street { get; set; }
    
    [ProtoMember(2)]
    public string City { get; set; }
    
    [ProtoMember(3)]
    public string PostalCode { get; set; }
}

// Serialization and deserialization
class Program
{
    static void Main()
    {
        var person = new Person
        {
            Id = 1,
            Name = "John Doe",
            BirthDate = new DateTime(1990, 1, 1),
            Address = new Address
            {
                Street = "123 Main St",
                City = "New York",
                PostalCode = "10001"
            }
        };

        // Serialization
        using (var file = File.Create("person.bin"))
        {
            Serializer.Serialize(file, person);
        }

        // Deserialization
        Person loadedPerson;
        using (var file = File.OpenRead("person.bin"))
        {
            loadedPerson = Serializer.Deserialize<Person>(file);
        }

        Console.WriteLine($"Name: {loadedPerson.Name}");
    }
}

Contract-First Approach

// person.proto
syntax = "proto3";

package example;

message Person {
    int32 id = 1;
    string name = 2;
    string birth_date = 3;
    Address address = 4;
}

message Address {
    string street = 1;
    string city = 2;
    string postal_code = 3;
}

<!-- Add to .csproj file -->
<ItemGroup>
    <PackageReference Include="protobuf-net.BuildTools" Version="3.*" 
                      PrivateAssets="all" />
    <AdditionalFiles Include="person.proto" />
</ItemGroup>

Inheritance Support

[ProtoContract]
[ProtoInclude(7, typeof(Employee))]
[ProtoInclude(8, typeof(Customer))]
public abstract class Person
{
    [ProtoMember(1)]
    public int Id { get; set; }
    
    [ProtoMember(2)]
    public string Name { get; set; }
}

[ProtoContract]
public class Employee : Person
{
    [ProtoMember(1)]
    public string Department { get; set; }
    
    [ProtoMember(2)]
    public decimal Salary { get; set; }
}

[ProtoContract]
public class Customer : Person
{
    [ProtoMember(1)]
    public string CustomerNumber { get; set; }
    
    [ProtoMember(2)]
    public DateTime RegistrationDate { get; set; }
}

Collections and Generics

[ProtoContract]
public class OrderList
{
    [ProtoMember(1)]
    public List<Order> Orders { get; set; } = new List<Order>();
    
    [ProtoMember(2)]
    public Dictionary<int, Customer> CustomerMap { get; set; } 
        = new Dictionary<int, Customer>();
    
    [ProtoMember(3, OverwriteList = true)]
    public HashSet<string> Tags { get; set; } = new HashSet<string>();
}

[ProtoContract]
public class Order
{
    [ProtoMember(1)]
    public int OrderId { get; set; }
    
    [ProtoMember(2)]
    public decimal Amount { get; set; }
    
    [ProtoMember(3)]
    public DateTime OrderDate { get; set; }
}

// Usage example
var orderList = new OrderList();
orderList.Orders.Add(new Order 
{ 
    OrderId = 1, 
    Amount = 1000.0m, 
    OrderDate = DateTime.Now 
});

// Serialization to memory stream
using var ms = new MemoryStream();
Serializer.Serialize(ms, orderList);
var bytes = ms.ToArray();

Performance Tuning

// Advanced configuration with RuntimeTypeModel
var model = RuntimeTypeModel.Default;

// Pre-compile types
model.CompileInPlace();

// Or generate fully compiled model
var compiledModel = model.Compile();

// Custom serialization configuration
model[typeof(Person)]
    .Add(1, "Id")
    .Add(2, "Name")
    .Add(3, "BirthDate");

// Using streaming API
public static async Task SerializeLargeDataAsync<T>(
    Stream stream, IEnumerable<T> items)
{
    foreach (var item in items)
    {
        Serializer.SerializeWithLengthPrefix(stream, item, 
            PrefixStyle.Base128);
        await stream.FlushAsync();
    }
}

// Streaming deserialization
public static async IAsyncEnumerable<T> DeserializeLargeDataAsync<T>(
    Stream stream)
{
    while (stream.Position < stream.Length)
    {
        var item = Serializer.DeserializeWithLengthPrefix<T>(
            stream, PrefixStyle.Base128);
        if (item == null) yield break;
        yield return item;
    }
}

Handling Null Values

[ProtoContract]
public class NullableExample
{
    // Regular nullable type
    [ProtoMember(1)]
    public int? OptionalInt { get; set; }
    
    // Wrapped null value transmission
    [ProtoMember(2), NullWrappedValue]
    public int? WrappedInt { get; set; }
    
    // Null collection handling
    [ProtoMember(3), NullWrappedCollection]
    public List<string> OptionalList { get; set; }
    
    // Conditional serialization
    [ProtoMember(4)]
    public string ConditionalField { get; set; }
    
    public bool ShouldSerializeConditionalField()
    {
        return !string.IsNullOrEmpty(ConditionalField);
    }
}

These examples demonstrate the main features of protobuf-net. It's an excellent choice for .NET applications requiring high-performance binary serialization.