Library

Bond - Microsoft's Cross-Platform Serialization Framework

Overview

Bond is a schema-driven serialization framework developed by Microsoft. It supports high-performance binary formats and multiple protocols including JSON, and is available for multiple languages including C++, C#, and Python. While conceptually similar to Protocol Buffers, Bond offers a more flexible type system and rich customization options. It particularly excels in large-scale systems and microservice communication scenarios.

Details

Bond is a serialization library with a proven track record of powering data exchange in Microsoft's large-scale services. It uses a schema definition language (IDL) to define type-safe, version-compatible data structures. At compile time, it generates code for each target language from schemas, enabling efficient serialization and deserialization.

One of Bond's distinctive features is its support for multiple protocols. You can choose from fast binary formats (Compact Binary, Fast Binary), human-readable JSON format, XML, and others based on your use case. It also has excellent compatibility mechanisms for schema evolution, allowing safe addition, removal, and type changes of fields.

Support for generics is another notable feature, integrating naturally with C++ templates and C# generics. It provides streaming APIs for efficient processing of large datasets. Custom type conversions and behavior customization through attributes are also flexibly supported.

Advantages and Disadvantages

Advantages

• High Performance: Fast serialization with optimized binary formats
• Cross-Language: Supports C++, C#, Python, Java, and more
• Schema Evolution: Modify schemas while maintaining backward compatibility
• Multiple Protocols: Choose between binary, JSON, XML based on needs
• Type Safety: Compile-time type checking catches bugs early
• Customizable: Rich extension points and customization options

Disadvantages

• Learning Curve: Requires understanding schema language and code generation
• Build Complexity: Schema compiler integration complicates build process
• Community: Smaller community compared to Protocol Buffers
• Documentation: Limited official documentation with few examples
• Dependencies: Cannot avoid dependency on code generation tools
• Debugging: Generated code can be difficult to debug

References

• Bond Official Repository - GitHub
• Bond Getting Started Guide
• Bond Schema Language
• Bond Performance Benchmarks

Code Examples

1. Schema Definition (.bond file)

// person.bond
namespace Examples

struct Address
{
    0: string street;
    1: string city;
    2: string zipCode;
}

struct Person
{
    0: string name;
    1: uint32 age;
    2: optional<string> email;
    3: vector<Address> addresses;
    4: map<string, string> metadata;
}

struct Employee : Person
{
    10: string employeeId;
    11: double salary;
    12: nullable<Person> manager;
}

2. C# Serialization

using Bond;
using Bond.Protocols;
using Bond.IO.Unsafe;
using System.IO;

// Using generated classes
var employee = new Employee
{
    name = "John Doe",
    age = 30,
    email = "[email protected]",
    addresses = new List<Address>
    {
        new Address 
        { 
            street = "123 Main St",
            city = "Seattle",
            zipCode = "98101"
        }
    },
    employeeId = "EMP001",
    salary = 75000.0
};

// Serialize to Compact Binary format
var output = new OutputBuffer();
var writer = new CompactBinaryWriter<OutputBuffer>(output);
Serialize.To(writer, employee);
byte[] data = output.Data.Array;

// Deserialize
var input = new InputBuffer(data);
var reader = new CompactBinaryReader<InputBuffer>(input);
var deserialized = Deserialize<Employee>.From(reader);

Console.WriteLine($"Name: {deserialized.name}, ID: {deserialized.employeeId}");

3. JSON Format Serialization

using Bond.Protocols;
using Newtonsoft.Json;

class JsonExample
{
    static void UseJsonProtocol()
    {
        var person = new Person
        {
            name = "Jane Smith",
            age = 25,
            email = "[email protected]",
            metadata = new Dictionary<string, string>
            {
                ["department"] = "Engineering",
                ["location"] = "New York"
            }
        };

        // Serialize to JSON format
        var serializer = new SimpleJsonSerializer();
        string json = serializer.Serialize(person);
        Console.WriteLine("JSON: " + json);

        // Deserialize from JSON
        var deserializer = new SimpleJsonDeserializer();
        var deserialized = deserializer.Deserialize<Person>(json);
        
        Console.WriteLine($"Name: {deserialized.name}, Age: {deserialized.age}");
    }
}

4. C++ Streaming Processing

#include <bond/core/bond.h>
#include <bond/protocol/compact_binary.h>
#include <bond/stream/output_buffer.h>
#include "person_reflection.h"

void StreamingExample()
{
    // Stream large number of Person objects
    bond::OutputBuffer output;
    bond::CompactBinaryWriter<bond::OutputBuffer> writer(output);

    // Write multiple objects consecutively
    for (int i = 0; i < 10000; ++i)
    {
        Examples::Person person;
        person.name = "User" + std::to_string(i);
        person.age = 20 + (i % 50);
        
        bond::Serialize(person, writer);
    }

    // Streaming read
    bond::InputBuffer input(output.GetBuffer());
    bond::CompactBinaryReader<bond::InputBuffer> reader(input);

    while (!reader.IsEof())
    {
        Examples::Person person;
        bond::Deserialize(reader, person);
        
        // Process each object
        ProcessPerson(person);
    }
}

5. Schema Evolution Example

// version 1
struct Product
{
    0: string name;
    1: double price;
}

// version 2 - Added fields (backward compatible)
struct Product
{
    0: string name;
    1: double price;
    2: optional<string> category = "general";  // With default value
    3: vector<string> tags;  // New field
}

// Reading old version data with new schema
class SchemaEvolution
{
    static void HandleVersioning()
    {
        // Data serialized in v1 format
        byte[] oldData = GetOldVersionData();
        
        // Deserialize with v2 schema
        var input = new InputBuffer(oldData);
        var reader = new CompactBinaryReader<InputBuffer>(input);
        var product = Deserialize<Product>.From(reader);
        
        // New fields will have default values or be empty
        Console.WriteLine($"Category: {product.category ?? "general"}");
        Console.WriteLine($"Tags count: {product.tags?.Count ?? 0}");
    }
}

6. Custom Type Conversion and Attributes

// Schema with custom attributes
struct TimeSeries
{
    [JsonName("timestamp")]
    0: int64 unixTimestamp;
    
    [Bond.Id(1), Bond.Type(typeof(double))]
    1: float value;
    
    [Bond.Required]
    2: string sensorId;
}

// Custom transform implementation
class CustomTransform : ITransform
{
    public void Begin(ITransformContext context) { }
    public void End(ITransformContext context) { }
    
    public bool OnField(ITransformContext context, 
                       FieldInfo field, 
                       object value)
    {
        // Convert timestamp to DateTime
        if (field.Name == "unixTimestamp" && value is long timestamp)
        {
            var dateTime = DateTimeOffset.FromUnixTimeSeconds(timestamp);
            Console.WriteLine($"Timestamp: {dateTime}");
        }
        
        return true;
    }
}

// Usage example
var timeSeries = new TimeSeries
{
    unixTimestamp = DateTimeOffset.UtcNow.ToUnixTimeSeconds(),
    value = 23.5f,
    sensorId = "SENSOR-001"
};

// Apply custom transform
var transform = new CustomTransform();
var transcoder = new Transcoder<CompactBinaryWriter<OutputBuffer>>(transform);
transcoder.Transcode(timeSeries);