When an object is added to Hiperspace their key and value components are encoded using protobuf to minimize the space utilization. The protobuf keys are then transformed from tag-length-value encoding to tag-value-length encoding to allow partial match of keys when searching for a range of values

The simplest example is the encoded of Country in the into key and value.

entity ERP.Location.Country : Versioned
(
    Id              : Int16
)
{
    Name            : String,
    ShippingCharge  : Decimal,
    Valid           = Name = null ? false : true
};

with code var uk = new Country { Id = 1, Name = "United Kingdom", ShippingCharge = 0m }; is encoded into the key

(note version is included in the binary key because this is a versioned entity) and value

Address with a reference to the country is modeled as

entity ERP.Location.Address : Versioned
(
    Id              : Int32
)
{
    Street          : String,
    Town            : String,
    City            : String,
    County          : String,
    Country         : ERP.Location.Country,
    PostalCode      : String,
    Valid           = Country = null || PostalCode = null ? false : true
};

The code var address {Id = 2, Street = "250 Bishopsgate", City = "London," Country = uk, PostalCode = "EC2M 4AA" }; is encoded as the key

and value (Town and County are not encoded because they're null) with only key reference to Country

Adding the Customer definition

entity ERP.Client.Customer : Versioned
(
    Id              : Int32
)
{
    Name            : String,
    BillTo          : ERP.Location.Address,
    ShipTo          : ERP.Location.Address,
    Valid           = Name = null || BillTo = null || BillTo.Valid = false || ShipTo = null || ShipTo.Valid = false ? false : true
}
[
    CustomerOrders  : ERP.Sales.Order  (Customer = this)
];

with value var natwest = new Customer { Id = 3, Name = "NatWest", BillTo = address, ShipTo = address }; is encoded with the key

and value with only key references to Address for BillTo and ShipTo

Updates

If the Customer name is updated from "NatWest" to "Nat-West" a new version is added with the version 4, which appears in the index before the other version

Usage

For most use-cases, you'll only be interested in the Customer name, but if the full postal address is needed, it is quickly fetched from Hiperspace with minimal latency

Storage

Encoding is performed within the domain space, and passed to the storage driver is the binary key/values.

Index

The definition of ERP.Client.Customer above includes an extension property CustomerOrders : ERP.Sales.Order (Customer = this) that is not stored as part of the state of Customer that needs to be indexed so that access from Customer to Order is efficient.

Oder is more complicated to encode because it has references to Customer, Store and Salesperson; Each of which has an extension property that enables them to look up the orders for that customer, that store or that salesperson. The domain specific SetSpace<>, SetSpace<ERP.Sales.Order> in the case of order has a KeyPath<O,K> (or KeyPathVersion<O,K> when versioned) to encode and decode keys, but also has any number of IndexPath<O,K> (or IndexPathVersion<O,K> when the element is versioned) that are used to encode and Index values and used for Find queries.

In the case of alter, there are three indexes for Customer (internal class OrderCustomerOrdersPath : IndexPathVersion<ERP.Sales.Order, ERP.Sales.Order.KeyType>), Salesperson (internal class OrderSalesPath : IndexPathVersion<ERP.Sales.Order, ERP.Sales.Order.KeyType>) and Store (internal class OrderStoreSalesPath : IndexPathVersion<ERP.Sales.Order, ERP.Sales.Order.KeyType>) that are used when Binding an Element to Hiperspace, and the rules-based-optimiser when Finding orders using the foreign key.

entity ERP.Sales.Order : Versioned
(
   OrderNumber     : Int32
)
{
   Customer        : ERP.Client.Customer,
   Store           : ERP.Location.Store,
   Salesperson     : ERP.Sales.Salesperson,
   Lines           : Set <ERP.Sales.OrderLine>,
   Valid           = Customer = null ||  changed(Customer) = true ||
                     Store = null || _changeStore = true ||
                     count(Lines.LineNumber) = 0 || _changeLines = true
                     ? false : true
}
[
   Invoiced        = Invoice = null ? true : false,
   _changeStore    = changed(Store) = true && Invoiced = true ? true : false,
   _changeLines    = changedLines(this) = true && Invoiced = true ? true : false,
   Invoice         : ERP.Sales.Invoice 
];

An order created with new ERP.Sales.Order { OrderNumber = 42, Customer = new Customer { Id = 3}, Salesperson = new ERP.Sales.Salesperson { Id = 6}, Store = new Store { Id = 1 }, Lines = new HashSet<ERP.Sales.OrderLine> { new ERP.Sales.OrderLine { LineNumber = 1, Product = new Product {Id = 4 }, Units = 10m } } } is encodes as key

with value (note Lines : Set<ERP.Sales.OrderLine> is a set of values, so encoded into the value of the Order)

and indexes

Stored as

Summary

At the Hiperspace driver, the key and value are opaque series of bytes that can be searched for a range of values without knowledge of the domain meaning (other than handling version numbers).

Objects are encoded once (in the domain space) and stored without further transformation in the driver or on a server.

To avoid transposition errors, durable a Hiperspace should be opened with the MetaModel parameter that validates that the protobuf field numbers have not been reused for an alternate purpose.