This release enhances the support for alternate indexes to include nested values and handling of unique references to other entities.

This enhancement removes the need to define additional element values to support view access.

Alternate Index

The normal/best way to define an index is to include an access path. The following model defines a Customer and Account and implicitly an alternate index for navigation from Customer to all accounts.

entity Customer (Id : Int32){Name : String}[Accounts : Account (Customer = this)];
entity Account (Id : Int32){Customer : Customer};

The @AlternateIndex property of a key/value allows for indexes to be created where it is not possible to define an access path (e.g. to segment or aspect that may have multiple implementations. If Account is instead defined as a segment, it would not be possible to define an access path from Book to Account since the segment could be defined for any number of elements.

entity Customer = Node ()(Id : Int32) {Name : String} [Accounts : Account, TypeName = "Customer"];
segment Account = Node(), Edge() (Id : Int32) {Name : String, @AlternateIndex Book : Book} [TypeName = "Account", From = Book, To = this];
entity Book = Node() (Id : Int32) {Name : String}[TypeName = "Book"];

When Book is viewed as a graph Node, it has a Froms collection of Edge that includes all Accounts that refer to the Book. @AlternateIndex creates an access path to Account from Book.

ISO20022

ISO20022.Static.BranchAndFinancialInstitution includes a nested value ISO20022.Static.FinancialInstitution that could be a Bank, a LegalEntity or both.

entity ISO20022.Static.BranchAndFinancialInstitution 
    = Node (Name = FinInstnId.Nm, SKey = SKey, TypeName = "ISO20022.BranchAndFinancialInstitution")
    , ISO20022.Edges 
        ( From = this
        , To = Bank
        , Name = BankName
        , FromTypeName = "BranchAndFinancialInstitution-Bank"
        , ToTypeName =  "Bank-BranchAndFinancialInstitution"
        )
(
    @XmlElement BrnchId            : ISO20022.Static.BranchData,
    @XmlElement FinInstnId         : ISO20022.Static.FinancialInstitution,
)
[
    @XmlIgnore, JsonIgnore 
    Bank                           : ISO20022.Static.Bank (AnyBIC = FinInstnId.BICFI),
]		

This release adds the option to include @AlternateIndex for ISO20022.Static.FinancialInstitution.BICFI and have an index on every entity/segment/aspect that includes the FinancialInstitution reference. When ISO20022.Static.Bank is viewed as a graph Node it will include a edges to and from every BranchAndFinancialInstitution that referenes it.

Unique Reference

When an element has an extent reference to another element it is normally a set, in the top example Customer.Accounts is a set because any number of Account could refer to Customer, but if the parameter to the reference is the same as key of the referenced element it is included as single since it will never have more than one value. When an element includes a reference to an aspect (single value inheriting the owner’s key) the single value behaves like a nested value:

• Reading the Value does a lazy lookup from the SubSpace cache, or from the underlying HiperSpace
• Setting the Value copies the owner key to the aspect transparently to behave like a value but stored seperately.

This release add the ability set foreign references in the same way as aspect, but instead of binding the owner of the aspect, the parameter fields in the referencing element are updated.

For the ISO20022 example above the code var aBranch = new BranchAndFinancialInstitution { Bank = goldmanSachs }; will assign goldmanSachs.AnyBIC value to aBranch.FinInstnId.BICFI since that is semantically equvilent to var aBranch = new BranchAndFinancialInstitution { FinInstnId = new ISO20022.Static.FinancialInstitution { BICFI = goldmanSachs }};

Worked Example

The ISO20022 includes the defintion of Party that can optionally reference a Legal Entity, a Bank or both, in addtion to generic person and branch information. The entity can be viewed as a Node in graph (with the TypeName "ISO20022.Party") and up to four Edge:

• Party-Bank (if OrgId.AnyBIC has a value) - via Party Key
• Bank-Party (if OrgId.AnyBIC has a value) - via Party AnyBIC index
• Party-LegalEntity (if OrgId.LEI has a value) - via Party Key
• LegalEntity-Party (if OrgId.LEI has a value) - via Party LEI index

if we query SELECT node, edge.To AS Party FROM Nodes AS node, node.Tos /*implicit join*/ AS edge WHERE node.Name = 'Goldman Sachs' AND edge.TypeName = 'Bank-Party'; will return a result set of all parties that participate in an ISO20022 message for the Bank with the name Goldman Sachs. The query plan will perform:

1. Create a template Node with Name "Goldman Sachs"
2. Query SetSpace<Node> for all Node that has a index on Name (including node.TypeName = 'ISO20022.Bank' would limit the query to SetSpace<Bank>)
3. Create a template Bank with Name "Goldman Sachs"
4. Query SetSpace<Bank> using the Bank Name index
5. Create a template Edge with From set to the Node object for "Goldman Sachs" and TypeName 'Bank-Party'
6. Query SetSpace<edge> (since edge TypeName is set, only SetSpace<Party> is searched)
7. Create a template Party with Bank set to "Goldman Sachs" (the binder code adds an OrgId value, and set OrgId.AnyBIC to Bank.AnyBIC)
8. Query SetSpace<Party> using the Party AnyBIC index
9. Project a data frame of Node and Party

The following LINQ query will traverse the graph of nodes through all edges (upto a length 100) to find CreditTransferTransaction associated with "Goldman Sachs":

from node in space.Nodes 
where node.Name == "Goldman Sachs" 
let transfer = node.HiperEdges("*", 100, new HashSet<string> {"Fact:ISO20022.Cube.CreditTransferTransaction"}
select transfer;

For Hiperspace.DB the graph search will use GPU parallel search, otherwise a parallel pipeline is used.

Referenced libraries

External references have been updated to the latest version

This release is a minor update for XML and JSON serialisation, following testing with ISO20022. XML Serialisation (with dotnet) reflects its age and is does not support references for elements referenced more than once in an XML document.

This release adds anotations to Hiperspace and HiLang to exclude Stringified key (SKey) and owner references from XML and JSON serialisation.

Referenced libraries

External references have been updated to the latest version

This release is a significantly enhances support for XML Schema, an introduces generated mapping for ISO20022 messages.

ISO20022 includes a large family of 761 XSD schema files comprising 39954 separate type definitions (many of which are duplicated in each schema file, but some specific domain variations). As a proof-of-principle the message schema has been mapped as-is with payments messages augmented to show how a Graph and Cube functionality can be added with minimal changes to store messages in hiperspace. Messages stored in hiperspace are naturally compressed from the XML or JSON source using protobuf encoding that replaces type names with the equivalent #id and native types instead of text encoding.

Hilangc changes

Import of XSD schema has been added to the command-line tool

Hilang changes

The directive %reference (domain.schema.dll); imports elements from pre-compiled domain schema all.hilang uses the reference import to build a consolidated ISO20022 schema including element definitions from each domain specific models {acmt, admi, auth, caaa, caad, caam, cafc, cafm, cafr, cain, camt, canm, casp, casr, catm, catp, colr, fxtr, head, pacs, pain, reda, remt, secl, seev, semt, sese, setr, trck, tsin, tsmt, tsrv}

The directive %annotations; has been added to simplify maintenance of XSD mapping. The code

@XmlNameSpace ("urn:iso:std:iso:20022:tech:xsd:pain.001.001.12"), ValidationCheck
entity ISO20022.Pain001001.CustomerCreditTransferInitiationV12 #35044
(
    @XmlElement GrpHdr             : ISO20022.Pain001001.GroupHeader114 #1,
)
{
    @XmlElement SplmtryData        : List<ISO20022.Pain001001.SupplementaryData1> #2,
};

could be modeled as

@XmlNameSpace ("urn:iso:std:iso:20022:tech:xsd:pain.001.001.12"), ValidationCheck
entity ISO20022.Pain001001.CustomerCreditTransferInitiationV12 #35044
{
    @Key, XmlElement GrpHdr             : ISO20022.Pain001001.GroupHeader114 #1,
    @XmlElement SplmtryData             : List<ISO20022.Pain001001.SupplementaryData1> #2,
};

to transform the the GrpHdr element of ISO20022.Pain001001.CustomerCreditTransferInitiationV12 from a value to a key without reordering the memvbers of the element.
This is intended to simplify the tracking of a golden-source of model using code change-control tools

Hiperspace changes

Various Hiperspace structures have been changed from struct to class to avoid stack-space issues with huge models. netstandard2.0 has been added as a target for Hiperspace assembly to enable meta-data definitions to be read by Hilang during compilation (or generation using hilangc)

Interface functions that were marked as Obsolete in earlier releases have been removed.

Additional Transactional versions of the Bind, Get and Find functions have been added, initially using the old functions. These functions take a MVCC Transaction reference has an additional parameter and a DurableType to allow the functions to be used for:

Hiperspace Find will search for hiperspace using the key or index values. The new function extends the search parameters to include a collection of values that are not part of a key/index to reduce the number of elements returned from hiperspace to pre-filter elements close to source.

This release is a minor update to support the presentation Cube data in Graph views with Hiperspace.DB, and minor updated to external references.

Cube enhancement

Aggregation

Hiperspace Cubes are declarative and stream-calculated by Hiperspace.DB, and viewable in graphs as a Node with Measures and a TypeName begining with "Cube:" (for aggregates) or "Fact:" for leaves within a Cube hierarchy. This release changes the presentation of Cube nodes that do not relate to any underlying facts:

• Do not return a facts collection when the Facts property is zero
• Do not return leaf dimensions (to a Pivot-Table) when there are no child dimensions or Measures

Validation

Validation rules have been relaxed to create cubes even if it doesn't have any measures. Cubes provide summary information for a single or multiple Dimensions, but sometimes the only measure you need is the number of connections included:

entity Customer = Node () (Id : Int32) {Name : String} [TypeName = "Customer"];
entity Order  = Node (), Edge (From = Customer, To = this, TypeName= "Order"), Edge_ (From = this, To = Product, TypeName = "Ordered" )
  (Id : Int32) {Customer : Customer, Product : Product} [Name = Customer.Name + "-" + Product.Name, TypeName = "Order"];
entity Product = Node () (Id : Int32) {Name : String} [TypeName = "Product"];

If we're not interested in each order, a Cube can be used to simplify the graph to

by changing the model to

@CubeDimension entity Customer = Node () (Id : Int32) {Name : String} [TypeName = "Customer"];
@CubeFact entity Order (Id : Int32) {Customer : Customer, Product : Product};
@CubeDimension entity Product = Node () (Id : Int32) {Name : String} [TypeName = "Product"];

If any of the dimensions to a Cube can be viewed as a Node, the _Cube and _Fact elements become a Node with Drilldown Edge to each dimension. This change enables Cube to be used to simplify a graph, even when the only thing being measured is the number edges being replaced.

View enhancement

The addition of the Cube summaries enhances the presentation of Graphs but introduced a level of detail that might be a distration for analysis. If you have a trade cube with dimensions {Sector, Product, Country} and drilldown to the lowest level of detail, a Graph search will include each level of detail all with node "Cube:Contract_Cube" and edges "Cube:Drilldown" and "Cube:Dimension". Too enable more targeted analysis, node TypeName has been enhanced to include the number of dimensions included in each cube:

• Cube has TypeName "Cube:Contract_Cube:1" with Sector->Cube (TypeName "Cube:Drilldown:1"
• {Sector, Product} Cube has TypeName "Cube:Contract_Cube:2" with Sector->Cube (TypeName "Cube:Drilldown:2"
• {Sector, Product, Country} Cube has TypeName "Cube:Contract_Cube:3" with Sector->Cube (TypeName "Cube:Drilldown:3"

Element enhancement

The IElement interface of Element<TElement> has been updated to include an ISetSpace reference for the trnasformation of ICubeFact facts to a acyclic directed graph needed to render Cubes as a Pivot-Table with drilldown to futher details.

HiLang enhancement

Extent expressions do not normally need #id to be recorded because they are not stored (calculated when needed from an element), but this is not the case for cube measures. Source editing has been added to carry forward the same id as the model changes.

Release also includes updates to referenced libraries

This release is a minor update to support the presentation Cube data in Graph views with Hiperspace.DB, and minor updated to external references.

Fact enhancement

A specific ICubeFact<TFact> inteface has been added to include public IEnumerable<TFact> DrillDown(Type dimension); for generic drilldown from one Cube to the next level when viewed in Hiperspace.DB.

The ICubeDimension interface now has a GetCube() function to get the named ICubeFact for use with a PivotTable view of the Cube through Hiperspace.DB.

SubSpace enhancement

SubSpaceParameters has been enhanced to include TransactionalPolicy

Hilang enhancement

Added support support for Cube aggregation of views in addition to {entity, segment, aspect}.

Consider the model where there are seperate entitites for {FX, FI, EQ} Trades that have different values for Entities, Bonds, Foreign Exchange, but can all be viewed as a Trade that has a booking_id, and a valuation (NPV). This change adds support for Cube analytics of views combining all trades in all entities that provide the view.

type Banking.TradeBase
(
	Id : String
) 
{
	Book : Banking.Book,
	@CubeMeasure(Aggregate.Sum)
	NPV : Decimal
};

@CubeFact
view Banking.Trade : Banking.TradeBase;

entity Banking.FI.Trade : Banking.TradeBase = Banking.Trade();
entity Banking.FX.Trade : Banking.TradeBase = Banking.Trade();
entity Banking.EQ.Trade : Banking.TradeBase = Banking.Trade();

@CubeDimension
entity Banking.Book ( Id : String ) [ Trades : Banking.Trade (Book = this)];

When the schema is compiled with HiLang, Banking.Book has an additional property Trade_Cube that aggregates NPV over the full set trades that can be viewed as a Banking.Trade.

Enhanced List support

Keys in a hilang element can be:

• A Value including value elements defined in Hilang and primatives (e.g. String, Int32, Guid, etc)
• A Reference to an entity (only the Key-part of the referened entity is serialised)
• A List of Values (e.g. List<String>) but not a Set<> since a Set order can change

Key support for List<> in keys has been extended to include Keys of Lists of Values of Lists of Values, that can be placed in a SortedSet. Examples include FINIOS CDM that supports complex identifiers that include list of identifiers

value CDM.PartyIdentifier ( identifier : String );
entity CDM.Party ( partyId : CDM.PartyIdentifier );
value CDM.PriceQuantity;
value CDM.AssetIdentifier
(
    identifer : String,
    version : Int32
);
value CDM.Identifer
(
    assignedIdentifier : List<CDM.AssetIdentifier>,
    issuerReference : CDM.Party,
    issuer : String
);
entity CDM.TradeLot
(
    lotIdentifier : List<CDM.Identifer>
)
{
    priceQuantity : List<CDM.PriceQuantity>
};

In this example CDM.TradeLot has a key that is a List<CDM.Identifer> which itself contains a List<CDM.AssetIdentifier>. Lists are supported in HiLang models since the they can be unambiguously serialised to a store and retrived later. NB CDM.Identifer contains a reference to CDM.Party which is a entity so only the key is stored, with CDM.Party retrived only when needed.

This release is a minor update to support the presentation Cube data in Graph views with Hiperspace.DB, and minor updated to external references.

Fact enhancement

Every instance of a _Fact or _Cube element now includes a Facts property that excludes Node that is a view of a fact but have no underlying value.

Node Enhancement

Node now includes a Measures property, which is a List<MeasureValue> to eliminate the need to separately fetch measures when displaying Cube Node in a Hiperspace.DB graph display. Normally a graph showing "which client Sectors trade which products" would involve connecting the edges Sector->Client->Account->Trade->Leg->Asset->Instrument->Product or HiperEdge Sector=>Trade=>Product, and then aggregate results. When Sector and Product are @CubeDimension and Trade is a @CubeFact the aggregate values are available directly through Sector->Trade_Cube->Product with streaming calculation in Hiperspace.DB. Displaying aggregate values in a graph is so useful for analysis that it is worth extending the Node model to include them as an optional property.

NB This diagram is from a HiperGraph of the Sector "All", Product hierarchy aggregates are connected to the Sector hierarchy (without spider connections)

Hilang enhancement

• Added support for enum values in messages.
• Resolved issue with self-referencing messages message Dimension { Branches : List<Dimension> };
• Name resolution of types now uses global namespace resolution to avoid clashes.

SubSpace enhancement

Currently Blazer Web Assembly clients cannot (currently) support threads and therefore cannot perform synchronous IO - FindAsync() and GetAsync() requests can be routed (over gRpc) to a remote store, Find() and Get() request are therefore cache-local. the test for OperatingSystem.IsBrowser() has been replaced with a CachePolicy that default to CachingPolicy.Cache for Browser and CachingPolicy.Space otherwise. This allows message to be used to pre-fetch dependencies on the server-side in addition to browser.

This release includes minor enhancement to the graph functionality when used with Hiperspace.DB graph view. Hiperspace.DB provides a graph view that aggregates alll databases, so that graph shows all connections both local and foreign. In the following example Togaf.Has.WorkPackage includes a value ProjectKey that could be a stringified SKey reference to a project plan in another database. When the project is viewed as a graph in Hiperspace.DB, the graph includes paths between a project plan and strategic goals.

segment Togaf.Has.WorkPackage : Togaf.Base 
    = Node        ( SKey = SKey, Name = Name, TypeName = "AF-WorkPackage"),
      Edge        (From = owner, To = this, Name = Name, TypeName = "AF-Has-WorkPackage"),
      Togaf.Edge_ (To = owner, From = this, Name = Name, TypeName = "AF-WorkPackage-For"),
      Togaf.Edges (From = this, To = Project, Name = Name, FromTypeName = "AF-WorkPackage-Plan", ToTypeName = "AF-Plan-WorkPackage"),
      Graph.HiperEdge = StrategicEdges #2675 
{
    ProjectKey      : String #29 
}
[
    "All Togaf.Edges that can be projected as Transitative Togaf.Edges to a Business Goal"
    @Once
    StrategicEdges  = StrategicEdge(this),
    Project         = externalnode(ProjectKey)
];

For efficiency, the SKey property has been changed from a Base64 encoding of the key to add the option of a database prefix. The Node SKey "Togaf.ascxyz==" will only search for matching items "abcxyz==" in the Togaf database. Within the Togaf database, only the specific segment is searched because the key includes the #Id of the element set.

the following SQL query would return a table of all project plans and Goals that can be reached through the {component, service, function, process, capability, etc} that are in some way associated with the Work Package

SELECT project.Name AS "Project Name"
     , goal.Name AS "Goal Name"
     , project.SKey as "Project Id"
     , goal.SKey as "Goal Id"
FROM   Nodes as project
     , project.Tos as planedge
     , planedge.To as workpackage
     , workpackage.Tos as workedge 
     , workedge.To as goal
WHERE  project.TypeName = 'Plan'
  AND  planedge.TypeName = 'AF-Plan-WorkPackage'
  AND  workpackage.TypeName = 'AF-WorkPackage'
  AND  workedge.TypeName = 'Strategic-Goal'
  AND  goal.TypeName = 'AF-Goal';

NB the five table join uses implicit joins in Hiperspace.SQL because project.Tos and workpackage.Tos are Set<Edge> properties that are implicitly filtered by the source Node

This release introduces two significant enhancements to hyperspace, to support Mult-version concurrency control in Hiperspace.DB, and Mermaid Markdown documentation for #AI agents.

Multi-Version Concurrency Control (MVCC)

MVCC is rarely needed for normal Hiperspace transactions because it is possible to use CQRS to apply a change to elements in Hiperspace that are invisible to other sessions (using Horizon filters), but this is not practical for Cube aggregation, where it would be possible to have inconsistent aggregates while updates are applied in real-time.

This implementation of MVCC follows the ISO SQL standard that every database change is enrolled in a transaction and only visible to other sessions when the transaction is committed and backed-out on rollback. There are no nested transactions, since the implementation (in Hiperspace.DB) is efficient and combined with the @DeltaIndex functionality for efficient point-in-time access.

For local hiperspaces (e.g. Hiperspace.Rocks) transactions are automatic, and Rollback() will return false

public bool Commit() => Commit(Transaction);
public bool Rollback() => Rollback(Transaction);

MarkDown documentation with HiLang

The HILang schema compiler generates a Domain Specific DataBase that can be used in a client application (including web-assembly) for object persistence either to a local hiperspace (e.g. Hiperspace.Rocks) or remotely using Hiperspace.DB either with lazy loading of reference or eager loading using message. Message is analogous to stored procedures that execute on the server (using message key) and return full object value in response using a shared DSDB code.

Code is generated during compilation to produce a DSDB with a number of transformations:

• Segments (many) are specialized for the Element that references them
• Aspects (one) are specialized for the Element that references them
• @CubeFact elements are transformed adding properties for aggregation
• @CubeFact elements are transformed to provide _Cube aggregates and _Fact cube cell aggregates
• @CubeDimension elements are transformed to include DrillDown to aggregates via lazy property

These transformations are not visible to AI Agents because the generated code is ephemeral (using the Indonesian word hilang). For agent-based development, the HiLang schema can be generated to markdown files using the %markdown directive in source.

Schema.md

For each .hilang schema file a matching .md file is generated containing formatted documentation in a AI friendly form, with a mermaid diagram of the data-model. This documentation file matches the structure of the .hilang schema

Schema-physical.md

For each .hilang schema file a matching .md file is generated containing formatted documentation in a AI friendly form, with a mermaid diagram of the data-model. This documentation file matches the generated classes, including all the transformations referenced above

Schema-cube.md

This markdown file contains only _Cube elements and the related dimensions

Schema-graph.md

This markdown file contains only graph (Node and Edge) elements and a diagram of relations

Cube Enhancements

A minor enhancement has been made to fully qualify references to Hiperspace.Functions. This prevents naming conflicts when a domain also uses Functions.

When one of a Cube’s Dimensions is Node, the Node view is added to the Cube along with Edge definitions that represent navigation between the Dimension and the Cube. In this release, the naming of these edges has been updated to ensure consistent navigation through a HiperGraph, using a common TypeName convention:

• TypeName Cube:Drilldown for navigation from a Dimension to the Cube.
• TypeName Cube:Dimension for navigation from the Cube to a Dimension.

Example

In the Cube Example Customer Sector and Product are Dimensions that can also be viewed as Nodes. The Contract Cube therefore has Node and Edges added as a views because.

• Contact (Cube) has one Account and Account has one Customer and Customer has one Sector (Dimension), it is inferred that Sector is a Dimension of Cube Contact.
• Contact (Cube) has one Instrument and Instrument has one Product, it is inferred that Product is a Dimension of Cube Contract.

When viewed through a HiperGraph the Cube becomes a Node that links Sector node to Product Node

If you drilldown from Product to Contact_Cube and drilldown to Sector, the shortest HiperEdge from Product to Sector is through the Contract_Cube slice that summarizes the connection between the two Nodes. We call this combination of HiperGraph and Cube "HiperCube", and is useful for AI prompts like "which sector uses most widgets?"

Cube enhancements

@CubeDimension and @CubeHierarchy updated to better support generic access to Cube data from Graph Pivot-Table view.

• @CubeDimension name parameter now refers to the property that provides the name of the element in a Pivot-Table dimension.
• @CubeHierarchy has additional parameters for Children and Level

Create GPU Accelerated Graph Search introduced a problem for cubes in that every single permutation of aggregate would be evaluated to produce the Graph cache resulting in:

• A dense MOLAP cube
• Expensive invalidation when cube facts change

The Cube model has been split into _Fact entity containing just base aggregates projected from @CubeFact elements, and _Cube elements that are calculated on demand for summaries.

Drilldown performed well when then was a very small number of dimensions and hierarchy levels, but poorly when the complexity increases. The mode has been changed from using Drilldown as an index for aggregate calculation to a scan of matching facts

Is Fact

IsFact attribute of _Cube and _Fact has been removed, as it is no longer needed. This is a help for BI access of Cube data via ODATA.

Cube Hierarchy

Keep hierarchy has been enhanced to include the name of the children property in addition to parent and add level to support aggregation of cube summaries

An example dimension is Organization that is structured as a hierarchy with Parent as a reference to the aggregation line, and Children as a collection of organization units below the current node. When referenced by a @CubeFact element, the aggregate values of all @CubeMeasures members show the aggregate value of the node and all child nodes

@CubeDimension, CubeHierarchy(Parent, Children, Tier)
entity Organization (Name : String) {Description : String, Parent : Organization, Tier : Int32} [Children : Organization (Parent = this)];

The generated code implements @CubeHierarchy using the ICubeHierarchy<TEntity> that now includes

Cube Measure

@CubeMeasure has been enhanced include Population Standard Deviation and Ranked Percentile aggregates. Aggregate.StdDev and Aggregate. Percentile use the Vector pseudo-aggregate member that includes every observation in the _Fact element for precise aggregation at the summary level.

@CubeFact, DeltaIndex 
aspect Valuation {@CubeMeasure(Aggregate.Sum) NPV : Decimal} 
[@CubeMeasure(Aggregate.StdDev) Deviation = NPV, @CubeMeasure(Aggregate.Percentile, 95) PFE = NPV];

is equivalent to

When the Hiperspace is hosted using Hiperspace.DB the _Fact and _Cube elements are automatically recalculated:

• When a Fact changes the _Fact entity is recalculated with its dimension references
• When an element that a Fact is dependent on changes the _Fact entity is recalculated
• When a Dimension changes, any _Cube elements that reference it are recalculated.

Because _Facts and _Cubes are recalculated using a delta of changed elements in real-time, it is possible for Facts to be calculated from other aggregated Facts. Both Facts and Cubes track history through time (using GetVersions() functions)

Relase also includes updates to reference assemblies

This release is focused on improvements to the Cube functionality to provide a foundation for viewing Cube's through Hiperspace.DB's Blazor. The Blazor explorer uses

• Cube Dimension Name to display the next of the element in a PivotTable view
• Cube Hierarchy Level to enable the PivotTable to view to drilldown through hierarchies of elements

For Portfolio Cube dimension with three levels in the hierarchy, the cube will be displayed in a Pivot -Table as {Portfolio 1, Portfolio 2, Portfolio 3}, and support drilldown from the highest level

@CubeDimension(Id), CubeHierarchy(Parent, Tier), Versioned
entity Cube.Portfolio 
	= Node (SKey = SKey, Name = Id, TypeName = "Portfolio"),
	  Cube.Edges (From = this, To = Parent, FromTypeName = "Portfolio-Parent", ToTypeName = "Portfolio-Child", Name = Id)
(Id : String) 
{ Parent : Cube.Portfolio, Tier : Int32 } 
[Children : Cube.Portfolio (Parent = this)];

Hiperspace

CubeDimension has been updated to include a reference to the name that should be used when the Cube is viewed in a Piviot-table view. CubeHierarchy has been updated to include a level indicator to allow Pivot-table views to navigate through hierarchy of elements

HiLang

###empty view enhancement The view Cube.Edge_ and Cube.Edges could be used as helpers to provide additional ways to project elements as graph views, but if they are not referenced elsewhere in the model, they will be pruned and not have any code generation.

"An additional edge helper for Cube.Edges"
view Cube.Edge_ : Edge = Edge();

"Bidirectional Edge, implemented with two Cube.Edges"
view Cube.Edges
    = Edge  (From = From, To = To, Name = Name, TypeName = FromTypeName),
      Cube.Edge_ (From = To, To = From, Name = Name, TypeName = ToTypeName)
(
    From            : Node      #7,
    To              : Node      #8,
    FromTypeName    : String    #9,
    ToTypeName      : String    #10
)
{
    Name        : String    #6,
};

This presents a problem for the generation of the Edge subspace, since it expects to add a base reference to the pruned view. A fix has been added to the HiLang compiler to remove the dependency. The compiler pipeline has been changed to continue process transformations to prevent spurious errors being created from validation that depends on prior transformations .

This release focuses on foundations to support of the migration of data from the current model of using key prefix for exceptions ("00" MetaModel, "0.." versioned, "00.." vector, "000.." sequence), to using the first-byte as a prefix for key-type. To address this need, dump / load functions have been added to SubSpace for JSON data interchange"

SubSpace

The JSON format is structured as { Key : Element { ....}, AsAt : timestamp, Value : Element { ....}}}

• IEnumerable<string> Dump() : dump the content of the hiperspace to JSON
• Load(IEnumerable<string> source) : load the JSON to Hiperspace

Change to the GetSequence and UseSequence functions to take a parameter to allow the use of KeyType for Sequences

Extended Session2(DateTime? AsAt = null) to Session2(DateTime? AsAt = null, DateTime? DeltaFrom = null) to allow the DeltaFrom parameter to be used in addition to AsAt.

Obsolete

Several interfaces function have been marked as Obsolete and will be removed in the next version.

Currently Hiperspace uses a key prefix convention to distinguish immutable and versioned elements, VectorSpaces (for nearest neighbor search) and sequences

Performance testing of Hiperspace.DB has highlighted that the rebuilding of the MetaMap used to translate between protobuf tag/length/value to tag/value/length for index search can be expensive top build when a store has experienced multiple schema changes. The next version will change the prefix to:

The #id for {Node, Edge, VectorNode, HiperEdge,PathMessage} will also be moved to a reserved set of low values with all domain keys moved to higher values

SetSpace

Added two functions to get a sequence counter from the driver:

• GetSequenceAsync<T>(T element) to get the current sequence number associated with the domain key
• UseSequenceAsync<T>(T element) to get a new sequence number for the with the domain key

NB While the main use of Sequence numbers is to provide an alternate (GPGPU friendly) sequence number for a Node, it can be applied to any key value.

// get an order number that is unique within all orders
var orderNo = await space.Orders.UseSequenceAsync (new Order { });
var orderNo = await space.Orders.UseSequenceAsync (new Order.KeyType { });

// get a line number within the owning Order {1,2,3..}
var lineNo = await space.Lines.UseSequenceAsync (new Line { Order = order }); 
var lineNo = await space.Lines.UseSequenceAsync (new Line.KeyType { Order = order.self }); 

Get and GetASync() functions now handle not found errors by returning null for scenarios where a reference object cannot be found. This is especially useful for views where not found is better handled by Horizon<> filters that

Bind Version

All versions of Bind Function have been marked as Obsolete, with a preference to use the optimistic locking instead (that include the original AsAt copied to AsWas for comparison). Versions of Bind functions that include the priorVersion DateTime value use optimistic concurrency control to prevent updates to a version that has been changed by another session

Nearest

Hiperspace provides nearest neighbor search using a VectorSpace aspect for geospatial and RAG search. These functions have been extended to include a maximum distance search

Hilang

The Hilang ahead of time compiler has been updated to remove indexes that are a subset of the element key, and will not be used

Hiperspace.Rocks

The RocksDB driver has been upgraded to verion 10.4.2.63147

Change method signatures for Path functions to use Route instead of RouteMap for remote calls.

Add Session2 to Subspace to support time-travel sessions over a graph space to support partial refresh of GPU cache for GPU parallel search of Graph methods without the need for recursive cycle detection.

Sequence support

Sequence numbers are a relational database feature that has largely been replaced by UUID (especially v7) since they do not require a sequence source and can be allocated within a client. They still have a use as an alternate key especially with GPGPU that need primitive points, but do not wish to use the expense of mapping UUID to int128 values that may not be supported by the device

Hiperspace implements sequences as unsigned 64-bit integers (1 to 18,446,744,073,709,551,616), and can be stored for any element key value

Hiperspace

Added UseSequenceAsync() for driver implementation, together with proxy methods in {BrowserSpace, FailoverSpace, GenerationSpace, PartitionSpace, SessionSpace}

Hiperspace.Heap

Implemented UseSequenceAsync() in Hiperspace.Heap and added additional driver CacheHeapSpace for use by caches that do not require history to be maintained for elements. CacheHeapSpace should only be used by caches since it lacks the ability to reconcile concurrent changes through version inspection.

LatestSpace is replaced by CacheHeapSpace and will be removed in a future release.

Hiperspace.Rocks

Implemented UseSequenceAsync()inHiperspace.Rocksand added additional driverCacheRockSpace` for use by caches that do not require history to be maintained for elements. CacheRockSpace should only be used by caches since it lacks the ability to reconcile concurrent changes through version inspection.

The underlying RocksDB driver has been updated.

SubSpace

SubSpace forwards the UseSequenceAsync() function to the underlying driver

SetSpace

Added two functions to get a sequence counter from the driver:

• GetSequenceAsync<TEntity>(TEntity? element) to get the current sequence number associated with the domain key
• UseSequenceAsync<TEntity>(TEntity? element) to get a new sequence number for the with the domain key

NB While the main use of Sequence numbers is to provide an alternate (GPGPU friendly) sequence number for a Node, it can be applied to any key value.

// get an order number that is unique within all orders
var orderNo = await space.Orders.UseSequenceAsync (new Order { });

// get a line number within the owning Order {1,2,3..}
var lineNo = await space.Lines.UseSequenceAsync (new Line { Order = order }); 

HiLang

Added code generation for the SubSpace methods.

OnInitialized

Addition of virtual void OnInitialized() function to SubSpace to allow domian subspaces to perform additional initialization after the Domain Space has been created by a generic container (e.g. Hiperspace.DB)

This release introduces GPU optimization for graph searches involving HiperEdge and other vector operations used in nearest-neighbor searches (commonly required when providing information for AI prompts).

It also enhances Hiperspace functions for LINQ queries by adding automatic null-coalescing for values selected from Hiperspace, supporting non-sargable join conditions, and improving performance for multi-table joins and HiperspaceDB cube aggregation.

CalculationGPU

Graph Theory is a branch of Mathematics that can be used to describe all knowledge as Node and Edge's between them; because there is a simple consistent model across domains it has wide application for visualizing information, but has the downside that many graph views appear as a tangled web of nodes and lines: often conveying complexity, but not clarity.

HyperGraph is a generalization of Graph that replaces the tangle of edges with a hyperedge that encapsulates all the intermediate nodes that are not interesting: I have a cousin (Robert), which can be described as "Me->(cousin)->Robert", but cousin is actually a hyperedge that is derived from "me->mother2->mother1->daughter->son where mother2 is not daughter"

HiperEdge is a HiperSpace implementation of hyperedge that encapsulates the source of the hyperedge for a simple view of connections or the intermediate nodes in the path. When viewed graphically a HiperEdge view looks like a tree of connections extending out from the subject Node.

Hiperspace provides functions to query HiperEdges using rules that project {source node type, edge type, end node type} to create HiperEdges whenever needed (e.g. as a member function of a Node).

Graph support

This version introduces ICalculationGPU for optional acceleration of Graph Queries using the hardware created for Ray Tracing,

Calculation support

Search for Nearest node proximity in a Vector Space is supported via GPU, together with aggregation of Vectors used for complex calculation.

Null Coalesce

Consider a cube analytics model with {Sector, Customer, Account, Trade} with dimensions Customer and Sector; with Trade Facts aggregated. Null-coalescing is necessary when querying dimensions associated with a Trade fact.

@CubeDimension, CubeHierarchy(Parent), Versioned
entity Sector 
  (Id : Int32) 
  {Name : String, Parent : Sector} 
  [Children : Sector (Parent = this), Customers : Customer (Sector = this)];

@CubeDimension, DeltaIndex
entity Customer 
  (Id : String) 
  {Sector : Sector}
  [Name : String, Accounts : Account (Customer = this)];
entity Instrument;

entity Account 
  (Id : String) 
  {Name : String, Customer : Customer};

@CubeFact, DeltaIndex
entity Trade : Versioned 
  (id : Int64) 
  {@CubeMeasure(Aggregation.Sum) Quantity : Decimal, Instrument : Instrument, Account : Account};

The query from trade in Space.Trades select trade.Account.Customer.Sector provides the Sector for a trade, but if any {Trade.Account, Account.Customer, Customer.Sector} is missing the expression will break. The work around is to manually add null-coalesce to the query:

from trade in Space.Trades 
select trade.Account ==  null ? null :  
       trade.Account.Customer == null ? null :
       trade.Account.Customer.Sector;

This release adds coalesce automatically. This is especially useful when Hiperspace.SQL is used to query a hiperspace from a scripting language where null-coalesce is not available, but we want to access the graph of properties.

select c.Id as SectorId, a.Id as AccountId
from Account as a join 
     Customer as c on a.Customer.Id = c.Id
group by c.Sector.Id, a.Id;

This query would have previously failed if a.Customer is missing.

Non-Saragable predicates

In Hiperspace the following query cannot be Sargable since Customer does not contain a Sector but a KeyRef<Sector.KeyType, Sector> with lazy loading of the element if needed.

select Sector.Name as "Sector Name", c.Name as "Customer Name"
from   Sector join
       Customer on Sector.Name = Customer.Sector.Name
order by Sector.Name, Customer.Name;

This query would previously have failed since Customer.Sector is a derived property of Customer and cannot be applied to a template for a Find() request of matching Customers

This release splits join processing into Sargable predicates and residual conditions and is constructed dynamically as compiled expressions.

This release adds NotFoundException to distinguish not found from cannot be found conditions. and extends the functionality of @AlternateIndex to support multiple alternate indexes on segments and aspects that are referenced by multiple entities.

Not Found

Prior to this release Get(...) calls did not distinguish between Not Found and not found because of IO error. To improve the handling of missing values several changes have been made:

• Additional Exception class NotFoundException
• KeyRef<> (reference to another element) changed to return null when a value cannot be found
• RefSingle<> (reference to an aspect) changed to return null when a value cannot be found

AlternateIndex

Alternate indexes are created automatically whenever there is a path from an element from another element, but can be added to support access from a view. The prime example is Edge which is defined (in the Hilang prelude) as

"edge between nodes"
view Edge 
( /* keys */
    From        : Node,
    To          : Node,
    TypeName    : String
)
{ /* values */
    Name        : String
};

@AlternateIndex enables an element to index the key/value that corresponds to the From key member for indexed access from a Node.Froms extension property

For the model

entity CostCentre (Id : Int32) [Costs : Cost (CostCentre = this)];
aspect Cost {CostCentre : CostCentre, Amount : Decimal};

with

entity Asset (...) {...} [Cost : Cost];
entity Project (...) {...} [Cost : Cost];

Concreate elements AssetCost and ProjectCost will create indexes AssetCostCostCentre.Index and ProjectCostCostCentre.Index

Source edit will change the source to

entity CostCentre (Id : Int32) [Costs : Cost (CostCentre = this)];
aspect Cost 
{
  @AlternateIndex("AssetCost", 42)
  ,AlternateIndex("ProjectCost", 43) 
  CostCentre : CostCentre, Amount : Decimal};

To ensure the index Id is not used for something else resulting in an incompatible model and store.

how does CostCentre know what (Asset/Project/ etc) the Cost is for? The aspect Cost is transformed to a view that is equvilent to

view Cost (owner : Any) {CostCentre : CostCentre, Amount : Decimal};

the (C#) hiperspace query

from centre in space.CostCentres 
select centre.Id, (from line in centre.Costs 
                   let asset = line.owner.Is<Asset>() ? Amount : 0 
                   let project = line.owner.Is<Project>() ? Amount : 0 
                   group line by line.CostCentre into totals
                   select new { Projects = totals.Sum(v => v.asset),
                                Assets = totals.Sum(v => v.project)})

Will return the total costs by type for each CostCentre

Inherited Index

This model defines an overall trade type with three different implementations for {FI, EQ, FX} that have different properties for the different asset-classes. Trade is referenced by Book, the extension property Book.Trades returns a collection of Trade that has a Book equal to the current Book. For efficient access, an index is created for the Trade.Book that is inherited by each implementation.

The syntax Banking.FI.Trade : Banking.Trade = Banking.Trade() means Banking.FI.Trade:

• Inherits keys / values / extensions / properties from Banking.Trade (via :)
• Can be viewed as a Banking.Trade (via =)

view Banking.Trade (Id : String) 
{Book : Banking.Book};
entity Banking.FI.Trade : Banking.Trade = Banking.Trade();
entity Banking.FX.Trade : Banking.Trade = Banking.Trade();
entity Banking.EQ.Trade : Banking.Trade = Banking.Trade();
entity Banking.Book (Id : String) [Trades : Banking.Trade (Book = this)];

Adding %ids to the model, with result in a source edit to a # id to each element, key/value and extension property, and @AlternateIndex property for each generated concrete index.

view Banking.Trade #45 (Id : String) 
{@AlternateIndex("Banking.EQ.Trade", 52) 
 ,AlternateIndex("Banking.FI.Trade", 48)
 ,AlternateIndex("Banking.FX.Trade", 50) 
 Book : Banking.Book};
entity Banking.FI.Trade : Banking.Trade = Banking.Trade() #49;
entity Banking.FX.Trade : Banking.Trade = Banking.Trade() #51;
entity Banking.EQ.Trade : Banking.Trade = Banking.Trade() #53;
entity Banking.Book #47 (Id : String #1) [Trades : Banking.Trade (Book = this) #54];

subsequent compilation of the model will result in the indexes using the same Id value when stored.
NB the #id can be of any value, but can never to reused for a different purpose once used with a Hiperspace.

Source Editing

When the directive %ids is added to a hilang model, the source code is edited to add #id values to ensure that the schema can be evolved without the risk of introducing incompatible changes.

know issue: Source editing lacks the context of other edits to a line of hilang source @AlternateIndex("Banking.EQ.Trade", 52) @AlternateIndex("Banking.FI.Trade", 48) Book : Banking.Book will create a syntax error the next time the schema is compiled because @ is the prefix for a comma-separated list of one-or-more attributes. The code needs to be edited to change subsequent @ to ,

This release adds the @AlternateIndex property to explicitly create an index for views and ad hoc queries that do not or cannot be defined in the normal way of using an access path.

Path Index

Normally indexes are created in Hiperspace by defining an extent that uses an attribute to retrieve a list of elements. The following hilang model defines a Person with a Parent and an extension Children that retrieves all Person that have a Parent that refers to this person.

entity Person (Id : Guid) {Parent : Person} [Children : Person (Parent = this)];

The Children extent causes an index to be created on Parent so that access to Children is fast and does not require an scan of all Person elements

Alternate Index

The following model defines two entities {Product, Order} that can be viewed as nodes in a graph. The segment Item defined the details of the Order with reference to the Product being ordered.

entity Product = Node(...) (Id : Guid);
entity Order = Node (...) (Id : Guid) [Items : Item];
segment Item = Edges (From = owner, To = Product, FromType = "Purchase", ToType ="Purchased")
(Line : Int32) 
{
  @AlternateIndex
  Product : Product,
  Quantity : Decimal
};

In a UML diagram, Order aggregates Item, and Item is associated with Product. In a Graph diagram Order and Product are both Nodes with two Edge between them for "Purchase" and "Purchased".

Product cannot refer to Item because it is a segment of its owner (in this case Order), and cannot therefore define a path from product to Item. When viewed as a graph the Order->(Purchase)->Product Edge uses the Item key to find all edges, but the Product->(Purchased)->Order Edge needs an @AlternateIndex to avoid a scan of all Items.

Result<> enhancement

The Hiperspace Result type has been updated to include Rust-Style continuation monads that can optionally follow a .Bind() (bind an element with hiperspace) with:

• .Then chaining function
• .Else error handling function

write.Roles
    .Bind(new Role
    {
        Name = role,
        RoleName = role,
    })
    .Then(r =>
    {
        Log?.LogInformation($"Added missing role {r.Name}");
        return Result.Ok(r);
    })
    .Else(r => Log?.LogError($"Failed to bind role {r.Value.Name} with status {r.Status} and reason {r.Reason}"));

This version updates to .NET 10 and modernizes the code to use name != null and ReferenceEquals(name, null) to name is not null.

The is null and is not null patterns are billed as

provide a clearer and more concise way to check for null values in C#. This change enhances readability and aligns with current C# best practices.

but the main advantage is that:

• You don't need to include ReferenceEquals in any operator == overload.
• name is not null is the same for reference (class) and value (struct) objects instead of ReferenceEquals(name, null) for classes and !name.HasValue for nullable values.

References

Reference libraries have also been updated

This release builds on the HiperEdge functionality provided by message to execute queries on a remote Hiperspace.DB and return the full set of elements back to a client without the need to return every intermediate object necessary. The prime example is the Hiperspace.DB opening page, which displays a summary bar-graphs of the number of nodes and edges in each database, and validates that changes to the (compiled) schema for databases does not change the definition of an already stored element set.

The graph-view functionality of Hiperspace.DB uses HiperEdge functions to recursively search (in parallel) all Node types that satisfy the view criteria and display as a navigable SVG graph of connections. These capabilities use the GraphFunctions to search Nodes using message-keys sent to the server and message-value provided on completion together with rendering information for Node shapes and color.

To support these use-cases, a couple of enhancements have been added to Hiperspace and HiLang to better support the functions.

SubSpace

IServiceProvider is an interface on the .NET Platform to inject Services into components (IoC) at runtime. A ServiceProvider property has been added to SubSpace to enable messages to enlist functionality when a message is received, and a parameter added to SubSpaceParameters for domain space construction.

FindPaths and FindPathsAsync functions now have domain-specific implementations that use a message to transmit the call through to a server where the parallel recursive search can be efficiently executed close the data, using many of the cores of the server.

Node

The Node element has been extended to include additional functions that provide HiperEdge search for related nodes across transitive edges

        /// <summary>
        /// Treat the Edges of TypeName as a HiperEdge and find all transitive paths for that type
        /// </summary>
        /// <param name="TypeName">The Edge type name</param>
        /// <param name="length">the maximum length of the path</param>
        /// <param name="targets">only return HiperEdges that end with a Node of the type matching one of these types</param>
        /// <returns>The full set of HiperEdges for this path</returns>
        public HashSet<HiperEdge> HiperEdges(string TypeName, int? length = null, HashSet<string>? targets = null)

        /// <summary>
        /// Create an inline HiperName from TypeNames and find all transitive paths for that type
        /// </summary>
        /// <param name="HiperName">The name of the HiperEdge being infered from the Edge TyopeNames</param>
        /// <param name="TypeNames">the set of Edge TypeNames that make up this HiperEdge</param>
        /// <param name="length">the maximum length of the path</param>
        /// <param name="targets">only return HiperEdges that end with a Node of the type matching one of these types</param>
        /// <returns>The full set of HiperEdges for this path</returns>
        public HashSet<HiperEdge> HiperEdges(string HiperName, IEnumerable<string> TypeNames, int? length = null, HashSet<string>? targets = null)

        /// <summary>
        /// Treat the Edges of TypeName as a HiperEdge and find all transitive paths for that type
        /// </summary>
        /// <param name="TypeName">the name given to this transitative hiperedge</param>
        /// <param name="rules">the set of meta edges (start-node type, end-node type, edge type) rules that define the transitative path</param>
        /// <param name="length">the maximum length of the path</param>
        /// <param name="targets">only return HiperEdges that end with a Node of the type matching one of these types</param>
        /// <returns>The full set of HiperEdges for this path</returns>
        public HashSet<HiperEdge> HiperEdges(string TypeName, HashSet<Rule> rules, int? length = null, HashSet<string>? targets = null)

        /// <summary>
        /// Treat the Edges of TypeName as a HiperEdge and find all transitive paths for that type
        /// </summary>
        /// <param name="TypeName">The Edge type name</param>
        /// <param name="length">the maximum length of the path</param>
        /// <param name="targets">only return HiperEdges that end with a Node of the type matching one of these types</param>
        /// <returns>The full set of HiperEdges for this path</returns>
        public async Task<HashSet<Graph.HiperEdge>> HiperEdgesAsync(string TypeName, int? length = null, HashSet<string>? targets = null, CancellationToken cancellationToken = default)

        /// <summary>
        /// Create an inline HiperName from TypeNames and find all transitive paths for that type
        /// </summary>
        /// <param name="HiperName">The name of the HiperEdge being infered from the Edge TyopeNames</param>
        /// <param name="TypeNames">the set of Edge TypeNames that make up this HiperEdge</param>
        /// <param name="length">the maximum length of the path</param>
        /// <param name="targets">only return HiperEdges that end with a Node of the type matching one of these types</param>
        /// <returns>The full set of HiperEdges for this path</returns>
        public async Task<HashSet<HiperEdge>> HiperEdgesAsync(string HiperName, IEnumerable<string> TypeNames, int? length = null, HashSet<string>? targets = null, CancellationToken cancellationToken = default)

        /// <summary>
        /// Treat the Edges of TypeName as a HiperEdge and find all transitive paths for that type
        /// </summary>
        /// <param name="TypeName">the name given to this transitative hiperedge</param>
        /// <param name="rules">the set of meta edges (start-node type, end-node type, edge type) rules that define the transitative path</param>
        /// <param name="length">the maximum length of the path</param>
        /// <param name="targets">only return HiperEdges that end with a Node of the type matching one of these types</param>
        /// <returns>The full set of HiperEdges for this path</returns>
        public async Task<HashSet<Graph.HiperEdge>> HiperEdgesAsync(string TypeName, HashSet<Rule> rules, int? length = null, HashSet<string>? targets = null, CancellationToken cancellationToken = default)

The Test cases demonstrate the use of these functions to provide server-side search for relations. The first example treats "Child" as a transitive HiperEdge, while the second infers an "Ancestors" HiperEdge from either Mother or Father.

                var descendants = node.HiperEdges("Child");
                var ancestors = node.HiperEdges("Ancestors", new[] {"Mother", "Father" });

This release extends the message functionality to behave more like a stored procedure call, binding all entity/segment/aspect/view results to the local subspace when it completes.

Initially the RemoteLabel flag was used to indicate which tier should invoke the message, and return results. Test with Hiperspace.DB highlighted that the flag is not needed - the message is now executed by the lowest Hiperspace that can provide a value.