Hiperspace

Tools for Hiperspace development

Posted by steve on July 28, 2023

When Ted Codd authored “A relational model of data for large shared data banks”, core memory was like modern processor cache memory while main memory was stored on disk (or tape) rather the large memory we have in modern computers.

Recent advances in memory technology driven by the need to address terabytes of memory for AI models are stretching the ability of current computer technology to address available memory. Modern AI and HPC are adapting the Relational view of key-addressable information, not to overcome the latency of spinning disks, but to address the need to efficiently scan large databanks of memory in parallel. RocksDB is one technology being adapted to this purpose.

Hiperspace takes the view that the best mix of technology is to apply a schema over the expanded (CXL and SSD) memory to provide transparent navigation from entity to entity with view projections and aspects (later additions) or segments (collections and history). Aspect/Segment allows for the experience of changeability without the need to synchronize information in complicated databases servers, because each fact is closed to change (immutable), but open to extension.

Hiperspace is a work-in-progress, but the first alpha tools are now available from Nuget package server for those that want to help.

Hiperspace and Hiperspace.Rocks provide basis for storing entities / segments / aspects as key-value pairs in expanded memory, while HiLang provides a Source Generator to translate a minimal data language to executables with a rich incremental development environment.

HiLang is completely implemented in F# using the FSLexYacc parser framework of the F# language, and (currently) a collection of 42 parse-tree visitors to {enrich, validate, transform, generate} the AST model to produce code with feedback interactively during coding.