evals/runSaturationEval.mjs), designed to measure how quickly different HDC strategies
lose discriminative power when many facts are superposed into a single composite representation.
We construct synthetic “books” as hierarchical bundles (records → chapters → book) and test whether
the resulting @Book vector supports pure holographic membership queries via the
reasoning equation UNBIND(Book, QueryKey), alongside symbolic query validation.
The suite is explicitly comparative: classic probabilistic VSA strategies are measured against
the lossless EXACT strategy, which provides an upper bound on retrievability.
In HDC/VSA systems, a bundle/superposition combines many items into one vector. As the number of bundled items grows, the composite vector may become less discriminative: unrelated candidates can become “accidentally” similar to a query, increasing ambiguity and false positives. We refer to this effect as saturation.
@Book composite alone support retrieval via UNBIND + cleanup?”
The suite uses DSL files in evals/saturation/books/ to simulate books:
Index-like facts that connect a book identifier, a key, and an idea.
Content is a bundle of record vectors. Optional ordered structure is kept separately.
Book content is a bundle of chapter content vectors (hierarchical superposition).
_Seq Variables (Structure vs. Membership)
Real books have order (ideas in a chapter; chapters in a book). In AGISystem2, the structural operator
__Sequence builds an ordered superposition by binding each element to a position marker
(Pos1, Pos2, …) before bundling.
For saturation testing we need a membership-oriented representation: the @Book vector
should behave like a superposition of queryable records. If we used __Sequence everywhere, the
“book content” would be expressed in a different algebra (everything becomes “positioned”), and the simple
fact-unbinding pattern would no longer reflect the intended query semantics.
*_Seq metadata, while using pure
bundle for @ChapterNN and @Book content vectors.
Each book file contains two markers parsed by the runner:
We treat Mentions(book, key, idea) as a 3-argument record with positional encoding (Pos1..Pos3).
For a query where book and key are known, we compute a query key and unbind:
The recovered ideaVec is then “cleaned up” by ranking against a bounded candidate set.
To approximate the question “is this idea in the book?”, we invert the missing slot:
If the idea is not present, the decoded key should not match any real key confidently.
Pure holographic decode typically needs cleanup against a candidate set. The runner simulates a “reverse-index narrowed” candidate pool of fixed size (default 10):
In parallel, the suite validates correctness via the query engine:
This ensures that failures in holographic decode are interpreted as saturation effects (representation/geometry), not “missing knowledge”.
Each strategy has its own “geometry” meaning (bits for Dense-Binary, k for Sparse-Polynomial, bytes for Metric, etc.). The suite provides fast/full/huge/extra-huge presets to explore scaling without changing code.
The runner prints a summary table with (names may vary by mode):
| Column | Meaning | Why it matters for saturation |
|---|---|---|
HDC |
Holographic decode A pass rate (book,key→idea) | Measures retrieval from the composite alone |
HMem |
Holographic membership pass rate (book,idea→key) | Direct proxy for “idea ∈ book?” |
Query, QMem |
Symbolic validation pass rates | Ground truth: should usually pass unless the DSL is wrong |
AvgPosM, AvgNegM |
Average margin (top1-top2 similarity) for POS/NEG | Lower margins indicate ambiguity and approach to saturation |
SimChk |
Similarity checks performed during cleanup | Cost proxy: cleanup is O(|candidates|) |
UnbChk (EXACT) |
Operation count proxy for polynomial unbind (subset checks) | EXACT cost scales with number of terms in the composite |
Time |
Total per-config time (learn+decode) | Throughput comparison across strategies |
In saturation experiments it is easy to accidentally count a decode as “correct” even when the representation carries no usable signal (e.g., every candidate has similarity 0.000, and the ranking depends on iteration order). To ensure the suite measures real retrieval, the runner uses a strict positive criterion and a conservative negative criterion:
top1Sim > 0, and top1Sim > top2Sim (a non-zero margin).HDC_MATCH(strategy)), to avoid confident hallucinations.Consequence: symbolic queries can be 100% correct while holographic decode reports failures—this is expected and indicates saturation or threshold mismatch, not missing data.
Higher geometry typically increases capacity (more nearly-orthogonal space), delaying saturation. However, strategies differ in how “capacity” manifests:
In XOR-based strategies, UNBIND is often implemented by BIND again (cancellation).
In EXACT, UNBIND is quotient-like: it searches for residual terms consistent with the component.
The saturation suite runs both default modes:
In this suite, most UNBIND calls use single-term components, so A/B are often similar; differences emerge in multi-term unbinding workloads.
Symbolic query engines use structured metadata and indexes over persisted facts. They can answer a query even when the holographic composite saturates, because they do not rely on approximate similarity in a single bundled representation. In contrast, holographic decode is intentionally “index-less” and therefore sensitive to saturation.
The suite includes multiple book families to test different scaling regimes:
book01..book30: increase chapters while keeping ideas/chapter fixed (hierarchical depth/size)book_10cap_100, book_10cap_1000: keep chapters fixed (10) but scale ideas/chapter and per-idea “width” (record density)
When comparing runs across presets (--fast/--full/--huge/--extra-huge), interpret changes as follows:
book01..book30) primarily increases the total number of superposed records, stressing bundling capacity.book_10cap_*) increases collisions within each chapter bundle, which can degrade cleanup margins earlier.