in reviewhumanPR #168Memory
hackathon: embx_semantic memory plugin (Layer 10) -- recall by meaning
A new embx_semantic plugin for Layer 10 (Memory). The default blackboard is an 80-line shared dict -- read a value under a key, get it back under the same key, nothing else.
Author
@jonhardwick-spec
github profile →- Status
- In review
- Opened on
- Jul 10
- Branch
- hackathon/jonhardwick-spec-embx-semantic-memory
Description
The pitch.
## What this is
A new `embx_semantic` plugin for Layer 10 (Memory). The default `blackboard` is an 80-line shared dict -- read a value under a key, get it back under the same key, nothing else. If agent-A files a fact under `airport-departure-gate` and agent-B asks for `departure-gate-airport`, blackboard returns `None`. For a swarm that's the whole problem: agents name the same thing differently and the shared state goes unused.
This plugin implements the same `Memory` Protocol surface (`read` / `write` / `subscribe` / `cas`) so it slots into any scenario that currently names `blackboard` or `lww_register`, but every `write` also embeds the key and a new `semantic_lookup` ranks stored entries by **cosine similarity** to the query. A read for a *related* key finds the stored value even when the literal keys disagree.
## Why embx fits the Memory layer
The embedding backend is **embx** (https://embx.net) -- a real semantic vector memory service. `write` posts the key to embx's `/v1/embeddings` endpoint (`frankenstein-22.8mb-int8`, 384-dim) and stores the returned vector alongside the value; `semantic_lookup` embeds the query and ranks stored entries by cosine similarity. In production this is genuine semantic recall over a learned embedding space -- the thing blackboard categorically cannot do, because it has no notion of similarity.
embx is the right fit because the Memory layer's job is exactly "shared state agents can find later," and "find" in a multi-agent system means "find even when the asking agent used different words." Semantic recall is the feature the reference plugin is missing.
## Determinism -- why there's a fallback
Nanda Town's contract is byte-reproducible traces under a fixed seed. A live LLM embedding model is neither deterministic across hardware nor offline, so it can't be the path CI runs. The plugin has a deterministic fallback:
- When `EMBX_BASE_URL` is **unset** (the default) the plugin uses a hash-based bag-of-tokens pseudo-embedding. It's order-independent (keys with the same token multiset score cosine 1.0, partial overlap scores fractional, disjoint tokens score zero), pure-function-of-the-input, and offline. That's enough to prove the plugin ranks by similarity instead of exact-matching -- which is what the test asserts.
- When `EMBX_BASE_URL` is **set** (production), it posts to the live embx service. If the call fails for any reason (network, timeout, non-200, bad JSON), it trips a lazy flag and falls back without raising -- the agent loop keeps running.
The real semantic power shows in production; the fallback keeps CI reproducible. Same code path, different config.
## How to test it
```bash
# CI path (deterministic, offline -- the default):
make ci-local # ruff, pyright strict, pytest
# Exercise the live embx path:
EMBX_BASE_URL=https://embx.net EMBX_API_KEY=embx_live_... uv run pytest \
packages/nest-plugins-reference/tests/test_embx_semantic.py -v
```
Point a scenario at it by flipping one line in the YAML:
```yaml
layers:
memory: embx_semantic # was blackboard or lww_register
```
## The test that proves it
32 new tests. The adversarial pair is the one that matters:
- `test_blackboard_cannot_recall_by_related_key` -- blackboard returns `None` for a different-but-related key and has no `semantic_lookup` method at all.
- `test_embx_recalls_by_related_key` -- this plugin finds the stored value under a reordered key, via cosine similarity.
The first fails without the plugin (blackboard can't do it); the second passes with it. That's the "fails without your change, passes with it" bar.
## Files
- `packages/nest-plugins-reference/nest_plugins_reference/memory/embx_semantic.py` -- the plugin
- `packages/nest-plugins-reference/tests/test_embx_semantic.py` -- 32 tests, all forcing the deterministic fallback
- `packages/nest-plugins-reference/pyproject.toml` -- entry-point registration (same pattern as `lww_register`)
- `packages/nest-core/nest_core/plugins.py` -- builtin
…Try it
Open PR on GitHubView diffCheckout locally
git fetch origin pull/168/head:pr-168
git checkout pr-168