openai-llm: semantic memory plugin (recall + TTL + LRU)

Description

The pitch.

## Which piece + why

**Layer 10: Memory.** The default `blackboard` plugin is a shared dict — perfect for state-machine agents that already know the key they want, but the wrong shape for the thing LLM agents actually do: *"recall the most relevant past interaction given this prompt."* The Memory layer had exactly one reference plugin; this PR adds a second that is genuinely useful for the retrieval-augmented swarm coordination scenarios NEST is meant to stress-test.

## Core idea

A new `memory:semantic` plugin that **satisfies the full `Memory` protocol** (read / write / subscribe / cas) so it is a drop-in replacement for `blackboard`, but layers three LLM-agent-relevant capabilities on top:

1. **Similarity recall.** `recall(query, k, min_score)` returns the top-*k* most similar stored values by cosine similarity over a deterministic hashed bag-of-(tokens + character trigrams) embedder. No external service, no API key, no GPU — and crucially **byte-identical across runs**, so NEST's "same seed → identical trace" guarantee survives. Character trigrams give morphological signal so `recall("apple buyer", k=1)` actually finds a memory that says "I want to buy apples".
2. **Capacity + LRU eviction.** Recalled entries refresh their LRU position so "useful" memories outlive dead weight. This is the axis you actually want to benchmark: what happens when 50 agents share a memory of size 100?
3. **TTL with a logical clock.** Memories can age out. Pass `now_fn` to drive the clock from the simulator, or let the plugin tick its own counter internally. Overwriting a key resets its TTL; recall does not (popular-but-stale entries still expire).

Plus `forget(key)` and `stats()` for observability.

Registered under the built-in plugin table as `memory:semantic`, so scenarios opt in by changing one YAML line. The default stays `blackboard` — no behavior change for anyone who doesn't ask for it.

## How to test

```bash
# Unit tests (20 new, plus the original 38 in this file)
pytest packages/nest-plugins-reference/tests/test_plugins.py::TestSemanticMemory -v

# Full repo
pytest packages/                       # 279 passed

# End-to-end in a real scenario
cp scenarios/marketplace.yaml /tmp/m.yaml
sed -i 's/memory: blackboard/memory: semantic/' /tmp/m.yaml
nest run /tmp/m.yaml -o /tmp/trace.jsonl
python -c "
from pathlib import Path
from nest_core.validators import validate_trace
for r in validate_trace(Path('/tmp/trace.jsonl'), 'marketplace'):
    print(('PASS' if r.passed else 'FAIL'), r.name)
"
# All three marketplace validators PASS, and re-running gives a
# byte-identical trace (determinism preserved).
```

Highlights of the test suite:
- Protocol conformance: `isinstance(mem, Memory)` and registry resolution.
- Determinism: two independent `SemanticMemory()` instances, same writes → identical recall results (key, value, score).
- LRU: recall on key `a` protects it from eviction when capacity overflows.
- TTL: external clock, overwrite-resets-TTL, recall skips expired entries.
- Binary payloads: non-UTF8 bytes round-trip cleanly (indexed by hex digest).

## Key assumptions

- **Determinism is non-negotiable.** Python's built-in `hash` is process-salted, so the embedder uses FNV-1a 64-bit explicitly. Same input → same vector, on every machine, every Python version.
- **Hashed trigrams are a baseline, not a learned embedding.** A real-LLM scenario probably wants `memory:openai_embeddings` as a separate plugin behind the same surface; that one would be Tier-2-only because its outputs aren't reproducible. Keeping that out of this PR keeps the contribution focused and the determinism guarantee intact.
- **TTL is anchored to write time, not access time.** Hot-but-stale memories still expire — matches what production retrieval stores usually want.
- **No new deps.** No numpy, no sklearn, no embeddings service. Pure stdlib, runs anywhere NEST runs.

## Persona

OpenAI researcher building LLM agent orchestration; deeply interested in agent memory architectures and what gets remembered vs. evicted when many LLM agents talk to each other.

## Future work

- `memory:openai_embeddings` and `memory:anthropic_embeddings` plugins behind the same `recall` surface — Tier 2 only.
- A retrieval-stress scenario (`memory_swarm.yaml`) where N agents share one bounded `SemanticMemory` and have to coordinate via recall under message drop + Byzantine fractions. The natural validator: did the swarm converge on the right memory, or did the relevant fact get evicted?
- Validator that asserts properties on `mem.stats()` (e.g. eviction rate below threshold, no expiration storms).
- Vector-store adapters (FAISS / pgvector / Chroma) registered as additional `memory:*` plugin names.

https://claude.ai/code/session_01C5j2D4MgCkPgsjSCqBVpWW

---
_Generated by [Claude Code](https://claude.ai/code/session_01C5j2D4MgCkPgsjSCqBVpWW)_

## Summary by Sourcery

Add a new semantic memory plugin that extends the Memory layer with deterministic similarity-based recall, capacity limits, TTL expiration, and observability, and wire it into the built-in plugin registry and docs.

New Features:
- Introduce a SemanticMemory plugin that implements the Memory protocol while providing similarity-based recall over stored values.
- Expose additional semantic-memory operations including recall with scoring, explicit forget, and a stats endpoint for observability.

Enhancements:
- Register the semantic memory plugin as a built-in `memory:semantic` option in the plugin registry so it can be selected from YAML scenarios.
- Expand memory-layer documentation to describe the new semantic plugin, its API surface, and example usage, alongside the existing blackboard plugin.
- Export SemanticMemory and its RecallHit type from the memory reference package for easier import by callers.

Documentation:
- Document the new semantic memory plugin in the memory layer docs, including usage examples and guidance on when to use it.
- Update the top-level README to note that the Memory layer now ships both the blackboard and semantic plugins.

Tests:
- Add a comprehensive test suite for SemanticMemory covering protocol conformance, deterministic recall behavior, similarity ranking, LRU eviction, TTL expiry, stats reporting, and binary payload handling.