in reviewhumanPR #124Memory
maharmuavia (distributed-systems-engineer): or_map -- add-wins observed-remove map CRDT (memory)
Persona: distributed-systems engineer. The thing I care about is *causal*
Author
@MaharMuavia
github profile →- Status
- In review
- Opened on
- Jul 9
- Branch
- hackathon/maharmuavia-observed-remove-map
Description
The pitch.
# memory: `or_map` — an add-wins observed-remove map CRDT (Problem 02)
**Persona:** distributed-systems engineer. The thing I care about is *causal*
consistency under concurrent delete — the failure mode that a plain register
hides.
## Motivation
`memory/lww_register` (merged) is a correct **LWW-Register**: it settles every
conflict by the total order `(lamport, node)`. That is exactly right for a
single-valued register — and exactly wrong for a shared **set/map of live
entries**, because a *delete* has to be modelled as a tombstone write, and a
tombstone that happens to carry the higher `(lamport, node)` **silently
overwrites a concurrent add**. An agent re-advertising a catalogue entry at the
same moment another agent retires the copy it had seen loses its entry. The
default `blackboard` is worse: delete is order-dependent `del` on a per-replica
dict, so replicas simply diverge.
There is no add-wins / observed-remove memory plugin in the tree, and
`docs/layers/memory.md` calls CRDTs out as wanted. This fills that gap with the
one capability `lww_register` structurally cannot express: **deletion under
concurrency**.
## Design
`OrMapMemory` is a **state-based ORSWOT** ("OR-Set Without Tombstones",
generalised from a set to a keyed map):
- every `write` mints a globally-unique **dot** `(node, counter)`;
- a per-replica **causal context** (a version vector: `node -> max counter
seen`) records everything the replica has observed;
- `remove(key)` drops only the dots this replica has *observed* — a concurrent
add carries a fresh dot **outside** the remover's context and therefore
survives (**add-wins**);
- the full-state join keeps a dot iff the other side still has it *or* has never
seen it, so the merge is **commutative, associative, and idempotent** — strong
eventual consistency with **no tombstones**.
Reads stay single-valued and deterministic: when concurrent writes leave several
live dots, `read` returns the payload of the largest dot under `(counter,
node)`, while all concurrent dots are retained internally so convergence and
trace inspection are exact.
The base `Memory` surface (`read`/`write`/`cas`/`subscribe`) is unchanged; the
delete verb and replication channel (`remove` / `export` / `merge` /
`export_all` / `merge_all`) are **additive**, so a caller that only speaks the
base protocol never has to know the values are CRDT entries.
## Tradeoffs
- **State-based (CvRDT), not op-based.** Simpler to reason about and matches the
problem brief ("pick state-based for this problem"); the cost is that a
per-key `export` carries the causal context. Removal propagation uses the
full-state `export_all`/`merge_all` channel, which is the correct anti-entropy
path for observed-remove.
- **No context compaction.** The version vector is not garbage-collected across
gossip rounds — fine at simulation scale, called out as future work.
- **Deterministic single-valued read** over a multi-value internal state: a
deliberate choice so traces replay byte-identically; the concurrent values
remain inspectable.
## Adversarial validators (the discriminating bar)
Two **capability-aware** validators in
`nest_plugins_reference/validators/or_map_validators.py`. They use the CRDT
channel + `remove` when present and otherwise model "observe" as a `write` and
"delete" as a tombstone `write`, so one call site discriminates all three
plugins:
| validator | `blackboard` | `lww_register` | `or_map` |
|---|---|---|---|
| `check_add_wins_survives_concurrent_remove` | **FAIL** (diverges) | **FAIL** (add lost to tombstone) | **PASS** |
| `check_convergence_under_churn` | **FAIL** (diverges) | PASS (converges, wrong value) | **PASS** |
`lww_register` passing churn-convergence but failing add-wins is the whole
point: it *is* a CRDT, so it converges — but it converges to the *deleted*
state, dropping the concurrent add. `or_map` keeps it.
## Scenario
`scenarios/memory_or_map_add_remove.yaml` — 9 replicas each write the same ke
…Try it
Open PR on GitHubView diffCheckout locally
git fetch origin pull/124/head:pr-124
git checkout pr-124