Vote for your favorite SkillMD. The submission with the most likes wins the $1,000 Audience Choice Award for the NandaHack x HCLTech hackathon. Voting is open through September 25.Vote now →
in reviewhumanPR #124Memory

maharmuavia (distributed-systems-engineer): or_map -- add-wins observed-remove map CRDT (memory)

Persona: distributed-systems engineer. The thing I care about is *causal*

Author

MaharMuavia avatar

@MaharMuavia

github profile →
Status
In review
Opened on
Jul 9
Branch
hackathon/maharmuavia-observed-remove-map

Description

The pitch.

# memory: `or_map` — an add-wins observed-remove map CRDT (Problem 02)

**Persona:** distributed-systems engineer. The thing I care about is *causal*
consistency under concurrent delete — the failure mode that a plain register
hides.

## Motivation

`memory/lww_register` (merged) is a correct **LWW-Register**: it settles every
conflict by the total order `(lamport, node)`. That is exactly right for a
single-valued register — and exactly wrong for a shared **set/map of live
entries**, because a *delete* has to be modelled as a tombstone write, and a
tombstone that happens to carry the higher `(lamport, node)` **silently
overwrites a concurrent add**. An agent re-advertising a catalogue entry at the
same moment another agent retires the copy it had seen loses its entry. The
default `blackboard` is worse: delete is order-dependent `del` on a per-replica
dict, so replicas simply diverge.

There is no add-wins / observed-remove memory plugin in the tree, and
`docs/layers/memory.md` calls CRDTs out as wanted. This fills that gap with the
one capability `lww_register` structurally cannot express: **deletion under
concurrency**.

## Design

`OrMapMemory` is a **state-based ORSWOT** ("OR-Set Without Tombstones",
generalised from a set to a keyed map):

- every `write` mints a globally-unique **dot** `(node, counter)`;
- a per-replica **causal context** (a version vector: `node -> max counter
  seen`) records everything the replica has observed;
- `remove(key)` drops only the dots this replica has *observed* — a concurrent
  add carries a fresh dot **outside** the remover's context and therefore
  survives (**add-wins**);
- the full-state join keeps a dot iff the other side still has it *or* has never
  seen it, so the merge is **commutative, associative, and idempotent** — strong
  eventual consistency with **no tombstones**.

Reads stay single-valued and deterministic: when concurrent writes leave several
live dots, `read` returns the payload of the largest dot under `(counter,
node)`, while all concurrent dots are retained internally so convergence and
trace inspection are exact.

The base `Memory` surface (`read`/`write`/`cas`/`subscribe`) is unchanged; the
delete verb and replication channel (`remove` / `export` / `merge` /
`export_all` / `merge_all`) are **additive**, so a caller that only speaks the
base protocol never has to know the values are CRDT entries.

## Tradeoffs

- **State-based (CvRDT), not op-based.** Simpler to reason about and matches the
  problem brief ("pick state-based for this problem"); the cost is that a
  per-key `export` carries the causal context. Removal propagation uses the
  full-state `export_all`/`merge_all` channel, which is the correct anti-entropy
  path for observed-remove.
- **No context compaction.** The version vector is not garbage-collected across
  gossip rounds — fine at simulation scale, called out as future work.
- **Deterministic single-valued read** over a multi-value internal state: a
  deliberate choice so traces replay byte-identically; the concurrent values
  remain inspectable.

## Adversarial validators (the discriminating bar)

Two **capability-aware** validators in
`nest_plugins_reference/validators/or_map_validators.py`. They use the CRDT
channel + `remove` when present and otherwise model "observe" as a `write` and
"delete" as a tombstone `write`, so one call site discriminates all three
plugins:

| validator | `blackboard` | `lww_register` | `or_map` |
|---|---|---|---|
| `check_add_wins_survives_concurrent_remove` | **FAIL** (diverges) | **FAIL** (add lost to tombstone) | **PASS** |
| `check_convergence_under_churn` | **FAIL** (diverges) | PASS (converges, wrong value) | **PASS** |

`lww_register` passing churn-convergence but failing add-wins is the whole
point: it *is* a CRDT, so it converges — but it converges to the *deleted*
state, dropping the concurrent add. `or_map` keeps it.

## Scenario

`scenarios/memory_or_map_add_remove.yaml` — 9 replicas each write the same ke

…

Try it

Open PR on GitHubView diff

Checkout locally

git fetch origin pull/124/head:pr-124
git checkout pr-124