in reviewhumanPR #151Coordination
distributed-sys-engg: deterministic coordination round ids (ADR-004 fix)
Persona: distributed-sys-engg — reliability/determinism focus.
Author
@rakshit-tresata
github profile →- Status
- In review
- Opened on
- Jul 10
- Branch
- hackathon/distributed-sys-engg-deterministic-coordination-ids
Description
The pitch.
# Deterministic coordination round ids (ADR-004 fix for `contract_net` + `hotstuff`)
**Persona:** `distributed-sys-engg` — reliability/determinism focus.
Both reference coordination plugins, including the default `contract_net`,
mint their round id with `uuid.uuid4()`. That draws from `os.urandom`, so it
is unseedable. This breaks ADR-004 (seeded-determinism), Nanda Town's rule
that a seeded run must replay to a byte-identical trace.
## Motivation
`ADR-004-seeded-determinism` requires that the same seed replay to an
identical trace, so operators can diff two runs or replay a failure. Every
scenario and replay tool in this repo depends on that guarantee holding.
Two reference coordination plugins break it:
- `coordination/contract_net.py` — the default coordination plugin.
- `coordination/hotstuff.py`.
Every call to `propose()` returns a `Round` (and downstream `Bid` / `Outcome`,
which copy `round.id`) with a fresh random id from `uuid4()`. Two runs of the
same seed produce different coordination ids. The plugin's public output is
not reproducible.
Stated plainly: this is a latent bug today. None of the shipped scenarios
(marketplace, auction, voting, consensus) serialize the round id into their
trace, which is why it's gone unnoticed so far. But `propose()` is still a
public API with nondeterministic output, and any scenario or validator that
starts using round ids directly (e.g. the fair-ordering or sealed-bid
coordination work already in flight) will silently inherit a trace that can't
be diffed or replayed.
## The change
Add `coordination/_ids.py::derive_round_id(agent_id, task_id, seq)` and use it
in both plugins. The id is:
- Deterministic: a pure function of `(proposing agent, task, monotonic
per-proposer sequence)`. A seeded run replays byte-for-byte.
- Injective on its inputs: the pre-image is a length-prefixed (netstring-style)
encoding, so distinct triples can't alias onto one digest through boundary
ambiguity (`("ab","c")` vs `("a","bc")`).
- Free of ambient state: no `uuid4`, no wall clock, no object identity, no
dict-ordering dependence.
Each plugin keeps a private `_round_seq` counter. The sim drives `propose()`
in a seeded, deterministic order, so the counter, and thus the id, reproduces.
Diff: `+177 / -5` across `_ids.py`, `contract_net.py`, `hotstuff.py`, and a new
test file. No code outside the coordination layer is touched.
## Tests — fail without the change, pass with it
`packages/nest-plugins-reference/tests/test_coordination_determinism.py`:
- Replay identity: proposing the same task sequence twice yields
byte-identical ids. This is the check that fails against `uuid4`.
- Uniqueness: successive rounds and distinct proposers still get distinct ids
(no functional regression).
- Derivation properties: purity, no ambient state, injective encoding
(`("ab","c") != ("a","bc")`), and a Hypothesis property over arbitrary
inputs.
Reverting just the two-line `propose()` change (restoring `uuid4`) makes
`TestReplayDeterminism` fail with, e.g., `round-8c683f74d9f1207a` (expected)
vs a random `f36c1ab2-6955-462d-92b9-9aece726f3b6`.
## Verification
```bash
make ci-local # uv sync, ruff, ruff format --check, pyright, pytest -v
# or just this PR's test:
uv run pytest -q packages/nest-plugins-reference/tests/test_coordination_determinism.py
```
Local run: `ruff` clean, `ruff format` clean, full suite **954 passed, 0
failed** (this branch), and the new test fails on the pre-change code as above.
## Out of scope / follow-up
`negotiation/alternating_offers.py` has the identical one-line `uuid4` offer-id
bug. It lives in a different layer, so I've left it out of this coordination-
scoped PR; happy to send a sibling PR for the negotiation layer.
Try it
Open PR on GitHubView diffCheckout locally
git fetch origin pull/151/head:pr-151
git checkout pr-151