in reviewhumanPR #294Coordination
[Fellowship Challenge] Partition fault: consensus stays safe while the town is split
Submission for the Hexaware x Project NANDA Fellowship Technical Challenge
Author
@Paramjeet-singh-neu
github profile →- Status
- In review
- Opened on
- Sep 30
- Branch
- challenge/Paramjeet-singh-neu-consensus-partition
Description
The pitch.
Submission for the Hexaware x Project NANDA Fellowship Technical Challenge **Gap** The Lab transport can drop, duplicate or delay a single message, but it can't split the town so two groups of agents can't reach each other. Imported scenarios that declare `network_partition` just get reported as not modeled. **Layer** Transport (Lab), exercised by a new scenario, `consensus_partition`, and its control, `consensus_partition_quorum_defect`. **Fault** `action: partition` with `groups`, `start` and `heal`. While the cut lasts, anything sent between groups is dropped by the town, recorded as `message_dropped` with fault `partition` and the sender. After the heal, new messages go through and the proposer's normal retries take over. A message is judged when it's sent. The declaration fails closed: every agent in exactly one group, start before heal, heal before `max_time`, and for now a partition has to be the only fault in its scenario. **Invariant** The proposer can't commit without a majority of the configured acceptors, it does commit after the heal, and every acceptor ends up with the same value. **Negative control** The same partition, with a proposer that shrinks its quorum to whoever answered (`defect: shrink_quorum_on_timeout`, off unless named; unknown names raise). On a healthy network it never shows up. **Predictions vs results** I wrote the predictions into this PR before running either scenario. | Run | Stage | Predicted | Result | |---|---|---|---| | `consensus_partition` | `partition_enforced` | pass | pass: 14 drops, none delivered | | | `quorum_commit` | pass | pass | | | `agreement` | pass | pass | | | `progress_after_heal` | pass, commit around t=10.5–11.5 | pass, commit at t=11.2 | | quorum defect | `partition_enforced` | pass | pass: 4 drops | | | `quorum_commit` | fail, 1 of 3 acceptors | fail | | | `agreement` | fail | fail | | | `progress_after_heal` | fail | fail | **Why two new stages** One test makes the partition leak on purpose. `quorum_commit` and `agreement` still pass on that run, while `partition_enforced` and `progress_after_heal` fail. The existing checks can't tell a working partition from a leaky one. **Evidence** - Bundle fingerprints (seed 42): - `consensus_partition`: `sha256:3d5634df7cdda4482d143aee7cceaf21ba2dac48b4371eaf7e8cf7f8c96acd12` - `consensus_partition_quorum_defect`: `sha256:a08ee505ce7bb97d9ee423ed1d3c7deb212bf4d3c13caf9bae23683234d18efc` - `nandatown verify` passes on both. - Campaigns, 20 trials each: 20/20 passed, and 20/20 failed with the same stage pattern every time. The scenario makes no random choices, so this shows the result doesn't depend on the seed. It isn't a reliability claim. - The 7 existing scenarios give identical events and verdicts at seed 42 compared with `main@36e7f6c` (run IDs and evaluation time aside), so `LAB_EVALUATOR_VERSION` stays at lab-0.2.6. - Full suite: 1801 passed. The new tests also pass on Python 3.11, and the wheel ships both scenarios and runs them outside the checkout. **What this doesn't prove** - There's a single proposer, so two conflicting values can't happen. This tests the majority rule, not full split-brain protection. - Lookup is an in-process call, so a cut-off agent can still discover agents it can't reach. - It's Lab only, on one machine, not a real network. - A partition can't be combined with other faults yet, because the order relative to per-message counters isn't defined. **What's next** - Partitions between real processes in the Track. While looking at that, I found the coordinator is the only path for recording evidence: kill it mid-run and no bundle is written at all. - Map imported `network_partition` declarations onto this fault instead of reporting them as not modeled. - Re-test partition-tolerant protocols like legacy #33, now that the town can cut the network. **Commits** 1. Declare timed partition faults, failing closed, with round-trip tests 2. …
Try it
Open PR on GitHubView diffCheckout locally
git fetch origin pull/294/head:pr-294
git checkout pr-294