mergedhumanPR #239Privacy
fix(lab): require scenario coverage before a passing verdict
A registered Lab validator can return no scenario checks and still receive a passing verdict from generic checks. This change makes missing scenario coverage explicit before those generic checks are appended.
Description
The pitch.
## Summary A registered Lab validator can return no scenario checks and still receive a passing verdict from generic checks. This change makes missing scenario coverage explicit before those generic checks are appended. - Zero selected-validator results become a Town evaluator error; entirely untested scenario results remain incomplete. - Empty ledger and declared-privacy evidence no longer pass. Completed no-money runs remain valid. - Literal native/adapted stage-set regressions detect coverage drift, and the redaction test now examines actual nested values instead of an always-true assertion. - Lab evaluator version becomes `lab-0.2.1`. New bundles replay; older Lab evaluator versions explicitly report that reproducibility was not checked rather than silently using changed rules. ## Attribution and scope Preserves useful coverage requirements from #219 by `dhyantsoni` (reviewed head `8135ab72ef7d27b310fada582346d381481ee8a2`) and empty-evidence requirements from #164 by `rajpatilrobotics` (reviewed head `0071ae80d0e3be50eb53c7ef20b2222009ece57b`), reimplemented for current `main`. No legacy code was copied. Both original PRs remain open pending the broader rescue/disposition review. This is separate from #233's receipt eligibility rule. It does not add a general required-stage manifest or claim to detect every truncated trace. ## Verification - Regression RED before production changes: 4 failed, 27 passed. - Focused GREEN: 31 passed. - Independent full-suite run on Python 3.12: 196 passed, nine pre-existing warnings. - Independent spec/quality review approved with no findings. - Combined with the pending correctness fixes #233–#237, response limits and rerun correction: 267 tests passed. The old-version fixture was adjusted to preserve the bundle root fingerprint; verification rules were not weakened. Run from the checkout with development dependencies installed: ```sh python -m pytest -q tests/test_lab_coverage.py tests/test_sim.py python -m pytest -q ```
Try it
Open PR on GitHubView diffCheckout locally
git fetch origin pull/239/head:pr-239
git checkout pr-239