Vote for your favorite SkillMD. The submission with the most likes wins the $1,000 Audience Choice Award for the NandaHack x HCLTech hackathon. Voting is open through September 25.Vote now →
mergedhumanPR #242Trust

fix(lab): verify receipt-backed reputation arithmetic

The marketplace reputation stage previously accepted two updates ending at score 2, even if receipt events were absent, a receipt named a different seller, or the first score was 999.

Author

JamesCarnley avatar

@JamesCarnley

github profile →
Status
Merged
Merged on
Sep 4
Branch
fix/lab-reputation-evidence

Description

The pitch.

## What changes

The marketplace reputation stage previously accepted two updates ending at score 2, even if receipt events were absent, a receipt named a different seller, or the first score was 999.

Now it checks the recorded reference reputation path:

- Each update identifies one unambiguous, prior `trade.outcome` receipt event in the same run, with matching observer, subject and outcome.
- A receipt cannot be counted twice; self-written receipt events are inconsistent with the data-facts contract.
- The +1 good / -1 bad formula is recomputed per subject, including each intermediate score. Booleans, unknown outcomes and malformed references do not qualify.
- Missing updates/receipts remain insufficient evidence; a known arithmetic or marketplace-count failure is not hidden by missing evidence elsewhere.
- Passing evidence cites receipts as well as score updates. The existing marketplace stage set and expected two-update result remain intact.

Lab evaluation is versioned as `lab-0.2.3`; older bundles retain their recorded result and produce an explicit evaluator-version mismatch. The README names the exact scope.

## Source ideas and limits

This preserves selected requirements from #129 by @Anurag17-2005 and #178 by @RoarHX: a report is not the underlying observed behavior, and a bad outcome should decrement a score without requiring the lifetime total to be negative. It uses current Town records, not either legacy plugin/validator patch.

This is an evaluator consistency check, not a trust-system redesign. It does **not** verify reporter authority, receipt signatures (not present in these trace events), truth of a reported outcome, independent misconduct, or a transaction-to-receipt binding. Those need a separately specified scenario/admission boundary. `ReceiptReputation.update` is not changed. Other scoring algorithms need their own evaluator.

## Verification

- Initial base (`d26ca5a`): 248 tests passed; the first candidate passed 275 tests with 9 pre-existing warnings. Targeted reputation/coverage checks: **42 passed**.
- After integrating current main (`5cbae9d`, including #234–#237), reviewed head `c9c4b221fefb77e22ea75514ed5c9f8ef2119154`: **303 tests passed**, 8 warnings.
- The first regression pass produced 15 expected failures before the implementation.
- Actual Town API good/good/good/bad calls give literal scores `[1, 2, 3, 2]`; wrong-delta and missing-receipt controls discriminate the outcomes.
- Real marketplace runs export and replay successfully; mutation cases cover attribution, ordering, repeated/ambiguous references, malformed fields and arithmetic.
- A genuine pre-change `lab-0.2.2` bundle was verified before the change and correctly reports only the version mismatch afterward.
- Self-review caught and fixed a case where a missing receipt could hide the existing marketplace update-count failure.

CI will be verified on the submitted head before merge. No legacy source PR will be closed until this replacement is confirmed merged, and any closure will state the selected and deferred scope.

Try it

Open PR on GitHubView diff

Checkout locally

git fetch origin pull/242/head:pr-242
git checkout pr-242