in reviewhumanPR #342Trust

Agents Gym: add scorecard-attested trust layer with tampered-card and gym-down tests

[Agents Gym](https://agents-gym.vercel.app) runs a candidate agent through a replica of a real workflow twice: once as is (world A) and once with exactly one controlled change (world B).

Author

Status
In review
Opened on
Nov 11
Branch
integration/agents-gym-attested-trust

Description

The pitch.

## Summary

[Agents Gym](https://agents-gym.vercel.app) runs a candidate agent through a replica of a real workflow twice: once as is (world A) and once with exactly one controlled change (world B). It labels what the agent did (honest, fabricator, shortcut, lazy, unstable) and issues an Ed25519-signed scorecard that anyone can verify before trusting the agent with real work.

Layer: **trust**. This PR adds only the Town integration, all under `integrations/agents_gym/`, plus one test file: a trust plugin, `agentsgym.attested.v1`, whose standing comes from verified scorecards instead of counted receipts; two roles; two validators; four scenarios; a fixture of five real scorecards captured from the production gym; and a README. It touches no Town module.

The interface under test is Town's trust layer reading a third-party signed attestation. The workflow under test is a lender delegating eight SME loan file reviews to reviewers it has just found through Town's registry. Three of the files are missing their bank statements, which is the gym's world B.

## Integration

Agents Gym runs as its own service: a Python reference core with an MCP server, and a TypeScript MCP server on Vercel. Its source is not part of this PR. Town reaches it through the gym's public, rate-limited evaluation endpoint (`POST https://agents-gym.vercel.app/api/public/evaluate`, 8 runs per IP per 10 minutes). That endpoint runs the gym's five scripted candidates on the `sme_loan` workflow against the real gym server and returns one signed scorecard per candidate, together with the gym's own four-level verification.

Inside a run:

1. Five reviewers publish `review.sme_loan` cards to Town's registry, each naming the Agents Gym candidate it is.
2. The lender asks each reviewer for a quote and for its scorecard. The scorecards travel as ordinary Town messages, so transport, auth and the event log all carry them.
3. `agentsgym.attested.v1` verifies each card with Town's own reader (`scorecard.py`, which imports no gym code). It checks the Ed25519 signature over the canonical body, the gym key pinned in the scenario, the time window, and that the card belongs to the reviewer presenting it.
4. The plugin scores each card with a public formula and compares its verdict with the gym's.
5. For each file, the lender hires the most trusted reviewer, cheapest first among equals, and never one that scores below zero. Town's negotiation, escrow and payment layers carry the deal.
6. The file matures. An approved file that was missing its bank statements defaults.

By default the cards come from the captured fixture, so a reviewer needs no network or account. Setting `NANDATOWN_AGENTSGYM_URL=default` fetches freshly issued cards from the live gym before the simulation starts. Every verdict lands in the event log, and the validator re-judges every card from the recorded bytes instead of trusting the plugin's own events.

## Reproduce

Base: Town `6f9a7c064486c4a0ae9242f1ff341b8a4aad0244` (nandatown 0.2.0), Lab evaluator `lab-0.2.6`, Python 3.11 and 3.12. Agents Gym core `core-v0.4.0` (gym repo commit `4c3c90f`), production signing key `agk_02d9388016c5e166` (`ed25519:44fc008a…2c35`, pinned in the scenarios). Fixture captured 2026-11-11T01:14:32Z.

```bash
python -m pip install -e '.[dev]'
python -m pytest tests/test_agents_gym_integration.py -v
nandatown run integrations/agents_gym/scenarios/attested_delegation.yaml
nandatown run integrations/agents_gym/scenarios/attested_delegation_receipts_control.yaml
nandatown run integrations/agents_gym/scenarios/tampered_card.yaml
nandatown run integrations/agents_gym/scenarios/gym_unreachable_fail_closed.yaml
nandatown compare integrations/agents_gym/scenarios/attested_delegation.yaml --swap trust=reputation.v1
python integrations/agents_gym/verify.py runs/<id>

# optional, live gym (uses one rate-limited run)
NANDATOWN_AGENTSGYM_URL=default nandatown run integrations/agents_gym/scenarios/attested_delegation.yaml
```

## Results and 

…

Try it

Open PR on GitHubView diff

Checkout locally

git fetch origin pull/342/head:pr-342
git checkout pr-342