in reviewhumanPR #327Trust

AgentSchool: add a credential-verifying trust layer and a revoked-credential test

[AgentSchool](https://github.com/justinspar/agentschool) trains AI agents in one skill, examines them on work they have not seen, and issues a signed credential that other agents check before handing over a job.

Author

Status
In review
Opened on
Nov 10
Branch
integration/agentschool-credential-gated-delegation

Description

The pitch.

## Summary

[AgentSchool](https://github.com/justinspar/agentschool) trains AI agents in one skill, examines them on work they have not seen, and issues a signed credential that other agents check before handing over a job. It runs as its own service, so this pull request adds only the Town integration: one trust plugin, three scenarios, a fixture captured from a real training run, 49 tests and a README, all under `integrations/agentschool/`.

The interface under test is Town's trust layer against AgentSchool's HTTP tools for a hiring agent. The interaction under test is one Lab buyer deciding whether to delegate paid work to three sellers it has just met through Town's registry, where one holds an earned credential, one holds a revoked credential and one holds none.

## Integration

AgentSchool stays on `localhost:8765`, with each candidate agent serving its own card and `/challenge` endpoint. Town reads the credential format from the wire and calls `search_certified_agents`, `verify_credential`, `check_identity_proof` and `report_task_outcome`, so it imports no AgentSchool code and adds no dependency. The credential and the signature travel as ordinary Town messages, so Town's registry, auth, negotiation and payment layers all take part, and the run lands in one evidence bundle.

Town verifies every credential itself, including the holder's signature over a nonce **Town generated** and will not accept twice, then compares its verdict with AgentSchool's. Agreement is a stage in the report, and disagreement fails that stage.

The credentials in the fixture were earned. Three agents ran through AgentSchool as headless Claude Code processes and sat its held-out exam, which certifies only at 95 out of 100:

| Agent | Model | Pre-test | Exam | Credential |
|---|---|---|---|---|
| atlas | Sonnet 5.5 | 100.0 | 100.0 | Level 1 |
| brisk | Sonnet 5.5 | 94.5 | 100.0 | Level 1 |
| flash | Haiku 4.5 | 44.1 | 87.0 | none |

Flash failed both attempts, so the uncertified seller is an agent that could not pass rather than a placeholder. We then revoked brisk's credential with `agentschool revoke`, which is where the revoked case comes from.

## Reproduce

Town base commit `6f9a7c064486c4a0ae9242f1ff341b8a4aad0244`, nandatown 0.2.0, Python 3.11.14 and 3.12.12, Lab evaluator `lab-0.2.6`, Path profile `a2a-capability-fulfillment@0.3`. AgentSchool commit `393c4b7f4991da6dc10fb07efe020126454886eb`, issuer key `as-D_vV6jcrJs`, skill `data-driven-slides` 2.0.0, exam `harbor-pine-slide-design-v2`.

```bash
python -m pip install -e '.[dev]'
python -m pytest tests/test_agentschool_integration.py -v
nandatown run integrations/agentschool/scenarios/credential_gating.yaml
nandatown run integrations/agentschool/scenarios/credential_gating_ungated_control.yaml
nandatown run integrations/agentschool/scenarios/revoked_credential_fail_closed.yaml
nandatown verify runs/<id> --plugin integrations/agentschool/plugin.py
```

Those commands need no AgentSchool and no model account. `integrations/agentschool/README.md` holds the live-mode variables, the full stage tables and how to rebuild the fixture.

## Results and limitations

| Run | Expected | Observed |
|---|---|---|
| The main workflow, live | the seller that verifies is hired and paid, the other two refused | all 12 stages passed |
| The same workflow, offline | the same verdicts, and no claim a captured proof cannot support | passed, `holder_proof_fresh` and `outcome_reported` reported as not tested |
| The control, `gate: false` | the uncertified seller undercuts on price and wins the work | failed at `unverified_got_no_work`, naming `seller-uncertified` |
| Only the revoked credential on offer | nothing is delegated and the refusal is recorded | passed, no purchase order and no payment |
| `nandatown test-agent --url` | not known beforehand | failed at `protocol_invocation` with `a2a_http_status_404` |

AgentSchool's credential format and its four hiring tools work unchanged inside a Town run: Town rea

…

Try it

Open PR on GitHubView diff

Checkout locally

git fetch origin pull/327/head:pr-327
git checkout pr-327