in reviewhumanPR #327Trust
AgentSchool: add a credential-verifying trust layer and a revoked-credential test
[AgentSchool](https://github.com/justinspar/agentschool) trains AI agents in one skill, examines them on work they have not seen, and issues a signed credential that other agents check before handing over a job.
Author
@EmmaDinh
github profile →- Status
- In review
- Opened on
- Nov 10
- Branch
- integration/agentschool-credential-gated-delegation
Description
The pitch.
## Summary [AgentSchool](https://github.com/justinspar/agentschool) trains AI agents in one skill, examines them on work they have not seen, and issues a signed credential that other agents check before handing over a job. It runs as its own service, so this pull request adds only the Town integration: one trust plugin, three scenarios, a fixture captured from a real training run, 49 tests and a README, all under `integrations/agentschool/`. The interface under test is Town's trust layer against AgentSchool's HTTP tools for a hiring agent. The interaction under test is one Lab buyer deciding whether to delegate paid work to three sellers it has just met through Town's registry, where one holds an earned credential, one holds a revoked credential and one holds none. ## Integration AgentSchool stays on `localhost:8765`, with each candidate agent serving its own card and `/challenge` endpoint. Town reads the credential format from the wire and calls `search_certified_agents`, `verify_credential`, `check_identity_proof` and `report_task_outcome`, so it imports no AgentSchool code and adds no dependency. The credential and the signature travel as ordinary Town messages, so Town's registry, auth, negotiation and payment layers all take part, and the run lands in one evidence bundle. Town verifies every credential itself, including the holder's signature over a nonce **Town generated** and will not accept twice, then compares its verdict with AgentSchool's. Agreement is a stage in the report, and disagreement fails that stage. The credentials in the fixture were earned. Three agents ran through AgentSchool as headless Claude Code processes and sat its held-out exam, which certifies only at 95 out of 100: | Agent | Model | Pre-test | Exam | Credential | |---|---|---|---|---| | atlas | Sonnet 5.5 | 100.0 | 100.0 | Level 1 | | brisk | Sonnet 5.5 | 94.5 | 100.0 | Level 1 | | flash | Haiku 4.5 | 44.1 | 87.0 | none | Flash failed both attempts, so the uncertified seller is an agent that could not pass rather than a placeholder. We then revoked brisk's credential with `agentschool revoke`, which is where the revoked case comes from. ## Reproduce Town base commit `6f9a7c064486c4a0ae9242f1ff341b8a4aad0244`, nandatown 0.2.0, Python 3.11.14 and 3.12.12, Lab evaluator `lab-0.2.6`, Path profile `a2a-capability-fulfillment@0.3`. AgentSchool commit `393c4b7f4991da6dc10fb07efe020126454886eb`, issuer key `as-D_vV6jcrJs`, skill `data-driven-slides` 2.0.0, exam `harbor-pine-slide-design-v2`. ```bash python -m pip install -e '.[dev]' python -m pytest tests/test_agentschool_integration.py -v nandatown run integrations/agentschool/scenarios/credential_gating.yaml nandatown run integrations/agentschool/scenarios/credential_gating_ungated_control.yaml nandatown run integrations/agentschool/scenarios/revoked_credential_fail_closed.yaml nandatown verify runs/<id> --plugin integrations/agentschool/plugin.py ``` Those commands need no AgentSchool and no model account. `integrations/agentschool/README.md` holds the live-mode variables, the full stage tables and how to rebuild the fixture. ## Results and limitations | Run | Expected | Observed | |---|---|---| | The main workflow, live | the seller that verifies is hired and paid, the other two refused | all 12 stages passed | | The same workflow, offline | the same verdicts, and no claim a captured proof cannot support | passed, `holder_proof_fresh` and `outcome_reported` reported as not tested | | The control, `gate: false` | the uncertified seller undercuts on price and wins the work | failed at `unverified_got_no_work`, naming `seller-uncertified` | | Only the revoked credential on offer | nothing is delegated and the refusal is recorded | passed, no purchase order and no payment | | `nandatown test-agent --url` | not known beforehand | failed at `protocol_invocation` with `a2a_http_status_404` | AgentSchool's credential format and its four hiring tools work unchanged inside a Town run: Town rea …
Try it
Open PR on GitHubView diffCheckout locally
git fetch origin pull/327/head:pr-327
git checkout pr-327