in reviewhumanPR #335Transport
Agent-Hunter: add trial-then-hire and provider invoice-extraction A2A path profiles
[Agent-Hunter](https://github.com/chris-a-ackerman/agent-hunter) is a hiring service for AI agents. Before a requester agent hands off an invoice-extraction job, Agent-Hunter reviews the job, gives every candidate provider agent a short…
Author
@chris-a-ackerman
github profile →- Status
- In review
- Opened on
- Nov 10
- Branch
- integration/agent-hunter-trial-then-hire
Description
The pitch.
## Summary
[Agent-Hunter](https://github.com/chris-a-ackerman/agent-hunter) is a hiring service for AI agents. Before a requester agent hands off an invoice-extraction job, Agent-Hunter reviews the job, gives every candidate provider agent a short trial with known answers, scores the candidates with confidence intervals, and returns a ranked verdict: one recommended candidate, or a tie.
This PR adds Path profiles that test both sides of that hire in Town:
| Profile | Tests | Evaluator |
| -- | -- | -- |
| `a2a-trial-then-hire@0.1` | The hiring agent: one A2A `message/send` runs the whole hire and returns a verdict that must hold together | `path-trial-hire-0.1` |
| `a2a-trial-then-hire-unscorable@0.1` | Failure case: a job that can't be scored automatically must be rejected before any candidate is paid | `path-trial-hire-0.1` |
| `a2a-trial-then-hire-stock-timeout@0.1` | Boundary case: the same hire at Town's stock 15 s limit | `path-trial-hire-0.1` |
| `a2a-invoice-extraction@0.1` | A provider agent being hired: its output must match the requested schema, and a resent task must not run twice | `path-invoice-extraction-0.1` |
It also adds tests for both evaluators, a README section, an integration page (`docs/integrations/agent-hunter.md`), one pinned synthetic invoice, and recorded, verifiable evidence bundles under `examples/agent-hunter/`. `docs/integrations/` and `examples/agent-hunter/` are new paths; Town had no convention for third-party docs or bundles.
## Integration
```
Town (requester) --A2A message/send--> Agent-Hunter door (:8940)
review -> rubric -> trial -> score -> rank
--A2A message/send--> provider agents (mocks :8101, or real)
```
- **Agent-Hunter runs outside this repo.** Public repo: [`chris-a-ackerman/agent-hunter`](https://github.com/chris-a-ackerman/agent-hunter), pinned at [`735f2d8`](https://github.com/chris-a-ackerman/agent-hunter/commit/735f2d848ff0f088259def2ad249497e40bc3c75). None of its code is in this PR.
- **Live mode** uses real model calls and 7 real provider agents from a separate private service.
- **Offline mode** (`--offline`) runs the same hire with stub agents and 5 mock providers. It makes no model calls and needs no keys, so anyone can rerun it. Every offline hire result and the agent card say `"mode": "offline"`.
- **Town-side code changes:** `path_profiles.py` and `path_runner.py` (profiles and evaluators), the request bodies in `src/nandatown/path_requests/` (now package data, so they ship in the wheel, and the distribution check covers them), and the two new evaluator versions registered in `bundle.py`.
- The provider profile fetches its document from this fork at the tag `agent-hunter-fixture-v1`, so the URL doesn't change when the branch is rebased.
## Reproduce
| | Version |
| -- | -- |
| Nanda Town | based on `projnanda/nandatown` `main` at `6f9a7c0`; this branch's head; `nandatown 0.2.0` |
| Agent-Hunter | [`chris-a-ackerman/agent-hunter`](https://github.com/chris-a-ackerman/agent-hunter) at `735f2d848ff0f088259def2ad249497e40bc3c75` |
| Python | Town 3.11 and 3.12 (full suite passes on both); Agent-Hunter 3.12 via `uv` |
**Offline, no keys. Terminal 1 (Agent-Hunter):**
```bash
git clone https://github.com/chris-a-ackerman/agent-hunter && cd agent-hunter
git checkout 735f2d848ff0f088259def2ad249497e40bc3c75
uv sync
uv run python -m core.a2a --offline --port 8940
```
**Terminal 2 (Town, this branch):**
```bash
pip install -e ".[dev]"
DIGEST=$(python -c "from nandatown.a2a_adapter import fetch_card; from nandatown.records import fingerprint; print(fingerprint(fetch_card('http://127.0.0.1:8940')))")
nandatown test-agent --url http://127.0.0.1:8940 --path-profile a2a-trial-then-hire@0.1 --pin-card-digest "$DIGEST" --out runs/
nandatown test-agent --url http://127.0.0.1:8940 --path-profile a2a-trial-then-hire-unscorable@0.1 --pin-card-digest "$DIGEST" --out runs/
nandatown test-agent --
…Try it
Open PR on GitHubView diffCheckout locally
git fetch origin pull/335/head:pr-335
git checkout pr-335