mergedhumanPR #159Auth
dhritimandas: capability-conformance registry gate
AgentCard.capabilities (nest_core/types.py:82) is a bare, self-asserted list[str]. Registry.lookup() does string matching against this list.
Author
@dhritimandas
github profile →- Status
- Merged
- Merged on
- Jul 10
- Branch
- hackathon/dhritimandas-capability-conformance
Description
The pitch.
## Problem
`AgentCard.capabilities` (`nest_core/types.py:82`) is a bare, self-asserted `list[str]`. `Registry.lookup()` does string matching against this list. Every existing defense in the repo verifies who published a card (signatures, identity, `ed25519_rotating`) or scores an agent's overall behavior (`agent_receipts`, `score_average`), but nothing checks a specific capability claim against whether the agent has ever fulfilled it. An agent can register `capabilities=["sell"]` and remain discoverable via that claim forever, even if it never completes a single sale.
## Change
- `VerifiedCapabilitiesRegistry` (`nest_plugins_reference/registry/verified_capabilities.py`), registered as `("registry", "verified_capabilities")`: wraps an inner registry and tracks defection counts per `(agent_id, capability)` pair. A pair with no evidence is never excluded (bootstrap allowance). Once a pair reaches `defection_threshold` (default 1, justified in the docstring against the scenario's zero message-drop rate) defections, `lookup` stops returning that agent for queries naming that capability — other capabilities of the same agent are unaffected. `report_fulfillment` resets the count to zero (re-admission, not a permanent blacklist); this is a documented trade-off, not an oversight — a periodic fulfiller could evade indefinitely, out of scope here.
- `scenarios/capability_spoofing.yaml` + `capability_spoofing_factory`: 3 honest sellers, 2 always-silent spoofers, 1 bait-and-switch seller (fulfills once, then goes silent), 5 buyers. Buyers discover sellers via `registry.lookup(Query(capabilities=["sell"]))`, send one outstanding `buy:` at a time, and self-schedule a `TIMEOUT:<round>` tick. A `sold:` response before the timeout reports a fulfillment; the timeout firing first reports a defection.
- `check_capability_conformance` (`nest_plugins_reference/validators/capability_validators.py`): registry lookups aren't traced (confirmed against `nest_core/sim/simulator.py`), so this validates directly against `ScenarioRunner.resolved_plugins["registry"]`, following the same `ValidatorReport` pattern as `registry_byzantine_validators`/`gossip_validators`. Checks two things together, so a gate that blocks everyone can't pass by accident: no spoofer is still discoverable, and every honest seller still is.
## Test
Literal output, same scenario YAML, only `layers.registry` overridden:
```
registry='in_memory' passed=False detail="still discoverable for 'sell': ['baitswitch-0', 'spoofer-0', 'spoofer-1']"
registry='verified_capabilities' passed=True detail="3 spoofer(s) excluded from 'sell', 3 honest agent(s) still discoverable"
```
`test_verified_capabilities.py` (7 unit tests): collection error (`ModuleNotFoundError`) before the plugin existed; all 7 pass after — bootstrap allowance, per-capability exclusion isolation, re-admission on fulfillment, below-threshold tolerance, honest-agent non-interference, deregister delegation. `test_capability_spoofing_scenario.py`: FAIL/PASS matrix above at seeds 42/7/1337, plus a byte-identical-trace determinism test. Full `make ci-local`: 1164 passed, 1 skipped, 1 deselected.
## Fit
A2A v1.0 ships signed Agent Cards and NANDA's Verified AgentFacts paper names capability spoofing as an open problem; a signature proves the publisher, not the capability. `agent_receipts` scores an agent's overall reputation from receipts — this gates *discovery*, per *capability*, independent of an agent's standing on any other claim.Try it
Open PR on GitHubView diffCheckout locally
git fetch origin pull/159/head:pr-159
git checkout pr-159