Vote for your favorite SkillMD. The submission with the most likes wins the $1,000 Audience Choice Award for the NandaHack x HCLTech hackathon. Voting is open through September 25.Vote now →
mergedhumanPR #159Auth

dhritimandas: capability-conformance registry gate

AgentCard.capabilities (nest_core/types.py:82) is a bare, self-asserted list[str]. Registry.lookup() does string matching against this list.

Author

dhritimandas avatar

@dhritimandas

github profile →
Status
Merged
Merged on
Jul 10
Branch
hackathon/dhritimandas-capability-conformance

Description

The pitch.

## Problem

`AgentCard.capabilities` (`nest_core/types.py:82`) is a bare, self-asserted `list[str]`. `Registry.lookup()` does string matching against this list. Every existing defense in the repo verifies who published a card (signatures, identity, `ed25519_rotating`) or scores an agent's overall behavior (`agent_receipts`, `score_average`), but nothing checks a specific capability claim against whether the agent has ever fulfilled it. An agent can register `capabilities=["sell"]` and remain discoverable via that claim forever, even if it never completes a single sale.

## Change

- `VerifiedCapabilitiesRegistry` (`nest_plugins_reference/registry/verified_capabilities.py`), registered as `("registry", "verified_capabilities")`: wraps an inner registry and tracks defection counts per `(agent_id, capability)` pair. A pair with no evidence is never excluded (bootstrap allowance). Once a pair reaches `defection_threshold` (default 1, justified in the docstring against the scenario's zero message-drop rate) defections, `lookup` stops returning that agent for queries naming that capability — other capabilities of the same agent are unaffected. `report_fulfillment` resets the count to zero (re-admission, not a permanent blacklist); this is a documented trade-off, not an oversight — a periodic fulfiller could evade indefinitely, out of scope here.
- `scenarios/capability_spoofing.yaml` + `capability_spoofing_factory`: 3 honest sellers, 2 always-silent spoofers, 1 bait-and-switch seller (fulfills once, then goes silent), 5 buyers. Buyers discover sellers via `registry.lookup(Query(capabilities=["sell"]))`, send one outstanding `buy:` at a time, and self-schedule a `TIMEOUT:<round>` tick. A `sold:` response before the timeout reports a fulfillment; the timeout firing first reports a defection.
- `check_capability_conformance` (`nest_plugins_reference/validators/capability_validators.py`): registry lookups aren't traced (confirmed against `nest_core/sim/simulator.py`), so this validates directly against `ScenarioRunner.resolved_plugins["registry"]`, following the same `ValidatorReport` pattern as `registry_byzantine_validators`/`gossip_validators`. Checks two things together, so a gate that blocks everyone can't pass by accident: no spoofer is still discoverable, and every honest seller still is.

## Test

Literal output, same scenario YAML, only `layers.registry` overridden:

```
registry='in_memory'             passed=False detail="still discoverable for 'sell': ['baitswitch-0', 'spoofer-0', 'spoofer-1']"
registry='verified_capabilities' passed=True  detail="3 spoofer(s) excluded from 'sell', 3 honest agent(s) still discoverable"
```

`test_verified_capabilities.py` (7 unit tests): collection error (`ModuleNotFoundError`) before the plugin existed; all 7 pass after — bootstrap allowance, per-capability exclusion isolation, re-admission on fulfillment, below-threshold tolerance, honest-agent non-interference, deregister delegation. `test_capability_spoofing_scenario.py`: FAIL/PASS matrix above at seeds 42/7/1337, plus a byte-identical-trace determinism test. Full `make ci-local`: 1164 passed, 1 skipped, 1 deselected.

## Fit

A2A v1.0 ships signed Agent Cards and NANDA's Verified AgentFacts paper names capability spoofing as an open problem; a signature proves the publisher, not the capability. `agent_receipts` scores an agent's overall reputation from receipts — this gates *discovery*, per *capability*, independent of an agent's standing on any other claim.

Try it

Open PR on GitHubView diff

Checkout locally

git fetch origin pull/159/head:pr-159
git checkout pr-159