Vote for your favorite SkillMD. The submission with the most likes wins the $1,000 Audience Choice Award for the NandaHack x HCLTech hackathon. Voting is open through September 25.Vote now →
in reviewhumanPR #230Transport

feat(agent-test): sign test results and verify them offline

A town.test-result/1 document binds what it ran against: the profile by profile.digest, the run by execution.seed and execution.scenario, the trace by an artifact digest. The document carrying those bindings is itself unsigned.

Author

Sharathvc23 avatar

@Sharathvc23

github profile →
Status
In review
Opened on
Aug 23
Branch
feat/town-agent-test-attestation

Description

The pitch.

## Problem

A `town.test-result/1` document binds what it ran against: the profile by `profile.digest`, the run by `execution.seed` and `execution.scenario`, the trace by an artifact digest. The document carrying those bindings is itself unsigned.

So a reader who did not run it cannot distinguish an untouched result from an edited one. Changing `evaluation.verdict` is a text edit, and it leaves every one of those internal digests valid. That is fine while results stay on the machine that produced them, and it stops being fine as soon as results are compared between participants or aggregated over time, because any such view inherits the trust assumptions of whoever supplied each file.

## Change

Adds `nest_core.agent_test.attestation`: a detached statement over a result.

- digests the result **as received** with `jcs_digest`, restates the facts a reader needs in order to interpret the verdict, and signs the statement with Ed25519
- `verify_attestation(result, attestation)` recomputes the digest and checks the signature, reading nothing but the two objects it is handed — no network, no service, no registry
- the result document is never modified

Detached rather than embedded because `TestResult` is `extra="forbid"`, so no existing schema changes.

Two things are deliberately **not** re-implemented:

- **Consistency is inherited.** Signing parses through `TestResult`, so `_result_state_is_consistent` and the `Literal` status constraints already gate what can be signed. A self-contradicting document is refused because the contract refuses it, not because this module carries a second copy of those rules to drift out of step with it.
- **Canonicalization and signature verification are `nest_core.canonical`'s.** `jcs_digest`, `issuer_signed_payload`, and `verify_receipt_signature` are used as they stand, so an attested digest and a sealed one can never disagree about what a document is. The statement is a receipt in the shape this repository already signs.

One thing is added that the contract cannot know about. A hosted model can be revised without notice, mutating what was tested while every local digest stays valid — so `mutable_dependencies` is a required argument, and an empty sequence is a positive claim rather than a default. It is the difference between "nothing was unfrozen" and "nobody said", and it is why an attested verdict is a statement about a run at a time rather than a standing certificate.

```python
attestation = build_attestation(
    result, signing_key=key, mutable_dependencies=[
        MutableDependency("a-hosted-model", "hosted-model", "2026-08-23T00:00:00Z"),
    ]
)
verify_attestation(result, attestation).ok  # False if either object was touched
```

**No new dependencies.** `cryptography` is already a hard dependency of `nest-core`.

## Verification

- `make ci-local` — all 5 checks passed; 1463 passed, 1 skipped, 1 deselected
- pyright strict clean
- 19 tests, using this branch's existing `result-pass` / `result-incomplete` fixtures rather than new synthetic documents, so the tests exercise the contract as shipped
- six parametrised tamper cases (verdict, seed, target label, coverage status, trace digest, profile digest) confirm the digest covers the whole document
- three parametrised cases confirm the contract, not this module, is what rejects an unsupported verdict or an unknown status
- offline verification confirmed by running it inside a network namespace with no network configured
- 3 files, 484 insertions, 0 deletions

## Scope

Library only. No CLI command, no transport, no runtime, no changes to the driver path — matching this PR's base, which defines contracts only. Wiring a `nest` subcommand is small and belongs after the CLI PRs in this stack land.

## Stack

**Depends on #221.** Targets `feat/town-agent-test-contract-1` because that is the narrowest true dependency: this consumes the contract that PR defines and nothing later in the stack.

**Merge note:** after #221 merges into `main`,

…

Try it

Open PR on GitHubView diff

Checkout locally

git fetch origin pull/230/head:pr-230
git checkout pr-230