in reviewhumanPR #365Identity
Signed Answers: add outlet source-package Lab integration with duplicate-delivery and altered-package tests
Signed Answers from News Outlets for AI Agents lets a news outlet run its own agent. A reader’s agent asks it about the outlet’s reporting over A2A. The outlet’s agent checks a subscriber credential.
Author
@jdzhrch
github profile →- Status
- In review
- Opened on
- Nov 12
- Branch
- integration/signed-answers-source-package
Description
The pitch.
## Summary Signed Answers from News Outlets for AI Agents lets a news outlet run its own agent. A reader’s agent asks it about the outlet’s reporting over A2A. The outlet’s agent checks a subscriber credential. Then it returns a signed source package: the exact sentences from the story, the document each sentence cites, dates, correction status and usage terms. The signature proves the package came from the outlet unchanged. It makes no claim about truth. This PR adds only the Town side, under `examples/signed_answers/`, plus one test file. It adds two Lab roles, a validator, four scenarios, a stand-in outlet for offline runs, a runner, saved evidence from two live runs and a README. It changes no Town source file. ## Integration The outlet’s agent runs as its own hosted service. Its code is in a private repository and is not in this PR. Inside a run: 1. The outlet’s gateway reads the agent card of the real service. It publishes a card in Town’s registry with the outlet’s signing key. 2. Each reader looks up `news.source_package` in the registry. It sends a signed Town message with one question. 3. Town’s auth layer verifies the sender. The gateway maps a verified subscriber to the credential, so no reader holds it. It forwards the question to the real agent with A2A JSON-RPC `SendMessage` over HTTPS. 4. The gateway downloads the package and sends it back as a signed Town message. 5. The reader checks the package id and the Ed25519 signature under the key on the Town card. 6. The reader opens the story on the publisher’s own site. It looks for each sentence of the package, word for word. The interface under test is the outlet agent’s A2A interface, used by Town agents through Town’s registry, signed messages, duplicate fault and privacy layer. The package text and the question are redacted, so the bundles hold ids and check results only. ## Reproduce - Town commit: `6f9a7c064486c4a0ae9242f1ff341b8a4aad0244` - Outlet agent: commits `24816cc` (before the fix) and `f69f3c9` (after) of the private project repository, deployed as a hosted service - Python 3.12 ```bash python3 -m venv .venv && source .venv/bin/activate python -m pip install -e '.[dev]' ``` Offline. No network, model or credential. About 15 seconds. ```bash python -m pytest -q tests/test_signed_answers_example.py python examples/signed_answers/run_check.py ``` Live, against the real outlet agent. The credential below is a test credential for this assignment. It allows 30 requests an hour and I will revoke it after grading. ```bash export SIGNED_ANSWERS_URL=https://source-agent-905861176071.us-west1.run.app export SIGNED_ANSWERS_TOKEN=mlsub_test_nJ84b9ZM5RfplDL8QRn4hkEUt6lKw9aw export SIGNED_ANSWERS_QUESTION="What did you report in September 2023 about staffing cuts at San Francisco State University? I want the enrollment figures and whether they were corrected." python examples/signed_answers/run_check.py --live ``` Check the saved bundles: ```bash python examples/signed_answers/run_check.py --verify examples/signed_answers/evidence ``` `nandatown verify` alone reports a replay mismatch for these bundles. It does not load a plugin validator. `run_check.py --verify` loads the validator first and then runs the same check. ## Results and limitations Two live runs on 2026-11-11. The first run found a failure in my agent. I fixed it and ran again. Both sets of bundles are in `examples/signed_answers/evidence/`. | Case | Expected | Before the fix | After the fix | |---|---|---|---| | `clean` | The subscriber’s reader gets a package and it verifies. Every sentence is in the published story. The stranger’s reader gets none. | Passed, 5 of 5 sentences found | Passed, 5 of 5 sentences found | | `duplicate` | One request delivered twice returns the same package. | **Failed.** Two valid packages with two different ids. | Passed. The same package id both times. | | `duplicate_remembered` | The gateway answers the redelivery from its own record. | Passed | Pass …
Try it
Open PR on GitHubView diffCheckout locally
git fetch origin pull/365/head:pr-365
git checkout pr-365