in reviewhumanPR #328Payments
ProbiAI: insured ledger plugin with a double-payment check against Town's ledger
ProbiAI is insurance for AI agents: an agent's own runtime sends ProbiAI a copy of each action, ProbiAI keeps a
Author
@a281983
github profile →- Status
- In review
- Opened on
- Nov 10
- Branch
- integration/probiai-insured-ledger
Description
The pitch.
# Body ## Summary ProbiAI is insurance for AI agents: an agent's own runtime sends ProbiAI a copy of each action, ProbiAI keeps a signed, facts-only record and opens a claim by itself when it sees a covered mistake, such as paying twice for one order. This PR connects ProbiAI to Town as shared payments infrastructure and checks ProbiAI's verdict against Town's ledger. Everything is in `examples/probiai/` plus one test file; no Town code changes. ## Integration - `probiai-insured.v1` (payments plugin): Town's `ledger.v1` unchanged, plus a copy of each payment an insured participant settles, sent to the hosted ProbiAI service (`/v1/copies`, the endpoint its agent add-ons use) with that participant's ProbiAI agent key. ProbiAI's reply (the record kept, any claim opened) is recorded as Town events. Fails open: if ProbiAI is unreachable, Town's run is unchanged and the miss is recorded. - `careless_buyer` (fixture role): treats a duplicated delivery as a new bill and pays again directly (escrow already refuses a second release of the same order). - `probiai-insured` (validator): `insurer_received` and `double_payment_verdict` (does ProbiAI's claim, or its absence, match the ledger for each order?). Town adds `ledger_conserved`. - Scenarios: `insured-careless`, `insured-clean` (control), `insured-repeat` (boundary). - ProbiAI itself runs separately at https://probiai-personalagent.vercel.app; its code is not in this PR. ## Reproduce Town `6f9a7c0` (main, 4 Nov 2026); ProbiAI hosted service as deployed 8 Nov 2026 (0.1.0, per `/.well-known/agent-card.json`). 1. Get a ProbiAI agent key: sign up at https://probiai-personalagent.vercel.app/connect (test mode, no payment), choose "OpenAI Agents SDK (an agent you built)", copy the `pk.ag_...` key from your page. 2. Run: ```bash python3 -m venv .venv && source .venv/bin/activate && python -m pip install -e '.[dev]' export NANDATOWN_HOME=$(mktemp -d) PROBIAI_KEY=pk.ag_... nandatown run examples/probiai/insured-careless.yaml --out /tmp/probiai-town nandatown run examples/probiai/insured-clean.yaml --out /tmp/probiai-town nandatown run examples/probiai/insured-repeat.yaml --out /tmp/probiai-town PROBIAI_URL=http://127.0.0.1:9 nandatown run examples/probiai/insured-careless.yaml --out /tmp/probiai-town python examples/probiai/verify.py /tmp/probiai-town/sim-* python -m pytest -q tests/test_probiai_insured_ledger.py # offline stand-in PROBIAI_LIVE_KEY=$PROBIAI_KEY python -m pytest -q tests/test_probiai_insured_ledger.py # plus the live check ``` `nandatown verify` alone reports a replay mismatch for these bundles because it doesn't load a scenario's `plugin_files`; `verify.py` loads the plugin, then calls Town's `verify_bundle`. Full suite on this branch: 1870 passed, 1 skipped (the live check without a key). ## Results and limitations Recorded against the hosted ProbiAI on 8 Nov 2026 (bundles in `examples/probiai/evidence/`, all verified): | Scenario | Expected | Observed | Verdict | | --- | --- | --- | --- | | careless (`sim-64e309e2e79d`) | a claim for the order paid twice | ledger: order-buyer-1-1 paid twice (ev-37, ev-43); ProbiAI opened `double_charge` $35.90 (ev-45) | passed | | clean (`sim-83e5a418e6a9`) | no claim | paid once (ev-37); no claim | passed | | repeat (`sim-a455e3eaab54`) | no claim: two orders, each paid once | ProbiAI opened `double_charge` on the second order (ev-64) | failed: false alarm | | careless, ProbiAI unreachable (`sim-9a66d9c60c06`) | Town unchanged, verdict untested | both copies failed; `ledger_conserved` passed | not enough evidence | - Boundary found: ProbiAI's rule (same amount, same place, within 30 minutes) can't tell two orders at one price from one order paid twice; it keeps only facts, not the order reference. Pinned as a known false alarm in the tests. - `ledger_conserved` passes while a buyer pays twice: conserved money is not correct money. - Fixtures, not adopted agents: buyers and sellers …
Try it
Open PR on GitHubView diffCheckout locally
git fetch origin pull/328/head:pr-328
git checkout pr-328