Vote for your favorite SkillMD. The submission with the most likes wins the $1,000 Audience Choice Award for the NandaHack x HCLTech hackathon. Voting is open through September 25.Vote now →
mergedhumanPR #270Other

fix(path): record undecodable terminal output as the subject's

One of nine independent PRs from a correctness audit of main at df0b5f1. Each merges cleanly with the others in any order; together they pass 908 tests locally on Python 3.11 and 3.14 (CI covers 3.11 and 3.12).

Author

JamesCarnley avatar

@JamesCarnley

github profile →
Status
Merged
Merged on
Sep 14
Branch
fix/path-undecodable-output

Description

The pitch.

One of nine independent PRs from a correctness audit of `main` at `df0b5f1`. Each merges cleanly with the others in any order; together they pass 908 tests locally on Python 3.11 and 3.14 (CI covers 3.11 and 3.12).

Commits: `8958101`

### Problem

Under the strict Path profiles (the `@0.3` default and `a2a-quote-intent@0.2`),
some subject output was misattributed.

- JSON nested past the interpreter's recursion limit produced an ERROR verdict
  blaming Town.
- An integer over Python's 4,300-digit limit, or a lone surrogate, recorded a
  phantom second failed `protocol_exchange`, so `protocol_invocation` failed.

### Change

- Strict profiles record these cases as `fulfillment_unparseable`, so
  `protocol_invocation` passes and `semantic_result` fails.
- Only decoding and digesting are guarded, so a fault in Town's own event
  recording is not blamed on the subject.

### Compatibility

- `evaluate_path`, profiles and evaluator versions are unchanged.
- Legacy profiles keep their recorded behavior, pinned by a characterization
  test that passes on `df0b5f1`.
- Deeply nested, non-finite and unencodable echoed values are handled by
  the follow-up commits described below.

### Verification

- Full suite on the branch: 828 passed.
- Five new strict cases fail on `df0b5f1`.
- A recording-fault test guards against over-catching.
- Live localhost fixture runs: strict profiles FAILED at `semantic_result`,
  legacy `@0.2` still ERROR, and the bundles verify.

### Also fixed: deeply nested values echoed into evidence (commits `3cfd70a`, `0b097f9`)

**Problem.** Valid JSON nested roughly 200–999 levels deep inside a value that
Town copies into an event crashed `run_path_test` while it was writing the
bundle, under every Path profile, and this predates this PR. The affected
values were:

- `total_cents` and `request_id`;
- the quote-intent fields;
- the task `id`, `kind` and `state`;
- the agent card `name` and `version`.

The run left a partial bundle with no attestation and a traceback. It exited 1,
the same code as "failed", which hid a Town fault.

**Fix.** Any such value nested more than 64 levels deep is now recorded as a
small marker, `{"unrecorded": "nesting too deep", type, json_length,
fingerprint}`. The limit of 64 was chosen from these measurements:

- pydantic refuses to write an event line past about 254 levels;
- it refuses to read one back past about 198;
- the value itself already sits up to three levels deep in the line.

Values within the bound are recorded exactly as before, for legacy and strict
profiles. `content_digest` and the card digest still cover the full output.

**How the marker is judged.** It is judged the way the real value would be: it
never equals an expected value.

- A nested `kind` fails `protocol_invocation` as the agent's fault.
- A nested `state` does the same under the strict profiles.
- A nested `total_cents` fails `semantic_result`.
- No case becomes a Town ERROR.

This replaces the note above saying that this limit "is not addressed".

**Also marked (`4af3e16`): NaN, ±Infinity and lone surrogates.** Before this
commit, both kinds of value broke evidence in these ways:

- Python's JSON parser accepts `NaN` and `Infinity`, but pydantic recorded them
  as `null`. Replay then differed from the original run. For example, a legacy
  run with a NaN state passed, but `verify` later rejected its bundle.
- A lone surrogate crashed the bundle write. This also affected the
  unparseable-output preview.

Both are now recorded as markers:

- a non-finite number becomes `{"unrecorded": "non-finite number", …}`, with a
  deterministic fingerprint;
- an unencodable string becomes `{"unrecorded": "unencodable string", …}`, with
  no digest.

The rule is applied uniformly, including to the card name and version. For
those rare NaN inputs, the recorded bytes change from a misleading `null` to
the marker, and stage outcomes are unchanged. Ordinary bundles stay
byte-identical.

**Independent review.** The reviewer 

…

Try it

Open PR on GitHubView diff

Checkout locally

git fetch origin pull/270/head:pr-270
git checkout pr-270