Vote for your favorite SkillMD. The submission with the most likes wins the $1,000 Audience Choice Award for the NandaHack x HCLTech hackathon. Voting is open through September 25.Vote now →
mergedhumanPR #276Other

fix(bundle): read JSON Lines records at LF only

Follow-up to #264–#272, from the same correctness audit of main at df0b5f1.

Author

JamesCarnley avatar

@JamesCarnley

github profile →
Status
Merged
Merged on
Sep 14
Branch
fix/bundle-jsonl-line-separators

Description

The pitch.

Follow-up to #264–#272, from the same correctness audit of `main` at `df0b5f1`.
It merges cleanly with the other open PRs that touch `bundle.py` or
`path_runner.py`.

### Problem

`load_bundle` split `intents.jsonl` and `events.jsonl` into records with
`str.splitlines()`. That method also breaks lines at U+2028, U+2029 and U+0085.
All three are legal unescaped inside JSON strings, and pydantic writes them raw
into `events.jsonl`.

So any recorded string containing one broke the bundle. Examples include an A2A
agent card named `Acme<U+2028>Seller`, a task id or state, and a Track
participant's ack note. The run crashed while rendering `report.md` and left a
partial bundle with no report or attestation. `verify`, `receipt`, `proof` and
`mirror` then failed on it.

### Change

Records are now split at `"\n"` only, and empty lines are still skipped. The
file is still read in text mode, so CRLF bundles load as before.

Other line-oriented readers were audited and need no change: the `checks.jsonl`
readers iterate text-mode files, the MCP adapter reads stdin and `readline`,
and every other bundle consumer goes through `load_bundle`.

### Compatibility

The change is reader-only. Recorded bytes, hashes, fingerprints and the bundle
format are unchanged, and every bundle Town's writers produce loads the same
records. A hand-built file that separates records with U+2028 or another non-LF
line boundary outside a JSON string is now rejected, as JSON Lines requires.

### Also: evidence files are UTF-8 (`eda0d59`)

Bundle records, the manifest, `report.md`, receipts, attestations and mirror
copies used to be read and written with the locale's default encoding. On a
non-UTF-8 locale this broke. Examples are Windows cp1252, and `LC_ALL=C` with
UTF-8 mode off. Writing a raw U+2028, or a name like "Agent Zoë — 東京", raised
`UnicodeEncodeError`. A bundle written on one machine might also fail to read on
another. These files now always use UTF-8.

- **UTF-8 systems:** bytes and hashes are unchanged, checked on pinned Lab, Track
  and mirror outputs.
- **New test:** it runs the full chain in a child process with an ASCII default
  encoding: write, attest, receipt, verify, mirror and recover. It asserts the
  child really is non-UTF-8, so it cannot pass vacuously.
- **Older bundles:** one written on a non-UTF-8 code page with non-ASCII text now
  reads as invalid everywhere. Before, it read only on a machine with that code
  page.

### Verification

- New `tests/test_record_line_separators.py` covers Lab, Track and Path bundles
  for each of the three characters, plus a CRLF guard. It fails on `main` and
  passes with this change.
- A real Path CLI run against a local agent whose card name contains U+2028
  passes all six stages. `verify`, `receipt`, `verify-receipt`, `proof`,
  `mirror` and `recover` then all exit 0.
- A real Track run whose seller note contains U+2028 verifies.
- Full suite: 831 passed, twice.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Try it

Open PR on GitHubView diff

Checkout locally

git fetch origin pull/276/head:pr-276
git checkout pr-276