mergedhumanPR #276Other
fix(bundle): read JSON Lines records at LF only
Follow-up to #264–#272, from the same correctness audit of main at df0b5f1.
Author
@JamesCarnley
github profile →- Status
- Merged
- Merged on
- Sep 14
- Branch
- fix/bundle-jsonl-line-separators
Description
The pitch.
Follow-up to #264–#272, from the same correctness audit of `main` at `df0b5f1`. It merges cleanly with the other open PRs that touch `bundle.py` or `path_runner.py`. ### Problem `load_bundle` split `intents.jsonl` and `events.jsonl` into records with `str.splitlines()`. That method also breaks lines at U+2028, U+2029 and U+0085. All three are legal unescaped inside JSON strings, and pydantic writes them raw into `events.jsonl`. So any recorded string containing one broke the bundle. Examples include an A2A agent card named `Acme<U+2028>Seller`, a task id or state, and a Track participant's ack note. The run crashed while rendering `report.md` and left a partial bundle with no report or attestation. `verify`, `receipt`, `proof` and `mirror` then failed on it. ### Change Records are now split at `"\n"` only, and empty lines are still skipped. The file is still read in text mode, so CRLF bundles load as before. Other line-oriented readers were audited and need no change: the `checks.jsonl` readers iterate text-mode files, the MCP adapter reads stdin and `readline`, and every other bundle consumer goes through `load_bundle`. ### Compatibility The change is reader-only. Recorded bytes, hashes, fingerprints and the bundle format are unchanged, and every bundle Town's writers produce loads the same records. A hand-built file that separates records with U+2028 or another non-LF line boundary outside a JSON string is now rejected, as JSON Lines requires. ### Also: evidence files are UTF-8 (`eda0d59`) Bundle records, the manifest, `report.md`, receipts, attestations and mirror copies used to be read and written with the locale's default encoding. On a non-UTF-8 locale this broke. Examples are Windows cp1252, and `LC_ALL=C` with UTF-8 mode off. Writing a raw U+2028, or a name like "Agent Zoë — 東京", raised `UnicodeEncodeError`. A bundle written on one machine might also fail to read on another. These files now always use UTF-8. - **UTF-8 systems:** bytes and hashes are unchanged, checked on pinned Lab, Track and mirror outputs. - **New test:** it runs the full chain in a child process with an ASCII default encoding: write, attest, receipt, verify, mirror and recover. It asserts the child really is non-UTF-8, so it cannot pass vacuously. - **Older bundles:** one written on a non-UTF-8 code page with non-ASCII text now reads as invalid everywhere. Before, it read only on a machine with that code page. ### Verification - New `tests/test_record_line_separators.py` covers Lab, Track and Path bundles for each of the three characters, plus a CRLF guard. It fails on `main` and passes with this change. - A real Path CLI run against a local agent whose card name contains U+2028 passes all six stages. `verify`, `receipt`, `verify-receipt`, `proof`, `mirror` and `recover` then all exit 0. - A real Track run whose seller note contains U+2028 verifies. - Full suite: 831 passed, twice. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Try it
Open PR on GitHubView diffCheckout locally
git fetch origin pull/276/head:pr-276
git checkout pr-276