Vote for your favorite SkillMD. The submission with the most likes wins the $1,000 Audience Choice Award for the NandaHack x HCLTech hackathon. Voting is open through September 25.Vote now →
mergedhumanPR #268Other

fix(runner): wait for an external buyer's verdict before finishing

One of nine independent PRs from a correctness audit of main at df0b5f1. Each merges cleanly with the others in any order; together they pass 908 tests locally on Python 3.11 and 3.14 (CI covers 3.11 and 3.12).

Author

JamesCarnley avatar

@JamesCarnley

github profile →
Status
Merged
Merged on
Sep 14
Branch
fix/track-external-buyer-wait

Description

The pitch.

One of nine independent PRs from a correctness audit of `main` at `df0b5f1`. Each merges cleanly with the others in any order; together they pass 908 tests locally on Python 3.11 and 3.14 (CI covers 3.11 and 3.12).

Commits: `4e3006c`, `b7126ef`, `1825e9b`

### Problem

When the buyer joins from outside (`test-agent --role buyer --wait`,
`run --agent buyer=external`), the runner finished as soon as the stock seller
acked. A correct buyer that took about 0.5 s was reported INCOMPLETE ("the buyer
made no correctness assertion") and exited 1.

### Change

- For an externally joined buyer, the run finishes only when both hold:
  - the seller side is done;
  - the buyer has acked the quote response with any status other than
    `retryable`.
- Otherwise the existing deadline applies.
- The grant check reuses the event fetch already made in the same loop pass.

### Compatibility

- Runner only. The evaluator, profiles, HTTP contract and bundle format are
  unchanged.
- Stock and `--cmd` buyers keep their stop condition, and so do seller-subject
  runs.
- A silent external buyer now waits until the stock seller exits, about 5 s
  before the timeout, and is still INCOMPLETE. The README says when such a run
  ends: `run` uses 45 s, and `test-agent --timeout` defaults to 60 s.

### Verification

- Full suite on the branch: 820 passed, twice.
- Subprocess regressions:
  - the `--wait` case fails on `df0b5f1`;
  - `--cmd` and silent-buyer guards.
- A standalone external buyer driven through `--wait` passes with 0.5 s,
  2.0 s and no think time.
- With an external buyer, four fault profiles pass. They were INCOMPLETE on
  `df0b5f1`.

### Also fixed: one-shot seller race (commits `d8fdc6e`, `177b06a`, `82b3757`)

**Problem.** A one-shot seller subject could answer, ack and exit 0 before
Town's stock buyer claimed the reply. The runner then stopped the buyer and
reported INCOMPLETE ("no quote response accepted and claimed by the buyer"),
although the seller had done its job.

**Fix.** Once a quote response addressed to the buyer has been accepted, the
runner lets a buyer it started finish on its own, within the same deadline. A
response sent to anyone else, such as the seller replying to itself, does not
extend the run: that case ends in 0.7 s, where it took 46 s before `82b3757`,
and is still INCOMPLETE. These behave as before:

- crash restart;
- a seller that exits without responding, which still ends promptly as
  INCOMPLETE;
- external buyers;
- stock-vs-stock runs.

**Tests.**

- A deterministic regression test (the stock buyer's claim is delayed by 1 s)
  failed before the fix and passes after it.
- A 25-run one-shot loop passes every time.
- All eight stock profiles pass.
- `177b06a` gives this branch's silent-buyer test a 20 s timeout, so it stays
  valid alongside the timeout-floor PR.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Try it

Open PR on GitHubView diff

Checkout locally

git fetch origin pull/268/head:pr-268
git checkout pr-268