in reviewagentPR #8Transport
linux-kernel: per-hop latency models for the in-memory transport
Transport (layer 1) — reference plugin extension + simulator wiring.
Author
@linux-kernel
github profile →- Status
- In review
- Opened on
- May 26
- Branch
- hackathon/linux-kernel-latency-transport
Description
The pitch.
## Layer picked
**Transport** (layer 1) — reference plugin extension + simulator wiring.
## Why
The README has a section titled "Determinism & what the clock does" that explicitly flags this gap:
> The bundled `in_memory` transport delivers at `time = now`, so the virtual clock stays at `0.0` and `mean_latency` / `duration` will both be `0.0` in your trace... Latency *numbers* become meaningful only when ... you write a transport plugin that introduces per-hop delay.
For a testing tool that brags about "diff a trace against properties you care about," a transport that always reports zero latency is a sharp edge. NEST already exposes `mean_latency` as a built-in metric; it's just always 0.0. This PR fixes that without breaking anything.
## Core idea
A tiny, composable latency-model layer wired into the existing in-memory transport. One thing, well.
- `Simulator(latency_model=...)` — optional callable `(rng, sender, receiver) -> delay_seconds`. The simulator gets a **dedicated, seed-derived `_latency_rng`** so adding latency never perturbs the per-agent or failure RNG streams — existing traces stay byte-identical.
- `InMemoryTransport.send` schedules deliveries at `now + delay`, clamped at zero so the clock can never go backwards.
- New scenario field `transport_config:` (parsed by `runner._build_latency_model_from_config`) so users get realistic latency from YAML alone.
Six pure, composable models in `nest_plugins_reference.transport.latency`:
| `kind` | Parameters | Use case |
|---------------|-------------------------------------|----------|
| `constant` | `mean` | Every hop same. |
| `uniform` | `low`, `high` | Bounded jitter. |
| `exponential` | `mean` | Long-tail packet network. |
| `normal` | `mean`, `stddev`, `min_delay` | Tight RTT distribution. |
| `pair_matrix` | `matrix: {"a,b": delay, ...}`, `default` | Heterogeneous topology (intra-DC fast, inter-DC slow). |
| `zero` | — | Explicit legacy default. |
Plus `with_jitter(base, jitter)` to wrap any model in ±uniform jitter.
## How to test
```bash
pip install -e packages/nest-core -e packages/nest-sdk -e packages/nest-plugins-reference -e packages/nest-cli -e packages/nest-scenarios -e packages/nest-mocks
pip install pytest pytest-asyncio hypothesis
# All 237 tests pass (193 baseline + 36 new + 8 runner-level).
pytest packages/nest-core/tests packages/nest-plugins-reference/tests -q
# End-to-end smoke: copy the marketplace YAML, append a latency block, run it.
cp scenarios/marketplace.yaml /tmp/m.yaml
cat >> /tmp/m.yaml <<EOF
transport_config:
kind: exponential
mean: 0.020
jitter: 0.005
EOF
nest run /tmp/m.yaml -o /tmp/m.jsonl
# Now mean_latency is ~0.02 instead of 0.0; virtual clock advances.
python -c "
import json, statistics
ev=[json.loads(l) for l in open('/tmp/m.jsonl') if l.strip()]
sends={}; ds=[]
for e in ev:
c=e.get('corr','')
if not c: continue
if e['kind']=='send': sends[c]=e['ts']
elif e['kind']=='receive' and c in sends: ds.append(e['ts']-sends[c])
print('mean_latency:', round(statistics.fmean(ds),5))
print('clock_max :', round(max(e['ts'] for e in ev),5))
"
```
Run it twice — the traces are byte-identical (`md5sum` confirms).
## Key assumptions / design notes
- **Backward-compatible by construction.** No `transport_config` ⇒ no latency model ⇒ `time = now` ⇒ existing traces are byte-for-byte unchanged. I verified the bundled `marketplace` baseline trace MD5-matches across a fresh run on this branch.
- **Determinism preserved.** The latency RNG is derived from `seed` directly (`(seed * 2654435761 + 0xA1A7C7) & ((1<<63)-1)`), not from the master RNG sequence. Adding/removing the latency feature does not change the per-agent or failure-RNG streams for an existing scenario.
- **Clamp at zero.** All models clamp the delay to `max(0.0, ...)` so the vi
…