in reviewhumanPR #318Registry
Add opt-in shared-dependency resilience experiment for Lab
Different registered agents can share the same backend, so switching agent identities may not escape a correlated failure.
Author
@Shereen20112003
github profile →- Status
- In review
- Opened on
- Nov 3
- Branch
- feat/shared-dependency-experiment
Description
The pitch.
## Motivation Different registered agents can share the same backend, so switching agent identities may not escape a correlated failure. This contribution adds an opt-in, reproducible Lab experiment for measuring that gap, including incomplete dependency disclosure and counterexamples where dependency-aware selection is not better. Built as part of the Nanda Town contribution exercise. The original independent project and published evidence are at [SA_NANDATOWN](https://github.com/Shereen20112003/SA_NANDATOWN). ## Scope This proposes the **full deterministic experiment package**, not just a scenario stub: - Isolated `nandatown.experiments.resilience` package with an opt-in `nandatown-resilience` CLI and module entry point. - Scenario construction, consumer/provider roles, and five policies: price, random, load, dependency-aware, and a dependency-label oracle. - Clean, shared-backend, and independent provider-local failure conditions with configurable dependency disclosure. - A versioned evaluator, signed-bundle verification, campaign execution, and reports. - 104 imported regression tests plus four integration/packaging tests. - Usage, findings, assumptions, provenance, and the original MIT license notice. The implementation uses the existing Lab engine, registry, signed messages, coordination, ledger, and evidence bundles. It does not modify default scenarios or add a deployed NANDA Index / Track / Path integration. The live Azure/coco workbench remains in the independent repository and is **not included here**. No model API credentials or cloud inference are required by this experiment. Generated campaign bundles are linked rather than copied into upstream. ## Published findings and limits The original published campaign covers 450 runs / 72,000 tasks (five policies, three failure conditions, three disclosure levels, ten seeds). - With full disclosure during the shared-outage stress cohort, dependency-aware completion was 100%, versus 12.5% for price-first. - Under clean operation, dependency-aware selection incurred a 30% cost premium. - Under independent provider-local outages, dependency-aware completion was 69.38%, versus 80% for price-first; the difference's confidence interval includes zero. - Benefits depend on truthful metadata and a shared reservation view. Unknown dependencies are conservatively grouped. - The oracle knows dependency labels, not future outages, and is not an optimality claim. These are simulation-specific results, not a claim of novel routing primitives, production reliability, real-world truth, or security. The full 450-run campaign was not rerun for this namespace integration. ## Validation On Windows / Python 3.12: - 104 imported regression tests and four new integration tests passed. - 305 existing upstream tests passed. - Three existing upstream failures were reproduced against clean upstream HEAD: Windows symlink privilege and two CRLF-sensitive fixture hashes. - Wheel and sdist builds, license/content checks, installed-layout execution, native plugin swapping, and marketplace regression checks passed. - `git diff --check` passed. The existing distribution-check script also has Windows path-separator false positives; these were not changed. Targeted archive/content checks were used to verify this contribution's distribution contents. Test selections (with the checkout's `src` on `PYTHONPATH`): ```sh python -m pytest -q tests/experiments/test_resilience.py tests/experiments/test_resilience_cli.py python -m pytest -q tests/experiments/test_resilience_bundling.py tests/test_sim.py tests/test_layers.py tests/test_lab_auction.py tests/test_lab_capped_influence.py tests/test_lab_consensus.py tests/test_lab_coverage.py tests/test_lab_reputation.py tests/test_registry_owned.py tests/test_scenario_faults.py tests/test_schema_sync.py tests/test_bundle.py tests/test_tooling.py ``` The three baseline failures were: - `tests/test_bun …
Try it
Open PR on GitHubView diffCheckout locally
git fetch origin pull/318/head:pr-318
git checkout pr-318