in reviewhumanPR #318Registry

Add opt-in shared-dependency resilience experiment for Lab

Different registered agents can share the same backend, so switching agent identities may not escape a correlated failure.

Author

Shereen20112003 avatar

@Shereen20112003

github profile →
Status
In review
Opened on
Nov 3
Branch
feat/shared-dependency-experiment

Description

The pitch.

## Motivation

Different registered agents can share the same backend, so switching agent identities may not escape a correlated failure. This contribution adds an opt-in, reproducible Lab experiment for measuring that gap, including incomplete dependency disclosure and counterexamples where dependency-aware selection is not better.

Built as part of the Nanda Town contribution exercise. The original independent project and published evidence are at [SA_NANDATOWN](https://github.com/Shereen20112003/SA_NANDATOWN).

## Scope

This proposes the **full deterministic experiment package**, not just a scenario stub:

- Isolated `nandatown.experiments.resilience` package with an opt-in `nandatown-resilience` CLI and module entry point.
- Scenario construction, consumer/provider roles, and five policies: price, random, load, dependency-aware, and a dependency-label oracle.
- Clean, shared-backend, and independent provider-local failure conditions with configurable dependency disclosure.
- A versioned evaluator, signed-bundle verification, campaign execution, and reports.
- 104 imported regression tests plus four integration/packaging tests.
- Usage, findings, assumptions, provenance, and the original MIT license notice.

The implementation uses the existing Lab engine, registry, signed messages, coordination, ledger, and evidence bundles. It does not modify default scenarios or add a deployed NANDA Index / Track / Path integration.

The live Azure/coco workbench remains in the independent repository and is **not included here**. No model API credentials or cloud inference are required by this experiment. Generated campaign bundles are linked rather than copied into upstream.

## Published findings and limits

The original published campaign covers 450 runs / 72,000 tasks (five policies, three failure conditions, three disclosure levels, ten seeds).

- With full disclosure during the shared-outage stress cohort, dependency-aware completion was 100%, versus 12.5% for price-first.
- Under clean operation, dependency-aware selection incurred a 30% cost premium.
- Under independent provider-local outages, dependency-aware completion was 69.38%, versus 80% for price-first; the difference's confidence interval includes zero.
- Benefits depend on truthful metadata and a shared reservation view. Unknown dependencies are conservatively grouped.
- The oracle knows dependency labels, not future outages, and is not an optimality claim.

These are simulation-specific results, not a claim of novel routing primitives, production reliability, real-world truth, or security. The full 450-run campaign was not rerun for this namespace integration.

## Validation

On Windows / Python 3.12:

- 104 imported regression tests and four new integration tests passed.
- 305 existing upstream tests passed.
- Three existing upstream failures were reproduced against clean upstream HEAD: Windows symlink privilege and two CRLF-sensitive fixture hashes.
- Wheel and sdist builds, license/content checks, installed-layout execution, native plugin swapping, and marketplace regression checks passed.
- `git diff --check` passed.

The existing distribution-check script also has Windows path-separator false positives; these were not changed. Targeted archive/content checks were used to verify this contribution's distribution contents.

Test selections (with the checkout's `src` on `PYTHONPATH`):

```sh
python -m pytest -q tests/experiments/test_resilience.py tests/experiments/test_resilience_cli.py
python -m pytest -q tests/experiments/test_resilience_bundling.py tests/test_sim.py tests/test_layers.py tests/test_lab_auction.py tests/test_lab_capped_influence.py tests/test_lab_consensus.py tests/test_lab_coverage.py tests/test_lab_reputation.py tests/test_registry_owned.py tests/test_scenario_faults.py tests/test_schema_sync.py tests/test_bundle.py tests/test_tooling.py
```

The three baseline failures were:

- `tests/test_bun

…

Try it

Open PR on GitHubView diff

Checkout locally

git fetch origin pull/318/head:pr-318
git checkout pr-318