closedhumanPR #35Data Facts

provenance-engineer: content-addressed datafacts with provenance chains & signed freshness proofs

Problem 08 (datafacts). The default datafacts_v1 plugin addresses datasets by publisher-chosen name (df://<name>), grants access unconditionally, and "proves" freshness with a bare time.time() check.

Author

Ngoga-Musagi avatar

@Ngoga-Musagi

github profile →
Status
Closed without merging
Opened on
Jun 23
Branch
hackathon/Ngoga-Musagi-datafacts-provenance

Description

The pitch.

## Problem

Problem 08 (datafacts). The default `datafacts_v1` plugin addresses datasets by **publisher-chosen name** (`df://<name>`), grants access unconditionally, and "proves" freshness with a bare `time.time()` check. That makes three classes of provenance attack silently undetectable in any trace:

- **Substitution** — republish *different* bytes under an existing name.
- **Stale-freshness** — claim a dataset is current by re-touching a timestamp without re-publishing.
- **Broken provenance** — a derived dataset whose ancestry references a hash nobody ever published.

## Design

A new `content_addressed` datafacts plugin where **the address is the content** and **freshness is a signature, not a timestamp**:

- `publish()` returns `df://sha256-<hex>` over the canonical content (volatile timestamps excluded so re-publishing identical content is idempotent). Different bytes ⇒ different address ⇒ **substitution is impossible by construction** — you cannot overwrite.
- `DatasetMetadata.parents` (new **optional** field, explicitly permitted by the problem) is folded into the child's content hash → a **Merkle-DAG**: a forged ancestor changes every descendant's address. `verify_provenance()` walks the DAG and rejects dangling parents; `publish()` raises `BrokenProvenanceError` on an unpublished parent.
- A `FreshnessProof` binds *(content-hash, tick, signer)* under a signature from the identity layer (`did_key`, taken via constructor — the same pattern as the EigenTrust PRs). A verifier validates it **without trusting wall-clock time**.

## API fit & tradeoffs

- Implements the `DataFacts` protocol **exactly** (`isinstance(plugin, DataFacts)` passes). The signed-proof machinery is added through **new** methods (`issue_freshness_proof`, `verify_freshness_proof`, `content_id`, `verify_provenance`) so `verify_freshness() -> bool` keeps the reference return type. The "proof object" the brief mentions is the new method, deliberately, so the plugin stays drop-in.
- Per-agent instances share one content-addressed **store** (substitution resistance is only meaningful swarm-wide) but each signs with its **own** identity — mirroring the merged `identity_rotation` provisioning pattern. Capability-gated so the same scenario degrades honestly under `datafacts_v1`.

## Adversarial validators (the charter's bar)

Three validators for scenario type `provenance_supply_chain`, each catching one attack class. From the **same** scenario YAML they **FAIL against `datafacts_v1`** and **PASS against `content_addressed`**:

| Validator | Catches |
|---|---|
| `validate_datafacts_content_addressing` | substitution / name-addressing |
| `validate_datafacts_freshness_proofs` | stale, unsigned freshness claims |
| `validate_datafacts_provenance_chain` | dangling ancestry, unverified chains |

## Verify

```bash
uv sync && uv run ruff check . && uv run ruff format --check . && uv run pyright && uv run pytest -v
# Adversarial discrimination — same YAML, two plugins:
uv run nest run scenarios/provenance_supply_chain.yaml -o ./traces/prov.jsonl
uv run python -c "from pathlib import Path; from nest_core.validators import validate_trace; \
  [print('PASS' if r.passed else 'FAIL', r.name, '—', r.detail) \
   for r in validate_trace(Path('traces/prov.jsonl'),'provenance_supply_chain')]"
# -> 3x PASS on content_addressed; flip the YAML to `datafacts: datafacts_v1` -> 3x FAIL
```

## Persona — `provenance-engineer`

The persona is in the code, not the label: content addressing as substitution resistance, parents folded into the hash to make the provenance DAG tamper-evident, a freshness proof that binds the content hash under a signature, and a validator that anchors freshness to *externally observed* ticks rather than attacker-claimed ones.

## Determinism & secrets

Pure-Python (`hashlib` / `json` + simulation-crypto `did_key`), **no new dependencies, no secrets/API keys**. Logical-tick clock, seeded only — `test_trace_is_byte_identical_under_replay` asserts byte-i

…

Try it

Open PR on GitHubView diff

Checkout locally

git fetch origin pull/35/head:pr-35
git checkout pr-35