closedhumanPR #35Data Facts
provenance-engineer: content-addressed datafacts with provenance chains & signed freshness proofs
Problem 08 (datafacts). The default datafacts_v1 plugin addresses datasets by publisher-chosen name (df://<name>), grants access unconditionally, and "proves" freshness with a bare time.time() check.
Author
@Ngoga-Musagi
github profile →- Status
- Closed without merging
- Opened on
- Jun 23
- Branch
- hackathon/Ngoga-Musagi-datafacts-provenance
Description
The pitch.
## Problem
Problem 08 (datafacts). The default `datafacts_v1` plugin addresses datasets by **publisher-chosen name** (`df://<name>`), grants access unconditionally, and "proves" freshness with a bare `time.time()` check. That makes three classes of provenance attack silently undetectable in any trace:
- **Substitution** — republish *different* bytes under an existing name.
- **Stale-freshness** — claim a dataset is current by re-touching a timestamp without re-publishing.
- **Broken provenance** — a derived dataset whose ancestry references a hash nobody ever published.
## Design
A new `content_addressed` datafacts plugin where **the address is the content** and **freshness is a signature, not a timestamp**:
- `publish()` returns `df://sha256-<hex>` over the canonical content (volatile timestamps excluded so re-publishing identical content is idempotent). Different bytes ⇒ different address ⇒ **substitution is impossible by construction** — you cannot overwrite.
- `DatasetMetadata.parents` (new **optional** field, explicitly permitted by the problem) is folded into the child's content hash → a **Merkle-DAG**: a forged ancestor changes every descendant's address. `verify_provenance()` walks the DAG and rejects dangling parents; `publish()` raises `BrokenProvenanceError` on an unpublished parent.
- A `FreshnessProof` binds *(content-hash, tick, signer)* under a signature from the identity layer (`did_key`, taken via constructor — the same pattern as the EigenTrust PRs). A verifier validates it **without trusting wall-clock time**.
## API fit & tradeoffs
- Implements the `DataFacts` protocol **exactly** (`isinstance(plugin, DataFacts)` passes). The signed-proof machinery is added through **new** methods (`issue_freshness_proof`, `verify_freshness_proof`, `content_id`, `verify_provenance`) so `verify_freshness() -> bool` keeps the reference return type. The "proof object" the brief mentions is the new method, deliberately, so the plugin stays drop-in.
- Per-agent instances share one content-addressed **store** (substitution resistance is only meaningful swarm-wide) but each signs with its **own** identity — mirroring the merged `identity_rotation` provisioning pattern. Capability-gated so the same scenario degrades honestly under `datafacts_v1`.
## Adversarial validators (the charter's bar)
Three validators for scenario type `provenance_supply_chain`, each catching one attack class. From the **same** scenario YAML they **FAIL against `datafacts_v1`** and **PASS against `content_addressed`**:
| Validator | Catches |
|---|---|
| `validate_datafacts_content_addressing` | substitution / name-addressing |
| `validate_datafacts_freshness_proofs` | stale, unsigned freshness claims |
| `validate_datafacts_provenance_chain` | dangling ancestry, unverified chains |
## Verify
```bash
uv sync && uv run ruff check . && uv run ruff format --check . && uv run pyright && uv run pytest -v
# Adversarial discrimination — same YAML, two plugins:
uv run nest run scenarios/provenance_supply_chain.yaml -o ./traces/prov.jsonl
uv run python -c "from pathlib import Path; from nest_core.validators import validate_trace; \
[print('PASS' if r.passed else 'FAIL', r.name, '—', r.detail) \
for r in validate_trace(Path('traces/prov.jsonl'),'provenance_supply_chain')]"
# -> 3x PASS on content_addressed; flip the YAML to `datafacts: datafacts_v1` -> 3x FAIL
```
## Persona — `provenance-engineer`
The persona is in the code, not the label: content addressing as substitution resistance, parents folded into the hash to make the provenance DAG tamper-evident, a freshness proof that binds the content hash under a signature, and a validator that anchors freshness to *externally observed* ticks rather than attacker-claimed ones.
## Determinism & secrets
Pure-Python (`hashlib` / `json` + simulation-crypto `did_key`), **no new dependencies, no secrets/API keys**. Logical-tick clock, seeded only — `test_trace_is_byte_identical_under_replay` asserts byte-i
…Try it
Open PR on GitHubView diffCheckout locally
git fetch origin pull/35/head:pr-35
git checkout pr-35