Vote for your favorite SkillMD. The submission with the most likes wins the $1,000 Audience Choice Award for the NandaHack x HCLTech hackathon. Voting is open through September 25.Vote now →
in reviewhumanPR #59Data Facts

interview-integrity: Problem 08 (datafacts) — ogha_facts plugin: audience ACLs + pre-hash PII redaction + screening-evaluation scenario & validators

I run the AI services for a verified-screening business — the filter at the *front* of a client's hiring funnel. A human interviewer conducts a recorded screening interview; the audio is transcribed, and the interviewer files an evaluation…

Author

puja-ankitha-ivaturi-ogha avatar

@puja-ankitha-ivaturi-ogha

github profile →
Status
In review
Opened on
Jul 3
Branch
hackathon/puja-ankitha-ivaturi-ogha-interview-evaluation

Description

The pitch.

## Problem 08 — datafacts (persona: interview-integrity engineer)

I run the AI services for a **verified-screening** business — the filter at the *front* of a client's hiring funnel. A human interviewer conducts a recorded screening interview; the audio is transcribed, and the interviewer files an evaluation — a `PASSED`/`FAILED` verdict, their gist notes, and supporting **Q&A pulled verbatim from the transcript** so the client sees exactly what was asked and answered — that we deliver to the company that posted the job. We only screen; candidates who pass go on to the client's own rounds (the hiring decision is theirs, not ours). Our value proposition is that the report is *genuine and grounded*: the right candidate, from the assigned interviewer, its evidence **provably drawn from a real interview** — not swapped, fabricated, leaked, or stale. This PR models that pipeline on-protocol as the chain **recording → transcript → report** and hardens every hop.

### What this adds over the merged `cid_facts`

`cid_facts` (merged in #31) already solves the three properties Problem 08 asks for — content-addressed URLs, a `parents` provenance DAG, and identity-signed freshness proofs. **Re-implementing those would be duplicate work**, so I don't. `ogha_facts` **subclasses** `CidFacts` and adds the two capabilities the `datafacts.md` wishlist explicitly lists as unclaimed (*"fine-grained ACLs"*, redaction), which an evaluation-delivery pipeline genuinely needs:

1. **Audience-scoped ACLs.** `CidFacts.request_access` is binary — owner-or-public gets `read`, everyone else is hard-denied with `PermissionError`. That's wrong for a verdict: it's private, but has *several* legitimate readers (the interviewer, the service, the posting company, the candidate), while every *other* company must still be able to audit that a record exists without reading it. `ogha_facts.request_access` returns `read` to the content-bound ACL audience and a **`metadata`-tier grant (never a hard denial)** to everyone else — fail-*closed* on content, fail-*open* on existence. The ACL is part of the hashed content, so *who may read a verdict cannot be changed without minting a new URL*.

2. **Pre-hash PII redaction.** A content hash is permanent — once a candidate's SSN, salary, phone, or DOB is folded into the hash and the URL is handed out, it can't be unpublished without breaking every provenance link to it. So redaction must run **before** hashing. `publish()` scrubs free-text fields through `redact_pii` (a fixpoint loop guaranteeing `scan_pii(redact_pii(t)) == {}`) before delegating to `CidFacts.publish`. This is our production `PiiDetectionGuardrail` re-expressed as pure-Python regexes so Tier 1 stays deterministic.

3. **Verbatim quote-grounding (the headline check).** The report's supporting Q&A must be *actually in the transcript it cites* — an interviewer cannot put words in the candidate's mouth. `ungrounded_quotes(qa, transcript)` (deterministic substring match, no model) flags any excerpt not present. The adversarial twist is what ties this to the datafacts layer: an attacker who fabricates a Q&A **and tampers the transcript to cover it** is caught **only** under content-addressing — the report cites the transcript by content hash, so the tampered transcript is a *different* dataset and the fabrication doesn't ground. Under name-addressing (`datafacts_v1`) the tampered transcript overwrites the original at the same URL and the lie passes. So the new `evaluation_quotes_grounded` validator FAILS on `datafacts_v1`, PASSES on `ogha_facts`.

**Nothing fabricated.** `evaluation_dataset` also enforces: the verdict must be exactly `PASSED` or `FAILED` (any other string raises), and the **transcript is a required provenance parent** — so a report whose transcript was never published (a verdict with no interview behind it) is rejected at `publish` time.

*Honest boundary:* the transcript stands in for what a speech-to-text system (our Track-2 product) produces; t

…

Try it

Open PR on GitHubView diff

Checkout locally

git fetch origin pull/59/head:pr-59
git checkout pr-59