P3 auditing: auditor evaluation mechanism — commit-then-reveal, encrypted test data, deviation accountability

[umeradl]

Part of the P3 mechanism-design push (#36).

Summary

Auditor rewards and slashing cannot depend only on whether an auditor submitted a score — they must depend on whether the score is honest, reproducible, and policy-compatible. Once client rewards, validator rewards, and accepted aggregation all hang off scoring, DIN needs an explicit mechanism for evaluating the auditors themselves.

Scope (from issues/auditor-evaluation-mechanism.md and MECHANISM_DESIGN §6)

• Cross-auditor median as the accountability reference: |auditor score − median| > threshold ⇒ S3 slash candidate (threshold empirically validated first, see the scoring issue)
• Commit-then-reveal for auditor score submission so auditors cannot copy each other's scores — the white paper specifies this (secret sharing / ECDH); verify it is in the P3 hardening scope (gap item 11) — ✅ Resolved (Abraham, 2026-07-21): in scope for P3. Prevents validators copying each other's scores; called out as critical for security.
• Audit test data distribution: compare the current assignAuditTestDataset flow against the white paper §5.2.3b scheme (single encrypted CID, per-validator key encryption, on-chain key mapping) — open a design decision if the current flow is plaintext or centrally shared (gap item 5) — ✅ Resolved: adopt in 2.0. Current flow is effectively plaintext; upgrade it to the paper's encrypted-CID + per-validator-key scheme.
• Test-set resampling strategy: reserved pool (~40%), resampled half per round, full pool in the final round (gap item 6) — ✅ Resolved: adopt in 2.0. Called out as critical for evaluation integrity.
• Aggregator auditing: T1/T2 outputs verifiable by recomputation; any validator may open an S4 dispute (no proactive re-audit of every batch)

Design sources

issues/auditor-evaluation-mechanism.md · MECHANISM_DESIGN §6 · whitepaper-summary §4.4, §8 gap items 5, 6, 11

Roadmap anchors: P3-4.1 (S3 feeds slashing spec), P3-SCR (threshold data), P3-6.2 (adversarial testing of score manipulation)