docs: add observability tables reference

[Aliciapet11]

Summary

Add documentation for the 4 database tables added by the observability layer (APPENG-5370). Includes schema diagram, column descriptions with types and nullability, indexes, constraints, and relationships.

Tables documented:

• scorecards — unified gate verdict per pipeline run
• gate_results — per-gate normalized results
• certifications — per-level certification results
• observability_metrics — token usage and timing per run

Requested by Guy.

[GuyZivRH]

Consolidated Review: PR #56 — docs: add observability tables reference

What Looks Good

• Schema diagram gives clear visual orientation of relationships
• Pipeline integration section accurately explains how/when tables are populated
• Column-level accuracy is high — types, nullability, indexes, CASCADE behavior all match abevalflow/db/models.py
• Underscore filename follows repo convention
• enable-scorecard default true is accurate
  This is clearly written by carefully reading the model file, not guessed.

---

Please Fix (Accuracy Issues)

1. eval_engine values omit aeh (line 48)

Doc lists harbor, ase, mcpchecker, a2a, both — but aeh is also a valid value, used throughout scripts/validate.py, analyze.py, publish.py, and set in abevalflow/engines/aeh.py. This is a significant recent addition to omit.

2. gate_name / gate_type omit behavioral (lines 74-75)

Doc says evaluation, security, or quality only. After PR #51, there's also behavioral (GateType.BEHAVIORAL, e.g. policy_key=edge-case in abevalflow/gates/behavioral/edge_case.py).

3. observability_metrics cardinality (schema diagram)

Diagram says 1:1 via pipeline_run_id, but the unique constraint is on (pipeline_run_id, attempt_number), not pipeline_run_id alone. Prefer "one row per run attempt" so retries don't look like a schema bug.

Should Fix (Misleading Today)

4. gate_results token/duration columns (lines 84-86)

duration_ms, prompt_tokens, completion_tokens are documented without caveat, but:

• None of these are actually populated today — map_gate_results() in store_results.py doesn't set them
• The GateResult Pydantic schema doesn't even have prompt_tokens/completion_tokens fields
• @timed_gate decorator is explicitly for "future use" per its docstring
  The duration_ms "(populated by Phase A when available)" caveat is close but overstates current reality. Add (reserved, not yet populated) to all three to avoid misleading readers into expecting per-gate token data.

---

Optional Nits

• Add soft link from Docs/results_persistence_and_observability_plan.md or README for discoverability
• Note that schema arrows are logical (no FK from evaluation_runs to scorecards)
• Could add example queries (pass rate by gate type, token cost over time)

---

Summary

Category	Items
Must fix	aeh enum value, behavioral gate type/name, cardinality wording
Should fix	Token/duration "not yet populated" caveat
Optional	Discoverability links, example queries
Approve after the three accuracy tweaks. Fine to merge as-is if tracked as follow-up, but for a reference doc I'd fix them in this PR.

[Aliciapet11]

All 4 items fixed:

• Item 1 — Added aeh to eval_engine values
• Item 2 — Added behavioral to gate_name and gate_type descriptions
• Item 3 — Fixed cardinality: "one per run attempt, via pipeline_run_id + attempt_number". Added note about logical references (no FK from evaluation_runs)
• Item 4 — Marked duration_ms, prompt_tokens, completion_tokens as "(reserved, not yet populated)"