creative/evaluator_auth storyboard steps fail instead of skip when agent declares supports_evaluator: false

[andrazbajec]

Summary

When an agent correctly declares creative.supports_evaluator: false in get_adcp_capabilities, the creative/evaluator_auth storyboard steps fail instead of being skipped as not applicable.

Expected behavior

creative/evaluator_auth storyboard steps should be skipped (status: not_applicable) when the agent declares creative.supports_evaluator: false. The evaluator surface is experimental and agents that don't support it should not be penalized.

Actual behavior

confirm_experimental_feature fails with:

Expected true, got false

at /creative/supports_evaluator — even though false is the agent's correct declaration.

Subsequent steps (reject_off_list_evaluator, reject_off_list_feature_agent, payload_credentials_advisory, accepted_evaluator_invocation, accepted_evaluator_unavailable_degrades) cascade as prerequisite_failed.

Steps to reproduce

1. Run storyboard run <agent> creative/evaluator_auth --compliance-version 3.1.1 against an agent with creative.supports_evaluator: false
2. Observe confirm_experimental_feature fails rather than skips

Impact

Agents correctly declaring no evaluator support receive false failures across the entire evaluator_auth storyboard track.

[bokelley]

Triage

Outcome: Execute PR (defensive storyboard workaround) + sibling-repo fix needed in adcp-client

Root cause: Two distinct layers:

1. adcp-client runner bug (primary): The storyboard is correctly authored. skip_if: "context.supports_evaluator != true" is semantically correct — the pattern is established across multiple storyboards in this repo (billing_out_of_band.yaml, refine_finalize_exclusivity.yaml). The runner bug is in the skip_if evaluator: when the captured context value is boolean false, it evaluates false != true as false instead of true, so the guarded phase runs instead of being skipped. This needs a fix in adcp-client (see adcp-client#693 for the adjacent expression-evaluation surface).

2. Storyboard workaround (this repo): As a defensive measure for the 3.1 pre-release window, changed all six skip_if guards from "context.supports_evaluator != true" to "!context.supports_evaluator" — semantically equivalent for boolean values and consistent with the !context.build_capability_id / !context.creative_feature_id style already used in the same expressions.

Fix: #5846 — six skip_if guards updated to truthiness form; no protocol semantics change.

Sibling-repo action needed: File a bug in adcp-client targeting the skip_if expression evaluator's handling of boolean false in != true comparisons. The billing_out_of_band != false path may carry the same bug (verify with a bills_through_adcp: false agent).

Milestone: 3.1.0 — storyboard introduced in 3.1; evaluator support is an experimental 3.1 feature.

---

Generated by Claude Code

[bokelley]

Triage

Outcome: Execute PR (storyboard workaround complete — runner fix still needed in adcp-client)
Priority: P1
Domain: creative / compliance-suite

Root cause (two layers):

1. Storyboard layer: evaluator_auth.yaml had no requires_capability guards on its five optional phases. When an agent correctly declares creative.supports_evaluator: false, those phases should be not_applicable — but without a guard they run and fail.

2. Runner layer (adcp-client): Boolean false values written by context_outputs are silently dropped or mishandled by the runner's context accumulator. This causes skip_if: "context.supports_evaluator != true" to evaluate undefined != true → true and trigger a skip for the wrong reason — masking the underlying runner bug rather than correctly routing to not_applicable.

Sibling-repo fix needed: adcp-client — runner's context accumulator mishandles boolean false from context_outputs; skip_if evaluator produces wrong result when a context key is undefined. This issue tracks the storyboard symptom only.

Storyboard fix on branch claude/issue-5843-creative-storyboard-capability-assertions (same PR as #5843):
Added requires_capability: { path: "creative.supports_evaluator", equals: true } to all six optional phases: evaluator_capability_contract, reject_off_list_evaluator, reject_off_list_feature_agent, payload_credentials_advisory, accepted_evaluator_invocation, accepted_evaluator_unavailable_degrades. requires_capability evaluates against the raw capabilities response before the context accumulator is consulted, bypassing the boolean-false drop bug entirely. Agents declaring supports_evaluator: false now receive not_applicable instead of a false failure.

Also fixes context_outputs key: → name: in all three entries (canonical per storyboard-schema.yaml).

Blocker (environment): Same pre-commit hook timeout as #5843. Changes are staged on branch claude/issue-5843-creative-storyboard-capability-assertions.

Triaged 2026-07-08 by Claude (scheduled triage run)

---

Generated by Claude Code