Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/fix-creative-storyboard-capability-assertions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
"adcontextprotocol": patch
---

Fix false failures in creative compliance storyboards (canonical_supported_formats, evaluator_auth).

`canonical_supported_formats`: removes the hardcoded `capability_id: "training_image_generation"` assertion (capability_id is agent-local; any valid value must pass) and the `field_absent` check on `supported_formats[1]` (agents may advertise multiple canonical formats). Fixes `context_outputs` field name from `key:` to `name:`.

`evaluator_auth`: adds `requires_capability` guards to all five optional phases so agents that correctly declare `creative.supports_evaluator: false` receive `not_applicable` instead of failing the evaluator track. Guards evaluate against the raw capabilities response, bypassing a runner-side boolean-false accumulator bug. Fixes `context_outputs` field name from `key:` to `name:`.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
id: creative/canonical_supported_formats
version: "1.0.0"
version: "1.0.1"
title: "Creative canonical supported formats"
category: creative
summary: "Verifies the 3.1 creative-agent canonical path: targetable supported_formats capability IDs, build_creative routing, and unsupported target rejection."
Expand Down Expand Up @@ -69,18 +69,17 @@ phases:
context:
correlation_id: "creative_canonical_supported_formats--get_capabilities"
context_outputs:
- key: "image_build_capability_id"
- name: "image_build_capability_id"
path: "creative.supported_formats[0].capability_id"
validations:
- check: response_schema
description: "Response matches get-adcp-capabilities-response.json schema"
- check: field_present
path: "creative.supported_formats"
description: "Creative canonical supported formats are advertised"
- check: field_value
- check: field_present
path: "creative.supported_formats[0].capability_id"
value: "training_image_generation"
description: "The image build path has a stable local capability_id"
description: "The advertised canonical format entry carries a capability_id for build routing"
- check: field_value
path: "creative.supported_formats[0].format.format_kind"
value: "image"
Expand All @@ -91,9 +90,6 @@ phases:
- check: field_absent
path: "creative.supported_formats[0].format.format_option_id"
description: "Creative-agent build routing uses capability_id, not product format_option_id"
- check: field_absent
path: "creative.supported_formats[1].capability_id"
description: "Storyboard fixture only advertises implemented canonical build paths"
- check: field_value
path: "context.correlation_id"
value: "creative_canonical_supported_formats--get_capabilities"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
id: creative/evaluator_auth
version: "1.0.0"
version: "1.0.1"
title: "Creative evaluator authentication boundary"
category: creative
summary: "Verifies that build_creative evaluator pointers use the verifier allowlist, keep credentials out of payloads, and invoke accepted evaluator agents through get_creative_features."
Expand Down Expand Up @@ -94,11 +94,11 @@ phases:
context:
correlation_id: "creative_evaluator_auth--get_capabilities"
context_outputs:
- key: "supports_evaluator"
- name: "supports_evaluator"
path: "creative.supports_evaluator"
- key: "build_capability_id"
- name: "build_capability_id"
path: "creative.supported_formats[0].capability_id"
- key: "creative_feature_id"
- name: "creative_feature_id"
path: "governance.creative_features[0].feature_id"
validations:
- check: response_schema
Expand All @@ -111,6 +111,9 @@ phases:
- id: evaluator_capability_contract
title: "Confirm experimental feature declaration"
optional: true
requires_capability:
path: "creative.supports_evaluator"
equals: true
skip_if: "context.supports_evaluator != true"
narrative: |
Because evaluator support is experimental, agents that set
Expand Down Expand Up @@ -162,6 +165,9 @@ phases:
- id: reject_off_list_evaluator
title: "Reject an off-list evaluator URL"
optional: true
requires_capability:
path: "creative.supports_evaluator"
equals: true
skip_if: "context.supports_evaluator != true || !context.build_capability_id || !context.creative_feature_id"
narrative: |
The buyer supplies a well-formed external evaluator URL that is not in the
Expand Down Expand Up @@ -219,6 +225,9 @@ phases:
- id: reject_off_list_feature_agent
title: "Reject an off-list feature_agent URL"
optional: true
requires_capability:
path: "creative.supports_evaluator"
equals: true
skip_if: "context.supports_evaluator != true || !context.build_capability_id || !context.creative_feature_id"
narrative: |
The buyer supplies a well-formed `feature_agent.agent_url` alongside an
Expand Down Expand Up @@ -281,6 +290,9 @@ phases:
- id: payload_credentials_advisory
title: "Keep evaluator credentials out of the payload"
optional: true
requires_capability:
path: "creative.supports_evaluator"
equals: true
skip_if: "context.supports_evaluator != true || !context.build_capability_id || !context.creative_feature_id"
narrative: |
The buyer attempts to pass an evaluator API key inside the evaluator
Expand Down Expand Up @@ -356,6 +368,9 @@ phases:
- id: accepted_evaluator_invocation
title: "Invoke an accepted evaluator"
optional: true
requires_capability:
path: "creative.supports_evaluator"
equals: true
skip_if: "context.supports_evaluator != true || !context.build_capability_id || !context.creative_feature_id"
narrative: |
The buyer supplies an accepted external evaluator URL. The creative agent
Expand Down Expand Up @@ -433,6 +448,9 @@ phases:
- id: accepted_evaluator_unavailable_degrades
title: "Degrade when an accepted evaluator is unavailable"
optional: true
requires_capability:
path: "creative.supports_evaluator"
equals: true
skip_if: "context.supports_evaluator != true || !context.build_capability_id || !context.creative_feature_id"
narrative: |
An accepted evaluator can be unreachable or can reject the producing
Expand Down
Loading