Skip to content

Validate agent nodes carry a required model id#12

Draft
posthog[bot] wants to merge 1 commit into
mainfrom
posthog-code/validate-agent-node-model
Draft

Validate agent nodes carry a required model id#12
posthog[bot] wants to merge 1 commit into
mainfrom
posthog-code/validate-agent-node-model

Conversation

@posthog

@posthog posthog Bot commented Jun 29, 2026

Copy link
Copy Markdown

Why

Users building WhatsApp workflows keep getting stuck adding an AI Agent node: session replays show them hitting "Debe seleccionar un modelo de IA" / "AI Model is required" and "System Prompt is required" walls, with two sessions abandoned. The agent-config step is essentially un-instrumented, so this friction is invisible in analytics and only shows up in replay.

In this repo node-types.md documents config.provider_model_id as required, but scripts/validate-graph.js only validated decide-node conditions — it never checked agent nodes. So a graph built via these skills could reach the exact state the product UI rejects, with no early feedback.

Changes

  • validate-graph.js: error when an agent node is missing config.provider_model_id (message points at list-provider-models.js); warn on an empty system_prompt. Adds an agentNodes stat.
  • node-types.md / SKILL.md: surface list-provider-models.js and the validation requirement so the model id is set before the node is saved.

Bundled agent assets still validate cleanly; npm run validate and npm run check:syntax pass.

Out of scope (product-UI team)

Three customer-facing bugs live in the workflow editor, not this repo, and need separate attention:

  • False-positive validation — System Prompt + AI Model errors firing even with default text present (signal e12856c0).
  • Model reverts on save — selecting claude-3-5-sonnet-latest, saving, and finding it silently reverted to the default google/gemini-2.5-flash (signal e858461c).
  • Stuck-loading model dropdown — the model dropdown initially hangs loading (signal 363c6d29).

Created with PostHog Code from an inbox report.

validate-graph.js only checked decide-node conditions, so an agent node
built via these skills could be saved with no provider_model_id and no
early feedback — the exact state the workflow editor later rejects with
"Debe seleccionar un modelo de IA" / "AI Model is required".

Extend the node loop to error when an agent node is missing
config.provider_model_id (pointing at list-provider-models.js), and warn
on an empty system_prompt. Surface the model lookup in the Graph rules of
SKILL.md and the agent notes in node-types.md so the id is set before the
node is saved.

Generated-By: PostHog Code
Task-Id: b0476de7-24fa-4962-9701-f3132536bc14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants