feat(workflow-executor): add AI-assisted and Full AI execution to the guidance step (PRD-148 pattern, PRD-18)#1741
Conversation
2 new issues
|
|
Coverage Impact ⬆️ Merging this pull request will increase total coverage on Modified Files with Diff Coverage (3)
🛟 Help
|
720c832 to
492febc
Compare
492febc to
cd8665f
Compare
| } | ||
|
|
||
| // An empty draft is not a submittable answer (even in Full AI) → degrade to manual input. | ||
| if (!draft.trim()) { |
There was a problem hiding this comment.
Claude Opus 4.8 (claude-opus-4-8): Should fix — draft is typed string, but invokeWithTools returns toolCall.args cast to T without validating it against the tool's zod schema (base-step-executor.ts:343-344). If the model emits a non-string response (a number, object, etc.), response ?? '' keeps that value and draft.trim() here throws a TypeError — and this line sits outside the try/catch at L39-46, so it surfaces as a generic "Unexpected error during step execution" (base-step-executor.ts:105) instead of the intended degrade-to-manual. That breaks the PR's "run never fails / degrade on bad AI output" guarantee for a malformed tool call. Coerce or guard the value (e.g. const draft = typeof raw === 'string' ? raw : '') or move the emptiness check inside the try, so this path degrades like the empty-draft case. No test covers a non-string tool arg.
There was a problem hiding this comment.
Good catch — fixed. askAiForResponse now coerces a non-string tool arg to '' (typeof response === 'string' ? response : ''), so a malformed AI response flows into the empty-draft path and degrades to manual instead of throwing a TypeError at draft.trim() outside the try/catch. Added a test: a { response: 123 } tool arg degrades to awaiting-input with pendingData: {} (no unexpected error). (pushed bb1f84f)
| executionType: z.literal(Manual).default(Manual).catch(Manual), | ||
| // No `.catch`; default Manual (not AWC) — the orchestrator owns the legacy default, so the | ||
| // executor fallback only fires on a missing field, where "never call AI" is the safe direction. | ||
| executionType: z.enum([Manual, AutomatedWithConfirmation, FullyAutomated]).default(Manual), |
There was a problem hiding this comment.
Claude Opus 4.8 (claude-opus-4-8): Preferential — dropping .catch(Manual) is a deliberate fail-loud choice (it matches the sibling trigger-action / load-related schemas and is covered by the "rejects an invalid executionType" test), but the failure mode is worth calling out: mapTask (step-definition-mapper.ts:34) calls .parse without wrapping, so an unrecognized executionType throws a raw ZodError (not InvalidStepDefinitionError). getAvailableRuns (forest-server-workflow-port.ts) only rescues instanceof WorkflowExecutorError, so such a run is silently dropped — logged, but not surfaced in malformed[] to the orchestrator. Only reachable if the server ever introduces a mode beyond today's 1:1 ServerStepExecutionTypeEnum, so risk is low. If you'd prefer fail-safe here: normalize unknown modes at the mapper, and update the now-inaccurate mapTask comment at step-definition-mapper.ts:22-23 ("each schema's .default().catch() handles missing or unsupported values" — no longer true for guidance).
There was a problem hiding this comment.
Acknowledged — keeping the fail-loud (no .catch) behavior for consistency with the sibling trigger-action / load-related schemas; the raw-ZodError-drop path is only reachable if the server ever emits a mode outside today's 1:1 ServerStepExecutionTypeEnum, so the risk is low and I'd rather not special-case guidance. I did fix the now-inaccurate mapTask comment (step-definition-mapper.ts:22-23) to reflect that the manual-accepting schemas drop .catch and reject out-of-enum values rather than coercing. If we later add a server mode, the cleaner move is a mapper-level normalization for all step types (out of scope here). (pushed bb1f84f)
… guidance step (PRD-18) Widen GuidanceStepDefinitionSchema to the 3-way execution mode. In AI-assisted the AI pre-fills the free-text response (persisted as pendingData with an aiGenerated flag for the front badge) and the human submits; in Full AI the AI writes and submits automatically (executionResult.generatedByAi). On AI failure or an empty draft the step degrades to Manual (empty field, no auto-skip), and the AI is never re-run on re-dispatch. Hoist withAiAssist + logAiDegrade from LoadRelatedRecordStepExecutor into BaseStepExecutor (2nd consumer), leaving load-related behavior unchanged. Relates to PRD-18. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
cd8665f to
bb1f84f
Compare

What
Adds the 3-way Execution mode (Manual / AI-assisted / Full AI) to the Guidance step. Part of PRD-18 — 3-PR set (executor + `ForestAdmin/forestadmin-server` orchestrator + `ForestAdmin/forestadmin` editor).
automated-with-confirmation) — AI pre-fills the free-text response from the step prompt + workflow context; persisted aspendingData.userInputwith anaiGeneratedflag (drives the front "AI" badge); the human edits/submits.fully-automated) — AI writes and submits automatically (executionResult.userInput+generatedByAi); the response becomes a variable reusable by downstream steps.Degrade / resilience (product-confirmed): on AI failure/timeout, or an empty draft, the step degrades to Manual (empty field,
pendingData: {}) — the run never fails, the step never auto-skips. Full AI always submits on a valid draft (no low-confidence degrade — a text generator, unlike Load Related Record). The AI is never re-run on re-dispatch of the same(runId, stepIndex).withAiAssist+logAiDegradeare hoisted fromLoadRelatedRecordStepExecutorintoBaseStepExecutor(2nd consumer); load-related behavior is unchanged.Tests
Full suite green (1412). Added: Manual never calls AI; submit path never calls AI; AI-assisted pre-fill + badge flag + re-dispatch no-regen + degrade (AI failure / empty draft) + no-retry-after-degrade; Full AI submit + replay + degrade; schema (3-way, no
.catch, rejects unknown); mapper (3 values + default manual). Load-related suite unchanged (hoist is behavior-neutral).Relates to PRD-18.
🤖 Generated with Claude Code
Note
Add AI-assisted and full AI execution modes to the guidance step executor
GuidanceStepExecutor.doExecutenow supports three modes:Manual(awaiting-input immediately),AutomatedWithConfirmation(AI drafts a response saved aspendingDatafor user review), andFullyAutomated(AI response saved directly asexecutionResult).pendingData, preventing future AI retries on re-dispatch.StepExecutionFormatters.formatGuidancenow attributes AI-generated responses to the AI rather than the operator in step summaries.withAiAssistandlogAiDegradehelpers are promoted toBaseStepExecutorand the bespoke implementations inLoadRelatedRecordStepExecutorare removed.GuidanceStepDefinitionSchemano longer coerces invalidexecutionTypevalues toManual; invalid values now fail validation.Macroscope summarized bb1f84f.