Skip to content

feat(workflow-executor): add AI-assisted and Full AI execution to the guidance step (PRD-148 pattern, PRD-18)#1741

Open
Scra3 wants to merge 1 commit into
mainfrom
feature/prd-18-ai-assisted-and-full-ai-guidance-task
Open

feat(workflow-executor): add AI-assisted and Full AI execution to the guidance step (PRD-148 pattern, PRD-18)#1741
Scra3 wants to merge 1 commit into
mainfrom
feature/prd-18-ai-assisted-and-full-ai-guidance-task

Conversation

@Scra3

@Scra3 Scra3 commented Jul 7, 2026

Copy link
Copy Markdown
Member

What

Adds the 3-way Execution mode (Manual / AI-assisted / Full AI) to the Guidance step. Part of PRD-18 — 3-PR set (executor + `ForestAdmin/forestadmin-server` orchestrator + `ForestAdmin/forestadmin` editor).

  • Manual — human types + submits (unchanged); AI never called.
  • AI-assisted (automated-with-confirmation) — AI pre-fills the free-text response from the step prompt + workflow context; persisted as pendingData.userInput with an aiGenerated flag (drives the front "AI" badge); the human edits/submits.
  • Full AI (fully-automated) — AI writes and submits automatically (executionResult.userInput + generatedByAi); the response becomes a variable reusable by downstream steps.

Degrade / resilience (product-confirmed): on AI failure/timeout, or an empty draft, the step degrades to Manual (empty field, pendingData: {}) — the run never fails, the step never auto-skips. Full AI always submits on a valid draft (no low-confidence degrade — a text generator, unlike Load Related Record). The AI is never re-run on re-dispatch of the same (runId, stepIndex).

withAiAssist + logAiDegrade are hoisted from LoadRelatedRecordStepExecutor into BaseStepExecutor (2nd consumer); load-related behavior is unchanged.

Tests

Full suite green (1412). Added: Manual never calls AI; submit path never calls AI; AI-assisted pre-fill + badge flag + re-dispatch no-regen + degrade (AI failure / empty draft) + no-retry-after-degrade; Full AI submit + replay + degrade; schema (3-way, no .catch, rejects unknown); mapper (3 values + default manual). Load-related suite unchanged (hoist is behavior-neutral).

Relates to PRD-18.

🤖 Generated with Claude Code

Note

Add AI-assisted and full AI execution modes to the guidance step executor

  • GuidanceStepExecutor.doExecute now supports three modes: Manual (awaiting-input immediately), AutomatedWithConfirmation (AI drafts a response saved as pendingData for user review), and FullyAutomated (AI response saved directly as executionResult).
  • On AI failure or empty/non-string response, the executor degrades to manual by persisting empty pendingData, preventing future AI retries on re-dispatch.
  • StepExecutionFormatters.formatGuidance now attributes AI-generated responses to the AI rather than the operator in step summaries.
  • withAiAssist and logAiDegrade helpers are promoted to BaseStepExecutor and the bespoke implementations in LoadRelatedRecordStepExecutor are removed.
  • GuidanceStepDefinitionSchema no longer coerces invalid executionType values to Manual; invalid values now fail validation.

Macroscope summarized bb1f84f.

@linear-code

linear-code Bot commented Jul 7, 2026

Copy link
Copy Markdown

PRD-148

PRD-18

@qltysh

qltysh Bot commented Jul 7, 2026

Copy link
Copy Markdown

2 new issues

Tool Category Rule Count
qlty Structure Function with many returns (count = 6): mapTask 2

Comment thread packages/workflow-executor/src/types/validated/step-definition.ts
Comment thread packages/workflow-executor/src/executors/guidance-step-executor.ts
@qltysh

qltysh Bot commented Jul 7, 2026

Copy link
Copy Markdown

Qlty


Coverage Impact

⬆️ Merging this pull request will increase total coverage on main by 0.01%.

Modified Files with Diff Coverage (3)

RatingFile% DiffUncovered Line #s
Coverage rating: A Coverage rating: A
packages/workflow-executor/src/executors/base-step-executor.ts100.0%
Coverage rating: A Coverage rating: A
...ow-executor/src/executors/summary/step-execution-formatters.ts100.0%
Coverage rating: B Coverage rating: A
...ages/workflow-executor/src/executors/guidance-step-executor.ts100.0%
Total100.0%
🚦 See full report on Qlty Cloud »

🛟 Help
  • Diff Coverage: Coverage for added or modified lines of code (excludes deleted files). Learn more.

  • Total Coverage: Coverage for the whole repository, calculated as the sum of all File Coverage. Learn more.

  • File Coverage: Covered Lines divided by Covered Lines plus Missed Lines. (Excludes non-executable lines including blank lines and comments.)

    • Indirect Changes: Changes to File Coverage for files that were not modified in this PR. Learn more.

@Scra3 Scra3 force-pushed the feature/prd-18-ai-assisted-and-full-ai-guidance-task branch from 720c832 to 492febc Compare July 7, 2026 15:08
Comment thread packages/workflow-executor/src/types/step-execution-data.ts
@Scra3 Scra3 force-pushed the feature/prd-18-ai-assisted-and-full-ai-guidance-task branch from 492febc to cd8665f Compare July 7, 2026 15:22
@Scra3 Scra3 marked this pull request as ready for review July 8, 2026 09:27
}

// An empty draft is not a submittable answer (even in Full AI) → degrade to manual input.
if (!draft.trim()) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Opus 4.8 (claude-opus-4-8): Should fixdraft is typed string, but invokeWithTools returns toolCall.args cast to T without validating it against the tool's zod schema (base-step-executor.ts:343-344). If the model emits a non-string response (a number, object, etc.), response ?? '' keeps that value and draft.trim() here throws a TypeError — and this line sits outside the try/catch at L39-46, so it surfaces as a generic "Unexpected error during step execution" (base-step-executor.ts:105) instead of the intended degrade-to-manual. That breaks the PR's "run never fails / degrade on bad AI output" guarantee for a malformed tool call. Coerce or guard the value (e.g. const draft = typeof raw === 'string' ? raw : '') or move the emptiness check inside the try, so this path degrades like the empty-draft case. No test covers a non-string tool arg.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed. askAiForResponse now coerces a non-string tool arg to '' (typeof response === 'string' ? response : ''), so a malformed AI response flows into the empty-draft path and degrades to manual instead of throwing a TypeError at draft.trim() outside the try/catch. Added a test: a { response: 123 } tool arg degrades to awaiting-input with pendingData: {} (no unexpected error). (pushed bb1f84f)

executionType: z.literal(Manual).default(Manual).catch(Manual),
// No `.catch`; default Manual (not AWC) — the orchestrator owns the legacy default, so the
// executor fallback only fires on a missing field, where "never call AI" is the safe direction.
executionType: z.enum([Manual, AutomatedWithConfirmation, FullyAutomated]).default(Manual),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Opus 4.8 (claude-opus-4-8): Preferential — dropping .catch(Manual) is a deliberate fail-loud choice (it matches the sibling trigger-action / load-related schemas and is covered by the "rejects an invalid executionType" test), but the failure mode is worth calling out: mapTask (step-definition-mapper.ts:34) calls .parse without wrapping, so an unrecognized executionType throws a raw ZodError (not InvalidStepDefinitionError). getAvailableRuns (forest-server-workflow-port.ts) only rescues instanceof WorkflowExecutorError, so such a run is silently dropped — logged, but not surfaced in malformed[] to the orchestrator. Only reachable if the server ever introduces a mode beyond today's 1:1 ServerStepExecutionTypeEnum, so risk is low. If you'd prefer fail-safe here: normalize unknown modes at the mapper, and update the now-inaccurate mapTask comment at step-definition-mapper.ts:22-23 ("each schema's .default().catch() handles missing or unsupported values" — no longer true for guidance).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged — keeping the fail-loud (no .catch) behavior for consistency with the sibling trigger-action / load-related schemas; the raw-ZodError-drop path is only reachable if the server ever emits a mode outside today's 1:1 ServerStepExecutionTypeEnum, so the risk is low and I'd rather not special-case guidance. I did fix the now-inaccurate mapTask comment (step-definition-mapper.ts:22-23) to reflect that the manual-accepting schemas drop .catch and reject out-of-enum values rather than coercing. If we later add a server mode, the cleaner move is a mapper-level normalization for all step types (out of scope here). (pushed bb1f84f)

… guidance step (PRD-18)

Widen GuidanceStepDefinitionSchema to the 3-way execution mode. In AI-assisted
the AI pre-fills the free-text response (persisted as pendingData with an
aiGenerated flag for the front badge) and the human submits; in Full AI the AI
writes and submits automatically (executionResult.generatedByAi). On AI failure
or an empty draft the step degrades to Manual (empty field, no auto-skip), and
the AI is never re-run on re-dispatch.

Hoist withAiAssist + logAiDegrade from LoadRelatedRecordStepExecutor into
BaseStepExecutor (2nd consumer), leaving load-related behavior unchanged.

Relates to PRD-18.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Scra3 Scra3 force-pushed the feature/prd-18-ai-assisted-and-full-ai-guidance-task branch from cd8665f to bb1f84f Compare July 8, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants