Skip to content

fix(agent): stop fabricated success confirmations (#363)#372

Open
AlanAAG wants to merge 1 commit into
V1.4.1from
fix/agent-fabricated-confirmations
Open

fix(agent): stop fabricated success confirmations (#363)#372
AlanAAG wants to merge 1 commit into
V1.4.1from
fix/agent-fabricated-confirmations

Conversation

@AlanAAG

@AlanAAG AlanAAG commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

CraftBot's agent was confirming actions it never performed: claiming it disabled an integration setting with no action call behind it, and claiming a Gmail "signature" was added when no signature capability exists anywhere in the codebase. Under user pushback, it escalated to inventing a new fabricated explanation on each retry instead of repeating the honest answer.

Root cause was three compounding gaps: no instruction to ground confirmation text in actual action evidence, an absolutist "never say you can't" prompt rule with no carve-out for genuine incapability, and stale/missing docs on what settings the agent can and can't actually change.

  • agent_core/core/prompts/context.py, action.py — add a non-negotiable honesty rule to every action-selection prompt: once a capability is genuinely checked and confirmed absent, say so plainly ("there's no way I can do that") instead of faking success; pushback from the user does not change the honest answer, so don't invent a new excuse on retry.
  • app/data/action/send_message.py — same reinforcement on the schema description, at the exact point the confirmation text gets composed.
  • agent_file_system/AGENT.md (+ app/data/agent_file_system_template/AGENT.md) —
    • Documents set_requirement for the first time (it was in use but entirely undocumented): check feasibility before locking a requirement in; only mark satisfied when grounded in real action evidence; flip to violated immediately if the user disputes it rather than silently retrying.
    • Corrects a claim that per-integration settings are editable via the six main app/config/*.json files (they aren't — most integrations have their own .credentials/<name>_config.json, or no runtime config at all).
    • New guidance for editing those per-integration config files directly: read-merge-write-verify, since a blind overwrite can wipe other saved fields (6 integrations have multi-field configs).
  • craftos_integrations/integrations/gmail/INTEGRATION.md, outlook/INTEGRATION.md — explicit "no signature feature exists, ever" note naming and banning the specific fabrications already observed in testing.
  • tests/test_prompt_honesty_rules.py — presence-guard tests (6 cases) for both the always-injected framework-prompt layer and the integrations have their own .credentials/<name>_config.json, or no runtime config at all).
    • New guidance for editing those per-integration config files directly: read-merge-write-verify, since a blind overwrite can wipe other saved fields (6 integrations have multi-field configs).
  • craftos_integrations/integrations/gmail/INTEGRATION.md, outlook/INTEGRATION.md — explicit "no signature feature exists, ever" note naming and banning the specific fabrications already observed in testing.
  • tests/test_prompt_honesty_rules.py — presence-guard tests (6 cases) for both the always-injected framework-prompt layer and the read-on-demand doc layer, so neither can silently regress.

Deploy note: the framework prompt changes (action.py, context.py, send_message.py) are loaded once at process start — a running CraftBot instance needs to be restarted to pick them up. The AGENT.md/INTEGRATION.md doc changes take effect immediately with no restart.

Test plan

  • python3 -m pytest tests/test_prompt_honesty_rules.py -v — 6/6 passing
  • python3 -m pytest tests/ -q — 101 passed; 4 pre-existing failures in test_trigger_router_and_parking.py confirmed present on clean main before this change (unrelated FakeLLM/prompt_name kwarg mismatch)
  • Live conversation test after restart: ask CraftBot to change a setting it can't (e.g. a nonexistent integration toggle) and confirm it says so instead of fabricating success
  • Live conversation test after restart: ask for a Gmail signature on a sent email, push back once, confirm the agent repeats the same honest answer instead of inventing a new explanation

Closes #363

@AlanAAG

AlanAAG commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator Author

Handles two specific error cases + general behaviour, if there are more specific issues where the agent wrongly confirms an action, please comment to be included in this pr

@AlanAAG AlanAAG requested a review from zfoong July 7, 2026 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant