UK chatbot beta launch: blog post, homepage CTA, featured tools

[vahid-ahmadi]

One PR consolidating the UK AI chatbot launch work (supersedes #1070 and #1072).

Blog post — "AI chatbot for policymaking" (beta)

• New post at app/src/data/posts/articles/uk-chat-cant-make-up-numbers.md + cover image, dated 2026-06-27.
• Framed as a beta launch; short noun-phrase section/subsection headings; six-tools figure (centred, grouped) and architecture diagram; numbered architecture/loop steps; key phrases bolded.
• Citations deep-linked to exact sentences (AI Playbook, Wikipedia, POST, NAO).
• posts.json: title "AI chatbot for policymaking", beta subtitle, date 27 Jun.

Homepage

• HeroCTA: second button "Try the AI chatbot" (navy, distinct from teal "Enter PolicyEngine") → PolicyEngine UK Chat.
• FeaturedResearchBanner: four cards refreshed to the latest UK tools, led by UK CliffWatch tool.
• HomeBlogPreview: excludes the chatbot launch post from "Expert policy analysis" (it's a product announcement, not policy analysis).

Apps

• Renamed UK CliffWatch → "UK CliffWatch tool" in apps.json and the home banner. (The CliffWatch apps.json entry itself already merged via Add UK CliffWatch to the apps registry #1071.)

Fixes

• MarkdownFormatter: restore ordered/unordered list markers that Tailwind preflight reset (numbered steps now show their numbers).

Note: the POST citation deep-link is best-effort — that page blocks automated fetching, so the exact-sentence highlight is unverified (falls back to page top if it doesn't match).

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
policyengine-app-v2	Ready	Preview, Comment	Jun 29, 2026 4:07pm
policyengine-calculator	Ready	Preview, Comment	Jun 29, 2026 4:07pm
policyengine-calculator-next	Ready	Preview, Comment	Jun 29, 2026 4:07pm
policyengine-website	Ready	Preview, Comment	Jun 29, 2026 4:07pm

[anth-volk]

I think the core idea here is strong, but I would rework the article so it reads more like a product launch and design discussion than a full technical breakdown.

A few specific suggestions:

1. Strengthen the title. The current title feels a bit lackluster for a launch post. I would consider a structure like Introducing PolicyEngine UK Chat: ..., where the second half explains the product promise. For example, something along the lines of Introducing PolicyEngine UK Chat: AI tax and benefit answers grounded in microsimulation. The exact wording can change, but I think the title should immediately tell readers that this is a new product and why it is different.

2. Make the introduction work harder as a launch. I would not start with PolicyEngine UK Chat is now in beta. That states the status, but it does not really announce the product or explain why readers should care. The intro should probably lead with the product name, the user problem, and the novelty: people increasingly ask AI systems tax and benefit questions, but predictive AI on its own is not reliable enough for questions where the number is the point. PolicyEngine UK Chat is novel because it combines a conversational interface with deterministic calculations from PolicyEngine UK. The current sentence about other chatbots guessing while this one computes is directionally right, but it buries the product motivation and undersells how unusual this design is.

3. Add a clearer “what this product is” section right after the intro. Before getting into architecture, I would add a section that explains what the user can actually do with the tool: ask household-level questions, ask about policy parameters, explore reforms, understand assumptions, and get answers in plain English with calculations performed by the model. An example would help here. Even a short illustrative example such as a user asking how a threshold or benefit change affects a household, followed by the kind of answer the product returns, would make the launch feel much more concrete.

4. Use the PolicyBench article as part of the argument. We have a stronger evidence base here than a generic claim that LLMs hallucinate. PolicyBench shows that predictive AI is not 100% accurate in tax and benefit determinations when used without a deterministic model underneath it. That is the precise reason this product needs to exist. I would bring that into the motivation: the problem is not just hallucination in the abstract, but measurable unreliability in exactly the kind of tax-benefit reasoning users want this chat to perform.

5. Reframe the “how it works” section around the product design, not tool inventory. The most important design idea is the boundary: predictive AI handles interpretation, planning, and explanation; deterministic software handles the calculations and other checkable artefacts. That is the real product insight. I would make the overview broader and less centered on the exact names and count of tools. If we keep a tool list, it should be secondary and should be checked against the current uk-chat tool surface, since that surface is changing. The article should avoid aging every time we add a lookup or calculation tool.

6. Be careful with claims that sound absolute. Lines like every figure comes from a computation, answers can be cited and reproduced, or references to reviewed Python should only stay if they are literally true across all relevant paths. Otherwise I would tighten them to something like: when the system provides model-derived tax or benefit figures, it is designed to route those through deterministic PolicyEngine calculations or clearly state when it cannot. That preserves the product promise without overclaiming.

7. Use a light-mode product image. The current visual direction should fit the surrounding site. I would use a light-mode screenshot or rendered image of the UK chat tool itself, since this is a product launch article and the reader should immediately see the product. If the graphic stays, I would consider making it demonstrate the deterministic boundary more clearly: user question -> model interpretation -> PolicyEngine calculation/lookup -> answer with assumptions. Otherwise, I would lean toward dropping the graphic in favor of a concrete product screenshot and example.

8. Trim internal implementation details from the limitations section. Some of the current limitations language gets pretty deep into internal implementation details, like specific tool fallback behavior. That may be useful for engineering docs, but most readers need a shorter user-facing version: the product is in beta, results depend on assumptions and model coverage, it is not professional advice, and users should verify important results.

9. Give the piece a stronger conclusion. The ending should invite readers to try the tool, remind them it is in beta, and tell them exactly how to leave feedback. The feedback mechanism should be explicit: for example, an in-app feedback link, a GitHub issue, a contact email, or whatever mechanism we want users to use. Right now the conclusion does not quite convert the reader from understanding the design into trying the product and helping us improve it.

[anth-volk]

One additional point: I would change the article URL slug away from uk-chat-cant-make-up-numbers. It feels a bit informal/negative for a product launch and does not align well with the stronger Introducing PolicyEngine UK Chat... framing. A launch-oriented slug like introducing-policyengine-uk-chat or policyengine-uk-chat-beta-launch would fit the article better.

[anth-volk]

Vahid, what do you think about treating the following as the basis for the article?

I think this is closer to the product-launch framing: it leads with what UK Chat is, keeps the design discussion focused on the predictive/deterministic boundary, and trims the deeper implementation details. We could still use more detail on what makes the tool unique, and we could possibly include more about policyengine-uk-compiled, though I am not yet sold that the launch post needs that level of engine implementation detail.

# Introducing PolicyEngine UK Chat: AI tax and benefit answers grounded in microsimulation

PolicyEngine is excited to announce PolicyEngine UK Chat, a new beta conversational interface for UK tax and benefit questions that connects AI to the PolicyEngine microsimulation model.

The product starts from a simple problem. People increasingly ask AI systems questions about tax and benefit reforms. Large language models can understand these question and explain answers clearly, but tax and benefit questions often require a degree of accuracy that models do not yet offer: the figure needs to come from a model whose assumptions can be inspected and verified.

PolicyEngine UK Chat is designed around that split. The AI reads the question, works out what kind of policy task the user is asking for, and writes the explanation based upon verifiable, deterministic calculations, parameter lookups, and output formatting from PolicyEngine UK. In this way, PolicyEngine UK Chat leverages the strengths of large language models, grounded by the accuracy and testability PolicyEngine UK offers.

Try the beta at [policyengine.org/uk/chat](https://policyengine.org/uk/chat).

## What it does

PolicyEngine UK Chat lets users ask UK tax and benefit questions in plain English, underpinned by the accuracy and verifiability of PolicyEngine UK. Users can:

- Look up current policy settings, including allowances, rates, thresholds, benefit amounts, limits, and tapers
- Calculate illustrative synthetic households to see modelled taxes, benefits, and net income under stated assumptions
- Explore parametric reforms across the UK population, including budgetary impact, programme changes, poverty measures, decile impacts, winners and losers, and caseloads
- Turn computed outputs into charts when a visual comparison is clearer than prose or a table
- Ask follow-up questions in the same thread, moving from a rule lookup to a household example or reform analysis without switching tools

<!-- OPTIONAL POLICYBENCH SECTION START
This section is optional. Remove from this heading through OPTIONAL POLICYBENCH SECTION END if the final launch post should avoid the benchmark discussion or keep the article shorter.
-->

## Why this needs more than a language model

PolicyEngine has been measuring this problem directly. In [PolicyBench](https://policyengine.org/us/research/introducing-policybench), PolicyEngine evaluated how accurately and efficiently AI models can compute taxes and benefits from household prompts without tools or lookups and scored their answers against deterministic PolicyEngine outputs.

The result is clear: unaided predictive AI is not yet accurate enough to calculate taxes and benefits without a deterministic model powering it. In the PolicyBench launch results, the top model, OpenAI GPT-5.5, came within $1 of PolicyEngine on 80.3% of scored outputs. Computed amounts were especially difficult. Federal and state income tax before credits scored far lower than many eligibility flags, because they require sequencing income concepts, thresholds, exclusions, and credits correctly.

PolicyEngine UK Chat addresses this finding by placing the deterministic PolicyEngine UK model at its heart. AI can interpret the user's question, but the tax and benefit facts come from PolicyEngine calculations, model parameters, and clearly stated assumptions.

<!-- OPTIONAL POLICYBENCH SECTION END -->

## How the product is designed

The central design principle is the boundary between predictive AI and deterministic software.

The predictive portions are the parts where language models are useful: interpreting the user's wording, deciding which type of analysis is being requested, drafting explanations, and suggesting follow-up questions. Those tasks are open-ended and language-heavy.

The deterministic elements are the parts where users need verifiable outputs: validating request shape, looking up parameters, constructing household inputs, running simulations, producing chart data, and returning structured results. These tasks should not depend on model memory.

That boundary is enforced in the runtime. If the opening question is not ready for calculation, a gateway step can route it to a lightweight response, a scope explanation, or a clarification question.

When a question is ready for computation, the model then enables calculation tools and a generated reference for the installed engine. That reference is built from the engine itself, including capabilities and parameter schema, so the model's view of the available calculation surface is tied to the deployed version of PolicyEngine rather than to a hand-written prompt that can drift.

The heart of PolicyEngine UK Chat is its scoped tool surface, which serves as the boundary between the predictive AI layer and PolicyEngine's own deterministic calculations. These tools ensure accurate outputs and lower model runtime. The current surface includes tools for parameter lookup, illustrative household calculation, economy-wide simulation, reform validation, and chart generation, thought the exact list of tools will change as the beta evolves.

## Try it out

PolicyEngine UK Chat is available now in beta. Try it with a tax or benefit question, a household scenario, or a reform you want to understand.

When an answer looks wrong, incomplete, or confusing, use the Report issue button in the chat. It opens a prefilled GitHub issue with a shared thread and your note, so we can inspect the question, the tools used, the calculated outputs, and the final explanation.

[vahid-ahmadi]

@anth-volk thanks — this was really helpful. I've reworked the post using your draft as the basis and pushed it (a98694c2). Point by point:

• Title — now "Introducing PolicyEngine UK Chat: AI tax and benefit answers grounded in microsimulation". Leads with the product and the promise.
• Intro as a launch — opens with the product name, the user problem (people increasingly ask AI tax/benefit questions), and the novelty (pairing a conversational interface with deterministic PolicyEngine UK calculations) instead of the beta status.
• "What this product is" — new What it does section listing concrete capabilities (look up settings, calculate illustrative households, explore parametric reforms, charts, follow-ups in-thread).
• PolicyBench — added a Why this needs more than a language model section that grounds the motivation in PolicyBench's measurable unreliability. I corrected the stat to match the post: GPT-5.5 matched PolicyEngine exactly on 80.3% of outputs, with income-tax-before-credits the hardest cases.
• "How it works" reframed — rebuilt around the predictive/deterministic boundary as the core insight. The tool surface is now secondary and explicitly flagged as changing during beta, so the post won't age each time we add a tool.
• Absolute claims — softened. No more "every figure comes from a computation" / "reviewed Python" framing; the boundary is described as what the runtime enforces.
• Limitations — trimmed to a short user-facing version (beta, depends on assumptions/coverage, not professional advice, verify important results); dropped the internal tool-fallback detail.
• Conclusion — Try it out section invites readers to try the beta and points to the in-app Report issue button (prefilled GitHub issue with the thread).
• Slug — renamed uk-chat-cant-make-up-numbers → introducing-policyengine-uk-chat across the article, cover image, posts.json, and the HomeBlogPreview filter.

A couple of things worth a look:

• Chat link — I pointed "Try the beta" at https://policyengine-uk-chat.vercel.app/ to match the homepage CTA's href. If we have a policyengine.org/uk/chat route planned, happy to switch to that.
• Visual — rather than a product screenshot I couldn't regenerate, I replaced the two dense SVGs with a single light-mode boundary diagram (question → AI interprets & plans → PolicyEngine computes → answer with assumptions). If you'd prefer a light-mode product screenshot as the cover/inline image, that's an easy swap once we have one.

🤖 Generated with Claude Code

[anth-volk]

@vahid-ahmadi a couple minor follow-ups:

• I'd add a brief segment on the PolicyBench description indicating that it was for the US
• Change "When an answer looks wrong..." to "If an answer..."

[vahid-ahmadi]

@anth-volk both done in 38ab5b1:

• PolicyBench is US-based — the "Why this needs more than a language model" section now opens by describing PolicyBench as "an evaluation built on the US tax and benefit system," and closes with a bridge sentence so the US figures don't read as out of place in a UK launch post: "The benchmark covers US policy, but the lesson carries directly to the UK: the harder the computation, the less a language model can be trusted to do it unaided."
• "When" → "If" — the feedback CTA now reads "If an answer looks wrong, incomplete, or confusing, use the Report issue button…"

[MaxGhenis]

CI is green and I don't see code-correctness blockers (agent-assisted review). The post renders (website posts.json/authors.json/articles are symlinks to the app-side files this PR edits), both bylines resolve, all four banner links return 200 on prod, and the PolicyBench figures match the live post (GPT-5.5 first at 80.3% exact, US-based, income-tax-before-credits hardest). HeroCTA uses design tokens, is UK-gated, and opens in a new tab safely.

Main thing: this PR is the launch, not a staged setup. On merge + deploy, three things go live immediately with no flag or date gate — the "Try the AI chatbot" button on the UK homepage, the refreshed featured banner, and the post at /uk/research/introducing-policyengine-uk-chat (also in the /uk/research list and sitemap). So we shouldn't merge until we want it public.

Decisions to make first:

1. The CTA and the post both point at https://policyengine-uk-chat.vercel.app/. policyengine.org/uk/chat is a 404 — no productionized route yet. The alias works, but linking the public homepage and a launch post to a bare *.vercel.app is off-brand, skips our analytics/SEO, and cuts against our multizone convention. I'd rather stand up /uk/chat as a multizone (appZoneRoutes.ts) or a custom domain first; if we ship the vercel.app link for beta, let's do it knowingly. Either way, make CHAT_URL an env var or internal path like CALCULATOR_URL so moving it isn't a code change.
2. The cover image is a dark-mode screenshot; the earlier ask was light-mode and the inline diagram is light. Real product screenshot is the right call — just recapture it light to fit the research surface.
3. Post date is 2026-06-30, already a week old. No future-date gate, so it publishes on merge but sorts below anything dated later. Set it to the actual launch day at merge.

Copy: drop "only" in "matched PolicyEngine exactly on only 80.3%" — the 47.1%/53.3% hardest-case numbers already make the point. "PolicyEngine is excited to announce" is the one line leaning promotional against our neutral tone; fine for a launch, but "PolicyEngine has launched" is tighter.

Non-blocking: HomeBlogPreview excludes the post via a hardcoded slug (could key off a tag); branch is 16 behind main, worth updating before merge. Also the title and date changed after the last review round, so the current head hasn't had a sign-off from that reviewer.