Skip to content

Fictional default OpenAI model IDs cause change summaries and triage to crash with 404 error #17

Description

@Adityakk9031

Describe the bug

The default model options configured for the openai and vercel_gateway AI providers are set to fictional/mock model IDs (gpt-5.4-nano, gpt-4.1-nano, and gpt-4.1-mini). Because these models do not actually exist on the OpenAI API, any check that triggers an AI change summary or triage using the default configuration fails immediately with a model-not-found HTTP 404 error.

Steps to reproduce

  1. Configure an OPENAI_API_KEY on the server or in per-account Settings.
  2. Enable AI Change Summary on any monitor and run a check that detects a change.
  3. The LLM completion request fails, throwing a model-not-found error, and no change summary is generated.

Expected behavior

The default models and recommendations should point to real, production-ready, and cost-effective models hosted by OpenAI (such as gpt-4o-mini as the default/recommended, and gpt-4o).

Code Location

In ai-models.ts:L5-L16:

export const AI_MODELS_BY_PROVIDER: Record<AiProvider, AiModelOption[]> = {
  openai: [
    { id: "gpt-5.4-nano", label: "GPT-5.4 Nano (recommended)" },
    { id: "gpt-4.1-nano", label: "GPT-4.1 Nano" },
    { id: "gpt-4.1-mini", label: "GPT-4.1 Mini" },
  ],
  vercel_gateway: [
    { id: "openai/gpt-5.4-nano", label: "GPT-5.4 Nano (recommended)" },
    { id: "openai/gpt-4.1-nano", label: "GPT-4.1 Nano" },
    { id: "openai/gpt-4.1-mini", label: "GPT-4.1 Mini" },
  ],
};

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions