Skip to content

Add Baseline CLI pipeline skeleton#337

Open
smsnedden wants to merge 1 commit into
cloudflare:mainfrom
smsnedden:baseline-pipeline-skeleton
Open

Add Baseline CLI pipeline skeleton#337
smsnedden wants to merge 1 commit into
cloudflare:mainfrom
smsnedden:baseline-pipeline-skeleton

Conversation

@smsnedden

Copy link
Copy Markdown
Contributor

Introduces the --baseline entry point and the initial Baseline pipeline infrastructure. When enabled, the pipeline runs after the performance navigation has completed, the browser has closed, and the HAR and metrics artifacts have been finalized.

The pipeline is intentionally minimal in this PR: it establishes option plumbing, execution ordering, error isolation, and the artifact-writing convention by producing baseline/meta.json. Feature detection, the separate detection navigation, Baseline status lookup, and final status-split reports follow in later PRs.

Changes

--baseline option

  • Adds a boolean --baseline CLI option, defaulting to false.
  • Threads the option through CLI normalization and the programmatic launchTest API.
  • Adds the option to LaunchOptions, DefaultOptions, CLIOptions, and the normalized internal TestConfig.
  • Preserves the normalized baseline: boolean guarantee through the browser runners.

Baseline pipeline (src/baseline.ts)

Adds runBaselinePipeline, invoked during post-processing after the HAR and metrics have been written.

For this initial slice, it writes:

baseline/meta.json

The manifest contains:

  • Tested URL
  • ISO timestamp
  • Telescope package version
  • Artifact schema version

The existing performance browser is not reused, and no separate detection browser is launched yet.

Artifact writer

Adds a reusable writeBaselineArtifact function that:

  • Namespaces all output beneath baseline/.
  • Supports nested intermediary paths such as baseline/detection/.
  • Produces deterministic JSON by sorting object keys.
  • Rejects paths that would escape the Baseline directory.
  • Accepts only JSON-compatible values through the shared JsonValue type.

Later slices can use the same writer for intermediary detection artifacts and the final reports split by Baseline status.

Error isolation

Baseline pipeline errors are logged without failing the performance test. Existing HAR, metrics, and other performance results remain available if Baseline processing fails.

Package version helper

Moves the existing package-version lookup from index.ts into a focused shared module so both the CLI version flag and Baseline metadata use the same implementation.

Tests

Adds coverage for:

  • --baseline appearing in CLI help.
  • CLI parsing with the flag enabled and omitted.
  • baseline: true, false, and omitted across the browser matrix using the local delay fixture.
  • Creation and contents of baseline/meta.json.
  • No Baseline directory when the option is disabled or omitted.
  • Byte-for-byte isolation of existing HAR and metrics artifacts.
  • Deterministic nested artifact output.
  • Rejection of paths outside the Baseline directory.
  • Pipeline execution after HAR and metrics creation.
  • Baseline errors being logged and swallowed without changing the successful performance result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant