Skip to content

Run validate on pull requests (with approval gate for forks) - #1774

Draft
aatchison wants to merge 1 commit into
mainfrom
ci/validate-on-pull-request
Draft

Run validate on pull requests (with approval gate for forks)#1774
aatchison wants to merge 1 commit into
mainfrom
ci/validate-on-pull-request

Conversation

@aatchison

Copy link
Copy Markdown
Contributor

What changed?

  1. validate.yml now triggers on pull_request (base main) in addition to push.
  2. Concurrency group is scoped per PR / ref instead of a single global validate group.

Why?

Fork PRs are never validated today. validate.yml triggers only on push, and a fork PR pushes to the fork, never to this repo — so no run is ever created here. Confirmed on #1746: zero workflow runs for its branch, and zero runs repo-wide in action_required or waiting. The "Approve and run workflows" button had nothing to act on, because there was no run.

main is excluded from the push trigger as well, so merging doesn't validate either. The next workflow to touch merged code is deploy-staging.yml, which deploys to stage. Net effect: an external contribution's first real exercise is a staging deployment.

#1746 (npmpnpm) is the worked example — an entire frontend build-system rewrite, approved by three reviewers, with no check ever having run against it. It also carried a bug where pnpm run build -- --mode $APP_ENV silently dropped the mode and would have shipped a bundle with empty VITE_* values to S3. Caught by hand; CI wouldn't have caught it either, but CI is the layer that should have had a chance.

Is it safe to run fork code?

Yes, and this repo is already configured for it:

  • On pull_request from a fork, GitHub withholds every repository secret and forces GITHUB_TOKEN to read-only, regardless of the permissions: block.
  • fork-pr-contributor-approval is already set to all_external_contributors, so each fork run waits for a maintainer to click Approve and run workflows. This PR makes that existing setting actually do something.
  • Every job in validate.yml references zero secrets.* and zero vars.* — lint, build, vitest, pytest, nginx config test. There is nothing for untrusted code to reach even inside the sandbox.
  • This is pull_request, not pull_request_target. The dangerous pattern is pull_request_target + checking out PR code, which gets secrets and a write token. pr-merged.yml uses pull_request_target correctly and is untouched here.

Concurrency fix

concurrency:
  group: validate          # before: global
  group: validate-${{ github.event.pull_request.number || github.ref }}   # after
  cancel-in-progress: true

A bare group name is global across the repo, so two people pushing to unrelated branches silently cancel each other's checks. Adding a second trigger would have made that worse.

Open question for reviewers

Internal branches with an open PR now get two runs — one from push, one from pull_request — since the concurrency keys differ by design. Options:

  • Accept the duplicate (what this PR does): every internal push stays validated, at 2× runner cost while a PR is open.
  • Drop the broad push trigger, keeping pull_request + push to main: no duplicates, but internal branches aren't validated until a PR exists.

Left as a draft deliberately — happy to switch to the second shape if that's the team's preference.

Applicable Issues

Follow-up from review of #1746.

🤖 Generated with Claude Code

`validate.yml` only triggers on `push`, and fork PRs push to the fork rather
than to this repo -- so no external contribution has ever been CI-validated
before merge. `main` is excluded from the push trigger too, so merging doesn't
validate it either; the next workflow to touch the code is deploy-staging,
which deploys. PR #1746 (the npm -> pnpm migration) merged an entire build
system change with zero checks ever having run against it.

Add a `pull_request` trigger. Fork PR runs get no secrets and a read-only
GITHUB_TOKEN, and this repo already sets fork-pr-contributor-approval to
all_external_contributors, so each run waits on "Approve and run workflows".
Every job in this workflow references zero secrets or vars, so there is nothing
for untrusted code to reach even inside the sandbox.

Also scope the concurrency group per PR / ref. The bare `validate` group is
global, so concurrent pushes on unrelated branches cancel each other's checks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant