Migrate frontend & e2e from npm to pnpm - #1746
Conversation
04bfae2 to
4793c63
Compare
npm to pnpm
npm to pnpmnpm to pnpm
|
Tagging @davinotdavid for review as the person who opened the issue and @rwood-moz as our e2e test expert. On my initial scan through, it looked good. |
|
Thanks for the tag! Tagging @devmount as I believed he had a plan queued to tackle this so it would be nice to compare notes to check completeness. Two parts the worry me a little bit:
|
|
@davinotdavid That's fair, and I appreciate the worry 🙂 To reduce scope and complexity, I've opened #1753 . |
devmount
left a comment
There was a problem hiding this comment.
@MooseTheRebel Thank you so much for taking the time and contribute to this repo! I have two notes below that apply to most of the workflow files and need approval from @rwood-moz.
Also these change requests:
- If we move to pnpm, we should also update the docs in frontend/README.md and test/e2e/README.md.
- We're currently still building the frontend with
npm installiirc (frontend/Dockerfile). We should use pnpm here too for consistency and have it available in the frontend container. There is also a frontend/deploy.dockerfile, @davinotdavid do you know if we still use this or can we delete it 🤔
rwood-moz
left a comment
There was a problem hiding this comment.
Thank you for your patch @MooseTheRebel.
From an E2E test point of view this LGTM. I pulled the PR onto my machine and ran E2E tests locally as well as in BrowserStack via pnpm, and they ran successfully. The GHA workflow changes also LGTM.
My only request is to please update the test/e2e/README as @devmount also mentioned.
Looks like you have some merge conflicts and you'll need to rebase, and also please await approvals from the others, but I'm approving in regards to the E2E changes. Thanks again!
aatchison
left a comment
There was a problem hiding this comment.
Review from Platform/Infra — thanks for driving the pnpm migration! 🙏
Ran a broad merge-safety + cross-repo build review. One critical blocker for our EKS/GHCR image build, a few merge-gating items, and two inline engines suggestions below. Note up front: allowBuilds: (map form) is correct pnpm 11 syntax — no change needed there.
🔴 Blocking — breaks our multi-arch GHCR / EKS build
This PR deletes frontend/package-lock.json, but the container builds still use npm and fail once it's gone:
frontend/deploy.dockerfile→npm ci(the multi-arch GHCR image that feeds our EKS deploy + Kargo promotion)frontend/Dockerfile→npm install(local dev)
Neither file is in this PR's diff, so I can't inline-suggest them, but they must migrate in lockstep. Proposed for frontend/deploy.dockerfile (replace the npm ci / npm run build steps):
RUN corepack enable
RUN cd /build/frontend && pnpm install --frozen-lockfile
RUN cd /build/frontend && pnpm run buildand frontend/Dockerfile (local dev): RUN corepack enable && pnpm install --frozen-lockfile. The GHCR workflows themselves need no change (docker-build only). Happy to open a companion PR with these so they land as a matched set.
🟠 Before merge
- Rebase onto
main+ regeneratepnpm-lock.yaml. The branch predates #1749's security bumps (eslint,@sentry/vite-plugin,@typescript-eslint/*,@vue/test-utils); merging as-is reverts them, and there's apackage-lock.jsonmodify/delete conflict. - Prove a clean install. pnpm 11's
strictDepBuildsdefaults totrue, so an incompleteallowBuildslist makespnpm install --frozen-lockfilefail hard (ERR_PNPM_IGNORED_BUILDS). CI hasn't run on this PR yet — please confirm a clean install + build for bothfrontend/andtest/e2e/, especially after the rebase regenerates the lockfile. test/e2eoverridesis silently dropped. Top-leveloverridesis an npm-only field; pnpm ignores it, souuidis no longer pinned to11.1.1(resolves to 8.x/9.x). Move it underpnpm.overridesinpackage.json(oroverrides:inpnpm-workspace.yaml) and regenerate the lock.- Rollback runway: consider keeping
package-lock.jsonfor one release as an escape hatch, and splitting the Node 20→24 bump into its own PR so the runtime upgrade and the package-manager migration can revert independently.
🟡 Nits
- Duplicate
@typescript-eslinttoolchain: addingtypescript-eslint@8.61.1while the explicit@typescript-eslint/eslint-plugin/parser@8.28.0pins remain resolves two full toolchains — drop the explicit pins or align versions. frontend/README.md,test/e2e/README.md, andfrontend/Dockerfilestill reference npm; nocorepack enableonboarding note.- Nice catch promoting the
@eslint/js/typescript-eslintphantom deps — exactly right for pnpm's strictnode_modules.
Inline engines suggestions below. Thanks again — glad to pair on the Dockerfile migration.
1465582 to
0160787
Compare
|
Hi all 👋 Thank you again for the open source work you do and for the detailed feedback on this PR! Hopefully, the changes I made match what you were hoping to see. If not, please let me know and I'm happy to make additional updates. The gist I created (migrate-to-pnpm.sh) might have been a bit of over-engineering. I started on it mostly to help myself with an issue on a different project 🙂 I try to keep my diffs small and my engineering right-sized, so please don't take this PR as a signal of my typical style. The linked PR is more my speed, and it's available for you all if you prefer just that smaller change. Let me know if there's anything I can do to help with this PR, or a different issue entirely where I could contribute! EDIT: Oh, also, I suppose someone will have to approve github actions to run. Apologies in advance if any of them fail (hopefully not 🤞 ). |
|
@MooseTheRebel Taking another look. Thanks for this effort! |
There was a problem hiding this comment.
Re-review — one new blocker.
🔴 Five suggestions inline below
Two nits with no inline anchor: .dockerignore's header comment still describes npm ci, and package.json's format script still uses npx.
Heads-up: green CI won't clear the --mode bug — it wouldn't catch it either.
aatchison
left a comment
There was a problem hiding this comment.
Verified on f3bcdbf0 — approving. ✅
Re-ran the checks locally rather than reading the diff: --mode now reaches vite in both deploy jobs (built with the real APP_ENV values, stage and prod — the stage Sentry DSN + PostHog key and the prod URLs bake in correctly), vitest now actually receives --run, clean pnpm install --frozen-lockfile, lint + build + 97/97 tests, and both images rebuild end to end.
CI still hasn't been authorized on this PR, so this approval rests on that local run rather than on green checks. @devmount's earlier change request is what's still gating the merge.
f3bcdbf to
427d614
Compare
427d614 to
51a2e5d
Compare
|
@aatchison That was fast! 😄 Thank you for the detailed feedback, and I'm sorry I didn't catch those things before. I took the additional step of simulating a "red/green" test locally, using act, and the following workflow files (attached below):
You can try these yourself, using: # Reproduce and verify the fix for the build --mode bug (deploy-staging.yml, both the
# stage and prod jobs use this same pattern). Runs the buggy command first (red),
# then the fixed command (green).
act workflow_dispatch -W mode-bug.yml --container-architecture linux/amd64
# Reproduce and verify the fix for lint -- --fix and test -- --run
act workflow_dispatch -W dashdash-bug.yml --container-architecture linux/amd64Both confirmed the bugs as you described — I also fixed the two nits (.dockerignore comment, package.json format script). |
|
@devmount Sorry for the spam, I tried using the "Re-request review" button to ping you, but it seems like Github isn't doing anything when I click that button 🤔 Please re-review this PR and let me know if there are any changes needed. Thank you again! 🙂 |
devmount
left a comment
There was a problem hiding this comment.
Thanks a lot for all your work on this! I'm currently on vacation and approved without having looked at the changes again to enable the merge.
|
CI has now genuinely run against this tree: fork PRs never trigger |
What changed?
Moves
frontend/andtest/e2e/fromnpmtopnpm. I rebuilt the branch and force-pushed onto a freshmain, so it's now 6 commits:Fix phantom devDependencies for pnpm compatibility— cherry-picked from Fix phantom devDependencies for pnpm compatibility #1769. Adds@eslint/jsandtypescript-eslintas explicitdevDependenciesinfrontend/, so pnpm's strictnode_modulesdoesn't break on eslint's config.Migrate frontend and e2e to pnpm— the core migration (pnpm import, newpnpm-lock.yaml/pnpm-workspace.yaml, droppedpackage-lock.json) for bothfrontend/andtest/e2e/, plus the matching CI workflow changes (validate.yml,deploy-production.yml,deploy-staging.yml, and the e2e nightly workflows):pnpm/action-setup,cache: 'pnpm', and the Node 20→24 bump pnpm 11 requires.Fix Dockerfiles and e2e uuid pnpm override— migratesfrontend/Dockerfileandfrontend/deploy.dockerfileto pnpm, and movestest/e2e's npm-onlyoverridesfield (pinninguuid) topnpm-workspace.yaml, since pnpm silently ignoresoverridesinpackage.json.Bump engines to Node 22, dedup typescript-eslint— raisesengines.nodeto>=22.13.0(pnpm 11 / vite 8's actual floor) in bothfrontend/andtest/e2e/, and drops the@typescript-eslint/{eslint-plugin,parser}pins that duplicate the bundledtypescript-eslintpackage.Update frontend and e2e READMEs for pnpm— updates both READMEs' install/run instructions fromnpm/npxtopnpm.Bump checkout and setup-node actions to v7— bumpsactions/checkoutandactions/setup-nodetov7across the 8 workflow files touched above. Keepscache: 'pnpm'explicit —setup-node's automatic caching via thepackageManagerfield only applies when it's set tonpm, not pnpm.Why?
Quoting @davinotdavid from the issue description:
Reviewer feedback addressed
From the previous round of review:
deploy.dockerfile/Dockerfilestill use npm — fixed in commit 3.test/e2eoverridessilently dropped by pnpm — fixed in commit 3.main+ regenerate lockfiles — done; branch is rebuilt directly on top of currentmain.pnpm install --frozen-lockfile— verified for bothfrontend/andtest/e2e/at each step.engines.nodefloor too low for pnpm/vite — fixed in commit 4.@typescript-eslinttoolchain — fixed in commit 4.frontend/README.mdandtest/e2e/README.md— fixed in commit 5.actions/checkout/actions/setup-nodeto a current version — fixed in commit 6.davinotdavid: isdeploy.dockerfilestill used? — yes: confirmed by aatchison's review, it feeds the multi-arch GHCR image for the EKS deploy + Kargo promotion pipeline.Reproduce this PR yourself
Commits 2, 3, 4, and 6 above were generated by
migrate-to-pnpm.sh. I wrote this as a scripted-patching experiment for a different project and figured it fit here too. I'm also trying to be careful about trust & security. I didn't want to hand over a pnpm lockfile I generated on my machine and ask you to take it on faith 🙂You can run the script yourselves to generate the lockfiles, not just reviewing a diff of ones I made. Tested against
main@e9cca499— ifmainhas since diverged non-trivially, the cherry-pick step fails with a clear message rather than silently producing something different from this PR. Requires Node >=22.13.0 and pnpm >=10 (e.g. viacorepack enable); the script checks both and fails before touching any files if they're missing.Recreate the diff, uncommitted:
This leaves you with one uncommitted diff (staged + unstaged) matching this PR exactly. Nothing is committed for you, so you can review,
git diff, or amend as you like.Or recreate the exact commits (needs local
gitsetup — creates local commits only, pushes nothing):This reproduces the PR's 6 commits, including the #1769 cherry-pick, so you can step through the migration with
git log -p.The script cherry-picks #1769's fix as step 0, then runs the pnpm migration, the Dockerfile/overrides fixes, the engines/dedup cleanup, the README updates, and the actions version bump. Each step checks whether it's already applied and skips if so, so it's safe to re-run — but it expects a clean working tree and won't run around unrelated local changes.
Limitations and Notes
git reset --hard HEAD && git clean -fd(uncommitted mode) or resetting to the pre-migration commit (commit mode) — same idea as reverting this PR or reviving the files from git history.node-version: 20tonode-version: 24— pnpm 11 requires Node ≥22.13 and fails outright on Node 20. Per rwood-moz's review, this is safe: the E2E jobs run in BrowserStack (not installing Playwright browsers locally), where the Node 24 / Playwright 1.59 interaction that blocks local browser installs doesn't apply.Applicable Issues
#1731
Screenshots
N/A