Skip to content

feat(pos-app): Maestro E2E on web via Vercel preview#303

Open
ignaciosantise wants to merge 10 commits into
mainfrom
feature/e2e-maestro-tests
Open

feat(pos-app): Maestro E2E on web via Vercel preview#303
ignaciosantise wants to merge 10 commits into
mainfrom
feature/e2e-maestro-tests

Conversation

@ignaciosantise

@ignaciosantise ignaciosantise commented Jan 8, 2026

Copy link
Copy Markdown
Collaborator

What

Moves the POS app's Maestro E2E from the Android emulator to the web build (react-native-web, driven by Maestro's headless Chromium). CI no longer builds anything — it runs against the app's Vercel pos-demo Preview deployment for the PR, cutting build time significantly.

Running against the preview (not a static serve -s dist) is required because POS web payments go through the app's /api/* Vercel serverless proxy (dapps/pos-app/api/), which only exists on a real deployment. This lets the payment flows run on web too.

Flows (dapps/pos-app/e2e/)

Flow Covers
payment-flow Start payment → enter $0.01 → charge → QR loads → tap QR copies the payment link
payment-cancel Create $0.01 → QR loads → Cancel → back to a fresh amount screen
invalid-amount Charge stays "Enter amount"/disabled at empty & 0; enables on a non-zero amount
keypad Decimal rules (single ., max 2 fractional digits) + backspace

common/create-payment-qr.yaml is a shared runFlow prelude. Flows run one process per flow (fresh browser) for isolation.

App changes to make it testable

main's POS had almost no testIDs. Added them across home/amount/scan/settings and made Button/SettingsItem forward testID. The decimal key is key-decimal (Maestro treats id:/text: selectors as regex, so key-. matched any key).

CI (.github/workflows/ci_e2e_tests_pos_web.yaml)

  • Triggers on deployment_status, gated to a successful Preview – pos-demo deployment (never Production/dev), plus a workflow_dispatch for a manual preview URL.
  • Checks out the deployed commit's flows, installs Maestro under xvfb, runs each flow (retry-once), uploads artifacts, Slack on failure.
  • Replaces ci_e2e_tests_pos_android.yaml.

Notes / to confirm

  • deployment_status runs the workflow copy from main, so this file takes effect once merged; use workflow_dispatch to try it before then.
  • Payment flows rely on the pos-demo Vercel project env (EXPO_PUBLIC_DEFAULT_MERCHANT_ID, EXPO_PUBLIC_DEFAULT_CUSTOMER_API_KEY, EXPO_PUBLIC_API_URL) being set — used to seed a merchant + proxy to the pay API.
  • Not covered (matches app TODO): wallet-connects-and-pays + success screen (needs payment simulation).

🤖 Generated with Claude Code

- Add payment flow E2E test (new sale -> enter amount -> QR code -> open URL)
- Add testID props to Button component and key interactive elements
- Add GitHub Action workflow for running Maestro tests on Android
- Update AGENTS.md with testing guidelines:
  - Avoid testing mocked components
  - Use testID for E2E tests
  - Minimal comments rule
  - No unused variables rule
Copilot AI review requested due to automatic review settings January 8, 2026 18:27
@vercel

vercel Bot commented Jan 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
pos-demo Ready Ready Preview, Comment Jul 20, 2026 7:59pm
react-native-examples Ready Ready Preview, Comment Jul 20, 2026 7:59pm

Request Review

@claude

claude Bot commented Jan 8, 2026

Copy link
Copy Markdown

Claude finished @ignaciosantise's task in 1m 30s —— View job


Review Complete

Found 3 issue(s)

Issue 1: App ID mismatch between workflow and E2E test

ID: payment-app-id-mismatch-a7b3
File: dapps/pos-app/e2e/payment-flow.yaml:1
Severity: HIGH
Category: configuration

The E2E test file uses appId: com.reown.mobilepos but the workflow builds the app with a different package name. This will cause the test to fail as Maestro won't be able to find the app.

Recommendation: Verify the correct app ID from dapps/pos-app/app.json and update the E2E test file to use the matching package name. The app ID should match the value in expo.android.package or the build configuration.

Issue 2: Insufficient E2E test coverage for error scenarios

ID: payment-edge-case-coverage-2f91
File: dapps/pos-app/e2e/payment-flow.yaml:1
Severity: MEDIUM
Category: code_quality

The payment flow test only covers the happy path. Critical edge cases are not tested: invalid amounts (negative, zero with validation bypassed, exceeding limits), network failures during payment initiation, QR code generation failures, and merchant configuration errors.

Recommendation: Add additional test files for error scenarios:

# e2e/payment-flow-invalid-amount.yaml
- tapOn: { id: "key-0" }
- tapOn: { id: "charge-button" }
- assertVisible: { text: "Amount is required" }

# e2e/payment-flow-no-merchant-config.yaml
# Clear merchant settings first, then verify error handling

Issue 3: Hardcoded amount assertion may fail with formatting variations

ID: payment-amount-assertion-fragile-8c42
File: dapps/pos-app/e2e/payment-flow.yaml:35
Severity: LOW
Category: code_quality

The assertion text: "0.01 USD" is fragile because it depends on exact formatting. If the gateway adds spacing, currency symbols, or uses different decimal formatting (0.01 USD vs $0.01 vs 0.01 USD), the test will fail.

Recommendation: Use a more flexible assertion pattern or verify the amount is visible within a specific container:

# Option 1: Check for amount components separately
- assertVisible: "0.01"
- assertVisible: "USD"

# Option 2: Use regex if Maestro supports it
- assertVisible:
    text: "0\\.01.*USD"

Automated Checks

🔒 External Domain URL Detected (Non-blocking)
URL: https://get.maestro.mobile.dev
Files:

  • .github/workflows/ci_e2e_tests_pos_android.yaml:72
  • dapps/pos-app/e2e/README.md:9

This change introduces URLs pointing to external domains. Please verify that these external dependencies are intentional and review for potential security, privacy, or compliance implications. Approved company domains are: reown.com, walletconnect.com, walletconnect.org

Note: Maestro is a legitimate E2E testing tool. Consider pinning to a specific version rather than using the latest version from the install script for better reproducibility and security.


Comment thread dapps/pos-app/e2e/payment-flow.yaml Outdated
Comment thread dapps/pos-app/e2e/payment-flow.yaml Outdated
Comment thread dapps/pos-app/e2e/payment-flow.yaml Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds Maestro E2E testing infrastructure to the POS app, enabling automated end-to-end testing of key user flows on Android. The implementation includes test files, component instrumentation with testIDs, a GitHub Actions workflow for CI, and updated developer guidelines.

  • Adds Maestro E2E testing framework with payment flow test coverage
  • Adds testID props to interactive components for test automation
  • Implements CI workflow with Android emulator and Slack notifications

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
dapps/pos-app/e2e/payment-flow.yaml Defines E2E test for payment flow from sale creation to QR code generation
dapps/pos-app/e2e/README.md Provides documentation for running and creating Maestro tests
dapps/pos-app/components/button.tsx Adds testID prop support to Button component
dapps/pos-app/components/numeric-keyboard.tsx Moves testID from child elements to parent Button for better E2E targeting
dapps/pos-app/app/scan.tsx Changes QR tap behavior to open URL in browser and adds testID
dapps/pos-app/app/index.tsx Adds testIDs to home screen buttons
dapps/pos-app/app/amount.tsx Adds testID to charge button
dapps/pos-app/AGENTS.md Documents testing guidelines and coding standards
.github/workflows/ci_e2e_tests_pos_android.yaml Implements CI workflow for running E2E tests on Android emulator

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread dapps/pos-app/e2e/README.md Outdated
Comment thread dapps/pos-app/app/scan.tsx Outdated
Comment thread .github/workflows/ci_e2e_tests_pos_android.yaml Outdated
Comment thread .github/workflows/ci_e2e_tests_pos_android.yaml Outdated
Comment thread dapps/pos-app/e2e/payment-flow.yaml Outdated
Brings POS app up to Expo SDK 56 and the web-ready build (react-native-web,
web:build export, DesktopFrameWrapper, Skia canvaskit setup), plus the
rn_cli_wallet web-Maestro reference pattern (ci_e2e_walletkit e2e-web job +
walletkit-build-and-maestro composite action).

POS source files that the branch had modified for testIDs (app/index.tsx,
app/amount.tsx, app/scan.tsx, components/button.tsx, components/numeric-keyboard.tsx,
AGENTS.md) were reset to main's rewritten versions; the branch's stale testIDs
(new-sale-button, key-dot, charge-button, qr-code) will be re-added cleanly
against main's structure. Branch-only E2E files (e2e/, ci_e2e_tests_pos_android)
are preserved and will be reworked into the web flow.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ignaciosantise
ignaciosantise marked this pull request as draft July 20, 2026 18:01
Migrate POS E2E from the Android emulator to the web build (react-native-web
driven by Maestro's Chromium web driver) to cut CI time. CI triggers on the
pos-demo Vercel *Preview* deployment and runs Maestro against the preview URL,
which serves the app's /api/* payment proxy — so payment flows work too.

- Make the app testable: forward testID through Button/SettingsItem; add ids
  across home/amount/scan/settings (main had almost none). Decimal key uses
  "key-decimal" (Maestro treats id/text selectors as regex, so "key-." matched
  any key).
- Flows: payment-flow, payment-cancel, invalid-amount, keypad, plus a shared
  common/create-payment-qr prelude (run per-flow in isolated browsers).
- Replace ci_e2e_tests_pos_android.yaml with ci_e2e_tests_pos_web.yaml
  (deployment_status trigger, Preview-only, per-flow xvfb-run maestro).
- Docs: e2e/README.md covers the /api-proxy requirement + web gotchas.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ignaciosantise ignaciosantise changed the title feat(pos-app): Add Maestro E2E testing infrastructure feat(pos-app): Maestro E2E on web via Vercel preview Jul 20, 2026
- common/create-payment-qr.yaml: add the required Maestro config section
  (url: ${APP_URL} + ---); a runFlow subflow without it fails with
  "Config Section Required" (the payment flows were red on it).
- ci: trigger on pull_request and derive the Vercel preview URL from the
  branch alias (pos-demo-git-<branch>-reown-com.vercel.app) instead of the
  deployment_status immutable URL; runs from the PR branch and matches the
  URL in the Vercel PR comment. Length-guarded (>63-char label -> use
  workflow_dispatch).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- POS E2E workflow -> "🧪 POS - E2E Tests"; job label mentions POS App
- Release workflows -> "🚀 Release {AppKit,Mobile POS,WalletKit}"

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- 🧪 WalletKit - E2E Tests
- 🚀 Release Merchant POS
- 🚀 Release Mobile POS (PoC)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Match WalletKit: add ready_for_review to the pull_request types and gate the
job on draft == false, so the POS E2E (which hits the real /api payment proxy)
doesn't run on every draft push.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ignaciosantise

Copy link
Copy Markdown
Collaborator Author

@claude review

@claude

claude Bot commented Jul 21, 2026

Copy link
Copy Markdown

Claude finished @ignaciosantise's task in 3m 57s —— View job


Review

  • Gather context
  • Read changed files
  • Review CI workflow
  • Review E2E test flows
  • Review app code changes
  • Post findings

🔄 Incremental Review (full review — no prior Claude review on this PR)

Overall: This is a well-thought-out shift from Android-emulator E2E to Vercel-preview web E2E. The isolation-per-flow design, testID naming rationale (avoiding key-. regex collision), cancel-before-first-poll fix in scan.tsx, and the 63-char label guard are all good. A couple of issues to fix before merging:

Found 2 issue(s)

Issue 1: README documents wrong CI trigger

ID: e2e-readme-wrong-trigger-3c91
File: dapps/pos-app/e2e/README.md:7
Severity: MEDIUM
Category: code_quality

Context:

  • Pattern: README line 7 says the CI "triggers on the deployment_status event: when Vercel reports a successful pos-demo deployment", but the actual workflow (ci_e2e_tests_pos_web.yaml:13-19) triggers on pull_request events, not deployment_status.
  • Risk: Developers reading the README will misunderstand when tests fire. The deployment_status approach runs the copy from main (noted in PR description as the reason for switching), so the workflow was intentionally changed to pull_request + a polling wait for the Vercel preview. The docs were never updated.
  • Impact: Confuses contributors debugging why tests fired immediately on push (before Vercel finishes) or why the deployment_status event they're looking for doesn't exist.

Recommendation:

**CI** (`.github/workflows/ci_e2e_tests_pos_web.yaml`) triggers on `pull_request`
events (opened, synchronize, etc.): the job then polls the Vercel branch-alias URL
until it's reachable (up to 10 min), then runs the flows. A manual
`workflow_dispatch` against an explicit URL is also available.

Fix this →


Issue 2: Maestro CLI installed without version pin

ID: ci-maestro-unversioned-install-a4f1
File: .github/workflows/ci_e2e_tests_pos_web.yaml:92
Severity: MEDIUM
Category: security

Context:

  • Pattern: curl -Ls "https://get.maestro.mobile.dev" | bash installs whatever version is latest at CI time with no pinning.
  • Risk: A Maestro release with a breaking change to YAML syntax or web driver behavior will silently break tests. The install script itself is an external execution vector — a supply chain compromise of get.maestro.mobile.dev would execute arbitrary code on the runner.
  • Impact: Unreproducible test failures across runs; runners execute untrusted code.
  • Trigger: Every PR that touches dapps/pos-app/**.

Recommendation: Pin a specific version via the MAESTRO_VERSION env var that the install script respects:

- name: Install Maestro
  env:
    MAESTRO_VERSION: "1.40.0"   # update deliberately when upgrading
  run: |
    curl -Ls "https://get.maestro.mobile.dev" | bash
    echo "$HOME/.maestro/bin" >> $GITHUB_PATH

Fix this →


Previous Issues

ID Status
payment-app-id-mismatch-a7b3 ✅ Resolved — web flows use url: not appId:, no app build in CI
payment-edge-case-coverage-2f91 ℹ️ Acknowledged — invalid-amount.yaml and keypad.yaml cover the non-payment edge cases; payment error scenarios are noted as future work (valid tradeoff)
payment-amount-assertion-fragile-8c42 ℹ️ Confirmed still present, but intentional — "\\$0.01" in YAML becomes regex \$0.01; the . matches any char but in practice the app always shows $0.01, so no false positives

Inline Notes (non-blocking)

  • scan.tsx cancel fix (undefined → "requires_action") is a correct and well-documented bug fix.
  • Per-flow Maestro process isolation to avoid localStorage leakage is the right call.
  • The 63-char Vercel alias guard with a clear error message is nice defensive programming.
  • key-decimal instead of key-. to avoid Maestro regex wildcard matching is well-explained and correct.
  • app.json versionCode bump (27→28) follows the AGENTS.md requirement. ✓

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants