CI: test firefox beta release over stable one

To catch issues earlier.

Playwright's Webkit and Chromium builds do not include beta channels
but already point to pre-release branches (see e.g. https://playwright.dev/docs/browsers#chromium).

Author: larabr

.github/workflows/tests.yml

@@ -32,7 +32,7 @@ jobs:
         run: npx playwright install --with-deps chromium
 
       - name: Install Firefox
-        run: npx playwright install --with-deps firefox
+        run: npx playwright install --with-deps firefox-beta
 
       # do not install webkit on ubuntu, since the X25519 WebCrypto implementation
       # has issues, see https://github.com/openpgpjs/openpgpjs/pull/1829 .

README.md

@@ -129,11 +129,11 @@ const sessionKey = await decryptSessionKey({
 
 ## Testing
 Headless Chrome (or Chromium), Firefox and Webkit are used for the tests.
-To install any missing browsers automatically, you can run `npx playwright install --with-deps <chromium|firefox|webkit>`. Alternatively, you can install them manually as you normally would on your platform.
+To install any missing browsers automatically, you can run `npx playwright install --with-deps <chromium|firefox|webkit|firefox-beta>`. Alternatively, you can install them manually as you normally would on your platform.
 
 The available test commands are:
 
 - `npm run test`: all browsers are tested in headless mode.
-- `npm run test:ci`: same as `test`, but Webkit is skipped if not installed.
+- `npm run test:ci`: similar to `test`, but beta browser versions are used if the vendor provides them; also, Webkit is skipped if not installed.
 - `npm run test:debug`: opens up a browser instance in your default browser to allow debugging the tests live
 

test/web-test-runner.config.mjs

@@ -61,16 +61,21 @@ export default {
             browsers: [
                 playwrightLauncher({
                     ...sharedPlaywrightCIOptions,
+                    // the Chromium project is ahead of the branded browsers, when the world is on Google Chrome N,
+                    // Playwright already supports Chromium N+1.
                     product: 'chromium'
                 }),
                 playwrightLauncher({
                     ...sharedPlaywrightCIOptions,
-                    product: 'firefox'
+                    product: 'firefox',
+                    launchOptions: { channel: 'firefox-beta' }
                 }),
                 // try setting up webkit, but ignore if not available
                 // (e.g. on ubuntu, where we don't want to test webkit as the WebCrypto X25519 implementation has issues)
                 existsSync(playwright.webkit.executablePath()) && playwrightLauncher({
                     ...sharedPlaywrightCIOptions,
+                    // Playwright's WebKit is derived from the latest WebKit main branch sources,
+                    // often before these updates are incorporated into Apple Safari and other WebKit-based browsers.
                     product: 'webkit'
                 })
             ].filter(Boolean)