feat(playwright-debug)!: add playwright-debug plugin for diagnosing failing tests#10
Merged
zizzfizzix merged 30 commits intomainfrom Mar 10, 2026
Merged
Conversation
Shift focus from one-off interactive browser automation to writing persistent, CI-ready E2E test suites using @playwright/test. Key changes: - SKILL.md: Complete rewrite with project-exploration-first workflow, @playwright/test patterns (test/expect/describe), playwright.config.ts generation, GitHub Actions + GitLab CI templates, and Page Object Model examples. Headless by default; tests written to project directory, not /tmp. - run.js: Replaced raw-script executor with a test runner that calls `npx playwright test` in the target project, with project-root detection, @playwright/test installation check, and spec-file targeting support. - package.json: Bump to v5.0.0, replace `playwright` dep with `@playwright/test`, update description and keywords. - plugin.json: Bump version and update description/keywords to match. https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
run.js:
- Remove forced CI=true override — let callers control the env
- Exit with 1 (not 0) when process is killed by a signal (status null)
SKILL.md:
- Fix auth fixture to use `authenticatedPage` name instead of shadowing
the built-in `page` fixture (antipattern that prevents unauthenticated
test cases in the same file)
- Add missing contentType: 'application/json' to route.fulfill example
API_REFERENCE.md:
- Replace raw playwright Basic Automation example (headless:false, slowMo)
with @playwright/test structure consistent with CI-first approach
- Replace removed test.describe.parallel with fullyParallel config pattern
- Fix GitHub Actions steps from v3 to v4; add browser install and artifact upload
- Replace broken PW_HEADER injection docs (run.js no longer wraps scripts)
with standard playwright.config.ts extraHTTPHeaders pattern
- Fix Infinite Scroll example to use waitForFunction instead of waitForTimeout
- Fix Accessibility example: replace unmaintained axe-playwright with
@axe-core/playwright (official Deque package)
package.json:
- Fix engines to node>=18 (required by @playwright/test v1.49)
- Restore browser install in setup script (regression from v4.1.0)
marketplace.json:
- Sync version (4.1.0 → 5.0.0) and update description/keywords to match
the CI test suite writing focus
lib/helpers.js:
- Add header comment clarifying helpers must be explicitly imported
(no longer auto-injected by run.js)
- Remove detectDevServers — irrelevant for CI suites where the server
is managed by playwright.config.ts webServer config
- Fix require('playwright') → require('@playwright/test')
https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
@playwright/test: 1.49.0 → 1.58.2 GitHub Actions: checkout/setup-node/upload-artifact v4 → v6/v6/v7 Node.js in CI configs: 20 → 22 GitLab CI Playwright image: v1.49.0-jammy → v1.58.2-jammy @axe-core/playwright: pin to ^4.11.1 in install instructions API_REFERENCE.md: fix install check (playwright → @playwright/test), add BASE_URL env var pattern and headless:true to Basic Configuration, fix reporter from plain 'html' to [list, html] tuple https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
Replace the old general-purpose automation README with content that reflects the current purpose: writing persistent, CI-ready @playwright/test suites for a user's project. Key changes: - New tagline and intro describing CI test suite writing - Updated features list (no more headless:false, /tmp, run.js executor) - Rewritten "How It Works" for the explore→write→commit workflow - Updated Quick Start and Usage Examples with CI-focused prompts - Added "What Gets Written to Your Project" table - Updated dependencies (Node>=18, @playwright/test 1.58.2) - Rewritten Troubleshooting (removed "browser doesn't open", added base URL and CI-specific issues) - Removed stale Configuration section (headless:false, slowMo:100ms) - Removed Agent Skills spec references (unrelated to current scope) https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
SKILL.md: - Rewrite Step 1 to use Read/Glob/Grep tools (not bash cat/ls) and explicitly read app source: routes/pages, auth components, key flows, existing test coverage — produces tests tailored to the real app - Add BASE_URL constant to playwright.config.ts template so port is defined once and both use.baseURL and webServer.url reference it - Add storageState globalSetup pattern (logs in once, reuses across all tests) — far faster in CI than per-test fixture login - Fix auth fixture: use correct Page type import instead of typeof base['_fixtures']['page'] (private API, breaks TypeScript strict) - Rename fixture from authenticatedPage to loggedInPage for clarity - Fix .gitignore guidance: remove /playwright/.cache/ (wrong path, wrong advice) and replace with /e2e/.auth/ for saved auth tokens; add CI browser cache instructions using actions/cache@v4 - Add Step 6 failure guidance: how to debug selectors, URL mismatches, timing issues, and what NOT to do (weakening assertions) helpers.js: - Fix scrollPage: remove waitForTimeout (anti-pattern per skill's own tips) - Fix authenticate: replace removed page.waitForNavigation() with page.waitForURL() (waitForNavigation was removed in Playwright 1.46) - Fix safeClick: use locator API instead of legacy page.click(selector) - Fix safeType: use locator API + pressSequentially instead of page.type - Fix extractTexts: use locator.evaluateAll instead of page.$$eval - Update module header to mark launchBrowser/createContext/getExtraHeadersFromEnv as legacy/low-value for @playwright/test suites API_REFERENCE.md: - Fix testDir './tests' → './e2e' (inconsistent with SKILL.md) https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
helpers.js deleted: - Every function was a thin wrapper over the locator API (safeClick, safeType, extractTexts) or dead code (getExtraHeadersFromEnv, createContext) or superseded by globalSetup+storageState (authenticate) or silently swallowed failures (waitForPageReady) or a one-liner (scrollPage, takeScreenshot) - authenticate used removed page.waitForNavigation() API - handleCookieBanner divided timeout across 8 selectors → 3s overhead per test - retryWithBackoff (the only genuinely useful function) is 15 lines; inline it in globalSetup when needed - lib/ directory removed as now empty SKILL.md: - webServer.command now uses production server in CI (npm run start) and dev server locally; add framework-specific examples (Vite=preview, CRA=serve) - GitHub Actions workflow: add explicit 'npm run build' step before tests so build failures appear as clear build errors, not confusing webServer timeouts - Fix GitHub Actions versions: checkout/setup-node/upload-artifact v6/v7 (non-existent) → v4 (current stable); these wrong versions would fail CI - Add browser cache step (actions/cache@v4) to avoid re-downloading browsers - Add CI Environment Variables section: table of required secrets, where to set them in GitHub/GitLab, instruction to read .env.example and tell user which secrets to configure - Step 1 Package & Config: add build/start script extraction and .env.example reading to identify all vars the CI workflow must supply - Update Tips: add build-before-test, storageState-for-auth, read-actual- components reminders API_REFERENCE.md: - Fix webServer.command to CI-aware pattern matching SKILL.md - Fix GitHub Actions versions v6/v7 → v4 - Add build step and CI env vars to workflow example https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
SKILL.md frontmatter:
- Add allowed-tools: Read, Write, Edit, Glob, Grep, Bash(npm install*),
Bash(npx playwright*), Bash(node ${CLAUDE_SKILL_DIR}/run.js*)
Prevents permission prompts on every file read/write during test generation
- Add argument-hint: [project-path or spec-file] [--headed] [--grep <pattern>]
Enables autocomplete guidance when invoking /playwright-skill
- Both fields are documented in official Claude Code skills spec
SKILL.md: remove Path Resolution preamble + fix $SKILL_DIR
- Remove 12-line "IMPORTANT - Path Resolution" block telling Claude to manually
locate and substitute the skill directory path
- Replace with official ${CLAUDE_SKILL_DIR} variable throughout
($SKILL_DIR was an ad-hoc convention; ${CLAUDE_SKILL_DIR} is the spec variable)
SKILL.md: trim to 489 lines (was 676, limit is 500)
- Move API Mocking, Form Submission, Navigation, Page Object Model patterns
to API_REFERENCE.md with a reference link — these are reference patterns,
not core workflow instructions
- Condense globalSetup auth section from three separate code blocks to
inline prose + two compact snippets
- Condense .gitignore section (cache note was already in CI workflow)
- Trim verbose YAML comments duplicating the CI Env Vars section
marketplace.json:
- Remove top-level "metadata" key — not in official schema
(fields were already correctly duplicated inside plugins[0])
skills/playwright-skill/package.json:
- Move @playwright/test from dependencies → devDependencies
(it's a build/test tool, not a runtime dependency)
- Replace "main": "run.js" with "bin": {"playwright-skill": "./run.js"}
(run.js is a CLI tool, not an importable module; bin is the correct field)
README.md:
- Remove lib/helpers.js from structure diagram (file was deleted)
- Add e2e/global-setup.ts to "What Gets Written" table
- Update "How It Works" to reflect auth/route code reading in Step 2
CONTRIBUTING.md:
- Fix "Always show headless: false by default" → headless: true (CI requires it)
- Fix "Add console.log statements for visibility" → do not add (clutters CI output)
- Update file structure diagram to reflect actual current layout (no lib/helpers.js)
- Update 500-line limit guidance
https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
…down files
run.js was a thin wrapper around `npx playwright test` that added:
- findProjectRoot: walk up to package.json — Claude already knows the project
root from Step 1 exploration; `cd <root> && npx playwright test` is equivalent
- hasPlaywrightTest: better error if @playwright/test missing — npx gives a
clear enough message; marginal value
- findPlaywrightConfig: warning if no config — npx handles this with defaults
- Formatted header output and unicode separator line — cosmetic
The entire value of run.js was `cd <project-root> && npx playwright test`.
Claude can do that directly. The wrapper was a holdover from when the skill
managed browser context and header injection; those responsibilities were
removed with helpers.js in earlier refactors.
package.json had no purpose without run.js — its only job was declaring
@playwright/test as a devDependency for the runner.
The skill is now SKILL.md + API_REFERENCE.md — no Node.js code, no setup step.
SKILL.md:
- Step 6: replace `cd ${CLAUDE_SKILL_DIR} && node run.js <root>` with
`cd <project-root> && npx playwright test` variants
- allowed-tools: remove Bash(node ${CLAUDE_SKILL_DIR}/run.js*)
(Bash(npx playwright*) already covers test execution)
README.md:
- Remove "Test runner included" feature bullet
- Remove `npm run setup` from all three installation paths
- Simplify: "skill has no dependencies of its own"
- Update "Validate Claude's Tests" section to use npx directly
- Update Dependencies section
- Update structure diagram
- Fix Troubleshooting: remove install-all-browsers script reference
CONTRIBUTING.md:
- Replace `npm run setup` in development workflow with `claude --plugin-dir ./`
- Update Testing checklist
- Update structure diagram
https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
Skill rename:
- skills/playwright-skill/ → skills/playwright-e2e/
- SKILL.md frontmatter: name: playwright-skill → playwright-e2e
(slash command becomes /playwright-e2e)
- plugin.json name: playwright-skill → playwright-e2e
- marketplace.json name (top-level and plugins[0]): → playwright-e2e
- Install command: /plugin install playwright-e2e@playwright-e2e
- README/CONTRIBUTING path references: skills/playwright-skill → skills/playwright-e2e
Author/owner update: lackeyjb → zizzfizzix
- plugin.json author.name
- marketplace.json owner.name, plugins[0].author.name
- All GitHub URLs: lackeyjb/playwright-skill → zizzfizzix/playwright-skill
(GitHub repo name stays playwright-skill; only skill name changes)
Original attribution preserved:
- LICENSE: added "Copyright (c) 2025 zizzfizzix" alongside original lackeyjb line
- plugin.json: contributors: [{name: "lackeyjb"}]
- marketplace.json: contributors: [{name: "lackeyjb"}]
https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
Mirrors @playwright/test package name — more natural for developers. Slash command is now /playwright-test. - skills/playwright-e2e/ → skills/playwright-test/ - SKILL.md frontmatter name: playwright-e2e → playwright-test - plugin.json name - marketplace.json name (top-level and plugins[0]) - All README and CONTRIBUTING references https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
Auth: replace globalSetup with setup project pattern - e2e/auth.setup.ts using `test as setup` from @playwright/test - playwright.config.ts projects: setup project + dependencies: ['setup'] on chromium project with storageState - globalSetup was the old approach; setup project is now canonical because: failures show traces/screenshots (not cryptic Node errors), retries work, multi-role auth is natural - Remove per-test fixture pattern (superseded; kept storageState opt-out example) playwright.config.ts template: - video: 'retain-on-failure' → 'on-first-retry' (match trace strategy per docs) - projects now shows setup project + storageState on chromium project GitHub Actions: - upload-artifact if: failure() → if: always() Playwright's own generated workflow uses always(); failure() misses the report on flaky tests that eventually pass Tips section: - Update storageState tip to reference setup project, not globalSetup - Expand waitForTimeout tip to mention expect.poll() as the recommended alternative for polling async conditions (e.g. API status checks) - Add eslint-plugin-playwright tip — officially recommended in Playwright best practices to enforce no-waitForTimeout, no-focused-tests at lint time SKILL.md: 489 → 460 lines (setup project is more concise than globalSetup) https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
- video: 'retain-on-failure' → 'on-first-retry' (matches SKILL.md)
- if: failure() → if: always() for artifact upload (catches flaky tests)
- page.type() → pressSequentially() (type() deprecated since v1.38)
- Mobile Testing: replace raw playwright API (require/browser.newContext)
with @playwright/test fixture pattern (projects config + context fixture)
- Selector priority: role selectors now listed first per official Playwright
docs; getByTestId() replaces page.locator('[data-testid=...]')
- Performance test: replace console.log with expect assertion
- Data-driven test: page.fill() → getByLabel/getByRole locator API
- POM example: .js → .ts, CSS selectors → getByLabel/getByRole
https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
…review - README: e2e/global-setup.ts → e2e/auth.setup.ts (stale from pre-setup-project era) - README: "artifact upload on failure" → "on every run" (matches if: always() change) - CONTRIBUTING: remove stale run.js reference from commit message example - SKILL.md: GitLab CI artifacts when: on_failure → when: always (match GitHub Actions) - API_REFERENCE.md: unpin @playwright/test and @axe-core/playwright versions https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
SKILL.md — Grep patterns used BRE alternation (\|) but Claude's Grep
tool runs ripgrep (Rust regex / ERE), where \| is a literal pipe, not
alternation. All four patterns would silently return zero results:
- createBrowserRouter\|<Route\|<Routes → createBrowserRouter|<Route|<Routes
- app.get\|router.get\|fastify.get → app\.get|router\.get|fastify\.get
(also escape . to avoid matching any character)
- login\|signin\|sign-in → login|signin|sign-in
- test(\|it(\|describe( → test\(|it\(|describe\(
(unclosed ( without \( would cause a regex parse error)
API_REFERENCE.md — "clear and type" example used page.locator('#username')
immediately after the selector best practices section saying to avoid ID
selectors; replaced with page.getByLabel('Username')
.gitignore — removed vestigial entries (no package.json/node_modules in
root); removed *.png/*.jpg/*.jpeg top-level globs that would block any
documentation images added to the repo; screenshots/ directory entry
already covers test-generated images
https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
…nt()
Best Practices section had selector priority backwards ("prefer data-testid,
use role-based") — the opposite of the Selectors section we updated; rewrote
all 5 items to reflect current guidance (role selectors first, storageState
auth, if:always() artifacts)
Mouse Actions section used legacy page.click(selector)/page.hover(selector)/
page.dblclick(selector) methods inconsistent with the rest of the reference;
updated to locator methods; replaced page.dragAndDrop() (older API) with
locator.dragTo() (current locator API)
Advanced Locator Patterns: .count() was a dangling expression with no
assignment or assertion — does nothing at runtime; replaced with
expect(...).toHaveCount(1)
https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
…cription
API_REFERENCE.md Form Interactions: page.selectOption(selector, ...) and
page.setInputFiles(selector, ...) were legacy page methods inconsistent with
the rest of the section; updated to locator API:
page.getByLabel('Country').selectOption(...)
page.getByLabel('Upload file').setInputFiles(...)
API_REFERENCE.md Waiting Strategies: page.click('button#load-users') was a
legacy page method + ID selector; updated to
page.getByRole('button', { name: 'Load users' }).click()
SKILL.md Key CI settings: "screenshot/video/trace: Artifacts on failure"
was inaccurate — video and trace use on-first-retry, not only-on-failure;
clarified each setting's trigger condition
https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
README Features: "artifact upload on failure" was the last remaining instance of this phrasing; updated to "on every run" with build step and secrets guidance mentioned (matches actual generated workflow) CONTRIBUTING Code Style: section listed JS coding conventions (variable names, single-responsibility functions) irrelevant now that the skill has no code; replaced with documentation-focused Content Guidelines that match the actual contribution surface: tone, example selector requirements, TypeScript validity https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
Advanced Locator Patterns: const row was declared twice in the same
code block (lines 155 and 166) — SyntaxError at runtime; renamed
second declaration to parentRow; also replaced button.edit CSS class
selector with getByRole('button', { name: 'Edit' })
Popup/Download patterns: page.click('button.open-popup') and
page.click('button.download') were legacy page methods with CSS class
selectors; replaced with getByRole('button', { name: ... }).click()
https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
README title: "Playwright Skill for Claude Code" → "playwright-test" (matches the skill name and invoke command; subtitle clarifies it's a Claude Code skill) Version: 5.0.0 → 1.0.0 — this is a complete overhaul of the plugin (helpers.js, run.js, package.json deleted; skill renamed; auth pattern rewritten; CI workflow corrected); starting fresh from 1.0.0 accurately represents the current state to marketplace users https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
Configures Release Please (v4) with the simple release type to automate: - CHANGELOG.md generation from conventional commits - Version bumps in .claude-plugin/plugin.json ($.version) - Version bumps in .claude-plugin/marketplace.json ($.plugins[0].version) - GitHub Release creation with generated release notes On push to main, Release Please opens a release PR. Merging it bumps both plugin metadata files, generates the changelog, and publishes the GitHub Release with the version tag. https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
Switches from GITHUB_TOKEN to a PAT so that CI checks run on Release Please's automated PRs. https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
Moves the plugin into plugins/playwright-test/ so multiple plugins can coexist in the same repo. Each plugin gets its own directory, plugin.json, CHANGELOG, and Release Please package entry. Scoped conventional commits route to the right plugin: feat(playwright-test): ... → bumps playwright-test only Release Please creates tags like playwright-test-v1.2.0 and a post-release workflow step automatically pins marketplace.json source.ref to that tag — so users are always on a frozen snapshot and main branch changes never silently reach them until a release. To add a second plugin later: 1. mkdir -p plugins/new-plugin/.claude-plugin 2. Add package entry to release-please-config.json 3. Add version entry to .release-please-manifest.json 4. Add plugin entry to .claude-plugin/marketplace.json https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
Renames playwright-test -> playwright-scaffold and trims the skill to cover only project setup: installs @playwright/test, generates playwright.config.ts, and creates CI workflow files. Spec writing will live in the playwright-spec plugin.
…iles Adds the playwright-spec plugin which focuses on exploring the app (routes, auth, components) and writing *.spec.ts files. Includes API_REFERENCE.md with advanced patterns. Assumes playwright.config.ts already exists (run playwright-scaffold first).
…ailing tests Adds the playwright-debug plugin which reads test output and failure artifacts (screenshots, traces), traces root causes back to app source, and fixes specs to match actual application behaviour.
- Use package-relative paths for changelog-path and plugin.json (from main) - Use /.claude-plugin/marketplace.json (repo-root-absolute, leading /) for marketplace version bumps - Add playwright-debug to release-please config, manifest, and marketplace - Add version.txt for all three plugins (required by simple release type) - Keep command: manifest in workflow and main's post-release version+ref sync script - Update CLAUDE.md to document correct path conventions https://claude.ai/code/session_01Pc7Yu5QeDDob2doj6V2iks
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.