AI Anonymous / Practical field guide / November 4, 2026

What did the test click?

Before you change a failing assertion in an AI-drafted browser test, check what the browser actually did.

In this field note · 6 sections
  1. Keep the trace from the failed attempt
  2. Read the assertion, then inspect the click
  3. Change the locator and keep the expected text
  4. Use the same review on a draft test
  5. Playwright documentation
  6. Keep the lab or share the field note

A test named “submitting a form shows Submitted” can still click Save draft. In this exercise, the failed assertion gives you the final text. Playwright’s Trace Viewer lets you inspect the action that produced it.

What we ran. A deliberately failing spec and its corrected counterpart on Windows, Node.js 24.14.0, Playwright 1.59.1 and Chromium. Both use a synthetic form with no server. The spec was authored for this exercise to illustrate a mistake an AI-drafted test could make. We measured no model behavior or performance.
AI Anonymous poster reading ‘What did the test click?’ A cursor points to an orange button on the front of three layered browser cards. JABSystems appears at the bottom.
Original AI-generated editorial illustration.

01 / Reproduce

Keep the trace from the failed attempt

Extract the practice ZIP into a fresh folder. With Node.js 22 or 24 and npm installed, open PowerShell in that folder. The first two commands install the locked test dependencies and the matching Chromium build. The lab itself uses an in-memory page.

npm ci
npx playwright install chromium
npx playwright test draft.spec.cjs --trace retain-on-failure --retries=0 --output=results-draft

The draft is meant to exit with code 1. Its assertion expects Submitted and receives Draft saved. retain-on-failure keeps the trace for this failed attempt with retries disabled. on-first-retry would need a retry before recording. Playwright trace modes.

The two buttons set different text in the same status element. Here is the complete failing spec. The supplied form.cjs provides the fixture.

const { test, expect } = require('@playwright/test');
const { form } = require('./form.cjs');

test('submitting a form shows Submitted', async ({ page }) => {
  await page.setContent(form);
  await page.getByRole('button', { name: 'Save draft', exact: true }).click();
  await expect(page.getByRole('status')).toHaveText('Submitted', {
    timeout: 1500,
  });
});

02 / Inspect

Read the assertion, then inspect the click

Run the following in the same PowerShell window. The first command finds the single ZIP created by this lab. The second opens it in the local viewer.

$trace = Get-ChildItem -LiteralPath .\results-draft -Recurse -Filter trace.zip
npx playwright show-trace "$($trace.FullName)"
  1. Select the failed toHaveText action. Compare the expected text with the received text.
  2. Select the earlier click in Actions. Inspect the locator and use the Action snapshot to identify the targeted button. It is Save draft.
  3. Switch to After. The status reads Draft saved. Compare that result with the test’s stated requirement.

Snapshots show the captured page state around an action. On a real application, verify backend effects separately if the requirement includes persistence or a transaction. Playwright Trace Viewer guide.

03 / Fix and rerun

Change the locator and keep the expected text

In this fixture the intended action is to submit. The supplied fixed.spec.cjs changes only the click below and still requires Submitted.

await page.getByRole('button', { name: 'Submit', exact: true }).click();

Close the viewer and run the corrected spec in a different output directory so Playwright’s cleanup keeps the draft trace available.

npx playwright test fixed.spec.cjs --trace retain-on-failure --retries=0 --output=results-fixed

In the tested run, the corrected spec passed and left no trace ZIP in results-fixed. The original failed run still had its ZIP. This exercise checks a single authored failure and correction in Chromium. The 1.5-second assertion timeout bounds the lab’s intentional failure.

04 / Apply

Use the same review on a draft test

Write down the behavior you want before asking an assistant to draft or repair a spec. For a failure, capture the expected state, the actual state, and the relevant action. Ask the assistant to identify the smallest change and explain which observation supports it. Review the assertion against the original requirement before rerunning.

Use a synthetic account and fixture before recording. A trace may contain DOM snapshots, screenshots, source, console output and network request or response headers and bodies. Keep traces private until you have checked what they contain. For a review handoff, a short, redacted description of the action and observed state may be enough. Playwright Network panel.

Primary sources

Playwright documentation

Downloads / Practice files

Keep the lab or share the field note

Explore more AI Anonymous field notes →

Share this post.

Found this useful? Pass it along. Download the artwork, add your own take, and tag @jabsystems.io so I can see your post.