End-to-End Testing with AI Agents

Use Claude Code, Cursor, or Codex to build Playwright end-to-end tests, verify real outcomes, and run equivalent ContextQA workflows through MCP.

Quick answer

Choose one important user journey, define its starting state and final business outcome, and ask your coding agent to write a browser test with explicit assertions. Run it against an approved environment, inspect the evidence, and verify the underlying state when the outcome requires it. With ContextQA MCP, describe the same journey, review the generated steps, execute the case, and retrieve actual result details. A completed sequence of clicks is not enough.

Start with a narrow flow such as creating and reopening a note. A successful toast does not prove the note was saved. Likewise, a booking thank-you page does not prove a calendar invitation or CRM meeting exists.

Prerequisites and environment contract

You need an approved staging application, an isolated test account, known seed data, a cleanup mechanism, and the correct browser runner. Retain the repository's existing Playwright version and lockfile. For a new learning project, follow Playwright installation.

The example below is an illustrative application contract, not a runnable test against ContextQA or a supplied demo app. Adapt it to an application you own before execution:

• An authenticated user can open /notes/new, fill fields labelled Title and Body, and click Save note.
• That action calls POST /api/notes, returns 201 and JSON with id, then shows the notes list.
• Each saved note has data-testid="note-<id>" and displays its title.
• Authenticated GET /api/notes/<id> returns the saved title and body; DELETE removes only that note and returns 204.
• TEST_BASE_URL identifies the approved test environment; TEST_AUTH_STATE points to an existing authenticated Playwright storage-state file for a disposable test account.

Do not commit the storage-state file: it can contain reusable authentication material. Use your project's documented authentication setup and ignored storage location. Playwright authentication.

1. Give the agent an outcome, not only steps

Inspect our app, existing Playwright configuration, and test fixtures.
Create an end-to-end test for an authenticated user creating a note.
Use a unique title and known body, save it through the UI, verify the
creation response, reload, and confirm it remains visible. Independently
read the stored note through the documented API and compare its content.
Clean up only this test's note, including when assertions fail.

Use our approved staging target and test account. Do not send messages,
charge cards, or create production records. Use semantic locators and
explicit expected results. Do not use fixed sleeps, weaken assertions,
or treat a toast as proof of persistence.

Run the test. Report the command, environment, final assertions, result
artifacts, retries, and any failed or blocked step. Preserve the first
failure if a later attempt passes.

Use the appropriate client setup for Claude Code, Cursor, or Codex. An agent can generate a test before it has an environment capable of running it; require that distinction in its handoff.

2. Configure the target explicitly

In a learning project with @playwright/test installed, this minimal playwright.config.ts rejects an unspecified target or session:

import { defineConfig } from '@playwright/test';

if (!process.env.TEST_BASE_URL || !process.env.TEST_AUTH_STATE) {
  throw new Error('Set TEST_BASE_URL and TEST_AUTH_STATE for the test account');
}
export default defineConfig({
  testDir: './tests',
  retries: 0,
  use: {
    baseURL: process.env.TEST_BASE_URL,
    storageState: process.env.TEST_AUTH_STATE,
    browserName: 'chromium',
    trace: 'retain-on-failure',
  },
});

Merge these concepts into an existing configuration rather than replacing its projects, fixtures, or CI settings. This example assumes cookie-based authentication: page.request shares the browser context's cookies for API readback and cleanup. A token stored only in localStorage does not automatically authenticate those requests; configure the API's approved authentication mechanism separately. Keep session files out of version control. The example retains failure traces and starts without automatic retries so initial failures remain obvious. Playwright configuration, trace viewer.

3. Write assertions around the business outcome

Save this adapted example as tests/notes.spec.ts:

import { randomUUID } from 'node:crypto';
import { test, expect } from '@playwright/test';

test('created note survives reload and matches stored content', async ({ page }) => {
  const title = `e2e-note-${randomUUID()}`;
  const body = 'A deterministic body for persistence verification.';
  let createdId: string | undefined;
  try {
    await page.goto('/notes/new');
    await page.getByLabel('Title', { exact: true }).fill(title);
    await page.getByLabel('Body', { exact: true }).fill(body);
    const responsePromise = page.waitForResponse((response) =>
      new URL(response.url()).pathname === '/api/notes' &&
      response.request().method() === 'POST');
    await page.getByRole('button', { name: 'Save note', exact: true }).click();
    const response = await responsePromise;
    expect(response.status()).toBe(201);
    const created = await response.json();
    expect(typeof created.id).toBe('string');
    expect(created.id.length).toBeGreaterThan(0);
    createdId = created.id;

    await expect(page).toHaveURL(/\/notes\/?$/);
    await page.reload();
    await expect(page.getByTestId(`note-${createdId}`)).toContainText(title);
    const persisted = await page.request.get(`/api/notes/${encodeURIComponent(createdId!)}`);
    expect(persisted.status()).toBe(200);
    expect(await persisted.json()).toMatchObject({ id: createdId, title, body });
  } finally {
    if (createdId) {
      let cleanupError: unknown;
      try {
        const cleanup = await page.request.delete(`/api/notes/${encodeURIComponent(createdId)}`);
        if (cleanup.status() !== 204) cleanupError = `cleanup returned ${cleanup.status()}`;
      } catch (error) { cleanupError = error; }
      expect.soft(cleanupError, 'delete only this test note').toBeUndefined();
    }
  }
});

Set the two environment variables in your test environment, install the configured browser if necessary, and run:

npx playwright install chromium
npx playwright test tests/notes.spec.ts

The cleanup assertion is soft so a cleanup problem is recorded without replacing an earlier application failure. Playwright's browser-context request client can share the browser session's cookies, which makes the independent API read useful for an authenticated workflow. It still verifies only the state exposed by that API; it does not prove a downstream warehouse or third-party sync. API testing.

4. Add meaningful failure cases

Start with cases suggested by the application's requirements: a required title left empty; an expired session; permission revoked before saving; a save failure that preserves the user's input; and retry after a network interruption without duplicate creation. Specify the expected outcome before asking the agent to implement each case.

Keep data independent between runs. A unique title avoids collisions, while a stored record ID makes cleanup precise. If creation succeeds but the response is lost, cleanup may need a test-run identifier and a bounded server-side lookup; document that limitation instead of deleting all notes in the account.

Use locators that describe the intended control. Playwright recommends resilient locators and assertions that wait for the expected UI state. Avoid converting a failing exact assertion into a broad text match just to make the run pass. Best practices, assertions.

5. Run the same outcome with ContextQA MCP

Connect ContextQA MCP, verify the intended workspace, and inspect the connected tools. The documented create_test_case accepts url and task_description; describe the actions and their expected results. Inspect generated steps before starting execution. Case management reference.

Prepare a ContextQA test for our approved staging notes workflow.
Use the dedicated account and unique test-run identifier. Create a note
through the UI, verify its exact title after reload, and verify persisted
content using the approved read-back mechanism. Clean up only that note.

Show the generated steps and assertions first. If the available tools
cannot perform the API read-back or cleanup, identify that gap and use
our approved companion check; do not mark those assertions passed.
Execute the reviewed case and retrieve the result ID and per-step
results. Report the saved-state evidence separately from UI navigation.

Use the documented execute_test_case, get_execution_status, and result tools. A STATUS_COMPLETED response means execution finished; inspect get_test_case_results and step evidence for the assertion verdict. Keep failed, blocked, and unverified outcomes distinct. Execution and results reference.

Debugging, retries, and portability

Inspect the first failing assertion, trace, relevant request, and seeded state before changing the test. A timeout might mean the element never appeared, the wrong account was used, or the app failed; more waiting does not distinguish those causes. Preserve cleanup failures as separate operational problems when they accompany an application failure.

If retries are enabled later, retain each attempt. Playwright distinguishes tests that pass initially from those that fail and subsequently pass; a retry should not erase that signal. Retry behavior.

For ContextQA-generated browser cases, follow export and portability. Inspect exported assertions, authentication, API checks, cleanup, and environment configuration, then rerun the project. An exported file is not evidence that the exported test passes or reproduces every managed-platform behavior.

Common questions

Should every feature have a long end-to-end scenario?

No. Keep broad input combinations in faster tests and select important cross-system journeys for browser coverage. Short, independent journeys are easier to diagnose.

Can ContextQA replace all Playwright fixtures automatically?

Do not assume that. Evaluate the actual migrated or exported artifact, including custom setup, authentication, API state checks, and assertions.

Next step: Set up ContextQA MCP and try one reviewed staging journey. To compare native Playwright evidence with an MCP-driven workflow, book an end-to-end testing demo.