Export Tests and Verify Portability

Export a ContextQA Playwright project through MCP, save the returned files locally, review credentials and manual steps, and prove assertion parity before CI.

Quick answer

ContextQA's Playwright project export returns a file manifest that an authorized coding client can save locally. Exporting code is the beginning of a portability check: review the generated assertions, credentials, fixtures, and manual checkpoints, then run a representative test outside ContextQA. A generated project or green exit code alone does not prove equivalent coverage, because some AI steps require manual work or become non-blocking verification placeholders.

Use this guide when evaluating vendor dependence, handing tests to a developer, or deciding whether exported tests can join repository CI. For incoming suites, use the migration guides.

Prerequisites

• ContextQA access to the specific source cases and their reviewed steps.
• An MCP-connected assistant with export tools available in its current manifest.
• A coding client with permitted local file access, or a supported way to save the returned files.
• An empty working directory outside committed source until credentials are reviewed.
• An approved non-production environment, scoped test account, and reproducible data.
• The source case IDs, application build, expected outcomes, and an execution baseline.

1. Define what needs to be portable

Keep these deliverables separate:

Deliverable	What it contains	What it does not establish
Code export	Generated test project or code files	Equivalent automated assertions or successful execution
Test-data export	Input rows or other documented data artifacts	A runnable test, credentials, or environment configuration
Result evidence	Run identifiers, logs, screenshots, videos, or traces when available	Future executability or permanent artifact retention
Migration	A conversion into another test representation	Live synchronization or a lossless round trip

Specify which you need. A manager assessing an exit path needs code, usable assertions, configuration, and ownership instructions. An engineer investigating yesterday's failure may only need the corresponding evidence, subject to its retention and access rules.

2. Choose the export route using the connected manifest

The current Playwright project implementation uses export_to_playwright with a nonempty test_case_ids list and optional project_name. One ID is a single-case project; several IDs produce multiple specs. It resolves prerequisites and reusable step groups while building the project.

The returned structure is a JSON file manifest:

{
  "project_name": "reporting-pilot",
  "file_count": 1,
  "files": [
    { "path": "README.md", "content": "Illustrative file content" }
  ],
  "summary": {}
}

This abbreviated example shows the structure, not a complete runnable project. Each real files entry has a relative path and content. The hosted server does not write these files to your laptop. The coding client saves them using its separately authorized filesystem capability.

The broader single-case tool, export_test_case_as_code, has a different interface and backend path. Do not substitute its parameters or promise Cypress, Selenium, or every language based on a generic export example. Confirm the requested format and usable output in your connected environment. The public MCP prompt library provides starting prompts; the actual tool schema and observed output determine which procedure applies.

3. Export a representative slice

Start with one happy path and one important negative or permission case. Include data-dependent or AI-heavy behavior before accepting a larger migration.

Inspect the current ContextQA export tool schema. Export only test cases
<ID1> and <ID2> as a Playwright TypeScript project named reporting-pilot.
Return the manifest and an inventory of generated files. Do not execute tests.
Identify manual checkpoints, assumed-success verification, loops, conditionals,
and any credentials. Do not print credential values in your explanation.

Save the returned files only under the approved empty directory. Reject absolute paths or traversal outside it, and do not overwrite an existing project. If the client cannot write files, retain the manifest securely and use a supported local client for the write step; a remote success message is not local file delivery.

Record the requested case IDs, generated specs, and any missing cases. Preserve the source baseline separately so generated changes can be reviewed against it.

4. Review the execution boundaries before installation

The current generator has concrete limits:

Source behavior	Export behavior to inspect	Required acceptance work
Supported typed actions with captured locators	Playwright actions	Check target, value, timing, and outcome
AI verification	Non-blocking aiVerify placeholder that assumes success	Replace with an assertion that can fail
AI agent action, missing locator, or unsupported action	Manual aiAgent checkpoint	Implement an automated action or declare manual coverage
AI conditional	Assumed-true branch; alternative branch is not executed	Implement and test the real condition and both branches
Loop	Body rendered once with a note	Restore the intended iteration data and assertions
Prerequisites and step groups	Expanded into the generated project	Verify ordering, parameters, isolation, and cleanup
Credentials	Values may appear in plaintext credentials.env	Review privately and move to approved secret handling

An ignored credential file is still sensitive on disk and in the manifest. Inspect generated files before sharing or committing them; preserve required variable names while replacing embedded values with scoped local or CI secrets.

Review the package manifest and dependency scripts before installing. Record the runtime and resolved lockfile so another engineer can reproduce the accepted project. Exported tests do not automatically retain ContextQA's managed execution, later edits, self-healing, or evidence storage.

5. Validate in layers

After reviewing the generated project and following its installation instructions:

npx playwright install chromium
npx playwright test --list

Use the project's actual browser requirements. --list checks test discovery; it does not run tests or type-check them. The current export does not include a complete TypeScript-check configuration. Add the approved TypeScript dependency, required types, and a tsconfig.json covering the generated files before running npx tsc --noEmit, or integrate the files into your existing configured check. See Playwright's TypeScript documentation.

Before a permitted test run, confirm the configured URL, account, side effects, and cleanup. For generated projects that include the current AI-step helper:

HEADLESS=1 AI_STEP_MODE=fail npx playwright test

The current generated configuration uses HEADLESS=1 for headless execution. AI_STEP_MODE=fail makes remaining manual aiAgent checkpoints fail rather than silently continue. It does not turn aiVerify placeholders into real assertions. Replace those separately. Do not use AI_STEP_MODE=log as release evidence: it skips manual actions.

Check the spec count, actual executed steps, skips, retries, and result report. A test that never reaches its critical assertion is incomplete even if the surrounding command succeeds.

6. Prove the important behavior survives

Run the source test and remediated export against the same build and seed data. Then introduce a reversible, known defect in a disposable build and rerun the unchanged tests. Both should fail for the relevant business reason. Restore the application and confirm the baseline again.

Record the outcome:

Acceptance item	Evidence required
Correct behavior	Source and exported run references for the same scenario
Defect detection	Seeded defect and failing assertion in both paths
Independent execution	Documented command, dependencies, data, and environment
No silent gaps	Disposition for every placeholder, branch, loop, skip, and manual action
Safe handoff	Sanitized repository and approved secret configuration
Maintenance ownership	Authoritative source, update process, and responsible reviewer

Keep the result as Needs remediation when any critical check is missing. A partially manual export can still help a migration assessment, but it should not count as unattended automated coverage. Use the pilot scorecard to capture that distinction.

Troubleshooting and FAQ

The tool rejected output_dir. Inspect its current schema. The manifest-based Playwright export takes case IDs and an optional project name; local saving is a client operation.

The exported run passes without checking the result. Search for aiVerify, skipped tests, and weakened assertions. Replace the relevant placeholders and prove a seeded defect fails.

Can exported code be re-imported with no loss? Do not assume round-trip parity. Use a dry-run migration and a fresh assertion comparison before changing the authoritative test source.

Can I export directly to Selenium or Cypress? Verify a working format-specific path and representative output in your workspace. The Playwright procedure here does not establish support or parity for other frameworks.

Sources and implementation reviewed September 25, 2026. Verify the tool schema in your connected client before using version-dependent capabilities.

Next step

Connect ContextQA MCP, select two representative cases, and keep the export result in a review branch. If portability is a purchase requirement, book an export evaluation demo and ask to compare source assertions, generated files, and an independent failure run for your own workflow.