Test Plans

Create and run ContextQA test plans, monitor execution capacity and live progress, stop active runs, and manage suites, devices, settings, and schedules.

Quick answer

A ContextQA test plan is the reusable configuration that runs one or more test suites against selected browsers or devices, an environment, and saved execution settings. Open Plans → Test Plans, select a plan, review the Running, Capacity, and Queued values, and choose Run Now. During execution, open Runs → Live execution to see running and queued cases; use Stop Run when the remaining work should not continue.

Who is this for? Testers, SDETs, and QA managers who turn reusable suites into browser or device-specific execution plans.

A test plan is the executable configuration in ContextQA. It maps one or more test suites to browsers or devices and defines parallelism, timeouts, screenshots, environments, knowledge, notifications, and recovery behavior.

Watch the suites-and-test-plans walkthrough.

Interactive course: configure and run a test plan

This second course segment continues from an organized suite into browser or device mapping, execution settings, and a repeatable plan run. Start with Part 1 if you have not created the suite yet.

What to notice

• A test plan keeps its suites, execution targets, environment, and recovery settings together.
• Saved configuration makes on-demand, scheduled, and pipeline-triggered runs comparable.
• Run history and live execution provide shared visibility without changing the underlying tests.

Before you begin

• Create at least one test suite. A suite can be empty while you configure a plan, but it will contribute zero executable cases.
• Open Plans → Test Plans.
• Decide which browser or device should run each suite and whether the plan needs an environment or knowledge base.

Find and review plans

The page has Test Plans and Schedules tabs. Search by ID, title, or label, use Filter to narrow the list, or combine a search term with filters to find a precise subset. ContextQA applies both conditions to the same result set. Clear the search term and active filter chips to return to the full list.

The execution-capacity card in the page header shows the organization name and three live values: Running, Capacity, and Queued. The ring summarizes running executions against the organization's parallel limit. Auto-refresh is on by default; use the refresh control to pause or resume it. Review this card before starting a large plan so you can tell whether it can begin immediately or will wait for capacity.

A plan row shows its name, suite and case totals, latest run result, and actions for Schedule, Reports, Clone, and Run.

Click a plan name to open its detail page. The header provides Run Now and a menu for Copy, Edit, and Delete. The redundant Run Manually action is no longer shown on the test-plan detail page. The summary shows suite and case totals and the current schedule state.

The detail page includes:

Tab	Use
Overview	Review plan metadata and the suite-to-browser/device mapping. Add suites or view their cases.
Runs	Search and review recent runs. During an active run, switch between Run history and Live execution.
Reports	Open run reports.
Schedules	Create and manage recurring execution.
CI / CD	Review pipeline integrations and programmatic triggers.
Settings	Update execution, notification, and recovery settings.
Execution History	Compare execution outcomes over time.
Version History	Audit plan configuration changes.

Test Plan overview with one suite assigned to Chromium

Review CI/CD options

Open a plan and select CI / CD to review the automation surfaces currently presented by your workspace.

The verified production page lists these integration targets: GitHub, GitLab, Bitbucket, Azure DevOps, CircleCI, Codeship, Bamboo, Jenkins, AWS, TravisCI, and Generic shell. It also presents two REST operations:

• Start Test Plan — returns a run ID for status polling.
• Check Status — returns status, result, and suite breakdown information.

Test Plan CI/CD tab with supported pipeline targets and REST operation cards

The verified production page does not publish REST endpoint paths. Do not build a pipeline from portal network calls or legacy examples that use /auth/login, /testplans/{id}/execute, or /executions/{id}/status. Use only a public API contract displayed in your workspace or supplied through an official ContextQA reference.

For an automated client, create a dedicated, narrowly scoped API token and store it in the pipeline's secret manager. The token is a credential; it does not define which endpoint or payload to use. Configure the plan's suites, browsers or devices, environment, and recovery settings in ContextQA before connecting the external pipeline.

Create a test plan

Click New Test Plan to open the three-step wizard.

Step 1: Create Test Plan

Enter the plan's identity and presentation settings:

• Test Plan Name (required)
• Labels and Description (optional)
• Hide on Dashboard
• Send Notification for email and connected Slack notifications when the plan runs
• Viewport: Desktop, Laptop, Tablet, or Mobile
• Resolution

Example:

Field	Example value
Name	Checkout PR gate
Description	Runs the checkout smoke suite before each production release. Owner: QA Platform.
Viewport	Desktop
Resolution	Full HD (1920×1080)

Click Next.

Step 2: Test Machines & Suites Selection

1. Click Add Test Suites.
2. Search by suite name and optionally filter by label.
3. Select suites in Available Test Suites, move them to Selected Test Suites, and click Add.

Selecting the checkout smoke suite for a Test Plan

1. Select one or more suite rows and click Add Machine/Device.
2. Enter a configuration name, such as Chrome desktop.
3. Choose the test lab type, platform, and browser or device. Browser choices can include Firefox, Safari, Microsoft Edge, Chromium, and Chrome (Beta), depending on workspace availability.
4. Click Create to attach the machine or device to the selected suites.
5. Repeat the mapping when different suites need different targets, then click Next.

A checkout smoke suite mapped to a Chromium machine

A plan can contain multiple machine or device configurations. The same suite can be mapped to more than one target for cross-browser or cross-device coverage.

Step 3: Test Plan Settings

Set Parallel Execution to the number of execution nodes the plan may use. Start with 1 while validating a new plan.

Expand Additional Settings to configure:

Setting	Purpose
Page Load Timeout	Seconds to wait for navigation to complete.
Element Timeout	Seconds to wait for a target element.
Environments	Supplies environment-specific URLs and variables.
Capture Screenshots	Controls when step screenshots are captured.
Knowledge Base	Supplies application context to the execution agent.
Proxy Profile	Routes the plan through a reusable organization proxy.
Remote Browser Profile	Executes the plan against a saved remote browser endpoint.
Show scrollbars during test runs	Inherits the organization default or overrides it for this plan. Enable it only when the scenario must verify a visible scrollbar.

Expand Recovery Actions to configure:

Failure condition	Available responses
Major test step failure	Abort and run the next test case; report and continue to the next test step.
Test step prerequisite failure	Abort and run the next test case; report and continue to the next test step.
Test case prerequisite failure	Abort the test plan; report and continue to the next test case.
Test case abort	Clean up and reuse the current session; start with a new session.
Test suite prerequisite failure	Abort the test plan; report and continue to the next suite.
Rerun on failure	None; all test cases; all failed and not-executed test cases. The targeted option includes cases skipped because an earlier failure stopped their execution.

Test Plan execution settings with additional settings and recovery actions

Click Create. ContextQA opens the plan's Overview page after saving it.

Update a test plan

Use either update path, depending on what you need to change.

Edit details, mappings, or creation settings

1. Open the plan.
2. Open the header menu and select Edit.
3. Move through the same three-step wizard. Change the name, description, labels, dashboard visibility, notifications, viewport, suite-to-device mappings, or plan settings.
4. Click Update on the final step.

Edit runtime settings directly

1. Open the plan and select Settings.
2. Change Parallel Nodes, screenshot capture, page or element timeouts, knowledge base, environment, notification toggles, scrollbar visibility, or recovery actions.
3. Click Save Changes. Use Discard to abandon unsaved changes.

To clear the plan's selected knowledge base, choose None in Knowledge Base and save. This removes the plan-specific selection instead of retaining its prior value.

ContextQA records plan changes in Version History. Use this tab to audit changes and, when available, compare or restore a prior plan version.

Change suite membership

• To add coverage from Overview, click Add Test Suite, choose suites, and assign the required browser or device configuration.
• To inspect membership without editing it, click View Test Cases on a suite row.
• To remove or remap suites, open Edit and use step 2 of the wizard.

After any membership change, confirm the suite and test case totals shown in the header.

Clone or copy a plan

Use Clone from the Test Plans list or Copy from the plan header menu when a new plan should start from an existing configuration. Give the copy a distinct name, then review its suites, devices, environment, notifications, and schedule before running it.

Run and monitor a plan

• Use Run Now for an immediate execution using the saved configuration.
• While the plan is queued or executing, the run control changes to Stop Run. Select it when the remaining cases should not continue.
• Open Runs and select Live execution to monitor the active plan. The summary shows the running and queued counts, run ID, and elapsed start time. Executing cards identify the test case, suite, and browser or device; the queue lists cases waiting for a slot.
• Select the run summary to open its report. After the run finishes, use Run history to search by ID, title, or label, refresh the list, and switch between list and card views.
• Use Reports for run evidence and Execution History for outcome trends.

Running tests changes execution state and can consume capacity. Verify the target environment, data, and device mapping before starting a run.

Run Now is disabled when the plan contains zero executable test cases. Add cases to a selected suite before trying again. An already queued run can still be stopped.

Schedule a plan

Open Schedules on the plan detail page or use Schedule from the list. Create a schedule with a name, start date and time, recurrence, and notification recipient. Existing schedules can be activated, paused, edited, run immediately, or deleted.

For a dedicated walkthrough, see Scheduling Tests.

Delete a test plan

1. Open the plan.
2. Open the header menu and select Delete.
3. Review the confirmation. ContextQA warns that the plan and all associated run reports will be permanently removed.
4. Type delete to enable Yes, Delete.
5. Click Yes, Delete only when the plan and its historical reports are no longer required.

Test Plan deletion requires typing delete before confirmation

Deleting a test plan cannot be undone. Deleting the plan does not delete its underlying suites or test cases, but it removes the plan and its associated run reports.

MCP automation

The ContextQA MCP server supports this workflow:

1. Discover or create suites with get_test_suites and create_test_suite.
2. Discover browsers or mobile devices with get_available_devices.
3. Create the plan with create_test_plan and map each device configuration to one or more suite IDs.
4. Run it with execute_test_plan.
5. Poll with get_test_plan_execution_status; use rerun_test_plan when a previous execution should run again.

The current MCP suite/plan tools do not expose update or delete operations. Use the portal for those actions. See Test Suites & Plans Tools for exact parameters and examples.

Troubleshooting

Add Machine/Device is disabled

Select at least one suite row first.

Next is blocked on the suite and machine step

Confirm that at least one selected suite has a browser or device configuration.

A suite contributes zero test cases

Open the suite and use Manage Test Cases to add compatible cases.

The plan is missing from the dashboard

Open Settings and turn off Hide on Dashboard.

Search and filters return no plans

Clear the search term and remove active filter chips one at a time. Search and filters are cumulative, so a plan must satisfy both when they are used together.

A run does not start

If the run controls are disabled, confirm that the plan has at least one executable test case. Otherwise, review the execution-capacity card for queued work, confirm that another run is not already active, and verify that the mapped browser or device is available.

Related pages

• Managing Test Suites
• Running Tests
• Scheduling
• Parallel Execution
• Environments
• Proxy and remote browser profiles
• Mobile Test Plans
• Integrations Overview

70% less manual test maintenance with AI self-healing. Book a Demo → — See ContextQA create and run test plans for your web application.