Ctrlk
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Tutorial: Your First Test with an AI Agent

Connect an AI assistant such as Claude Code or Cursor to the ContextQA MCP server, then create, run, and review your first test using natural-language prompts — no test code and no manual API calls.

Who is this for? Developers and SDETs who want to drive ContextQA from an AI assistant. You connect your client to the MCP server, ask the agent to build a test case in plain English, run it, and review the evidence — all from a chat prompt.

The ContextQA MCP server lets an AI assistant — Claude Code, Claude Desktop, Cursor, ChatGPT, or Codex — create and run tests on your behalf. You describe what you want in plain English, and the agent selects the right ContextQA tools, fills in the parameters, and reports the results. There are no test scripts to write and no API calls to wire up by hand.

In this tutorial you will:

  1. Connect the hosted MCP server to your AI assistant

  2. Verify the connection with a read-only prompt

  3. Create a test case from a natural-language description

  4. Run the test and poll for the result

  5. Review the evidence and interpret a failure

End result: A working test case in your ContextQA workspace that you created and executed entirely through your AI assistant, plus the skills to drive the full toolset from natural-language prompts.

Prerequisites

  • A ContextQA account with at least one workspace (sign up at contextqa.com)

  • An MCP-compatible client installed: Claude Code, Claude Desktop, Cursor, ChatGPT, or Codex

  • A web application you can test against — a public site or your own staging environment

  • Familiarity with core concepts — specifically test cases, steps, and executions

This tutorial uses Claude Code for the example commands because it connects with a single command. The prompts work identically in any MCP client; only the one-time connection step differs. See Installation & Setup for the exact configuration for Claude Desktop, Cursor, ChatGPT, and Codex.

Step 1: Connect the MCP server to your agent

The server is hosted at https://mcp.contextqa.com/mcp. There is nothing to download or run locally — you point your client at the URL and sign in through your browser.

In your terminal, add the server to Claude Code:

This registers a server named contextqa that uses HTTP transport and OAuth 2.0 authentication.

Verify it worked: Run claude mcp list and confirm contextqa appears in the output. The server shows as connected once you complete sign-in in the next step.

Using a different client? Claude Desktop and Cursor use a JSON config file, ChatGPT uses a custom connector, and Codex uses a TOML entry. Each points at the same https://mcp.contextqa.com/mcp URL. See Installation & Setup for per-client steps.

Step 2: Sign in and verify the connection

The first time your agent calls a ContextQA tool, it triggers the OAuth 2.0 sign-in flow. Trigger it now with a read-only prompt that lists your existing tests.

Ask your agent:

The first tool call opens a ContextQA login page in your browser:

  1. Sign in with your ContextQA account.

  2. Authorize the client's access when prompted.

  3. Wait for the browser to auto-redirect back to your client.

Don't close the browser or navigate away until the redirect completes. If you interrupt the callback, authorization fails and you must start sign-in again. See Authentication for the full flow.

Verify it worked: The agent returns a list of your test cases (or reports that the workspace has none yet). Either response confirms the connection is live. Access tokens last 8 hours and refresh automatically, so you stay signed in for the rest of your working day.

Step 3: Create a test case from a prompt

Now create a real test. Describe the flow you want in plain English, including the starting URL, the steps, and a name. The agent maps your description to the create_test_case tool.

Ask your agent:

Replace the URL, credentials, and steps with values for your own application. Write each step the way you would describe it to a teammate — ContextQA's AI engine interprets the intent and resolves the actual UI elements at run time.

Verify it worked: The agent reports that it created the test case and returns a test_case_id. Note this ID — the agent uses it to run the test in the next step. You can also open your ContextQA workspace and confirm Reports - Smoke Test appears in the test case list.

Tip: The more specific your step descriptions, the more reliable the generated test. If a test comes back with too few steps, add detail to your prompt — name the exact buttons, fields, and expected outcomes — and ask the agent to recreate it.

Step 4: Run the test and wait for the result

Execution is asynchronous. ContextQA runs the test in a cloud browser, which takes anywhere from a few seconds to several minutes depending on the number of steps. The agent starts the run with execute_test_case and then polls get_execution_status until the test finishes.

Ask your agent:

The agent:

  1. Calls execute_test_case with the test case ID from Step 3

  2. Polls get_execution_status every 15-30 seconds while the test runs

  3. Reports the final status — passed or failed — once execution completes

You don't manage the polling yourself. The agent waits for a terminal status and summarizes the outcome.

Verify it worked: The agent reports a final status of passed or failed. A passed result means every step and assertion succeeded. A failed result means a step or assertion didn't pass — continue to Step 5 to investigate.

Status
What it means

Passed

Every step and verification succeeded

Failed

A step or assertion didn't pass — investigate the evidence

Still running after several minutes

A complex flow is still executing; let the agent keep polling

Step 5: Review the results and diagnose a failure

Every execution produces a full evidence package: per-step screenshots, a video recording, network and console logs, a Playwright trace, and an AI root cause analysis. Your agent can retrieve any of these on request.

To review a passing run, ask:

The agent calls get_execution_step_details and returns each step with its pass/fail status, a screenshot URL, and the execution duration.

To diagnose a failure, ask:

The agent calls get_root_cause, which analyzes the screenshots, video, DOM state, and network logs and returns a plain-English explanation. A typical response identifies the failing step, the underlying cause, and a suggested fix — for example, a Submit button that stayed disabled because a required checkbox was never checked.

Verify it worked: For a passing test, you receive a step-by-step breakdown with screenshots. For a failing test, you receive a root cause explanation that points to a specific step and reason. From here, the agent can apply a self-healing suggestion, file a defect ticket, or update the test — see the Agent Integration Guide for the full bug-fix loop.

Summary

You drove ContextQA end to end from an AI assistant:

  1. Connected the hosted MCP server to your client with a single command

  2. Signed in through OAuth 2.0 and verified the connection with a read-only prompt

  3. Created a test case from a plain-English description

  4. Ran the test and let the agent poll for the result

  5. Reviewed the evidence and interpreted a root cause analysis

Your agent now has access to the full ContextQA toolset, and you can build the rest of your workflow from natural-language prompts.

Next steps

  • Generate tests from your existing artifacts: Ask the agent to create tests from a Jira ticket, an OpenAPI spec, a Figma design, or a git diff. See AI Test Generation.

  • Automate the bug-fix loop: Have the agent reproduce a failure, propose a self-healing fix, apply it, and re-run. See Agent Integration Guide.

  • Run tests in CI: Combine MCP-driven generation with a CI/CD pipeline so tests are created and executed on every pull request. See GitHub Actions.

  • Explore every tool: Browse the full parameter reference for all 67 tools. See the Tool Reference.

Related pages

  • MCP Server Overview — what the MCP server is and what it exposes

  • Installation & Setup — connect Claude Desktop, Cursor, ChatGPT, and Codex

  • Authentication — the OAuth 2.0 sign-in flow, sessions, and multiple accounts

  • Agent Integration Guide — tool ordering, async execution, and multi-step orchestration

  • Tool Reference — full parameter documentation for every tool

Drive ContextQA from your AI assistant — no test code required. Start Free Trial → — Or Book a Demo → to see the MCP server with your stack.

PreviousAuthenticationNextAgent Integration Guide

Last updated

Was this helpful?