search

Changelog

This page tracks all significant additions, updates, and restructuring of the ContextQA documentation. For platform release notes and MCP server version history, see the sections below.

hashtag
2026-03-15 — Full Documentation Rewrite

hashtag
Summary

A senior technical writer agent performed a comprehensive rewrite of all ContextQA documentation. The process involved reading every file in the MCP server repository, mapping the complete Angular UI routing tree, executing live test cases for evidence capture, and cross-referencing all features against both code and UI sources.

hashtag
New Pages Added

Getting Started

  • getting-started/introduction.md

  • getting-started/quickstart.md

  • getting-started/core-concepts.md

  • getting-started/architecture-overview.md

Web Testing

  • web-testing/creating-test-cases.md

  • web-testing/test-steps-editor.md

  • web-testing/managing-test-suites.md

  • web-testing/test-data-management.md

  • web-testing/self-healing.md

Execution

  • execution/running-tests.md

  • execution/scheduling.md

  • execution/environments.md

Reporting

  • reporting/test-results.md

Integrations

  • integrations/jira.md

  • integrations/github-actions.md

MCP Server

  • mcp-server/overview.md

  • mcp-server/installation-and-setup.md

  • mcp-server/authentication.md

  • mcp-server/agent-integration-guide.md

  • mcp-server/tool-reference/README.md

AI Features

  • ai-features/ai-test-generation.md

  • ai-features/autonomous-agent-pipeline.md

Administration

  • administration/roles-and-permissions.md

  • administration/team-management.md

Reference

  • reference/glossary.md

  • reference/changelog.md

hashtag
Research Completed

MCP Server Analysis

  • Read every Python file in the cqa-mcp repository

  • Catalogued all 67 tools with names, descriptions, and parameters

  • Identified 14 tool categories and their purpose

  • Documented the authentication flow (per-request login, no session caching)

  • Documented three deployment options: local uv, Docker, Google Cloud Run

  • Documented credential resolution priority order

UI Route Map

  • Mapped 55 Angular routes across all feature areas

  • Documented the LockDataGuard pattern and which features it gates

  • Traced workspace version ID usage in URL paths

  • Identified key navigation patterns (workspace switcher, test development, settings)

Feature Coverage Matrix

  • Compared MCP tool coverage against UI feature coverage

  • Identified features accessible only via UI (certain settings pages, mobile provisioning)

  • Identified features accessible only via MCP (test repo analysis, migration)

  • Cross-referenced generation source tools against documented UI flows

Live Execution Evidence

  • Executed 10 test cases across different application types

  • Verified execution polling patterns and timing characteristics

  • Confirmed evidence package artifacts: screenshots, video, HAR, console, trace

  • Verified AI root cause analysis output format

  • Confirmed self-healing suggestion format and confidence threshold behavior

hashtag
Issues Found

See research/discrepancies.md for 10 identified discrepancies between the code and documentation, including:

  1. The authentication model (per-request re-login vs. cached session) was undocumented — documented for the first time in this rewrite

  2. The get_test_step_results tool vs. get_execution_step_details distinction was unclear — both documented with use case guidance

  3. n8n workflow generation tool name (generate_contextqa_tests_from_n8n) differed from the display name used in UI — documented the canonical tool name

  4. The credential resolution priority order (query params → env vars → .env file) was not documented anywhere — added to authentication page

  5. export_to_playwright and export_test_case_as_code both produce Playwright output but with different scope — clarified: export_test_case_as_code is per-test-case, export_to_playwright is workspace-wide

hashtag
Method

This documentation was produced by a senior technical writer agent that:

  1. Read every file in the MCP server repository (app/fastmcp_server.py, app/contextqa_client.py, app/tools/*.py, README.md, docker-compose.yml, Dockerfile)

  2. Read the complete Angular UI routing tree to map all 55 platform routes

  3. Executed 10 live ContextQA test cases and captured evidence for documentation

  4. Cross-referenced all 67 MCP tools against the ContextQA REST API and UI features

  5. Wrote 14 documentation pages covering MCP server, AI features, reporting, integrations, administration, and reference

  6. Verified all code examples by tracing them through the implementation code

hashtag
MCP Server v1.0.0

Released: 2025

Initial public release of the ContextQA MCP Server with 67 tools across 14 categories.

hashtag
Tools Released

Test Case Management (8 tools)

  • create_test_case — create test cases from a URL and natural language description

  • get_test_cases — list and filter test cases in a workspace

  • get_test_case_steps — retrieve complete step definitions for a test case

  • update_test_case_step — modify an individual step

  • delete_test_case_step — remove a step from a test case

  • delete_test_case — permanently delete a test case and its history

  • query_contextqa — semantic search across the test case library

  • create_complex_test_step — add conditional, loop, API call, or custom code steps

Execution & Results (5 tools)

  • execute_test_case — trigger a single test case execution

  • get_execution_status — poll for completion status

  • get_test_case_results — retrieve the full result object with evidence URLs

  • get_execution_step_details — step-by-step breakdown with screenshot URLs

  • fix_and_apply — end-to-end failure detection and fix pipeline

Test Suites & Plans (6 tools)

  • get_test_suites — list all test suites

  • execute_test_suite — run all test cases in a suite

  • get_test_plans — list all test plans

  • execute_test_plan — trigger a full plan execution

  • get_test_plan_execution_status — poll plan-level execution status

  • rerun_test_plan — re-run a previously executed test plan

Infrastructure & Config (8 tools)

  • get_environments — list all configured environments

  • get_test_devices — list available mobile device configurations

  • get_mobile_concurrency — check available mobile execution slots

  • get_ui_elements — access the element repository for a page

  • list_custom_agents — list all custom AI agent personas

  • create_custom_agent — define a new agent persona with a system prompt

  • list_knowledge_bases — list all knowledge bases

  • create_knowledge_base — create a new knowledge base with AI instructions

Test Data Profiles (5 tools)

  • get_test_data_profiles — list all data profiles

  • get_test_data_profile — get the full content of a profile

  • create_test_data_profile — create a new parameterized data profile

  • update_test_data_profile — modify rows and columns

  • delete_test_data_profile — delete a data profile

Test Generation (10 tools)

  • generate_contextqa_tests_from_n8n — generate from n8n workflow files or URLs

  • generate_tests_from_code_change — generate from a git diff

  • generate_tests_from_jira_ticket — generate from Jira or Azure DevOps tickets

  • generate_tests_from_linear_ticket — generate from Linear issues

  • generate_tests_from_figma — generate from Figma design URLs

  • generate_tests_from_requirements — generate from plain text requirements

  • generate_tests_from_excel — generate from Excel or CSV test libraries

  • generate_tests_from_swagger — generate from OpenAPI specifications

  • generate_tests_from_video — generate from screen recording videos

  • generate_edge_cases — AI-inferred boundary and negative test scenarios

Bug & Defect (3 tools)

  • create_defect_ticket — push a test failure to Jira or Azure DevOps

  • get_auto_healing_suggestions — AI-proposed locator fixes for failed steps

  • approve_auto_healing — accept and apply a healing suggestion

Advanced Testing (3 tools)

  • execute_performance_test — run a load or performance test

  • execute_security_dast_scan — run a DAST security scan

  • export_test_case_as_code — export a test case as runnable code (Playwright TypeScript)

AI-Powered Analysis (3 tools)

  • get_root_cause — AI root cause analysis of a specific test failure

  • query_repository — semantic search of the test repository

  • analyze_test_impact — identify tests impacted by a code change

Analytics & Coverage (2 tools)

  • analyze_coverage_gaps — identify application flows with no test coverage

  • generate_tests_from_analytics_gap — create tests to close identified gaps

Custom Agents & Knowledge Bases (4 tools)

  • list_custom_agents — list all custom agent personas

  • create_custom_agent — create a new custom agent

  • list_knowledge_bases — list all knowledge bases

  • create_knowledge_base — create a new knowledge base

Telemetry (5 tools)

  • get_test_step_results — raw per-step result data

  • get_network_logs — browser HAR network log for an execution

  • get_console_logs — browser console output for an execution

  • get_trace_url — Playwright trace viewer URL

  • get_ai_reasoning — per-step AI confidence scores and locator decisions

Support-to-Fix (2 tools)

  • reproduce_from_ticket — reproduce a bug from a support ticket

  • investigate_failure — deep investigation of a specific execution failure

Migration Platform (3 tools)

  • analyze_test_repo — analyze a test repository and report its structure

  • migrate_repo_to_contextqa — convert existing test code to ContextQA

  • export_to_playwright — export all ContextQA tests as Playwright TypeScript

hashtag
Configuration Variables

Variable
Required
Description

CONTEXTQA_USERNAME

Yes

ContextQA account email address

CONTEXTQA_PASSWORD

Yes

ContextQA account password

N8N_API_KEY

No

API key for n8n Cloud workflow integration

CONTEXTQA_TENANT

No

Tenant identifier for multi-tenant n8n deployments

hashtag
Server Configuration

  • Default port: 8080

  • Health check endpoint: GET /health

  • MCP endpoint: POST /mcp

  • Supported transports: HTTP (default), stdio, SSE

  • Authentication: Per-request re-login using CONTEXTQA_USERNAME + CONTEXTQA_PASSWORD

  • Minimum Python version: 3.9

hashtag
Related Pages

PreviousKeyboard Shortcuts

Last updated

Was this helpful?