> For the complete documentation index, see [llms.txt](https://learning.contextqa.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://learning.contextqa.com/mcp-server/tool-reference/telemetry-and-agents.md). # Telemetry & Custom Agents {% hint style="info" %} **Who is this for?** SDETs, developers, and DevOps engineers integrating ContextQA with AI coding assistants (Claude, Cursor) or CI/CD pipelines. {% endhint %} This reference covers two groups: telemetry tools that surface low-level execution evidence (step results, network traffic, console logs, Playwright traces, AI reasoning), and the custom agents and knowledge base tools that allow you to configure AI behaviour for test execution. *** ## Telemetry *** ## get\_test\_step\_results Retrieves raw per-step execution data for a given result, including the action performed, pass/fail status, screenshot URL, executed result detail, and AI reasoning summary. **Category:** Telemetry | **Authentication required:** Yes ### Parameters | Name | Required | Type | Description | | ---------- | -------- | ------- | ---------------------------------------------------------------------------------------------- | | result\_id | ✅ | integer | Numeric ID of the test case result to retrieve steps for. Obtain from `get_test_case_results`. | ### Returns JSON array of step records. Each record includes `stepIndex`, `action`, `status` (`PASSED`/`FAILED`/`SKIPPED`), `screenshot_url`, `executedResult`, and a brief `aiReasoning` field. ### Example ```json { "result_id": 1042 } ``` ### Related Tools `get_test_case_results`, `get_ai_reasoning`, `get_network_logs`, `get_console_logs`, `investigate_failure` *** ## get\_network\_logs Returns the HAR-format network log capturing all HTTP requests and responses made during a test execution result; use this to debug API calls, check response codes, and inspect payload content. **Category:** Telemetry | **Authentication required:** Yes ### Parameters | Name | Required | Type | Description | | ---------- | -------- | ------- | --------------------------------------------------------------------- | | result\_id | ✅ | integer | Numeric ID of the test case result whose network traffic to retrieve. | ### Returns HAR-format JSON (`log.entries` array) with each entry containing `request` (method, URL, headers, body), `response` (status, headers, body), and timing data. ### Example ```json { "result_id": 1042 } ``` ### Related Tools `get_test_step_results`, `get_console_logs`, `get_trace_url`, `investigate_failure` *** ## get\_console\_logs Returns all browser console entries recorded during a test execution result, including log, warn, error, and uncaught exception messages. **Category:** Telemetry | **Authentication required:** Yes ### Parameters | Name | Required | Type | Description | | ---------- | -------- | ------- | ------------------------------------------------------------------- | | result\_id | ✅ | integer | Numeric ID of the test case result to retrieve console entries for. | ### Returns JSON array of console entries. Each entry includes `level` (`log`, `warn`, `error`, `exception`), `message`, `source`, and `timestamp`. ### Example ```json { "result_id": 1042 } ``` ### Related Tools `get_network_logs`, `get_test_step_results`, `get_trace_url`, `investigate_failure` *** ## get\_trace\_url Returns the Playwright Trace Viewer URL for a test execution result, enabling step-by-step visual replay with DOM snapshots and network activity. **Category:** Telemetry | **Authentication required:** Yes > **Note:** Access to trace files requires the Trace Viewer add-on licence. If your workspace does not have this licence, the call returns a `403` error. ### Parameters | Name | Required | Type | Description | | ---------- | -------- | ------- | ------------------------------------------------------------- | | result\_id | ✅ | integer | Numeric ID of the test case result to retrieve the trace for. | ### Returns JSON with `trace_url` (a direct link to the Playwright Trace Viewer) and `expires_at` (ISO 8601 timestamp indicating when the URL expires). ### Example ```json { "result_id": 1042 } ``` ### Related Tools `get_test_step_results`, `get_network_logs`, `investigate_failure` *** ## get\_ai\_reasoning Returns detailed per-step AI reasoning data for an execution result, including confidence scores, the locator strategy chosen, DOM similarity analysis, and any fallback decisions made by the AI agent. **Category:** Telemetry | **Authentication required:** Yes ### Parameters | Name | Required | Type | Description | | ---------- | -------- | ------- | --------------------------------------------------------------- | | result\_id | ✅ | integer | Numeric ID of the test case result to inspect AI reasoning for. | ### Returns JSON array of reasoning records, one per step. Each record includes `stepIndex`, `confidence` (0–1 float), `locatorStrategy`, `domSimilarityScore`, `fallbackUsed` (boolean), and `reasoningNarrative`. ### Example ```json { "result_id": 1042 } ``` ### Related Tools `get_test_step_results`, `get_ai_insights`, `investigate_failure` *** ## get\_ai\_insights Returns aggregated AI insights and analytics patterns across executions; optionally scoped to a specific result or workspace version to focus analysis. **Category:** Telemetry | **Authentication required:** Yes ### Parameters | Name | Required | Type | Description | | ----------- | -------- | ------- | ----------------------------------------------- | | result\_id | ❌ | integer | Scope insights to a specific test result. | | version\_id | ❌ | string | Scope insights to a specific workspace version. | ### Returns JSON with aggregated metrics including `flakiness_score`, `top_failure_categories`, `healing_frequency`, and `coverage_trend` data points. ### Example ```json { "version_id": "v_2025_q1" } ``` ### Related Tools `get_ai_reasoning`, `analyze_coverage_gaps`, `get_test_case_results` *** ## Custom Agents & Knowledge Base *** ## list\_custom\_agents Returns all custom agent personas defined in the workspace; use agent IDs as the `persona_id` parameter when executing test cases or test plans. **Category:** Custom Agents | **Authentication required:** Yes ### Parameters None. ### Returns JSON array of agent persona objects. Each object includes `id`, `name`, `description` (the system prompt), and `createdAt`. ### Example ```json {} ``` ### Related Tools `create_custom_agent`, `execute_test_case`, `execute_test_plan` *** ## create\_custom\_agent Creates a new custom AI agent persona with a name and a system prompt that defines its behaviour and knowledge scope during test execution. **Category:** Custom Agents | **Authentication required:** Yes ### Parameters | Name | Required | Type | Description | | ----------- | -------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | | name | ✅ | string | Display name for the agent persona. | | description | ❌ | string | System prompt defining the agent's behaviour. This plain-English instruction set guides how the AI navigates and interacts with the application during execution. | ### Returns JSON of the created agent persona including its assigned `id`, which can then be used as `persona_id` in execution calls. ### Example ```json { "name": "Accessibility Auditor", "description": "You are an accessibility testing agent. Focus on WCAG 2.1 AA compliance. Always check for keyboard navigation, ARIA labels, colour contrast issues, and screen reader compatibility on every page you visit." } ``` ### Related Tools `list_custom_agents`, `execute_test_case`, `execute_test_plan` *** ## list\_knowledge\_bases Returns all knowledge bases defined in the workspace; use knowledge base IDs as the `knowledge_id` parameter when executing test cases or test plans to give the AI agent additional domain context. **Category:** Knowledge Base | **Authentication required:** Yes ### Parameters None. ### Returns JSON array of knowledge base objects. Each object includes `id`, `title`, `prompt`, and `createdAt`. ### Example ```json {} ``` ### Related Tools `create_knowledge_base`, `execute_test_case`, `execute_test_plan` *** ## create\_knowledge\_base Creates a new knowledge base containing plain-English instructions that are injected into the AI agent's context during test execution to provide domain-specific guidance. **Category:** Knowledge Base | **Authentication required:** Yes ### Parameters | Name | Required | Type | Description | | ------ | -------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | | title | ✅ | string | Display name for the knowledge base. | | prompt | ✅ | string | Plain-English instructions for the AI agent. Describe domain rules, business logic, expected behaviours, or navigation patterns the agent should be aware of. | ### Returns JSON of the created knowledge base including its assigned `id`, which can then be used as `knowledge_id` in execution calls. ### Example ```json { "title": "E-commerce Checkout Rules", "prompt": "The checkout flow has three stages: Cart Review, Shipping Details, and Payment. Guest checkout is available but requires an email address. Promo codes are entered on the Cart Review page. Free shipping applies automatically for orders over $50. The payment page accepts Visa, Mastercard, and PayPal — no Amex." } ``` ### Related Tools `list_knowledge_bases`, `execute_test_case`, `execute_test_plan`, `create_custom_agent` *** --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://learning.contextqa.com/mcp-server/tool-reference/telemetry-and-agents.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.