Ctrlk

How to Debug a Failing Test

A step-by-step workflow for diagnosing why a test failed using ContextQA's execution evidence — AI root cause analysis, screenshots, video, network logs, console logs, and Playwright traces.

Who is this for? Testers, SDETs, and developers who need to determine why a test failed and decide whether to fix the test, file a bug, or address a flaky environment issue.

When a test fails, the goal is to answer three questions as fast as possible:

  1. What failed? — Which step, what error message?

  2. Why did it fail? — Application bug, test bug, flaky timing, or environment issue?

  3. What do I do next? — Fix the test, file a bug, or stabilize the environment?

ContextQA captures a full evidence package for every execution — AI root cause analysis, per-step screenshots, session video, network logs, console logs, and a Playwright trace. This guide walks you through a systematic workflow that uses each artifact to reach a diagnosis quickly.

Prerequisites

  • A ContextQA account with access to at least one workspace

  • A failed test execution to investigate (any test case or test plan run with a FAILED status)

  • Familiarity with test results and reports — specifically how to navigate to the detailed report view

Step 1: Open the failed execution report

  1. Navigate to Test Development in the left sidebar and click the failed test case, or navigate to Test Plans under the Test Plan & Results section and click the failed plan run.

  2. Click View Detailed Report in the execution summary banner.

  3. The detailed report opens showing every step in sequence with pass/fail indicators.

Locate the first step with a red X indicator. This is the step where the failure occurred. All subsequent steps may also show failures, but the root cause is almost always in the first failing step.

Step 2: Read the AI root cause analysis

Before inspecting any artifact manually, check what the AI has already determined.

  1. Click the AI Insights button (or Root Cause Analysis button) at the top of the report.

  2. Read the structured output:

Field
What it tells you

Summary

A plain-English explanation of what failed and why

Failure category

Application Bug, Test Bug, Flaky Failure, or Environment Issue

Affected step

The step number and action where the failure occurred

Suggested fix

A specific, actionable recommendation

If the AI classification is clear and the suggested fix is actionable, you may have your answer. Proceed to Step 7: Decide your next action.

If you need more context — the classification is Flaky Failure without a clear cause, or the summary is ambiguous — continue to Step 3 to inspect the evidence yourself.

Step 3: Inspect the failing step screenshot

Click the failing step row to expand it. Click the screenshot thumbnail to open the full-size image.

What to look for:

  • Is the expected element visible on the page? If the step tried to click a button that is not on the screen, the page may not have loaded fully or navigated to the wrong URL.

  • Is a dialog, overlay, or banner blocking the element? Cookie consent banners, modal dialogs, and notification popups are common causes of "element not interactable" failures.

  • Does the page show an error state? A 500 error page, a "session expired" message, or a form validation error visible in the screenshot tells you immediately whether this is an application issue.

  • Does the UI match what the test expects? If the test expects a "Submit Order" button but the screenshot shows "Place Order", the UI label has changed and the test step needs updating.

Tip: Compare the failing step screenshot with the screenshot from the previous step. The previous step shows the page state just before the failure, which is often more revealing than the failure screenshot itself.

Step 4: Watch the video recording

If the screenshot does not make the cause obvious, watch the session video.

  1. Click the Video tab at the top of the report.

  2. Click the failing step in the step list — the video jumps to the moment that step began.

  3. Watch the 10–15 seconds leading up to the failure.

What to look for:

  • Timing issues: The page is still loading or an animation is still playing when the step executes. This points to a timing-related flaky failure.

  • Unexpected navigation: The application redirected to a different page (login page, error page) before the step could execute.

  • Element movement: A page element shifts position after rendering — the AI clicked where the button was, but it moved before the click landed.

  • Multi-step context: Sometimes the failure at step 8 was actually caused by step 5 entering the wrong value. The video reveals this because you can see the full flow in sequence.

Step 5: Check the network and console logs

If the visual evidence (screenshot and video) does not explain the failure, the cause may be in the application's back end or JavaScript layer.

Network log

  1. Click the Network tab in the report.

  2. Filter by status code to show only 4xx and 5xx responses.

What you find
What it means

A 500 Internal Server Error on an API call

The application server crashed — this is an application bug

A 401 Unauthorized or 403 Forbidden

The test session lost authentication — check if credentials expired or a token timed out

A 422 Unprocessable Entity on a form submission

Server-side validation rejected the input — the test data may be invalid

A request that took more than 5 seconds

A slow API response may have caused a timeout in the UI step

A missing request (expected API call never made)

The UI interaction did not trigger the back-end call — the click or form fill may have failed silently

Console log

  1. Click the Console tab in the report.

  2. Look for red entries (errors).

What you find
What it means

Uncaught TypeError or Uncaught ReferenceError

A JavaScript error broke the page logic — this is an application bug

Failed to fetch or net::ERR_CONNECTION_REFUSED

The application could not reach a dependency (API, CDN, database) — this is an environment issue

[Violation] 'click' handler took 3,412ms

A slow event handler may be causing the UI to appear unresponsive, leading to timeout failures

Step 6: Use the Playwright trace for deep inspection

When screenshots, video, and logs are not enough — typically for "element not found" failures where the element appears to be on the page — the Playwright trace provides the definitive answer.

  1. Click View Playwright Trace in the report. The trace viewer opens at trace.playwright.dev in a new tab.

  2. In the timeline panel (top), navigate to the failing action.

  3. In the action detail panel (left), read the exact locator that was attempted and the error.

  4. In the DOM panel (right), inspect the actual HTML structure at the moment the locator was tried.

What to look for:

  • The element exists but has a different attribute. The locator targets [data-testid="submit-btn"] but the element now has [data-testid="submit-button"]. This is a test bug — update the locator or re-record the step.

  • The element exists but is hidden. The DOM shows the element with display: none or visibility: hidden. The element has not rendered yet (timing issue) or is conditionally hidden (application behavior change).

  • The element is inside an iframe. The locator searches the main document but the target element is in an embedded iframe. Add a step to switch to the iframe context.

  • Multiple elements match the locator. The trace shows that the locator matched three elements, and the one ContextQA interacted with was not the intended target. Make the locator more specific.

Note: ContextQA retains Playwright traces for 30 days after execution and screenshots and logs for 90 days. If you are investigating an older failure, the trace may no longer be available.

Step 7: Decide your next action

Based on your investigation, take the appropriate action:

Application bug

The test is correct, but the application behaved incorrectly.

  1. Click the Report Bug button in the failure reporting toolbar.

  2. Select your bug tracker project and issue type.

  3. ContextQA pre-populates the ticket with the failure description, screenshot, and a link to the full report.

  4. Click Create Issue.

  5. The test case remains as-is — it will pass once the application bug is fixed.

Test bug

The test steps need updating because the application changed intentionally.

  1. Open the test case in the Test Steps Editor.

  2. Update the failing step — change the locator, update expected text, or add a missing step.

  3. Re-run the test case to verify the fix.

  4. If the test uses a step group, update the step group instead so the fix propagates to all test cases that use it. See Step groups tutorial.

Flaky failure

The test fails intermittently due to timing or environment variability.

  1. Add an explicit wait step before the failing step. In the step editor, add a Wait action that waits for the target element to be visible or for a network request to complete.

  2. Re-run the test three to five times to confirm the fix eliminates the intermittent failure.

  3. If the failure persists, consider whether the application under test has a genuine performance issue that needs addressing.

Environment issue

The test environment is misconfigured or unavailable.

  1. Verify the environment is running and accessible at the configured base URL. Open the URL in a browser to confirm.

  2. Check Test Development → Environments (under Test Plan & Results in the left sidebar) to confirm the environment configuration is correct.

  3. If the issue is intermittent network connectivity, re-run the test. If it persists, check firewall rules, VPN configuration, or IP allowlisting.

Using debug mode for interactive diagnosis

For failures that are difficult to diagnose from the execution report alone, re-run the test in Debug mode to interact with it in real time.

  1. Open the test case.

  2. Click Debug (instead of Run).

  3. Watch the test execute step by step. When a step fails, ContextQA pauses and presents options:

Option
When to use it

Enter more detail

The step description is ambiguous — provide a clarification (e.g., "click the admin dropdown, not the admin link")

Update and Rerun

You know the fix — edit the step and retry from that point

Add a New Step and Rerun

A missing step caused the failure — insert a wait, a click, or a navigation step

Skip Step

The step is not critical and you want to test the remaining flow

Delete Step

The step is obsolete and should be removed from the test case

Debug mode is the fastest way to iterate on a failing test without re-running the entire sequence from scratch.

Quick reference: debugging decision tree

Related pages

Diagnose failures in seconds, not hours. Book a Demo → — See AI root cause analysis and the full evidence package with your application.

PreviousFailure AnalysisNextFlaky Test Detection

Last updated

Was this helpful?