> For the complete documentation index, see [llms.txt](https://learning.contextqa.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://learning.contextqa.com/business-use-cases/regression-risk-and-evidence.md).

# Regression Risk and Evidence

## Quick answer

Build a risk-based regression program with targeted execution, self-healing, root cause analysis, and auditable run evidence. Use this page to connect the business problem to a supported ContextQA workflow, its expected value, and its operating considerations.

## What this page covers

{% hint style="info" %}
**Business problem:** Full regression is slow, while a small smoke suite can miss high-impact failures. Release owners need a defensible way to choose tests and rapidly explain failures.
{% endhint %}

ContextQA combines business priority, usage context, suites, test plans, impacted-test analysis, and execution history. The goal is not to run fewer tests by default; it is to run the right scope for each decision and retain evidence for the result.

## Build a regression portfolio

Organize cases by purpose rather than maintaining one undifferentiated list:

| Portfolio           | When to run                                            | Example content                                                                 |
| ------------------- | ------------------------------------------------------ | ------------------------------------------------------------------------------- |
| Smoke               | Every deployment or environment check                  | Login, primary navigation, one critical transaction                             |
| Changed-area        | Pull request, feature branch, or targeted verification | Tests linked to changed requirements, services, pages, or dependencies          |
| Critical regression | Release candidate                                      | High-priority and high-usage business journeys, including negative cases        |
| Full regression     | Scheduled or milestone run                             | Supported browsers, devices, integrations, and longer-tail workflows            |
| Revisit/quarantine  | After triage                                           | Flaky, data-blocked, or environment-blocked cases with an owner and review date |

Use test suites to express feature ownership and test plans to define execution: environment, browser, device, schedule, parallelism, notifications, and release-gate behavior.

## Prioritize by risk

The RBT view plots usage frequency against business priority. Start with high-frequency, high-priority areas, then add recent code and requirement impact, defect history, and integration criticality.

Do not interpret the heatmap as a substitute for judgment. A rarely used workflow may still be legally, financially, or operationally critical. Document overrides and keep the priority metadata current.

## Run at the right point in delivery

* Trigger changed-area or smoke plans from CI/CD.
* Run broader plans on a schedule or before a release decision.
* Use parallel execution where the environment and test data can safely support concurrency.
* Add browsers, viewports, and devices according to supported-user data, not an arbitrary matrix.
* Send a concise result summary to the delivery channel and link to the detailed run.

## Triage with evidence, not guesswork

![ContextQA execution result showing passed steps, self-healing percentage, and screenshot, video, and trace evidence](/files/QGcky7B8xX1J3tpqwciI)

Review a failed run in this order:

1. **Failed step and assertion:** What was expected, and what actually occurred?
2. **Screenshot and video:** Was the application in the expected state? Did a modal, redirect, or loading condition intervene?
3. **Console and network:** Did a client error, failed request, timeout, or service response explain the visible failure?
4. **Trace:** Can the interaction and timing be replayed in detail?
5. **History:** Is the failure new, recurring, environment-specific, or flaky?
6. **Self-heal record:** Did the test use a recovered element, and was it the intended control?

Classify the result as a product defect, test defect, data issue, environment issue, dependency failure, or known flake. Root-cause summaries accelerate triage, but the attached evidence remains the audit trail.

## Release decision example

A team shipping a revised checkout can run:

* tests linked to the checkout requirements and code changes;
* high-priority, high-usage purchase and refund flows;
* payment API and persisted-order checks;
* supported browser and responsive viewport combinations;
* one declined-payment and one dependency-failure scenario.

The release owner receives the plan status, failing cases, root-cause context, and evidence links. Any excluded case should have an explicit reason and owner.

## Success measures

* Regression feedback time by plan type
* Critical business flows passing before release
* Failures triaged without rerun
* Product defects versus test, data, and environment failures
* Flaky tests and recurring failure clusters
* Self-healed steps reviewed and accepted
* Release incidents that had no corresponding pre-release test

## Related documentation

* [Build and schedule a regression suite](/execution/regression-suite-tutorial.md)
* [Analytics Dashboard](/reporting/analytics-dashboard.md)
* [Failure analysis](/reporting/failure-analysis.md)
* [Parallel execution](/execution/parallel-execution.md)
* [Video and screenshots](/execution/video-and-screenshots.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://learning.contextqa.com/business-use-cases/regression-risk-and-evidence.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.