> For the complete documentation index, see [llms.txt](https://learning.contextqa.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://learning.contextqa.com/ai-features/pr-impact-analysis.md).
# PR Impact Analysis
Use ContextQA PR Impact Analysis to map a GitHub pull request to affected, reusable, or missing tests with confidence and code-change evidence.
## Quick answer
ContextQA PR Impact Analysis reads a GitHub pull request, maps its code changes to the test repository, and produces a reviewable report grouped by recommended action. Start in **Settings → PR Impact Analysis**, define the base branch and path scope, then open a **PR Impact** row in **Assets → Requirements**. The initial production experience is an advisory, read-only report: it explains what is affected and why, but it does not change test cases or block the GitHub build.
## What this page covers
{% hint style="info" %}
**Who is this for?** QA engineers, SDETs, developers, and release owners who want change-based regression selection without relying on test names or tribal knowledge.
{% endhint %}
{% hint style="warning" %}
PR Impact Analysis is a limited-rollout capability. The settings route or PR Impact requirement type may be unavailable until ContextQA enables it for your workspace and connects an eligible GitHub repository.
{% endhint %}
## What PR Impact Analysis answers
For each pull request, the report answers three citation-friendly questions:
1. **Which tests are related to this change?** The report groups impacted tests by its recommended verdict.
2. **Why is each test included?** Expand a row to inspect code-change evidence, confidence, and the generated MCP prompt.
3. **How trustworthy is the recommendation?** The report identifies confidence and whether the match came from structural application mapping or semantic test knowledge.
This makes the analysis useful as both a release-review surface and an evidence trail for an AI agent preparing a regression run.
## Configure the analysis boundary
Open **Settings → PR Impact Analysis**.
The current settings page includes:
| Setting | What it controls |
| -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Autonomy tier** | The operating mode. **Read** produces an advisory report and makes no test changes. Other tiers may be visible but are not part of the initial read-only rollout. |
| **Confidence threshold** | Recommendations below the threshold are held rather than treated as high-confidence guidance. |
| **Base branch filter** | Limits analysis to pull requests targeting the named branch, such as `main`. |
| **Max files per analysis** | Caps the files evaluated in one run. Reports beyond the limit are marked partial. |
| **Path scope** | One glob per line for the repository or monorepo subtree to include. |
| **Post PR comment** | Posts or updates the ContextQA summary on the pull request when enabled. |
| **Post GitHub check** | Adds an advisory GitHub check. The check does not fail the build. |
| **Suppress paths** | Excludes generated, vendor, or other noisy paths from analysis. |
{% hint style="info" %}
The settings snapshot is stored with an analysis so a reviewer can understand the boundary used at that time, even after an administrator changes the live configuration.
{% endhint %}
## How a pull request becomes a coverage report
```mermaid
flowchart LR
A[GitHub pull request updated] --> B{Matches base branch
and path scope?}
B -- No --> C[No analysis created]
B -- Yes --> D[ContextQA maps changed code
to test knowledge]
D --> E[PR Impact requirement]
E --> F[Overview: actions and confidence]
F --> G[Verdict tabs: impacted tests]
G --> H[Evidence, confidence, and MCP prompt]
H --> I[Reviewer selects the regression scope]
```
The GitHub comment is updated in place as the pull request changes, so reviewers do not need to follow a chain of obsolete comments. A newer commit can supersede an earlier analysis; ContextQA preserves the earlier report as history and links to the newer run.
## Review a PR Impact report
1. Open **Assets → Requirements**.
2. Find the row whose type is **PR Impact** and open its `REQ-` link.
3. Confirm the pull-request repository, target branch, commit range, author, and changed-file count in the header.
4. Read the **AI summary** and check for a **Partial** or **Superseded** notice.
5. Use **Overview** to review the recommended-action totals and average confidence.
6. Open each verdict tab to inspect the affected test cases.
7. Expand **Evidence, confidence & prompt** on a row.
8. Use the report to select or update the regression scope through your normal reviewed workflow.
The expanded row can show:
* the linked test-case ID and title;
* priority and confidence;
* the reason the test was associated with the change;
* relevant code-change evidence;
* whether the signal is structural or semantic; and
* a generated MCP prompt that an authorized AI agent can use as a starting point.
{% hint style="warning" %}
The initial production report is read-only. It does not approve, dismiss, edit, create, retire, or run a test by itself. Apply any change through the normal test-case approval and execution controls available in your workspace.
{% endhint %}
## Read the result safely
* Treat **high confidence** as stronger evidence, not as proof that the test will pass.
* A **semantic** match means the diff aligns with test knowledge; a **structural** match means the changed file maps to an exercised route or component.
* A **partial** report did not analyze every changed file. Review the unprocessed scope before using it as a release gate.
* **No behavioural change detected** is a valid result. Confirm the repository and path settings before assuming no regression is possible.
* A failed analysis offers retry and full-suite fallback actions; it should not produce a false empty report.
## Measure selection quality with a run ledger
An impact report proposes a regression scope. To evaluate whether that scope is safe, compare it with a broader reference run on the same commit, environment, data revision, and test revision.
```mermaid
flowchart LR
A[Commit-specific impact report] --> B[Selected test run]
A --> C[Broader reference run]
B --> D[Compare failures and omissions]
C --> D
D --> E[Accept, tune, or fall back]
```
Use a time-bounded cohort of representative pull requests, including risky changes and changes that should not affect testable behavior. For each pull request, preserve the analysis settings snapshot and record:
| Field | What to capture |
| -------------------------------- | -------------------------------------------------------------------------- |
| Pull request and commit | Stable PR reference and full analyzed SHA |
| Analysis scope | Base branch, included paths, suppressed paths, and file cap |
| Report state | Complete, Partial, Superseded, Failed, or no behavioral change |
| Selected tests | Test-case references and selection reasons |
| Known relevant tests omitted | Tests a qualified reviewer expected but the report did not select |
| Selected tests judged irrelevant | Recommendations without a defensible relationship to the change |
| Unknown dependencies | Shared services, schemas, or runtime behavior the analysis could not bound |
| Fallback | Expanded set, broader suite, or explicit human disposition |
| Run evidence | Selected and reference run IDs, build, environment, and result |
| Reviewer | Person who classified the outcome |
### Use precise error definitions
For this evaluation:
* A **selection false negative** is a test judged relevant by the review protocol or a failure found in the broader reference run that the impact report did not select.
* A **selection false positive** is a selected test judged irrelevant under the same review protocol.
* An **unknown** is a dependency or outcome the reviewer cannot classify from the available evidence.
These are selection-quality labels, not claims that a product defect was or was not real. Write the review protocol before measuring so the same evidence is classified consistently.
### Copyable impact ledger
| Date | PR / SHA | Scope state | Selected count | Relevant omitted | Irrelevant selected | Unknown dependencies | Selected-run result | Reference-run result | Fallback used | Evidence / reviewer |
| ---- | -------- | ----------- | -------------: | ---------------: | ------------------: | -------------------: | ------------------- | -------------------- | ------------- | ------------------- |
| | | | | | | | | | | |
When a broader run finds a relevant failure outside the selected set, treat the event as a release-safety incident for the selection rule. Preserve the diff, analysis snapshot, omitted test, failure evidence, and remediation decision.
### Decide when to fall back
Use the broader regression scope when:
* the report is `Partial`, `Failed`, or based on a superseded commit;
* the changed file count exceeds the configured analysis boundary;
* a shared dependency is unknown;
* a critical workflow has no selected test and no accepted coverage disposition;
* the application build does not match the analyzed commit; or
* the cohort exceeds the team's accepted false-negative threshold.
Do not publish a universal precision or recall claim from a small internal sample. Report the cohort, scope, definitions, and unknowns with the result.
For the complete release-decision record, use the [agentic PR evidence checklist](/evaluation-and-governance/agentic-pr-evidence-checklist.md).
## FAQ
Does PR Impact Analysis change my test cases automatically?
Not in the initial read-only rollout. It produces recommendations, evidence, and an MCP prompt while leaving the test repository unchanged.
Does the GitHub check block a merge?
No. The current GitHub check is advisory and does not fail the build.
Why is a report marked partial?
The pull request exceeded the configured file limit or another analysis boundary. Review the report notice and expand the regression scope when needed.
## Related documentation
* [Requirements management](/web-testing/requirements-management.md)
* [Requirements coverage gaps](/ai-features/requirements-coverage-gaps.md)
* [GitHub Actions](/integrations/github-actions.md)
* [Running tests](/execution/running-tests.md)
* [MCP agent integration](/mcp-server/agent-integration-guide.md)
{% hint style="info" %}
**See AI change-impact analysis on your repository.** [**Book a Demo →**](https://contextqa.com/book-a-demo/?utm_source=learning.contextqa.com\&utm_medium=referral\&utm_campaign=docs_demand_gen\&utm_content=pr_impact_analysis) — Review how ContextQA turns a pull request into an evidence-backed regression scope.
{% endhint %}
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://learning.contextqa.com/ai-features/pr-impact-analysis.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.