PR Impact Analysis

Use ContextQA PR Impact Analysis to map a GitHub pull request to affected, reusable, or missing tests with confidence and code-change evidence.

Quick answer

ContextQA PR Impact Analysis reads a GitHub pull request, maps its code changes to the test repository, and produces a reviewable report grouped by recommended action. Start in Settings → PR Impact Analysis, define the base branch and path scope, then open a PR Impact row in Assets → Requirements. The initial production experience is an advisory, read-only report: it explains what is affected and why, but it does not change test cases or block the GitHub build.

What this page covers

Who is this for? QA engineers, SDETs, developers, and release owners who want change-based regression selection without relying on test names or tribal knowledge.

PR Impact Analysis is a limited-rollout capability. The settings route or PR Impact requirement type may be unavailable until ContextQA enables it for your workspace and connects an eligible GitHub repository.

What PR Impact Analysis answers

For each pull request, the report answers three citation-friendly questions:

1. Which tests are related to this change? The report groups impacted tests by its recommended verdict.
2. Why is each test included? Expand a row to inspect code-change evidence, confidence, and the generated MCP prompt.
3. How trustworthy is the recommendation? The report identifies confidence and whether the match came from structural application mapping or semantic test knowledge.

This makes the analysis useful as both a release-review surface and an evidence trail for an AI agent preparing a regression run.

Configure the analysis boundary

Open Settings → PR Impact Analysis.

PR Impact Analysis settings for confidence, repository scope, and GitHub reporting

The current settings page includes:

Setting	What it controls
Autonomy tier	The operating mode. Read produces an advisory report and makes no test changes. Other tiers may be visible but are not part of the initial read-only rollout.
Confidence threshold	Recommendations below the threshold are held rather than treated as high-confidence guidance.
Base branch filter	Limits analysis to pull requests targeting the named branch, such as main.
Max files per analysis	Caps the files evaluated in one run. Reports beyond the limit are marked partial.
Path scope	One glob per line for the repository or monorepo subtree to include.
Post PR comment	Posts or updates the ContextQA summary on the pull request when enabled.
Post GitHub check	Adds an advisory GitHub check. The check does not fail the build.
Suppress paths	Excludes generated, vendor, or other noisy paths from analysis.

The settings snapshot is stored with an analysis so a reviewer can understand the boundary used at that time, even after an administrator changes the live configuration.

How a pull request becomes a coverage report

The GitHub comment is updated in place as the pull request changes, so reviewers do not need to follow a chain of obsolete comments. A newer commit can supersede an earlier analysis; ContextQA preserves the earlier report as history and links to the newer run.

Review a PR Impact report

1. Open Assets → Requirements.
2. Find the row whose type is PR Impact and open its REQ-<id> link.
3. Confirm the pull-request repository, target branch, commit range, author, and changed-file count in the header.
4. Read the AI summary and check for a Partial or Superseded notice.
5. Use Overview to review the recommended-action totals and average confidence.
6. Open each verdict tab to inspect the affected test cases.
7. Expand Evidence, confidence & prompt on a row.
8. Use the report to select or update the regression scope through your normal reviewed workflow.

The expanded row can show:

• the linked test-case ID and title;
• priority and confidence;
• the reason the test was associated with the change;
• relevant code-change evidence;
• whether the signal is structural or semantic; and
• a generated MCP prompt that an authorized AI agent can use as a starting point.

The initial production report is read-only. It does not approve, dismiss, edit, create, retire, or run a test by itself. Apply any change through the normal test-case approval and execution controls available in your workspace.

Read the result safely

• Treat high confidence as stronger evidence, not as proof that the test will pass.
• A semantic match means the diff aligns with test knowledge; a structural match means the changed file maps to an exercised route or component.
• A partial report did not analyze every changed file. Review the unprocessed scope before using it as a release gate.
• No behavioural change detected is a valid result. Confirm the repository and path settings before assuming no regression is possible.
• A failed analysis offers retry and full-suite fallback actions; it should not produce a false empty report.

FAQ

Does PR Impact Analysis change my test cases automatically?

Not in the initial read-only rollout. It produces recommendations, evidence, and an MCP prompt while leaving the test repository unchanged.

Does the GitHub check block a merge?

No. The current GitHub check is advisory and does not fail the build.

Why is a report marked partial?

The pull request exceeded the configured file limit or another analysis boundary. Review the report notice and expand the regression scope when needed.

Related documentation

• Requirements management
• Requirements coverage gaps
• GitHub Actions
• Running tests
• MCP agent integration

See AI change-impact analysis on your repository. Book a Demo → — Review how ContextQA turns a pull request into an evidence-backed regression scope.