> For the complete documentation index, see [llms.txt](https://learning.contextqa.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://learning.contextqa.com/planning-and-decision-guides/ai-testing-change-decision-log.md).

# AI Testing Change-to-Decision Log

Decide whether an AI testing, agent, MCP, framework, or model-routing update should change a QA workflow, evaluation criterion, or existing guide.

> **Editorial owner:** ContextQA Documentation · **Last updated:** September 12, 2026

## Quick answer

Publish or act on an AI-testing update only when a verified release changes a supported workflow, risk, evaluation criterion, or integration boundary. Record the primary source and date, distinguish announced from observed behavior, run the affected fixture, and update the existing authoritative guide instead of creating a trend article for every launch.

## Use a materiality test

An update is material when it changes at least one of:

* inputs, outputs, or required assertions;
* tool discovery, permissions, or authentication;
* test generation or healing behavior;
* state, timeout, retry, or cancellation semantics;
* model/provider routing, retention, or cost controls;
* browser, device, framework, or dependency compatibility;
* evidence available to release reviewers; or
* a previously documented limitation.

Popularity, launch volume, social engagement, and “AI trend” language do not make a change material to QA.

## Create the change record

| Field                 | Required entry                                         |
| --------------------- | ------------------------------------------------------ |
| Product/component     | Exact name and provider                                |
| Release/source date   | Date shown in primary release documentation            |
| Source                | Official release note or current documentation         |
| Announced behavior    | Accurate paraphrase                                    |
| Observed behavior     | Fixture, version, result, and evidence—or `Not tested` |
| Affected workflow     | Named test, agent, CI, migration, or review path       |
| Risk/opportunity      | Concrete consequence                                   |
| Existing guide        | Canonical page to update                               |
| Decision              | No change, revalidate, revise, restrict, or stop       |
| Owner and review date | Named person/date                                      |

## Current examples of what merits review

As of September 12, 2026:

* Playwright documents planner, generator, and healer test agents, and instructs users to regenerate agent definitions after Playwright upgrades. Teams using those definitions should version them and rerun skill/fixture validation after upgrades.
* Claude Code’s MCP documentation covers remote HTTP, project trust and approval, scopes, tool availability, and dynamic discovery behavior. A client upgrade can therefore change setup or troubleshooting guidance even when the ContextQA server did not change.
* OpenRouter documents endpoint-specific provider policies plus controls such as ZDR and guardrails. A provider, endpoint, or policy change can require renewed approval for QA context routing.

These are examples of review triggers, not proof that every team must adopt the features.

## Verify before changing the workflow

1. Read the primary source and capture the exact version/date.
2. Identify one affected representative fixture.
3. Run the existing workflow unchanged and retain the baseline.
4. Apply the update in an isolated environment.
5. Run normal, seeded-failure, permission, and recovery checks that match the change.
6. Compare artifacts and human effort.
7. Update the canonical guide only when behavior or instructions materially changed.

If you did not run the change, label the entry **announced, not observed**.

## Avoid content debt

* Do not create a new URL for every model or framework release.
* Do not update the “modified” date when the substantive guidance is unchanged.
* Do not infer buyer demand from release frequency.
* Do not mass-produce keyword variants of the same answer.
* Archive or merge obsolete update pages into the relevant evergreen guide.

Google Search’s current generative-content guidance emphasizes accuracy, quality, and relevance; scaled low-value pages can violate spam policies regardless of how they were produced.

## Frequently asked questions

### Is an official announcement enough to change a QA workflow?

No. An announcement establishes what a provider claims, not what your application, permissions, data, and release path will experience. Record the claim, reproduce the affected workflow in an isolated fixture, and label it **announced, not observed** until evidence exists.

### How often should the change log be reviewed?

Review it when a depended-on product, model, framework, integration, or policy changes, and include a periodic review for quiet dependencies. The cadence should follow operational risk; a release-gating integration deserves more frequent review than an optional research tool.

### When should a change become a new documentation page?

Create a new page only when readers have a distinct decision or task that the existing guide cannot answer cleanly. Version updates and corrected limitations normally belong in the current authoritative page so search engines and AI systems do not encounter competing explanations.

## Related documentation

* [Playwright Test Agents](https://playwright.dev/docs/test-agents)
* [Claude Code MCP Documentation](https://code.claude.com/docs/en/mcp)
* [OpenRouter Provider Logging](https://openrouter.ai/docs/guides/privacy/provider-logging)
* [Google Search Guidance on Generative AI Content](https://developers.google.com/search/docs/fundamentals/using-gen-ai-content)
* [MCP Connection Troubleshooting](/mcp-server/troubleshooting.md)

{% hint style="info" %}
[**Review a material change with ContextQA →**](https://contextqa.com/book-a-demo/?utm_source=learning.contextqa.com\&utm_medium=referral\&utm_campaign=docs_demand_gen\&utm_content=ai_testing_change_log) when it affects a pilot or release gate, and bring the source, fixture, and acceptance evidence.
{% endhint %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://learning.contextqa.com/planning-and-decision-guides/ai-testing-change-decision-log.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.