> For the complete documentation index, see [llms.txt](https://learning.contextqa.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://learning.contextqa.com/getting-started/ai-testing-portfolio-lab.md).

# AI Testing Portfolio Lab

A practical, evidence-led lab for learning AI-assisted test design with a disposable application, explicit assertions, a seeded defect, and a reviewable results packet.

## Quick answer

Build a small, reproducible testing project around a disposable application you control. The useful output is not the number of generated tests; it is a traceable set of requirements, explicit assertions, a deliberately failing control, execution evidence, and a short explanation of what you accepted or changed.

## What this page covers

This optional learning lab helps testers, developers, and evaluators practice an evidence-led ContextQA workflow. It is not a certification, hiring promise, or substitute for testing the constraints of your own application.

## Choose a safe fixture

Use a local sample application, a public demo designed for automation, or a user-owned non-production environment. It must be legal to test and easy to reset. Do not use production accounts, customer records, or credentials copied into screenshots.

A strong fixture contains:

* a login or session boundary;
* one create-or-update workflow;
* one validation or error state;
* two roles or tenants if authorization matters;
* stable reset data;
* one defect you can deliberately enable and disable.

## Define the project brief

Write a one-page brief before generating anything:

```
Goal: Prove that a standard user can create an item and see it in their own account.
Out of scope: Billing, email delivery, and production data.
Environment: Disposable staging fixture.
Roles: Standard user and administrator.
Critical risks: Cross-tenant visibility and false success after a failed save.
Evidence required: Step assertions, screenshots, run video, result ID, and defect note.
Reset: Delete created records and restore the seeded-defect flag.
```

## Build a compact test portfolio

Create only enough coverage to demonstrate different reasoning problems:

| Test                | Purpose                         | Required assertion                               |
| ------------------- | ------------------------------- | ------------------------------------------------ |
| Happy path          | Prove the main user outcome     | Saved record appears with the expected values    |
| Negative validation | Prove invalid input is rejected | Error appears and no record is created           |
| Role boundary       | Prove least privilege           | Standard user cannot access the admin action     |
| Tenant boundary     | Detect data leakage             | Record created in tenant A is absent in tenant B |
| Seeded defect       | Prove the suite can fail        | The changed outcome produces a failed assertion  |

Use specific observable outcomes. “Complete checkout” is an instruction; “order confirmation shows the expected order number and the cart is empty” is testable.

## Execute in two controls

1. Reset the fixture and run the portfolio with the application behaving correctly.
2. Save the ContextQA result IDs and confirm the expected evidence exists.
3. Enable the seeded defect without changing the tests.
4. Run the affected test again.
5. Confirm the intended assertion fails and the evidence points to the changed behavior.
6. Restore the fixture and rerun to demonstrate recovery.

If the test passes while the defect is active, the project has exposed a weak assertion. Fix the test and repeat the controls; do not count the original run as success.

## Create the evidence packet

Keep the portfolio reviewable and safe to share:

* project brief and scope;
* requirement-to-test map;
* test names and explicit assertions;
* fixture and reset instructions;
* clean-run and seeded-failure result IDs;
* selected redacted screenshots or trace links;
* one failure-analysis note that separates observation from hypothesis;
* decisions: accepted, revised, retained elsewhere, or unresolved;
* limitations and the next experiment.

## Review rubric

| Dimension           | Strong evidence                                                       |
| ------------------- | --------------------------------------------------------------------- |
| Traceability        | Each test maps to a stated risk or requirement                        |
| Assertion quality   | Assertions verify business outcomes, not just clicks                  |
| Failure sensitivity | The seeded defect creates the expected failure                        |
| Boundary coverage   | Roles, tenants, data, and environment are explicit                    |
| Reproducibility     | Another person can reset and rerun the lab                            |
| Diagnostic quality  | Claims link to artifacts and distinguish facts from hypotheses        |
| Judgment            | The write-up explains what was changed, retained, or rejected and why |

## Extend the lab

After the core portfolio is reliable, add one focused extension: an API-to-UI chain, a CI negative-path gate, a self-healing acceptance experiment, or an MCP agent workflow. Add only one variable at a time so you can explain what changed.

## Related documentation

* [Quickstart Guide](/getting-started/quickstart.md)
* [AI Testing Pilot Scorecard](/evaluation-and-governance/ai-testing-pilot-scorecard.md)
* [AI Test Generation](/ai-features/ai-test-generation.md)
* [Test Results](/reporting/test-results.md)
* [Test Data Management](/web-testing/test-data-management.md)

{% hint style="info" %}
When this lab is repeatable, use the [30-Day AI Testing Pilot Scorecard](/evaluation-and-governance/ai-testing-pilot-scorecard.md) to evaluate a representative workflow from your own delivery process.
{% endhint %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://learning.contextqa.com/getting-started/ai-testing-portfolio-lab.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.