AI Testing Portfolio Lab
A practical, evidence-led lab for learning AI-assisted test design with a disposable application, explicit assertions, a seeded defect, and a reviewable results packet.
Quick answer
Build a small, reproducible testing project around a disposable application you control. The useful output is not the number of generated tests; it is a traceable set of requirements, explicit assertions, a deliberately failing control, execution evidence, and a short explanation of what you accepted or changed.
What this page covers
This optional learning lab helps testers, developers, and evaluators practice an evidence-led ContextQA workflow. It is not a certification, hiring promise, or substitute for testing the constraints of your own application.
Choose a safe fixture
Use a local sample application, a public demo designed for automation, or a user-owned non-production environment. It must be legal to test and easy to reset. Do not use production accounts, customer records, or credentials copied into screenshots.
A strong fixture contains:
a login or session boundary;
one create-or-update workflow;
one validation or error state;
two roles or tenants if authorization matters;
stable reset data;
one defect you can deliberately enable and disable.
Define the project brief
Write a one-page brief before generating anything:
Build a compact test portfolio
Create only enough coverage to demonstrate different reasoning problems:
Happy path
Prove the main user outcome
Saved record appears with the expected values
Negative validation
Prove invalid input is rejected
Error appears and no record is created
Role boundary
Prove least privilege
Standard user cannot access the admin action
Tenant boundary
Detect data leakage
Record created in tenant A is absent in tenant B
Seeded defect
Prove the suite can fail
The changed outcome produces a failed assertion
Use specific observable outcomes. “Complete checkout” is an instruction; “order confirmation shows the expected order number and the cart is empty” is testable.
Execute in two controls
Reset the fixture and run the portfolio with the application behaving correctly.
Save the ContextQA result IDs and confirm the expected evidence exists.
Enable the seeded defect without changing the tests.
Run the affected test again.
Confirm the intended assertion fails and the evidence points to the changed behavior.
Restore the fixture and rerun to demonstrate recovery.
If the test passes while the defect is active, the project has exposed a weak assertion. Fix the test and repeat the controls; do not count the original run as success.
Create the evidence packet
Keep the portfolio reviewable and safe to share:
project brief and scope;
requirement-to-test map;
test names and explicit assertions;
fixture and reset instructions;
clean-run and seeded-failure result IDs;
selected redacted screenshots or trace links;
one failure-analysis note that separates observation from hypothesis;
decisions: accepted, revised, retained elsewhere, or unresolved;
limitations and the next experiment.
Review rubric
Traceability
Each test maps to a stated risk or requirement
Assertion quality
Assertions verify business outcomes, not just clicks
Failure sensitivity
The seeded defect creates the expected failure
Boundary coverage
Roles, tenants, data, and environment are explicit
Reproducibility
Another person can reset and rerun the lab
Diagnostic quality
Claims link to artifacts and distinguish facts from hypotheses
Judgment
The write-up explains what was changed, retained, or rejected and why
Extend the lab
After the core portfolio is reliable, add one focused extension: an API-to-UI chain, a CI negative-path gate, a self-healing acceptance experiment, or an MCP agent workflow. Add only one variable at a time so you can explain what changed.
Related documentation
Last updated
Was this helpful?