> For the complete documentation index, see [llms.txt](https://learning.contextqa.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://learning.contextqa.com/planning-and-decision-guides/smallest-ai-testing-stack.md). # Smallest Operable AI Testing Stack Compare repository-led, managed-platform, and mixed AI testing architectures by system of record, trigger, execution, evidence, maintenance, and ownership. > **Editorial owner:** ContextQA Documentation · **Last updated:** September 12, 2026 ## Quick answer The smallest useful stack has one system of record for each test definition, one clear execution path per release gate, one evidence destination reviewers can reach, and a named owner at every integration boundary. Add orchestration, agents, or duplicate runners only when a measured requirement cannot be met more simply. ## Start with required capabilities List the applications, risks, browsers/devices, private-network needs, data constraints, CI triggers, evidence, retention, and portability requirements. Do not start by drawing every available tool. ## Pattern 1: repository-led ```mermaid flowchart LR A[Requirements] --> B[Test code in Git] B --> C[CI runner] C --> D[Framework reports and artifacts] D --> E[Release reviewer] ``` Choose this when code ownership, framework customization, and direct CI control are central. Your team owns fixtures, dependencies, runner capacity, maintenance, evidence hosting, and failure triage. ## Pattern 2: managed platform ```mermaid flowchart LR A[Requirements] --> B[Reviewed platform tests] B --> C[Managed execution] C --> D[Platform result and artifacts] D --> E[Release reviewer] ``` Choose this when managed execution and intent-oriented authoring can reduce a measured burden. Your team still owns test oracles, data, environments, access, triage decisions, and release authority. ## Pattern 3: mixed ```mermaid flowchart LR A[Requirements] --> B[Risk and ownership map] B --> C[Repository tests] B --> D[Managed platform tests] C --> E[Unified release readout] D --> E ``` Choose this when different layers need different tools—for example, component and protocol checks in Git, while high-change end-to-end workflows use a managed platform. Prevent duplicate coverage and conflicting systems of record. ## Complete the architecture record | Layer | System of record | Trigger | Execution owner | Evidence destination | Failure/maintenance owner | | ------------------- | ---------------- | ------- | --------------- | -------------------- | ------------------------- | | Requirements | | | | | | | Component/API tests | | | | | | | End-to-end tests | | | | | | | Specialist tests | | | | | | | Environments/data | | | | | | | Release decision | | | | | | For each connection, record authentication, scope, retry behavior, timeout, revocation, observability, and who is paged when it fails. ## Remove unnecessary layers Challenge every component: * Does it own unique coverage or duplicate another path? * Is it the source of truth or only a copy? * Does it improve a measured outcome? * Can the team diagnose and upgrade it? * What happens when it is unavailable? * Can the workflow exit without it? Avoid a workflow graph when a bounded sequence is sufficient. Avoid two reporting systems when one can carry the required evidence. Avoid model routing when direct provider access already meets policy and reliability needs. ## Acceptance test the stack Run one normal case, one seeded product failure, one unavailable dependency, one timeout, and one revoked-credential scenario. Confirm the correct owner receives an actionable, non-duplicated signal and the release outcome remains pass, fail, or unknown—not silently green. ## Frequently asked questions ### Does the smallest stack always use the fewest tools? No. It uses the fewest independently owned layers needed to meet the requirements. A specialist accessibility, performance, or protocol tool can reduce risk; an overlapping runner or duplicate evidence store without a distinct owner usually adds complexity. ### When is a mixed testing stack appropriate? Use a mixed model when different test layers have genuinely different execution, evidence, or ownership requirements. Define one system of record per test, prevent duplicate release signals, and identify how results become one reviewable verdict. ### What should be the system of record for tests? Choose the location where the authoritative test intent, history, ownership, and review occur. Exported or synchronized copies should identify their source and must not become independently edited duplicates without an explicit reconciliation process. ## Related documentation * [Anthropic: Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) * [Platform Architecture](/getting-started/architecture-overview.md) * [Agent Integration Guide](/mcp-server/agent-integration-guide.md) * [QA Agent Workflow Graph or Simple Loop](/planning-and-decision-guides/qa-agent-workflow-graph.md) {% hint style="info" %} [**Map the smallest ContextQA pilot architecture →**](https://contextqa.com/book-a-demo/?utm_source=learning.contextqa.com\&utm_medium=referral\&utm_campaign=docs_demand_gen\&utm_content=smallest_ai_testing_stack) using your current systems of record and ownership boundaries. {% endhint %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://learning.contextqa.com/planning-and-decision-guides/smallest-ai-testing-stack.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.