> For the complete documentation index, see [llms.txt](https://learning.contextqa.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://learning.contextqa.com/enterprise-application-testing/ship/ai-software-factory.md).

# AI Software Factory

Build an AI software factory that reproduces bugs, verifies coding-agent fixes, tests affected journeys, and gives release owners evidence before merge.

> **Documentation owner:** [Deep Barot](https://contextqa.com/academy/) · **Last updated:** November 11, 2026

## Quick answer

An AI software factory connects requirements, coding agents, testing, and delivery into a repeatable workflow. ContextQA and Ship provide its quality engineering layer: understand the application, reproduce reported behavior, identify journeys affected by a change, execute tests, and return evidence to the coding agent and release owner. A fix becomes a release candidate when its intended behavior and regression scope have been verified on the correct build.

## Key takeaways

* Connect coding agents to application context and independent execution evidence.
* Keep generated coverage, impact recommendations, and verified outcomes distinct.
* Return regressions to the coding agent and preserve human release review.

## What does ContextQA contribute?

| Capability                         | What it does                                                                             | Why it matters                                                                             |
| ---------------------------------- | ---------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| Application crawl                  | Explores a reachable application and proposes journeys and tests                         | Establishes behavioral context, including screens the coding agent may not have considered |
| Code analysis                      | Maps repository context to product features and proposed coverage                        | Connects implementation changes to business behavior                                       |
| Requirements generation            | Turns requirements, tickets, documents, and walkthroughs into reviewable test candidates | Makes acceptance criteria executable                                                       |
| Bug reproduction                   | Attempts the reported journey and captures the observed outcome                          | Gives the coding agent a concrete failure to fix                                           |
| Fix handoff through MCP            | Supplies reproduction evidence and relevant context to an authorized agent               | Reduces manual reconstruction of the failure                                               |
| PR impact analysis                 | Identifies existing tests to rerun and coverage to add or update                         | Focuses feedback on the change and its dependencies                                        |
| Browser, API, and mobile execution | Exercises real behavior and retains run evidence                                         | Tests what users and connected systems experience                                          |
| Root-cause analysis and auto-fix   | Investigates test failures and supports reviewed test repair                             | Helps separate product defects from automation maintenance                                 |
| Auth profiles and tunnels          | Provide reusable sign-in and access to private test applications                         | Make local and protected environments testable                                             |
| Reports and integrations           | Bring findings into delivery tools and preserve run history                              | Gives developers and release owners a shared decision record                               |

These capabilities work together, but each answers a different question. A generated test is proposed coverage. An impact report is a proposed scope. A completed run is execution evidence. A release decision combines that evidence with the remaining risks.

## How the workflow fits together

```mermaid
flowchart TD
    A[Requirement or reported bug] --> B[Application and code context]
    B --> C[Generate coverage or reproduce behavior]
    C --> D{Enough evidence?}
    D -- No --> E[Clarify steps, access, data or expectations]
    E --> C
    D -- Yes --> F[Coding agent implements a bounded change]
    F --> G[Pull request and deployed candidate build]
    G --> H[Ship analyzes affected journeys]
    H --> I[Run selected tests and critical regression coverage]
    I --> J{New regression or unresolved result?}
    J -- Yes --> K[Return evidence to agent or reviewer]
    K --> F
    J -- No --> L[Human release review]
    L --> M[Merge and post-deployment verification]
```

The loop can be driven through connected issue trackers, repository events, CI/CD, and MCP. Decide which transitions your integration starts automatically and which require a person. A coding-agent prompt or a demonstration skill does not by itself create a continuously operating service.

## Set up the factory

1. **Choose a bounded application and outcome.** Start with one important journey and one repository. Write expected results independently of the proposed implementation.
2. **Connect the test environment.** Configure its URL, data, network access, and test identity. Use a tunnel for a private or local application when appropriate.
3. **Establish sign-in.** Verify an auth profile or reusable prerequisite before crawling or running dependent tests. Keep secrets in managed fields.
4. **Build context.** Run application crawl and code analysis. Review generated coverage against the critical journeys, user roles, negative paths, and permissions.
5. **Create a baseline.** Execute accepted tests against the known base build. Triage failing tests before asking impact analysis to distinguish new regressions.
6. **Connect delivery signals.** Configure the repository, issue tracker, branch filters, and reproduction triggers. Keep the application build aligned with the ticket or PR being evaluated.
7. **Choose autonomy.** Set Read, Upsert, or Autonomous for test maintenance. Configure test execution timing separately. See [PR impact analysis](https://learning.contextqa.com/enterprise-application-testing/ai-features/pr-impact-analysis.md).
8. **Run a complete cycle.** Reproduce one issue, create a fix, deploy the PR build, execute affected journeys, and inspect the evidence before merging.
9. **Expand gradually.** Add workflows after the baseline and failure classifications are reliable. Track incomplete runs, omitted journeys, and maintenance work alongside successful fixes.

## Use any coding agent with an evidence contract

The initial implementation can use Claude Code, Codex, Cursor, or another authorized agent. The contract is the same: provide the intended behavior, exact build, reproduction, scope, and expected assertions; require the agent to return a patch and validation results. Connect through the [MCP setup guide](https://learning.contextqa.com/contextqa-mcp/overview/installation-and-setup.md).

```
Work on the specified issue in a new branch.
Read the reproduction evidence and expected business behavior first.
Make a focused fix and run the repository's relevant checks.
Open a PR and identify the candidate commit and deployment.
Use ContextQA to analyze impact and run the affected journeys.
Compare outcomes with the baseline; separate new regressions,
existing failures, test defects, and blocked checks.
Return the PR, tested commit, run links, and unresolved findings.
Leave merging and release approval to the release owner.
```

This is an example instruction, not a required command or built-in skill name. Discover the current tools in your connected MCP client before issuing operations.

## A useful example: a correct fix with a broken confirmation screen

A payments example illustrates why independent journey testing matters. Removing a duplicate amount conversion can correct the stored balance while leaving the confirmation screen with an incorrect amount. Tests should check the balance, transaction history, and displayed confirmation separately. The next fix must be tested again against the new commit.

The [example pull request](https://github.com/harshit-sharma44/testing-lab-react-payments/pull/13) contains two changes covering these distinct behaviors. Use it as a worked example, not a performance benchmark. Its run outcomes and test counts belong to that application and those commits.

For a hands-on procedure, follow [Test AI-generated code before merge](https://learning.contextqa.com/enterprise-application-testing/ship/testing-ai-generated-code.md). For the release decision, use [Release readiness](https://learning.contextqa.com/enterprise-application-testing/ship/release-readiness.md).

## Frequently asked questions

### Does an AI software factory replace code review?

No. Execution evidence complements review of architecture, security, maintainability, unit tests, and contracts. A journey can pass while a code-level problem remains outside its assertions.

### Does a high confidence score mean the application is correct?

No. Confidence explains an analysis or recommendation. A release decision needs completed tests, meaningful assertions, build identity, and disposition of unresolved outcomes.

### Can the workflow fix its own regressions?

The reviewer can return failure evidence to a coding agent for another bounded attempt. Each attempt produces a new candidate that must be verified. Define retry limits and an escalation path so the loop stops when access, requirements, or results remain ambiguous.

### What should we measure?

Measure time to first useful evidence, reproduction quality, verified fixes, newly introduced regressions, impacted tests omitted, human interventions, and total credits and elapsed time. Use representative applications and repeat runs. Do not equate the number of generated cases with coverage quality.

## Next steps

* [Test AI-generated code before merge](https://learning.contextqa.com/enterprise-application-testing/ship/testing-ai-generated-code.md)
* [Quarterly release testing](https://learning.contextqa.com/enterprise-application-testing/ship/quarterly-release-testing.md)
* [SAP test automation](https://learning.contextqa.com/packaged-application-testing/sap-testing.md)
* [Evaluate a QA agent skill](https://learning.contextqa.com/plan-evaluate-and-migrate/evaluation/qa-agent-skill-validation.md)

[Discuss an AI software factory workflow](https://contextqa.com/book-a-demo/?utm_source=github\&utm_medium=developer_referral\&utm_campaign=gitbooks_docs\&utm_content=ai_software_factory).

## Put the workflow into practice

Follow [From Slack or Linear to release readiness](https://learning.contextqa.com/enterprise-application-testing/ship/slack-linear-to-release.md) for a complete report-to-fix-to-validation example, a feature-ticket scenario, and a quarterly SAP upgrade scenario. The guide includes prompts, evidence requirements, decision criteria, and a verified public PR example that shows why an impact label cannot substitute for passing execution.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://learning.contextqa.com/enterprise-application-testing/ship/ai-software-factory.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
