OpenAPI Contract Testing with AI

Turn an OpenAPI specification into reviewed API tests, distinguish schema checks from business assertions, and add ContextQA MCP without assuming complete coverage.

Quick answer

Start with a versioned OpenAPI document and a small set of important operations. Validate the documented response shapes, then add independent business, authorization, and state assertions that the schema cannot establish. A coding agent can help draft this matrix and write repository tests; ContextQA can generate candidate API cases from a supported specification through MCP. Review and execute those cases before counting them as coverage.

What kind of contract are you testing?

An OpenAPI document describes an HTTP interface: operations, parameters, responses, schemas, and declared security requirements. A schema-valid response can still contain the wrong price, another tenant's record, or stale state. The OpenAPI specification defines the description format; it does not certify an implementation's business behavior.

Check	Example	Evidence you need
Description validation	Required path parameters and references are valid	A validator compatible with your OpenAPI version
Response contract	totalCents is an integer and currency is allowed	Status, media type, and body-schema checks
Business behavior	This fixture's quote totals exactly 1,200 cents	Expected amount derived from the requirement
Access control	A user cannot retrieve another tenant's quote	Separate identities and the documented denial outcome
State transition	Cancelling a quote prevents confirmation	A sequence with state readback and cleanup
Consumer compatibility	A change still meets a client's actual expectations	Consumer/provider agreement and verification

Consumer-driven contract testing is a related but distinct technique. For example, Pact verifies interactions expected by a consumer against a provider. Importing an OpenAPI file does not automatically create that consumer agreement or run Pact verification.

Prerequisites

Use a specification tied to the application build under test, an approved local or staging API, and deterministic data. Identify which operations are read-only and which create, delete, notify, or charge. Start with a limited operation set and a cleanup plan. Keep authentication values outside the spec, agent prompts, and committed test files.

Choose tools that support the document's declared OpenAPI and JSON Schema dialect. Do not change the version label to make a parser accept constructs it does not understand.

1. Build an operation-and-assertion matrix

For a quote service, write the expected behavior before generating tests:

Case	Fixture and action	Required outcome
Known quote	Read the synthetic quote with two 500-cent items and 200-cent shipping	Correct ID, USD, and exactly 1,200 cents
Unknown quote	Read an ID confirmed absent from the fixture	The documented not-found status and error body
Wrong response type	Service returns totalCents as a string	Contract check fails
Wrong calculation	Service returns integer 1,201 for the known fixture	Business assertion fails even though schema validation passes
Cross-tenant read, if supported	A different tenant requests the seeded quote	Documented denial without leaking quote data

Do not manufacture a 500 response because it appears in the specification. Testing service failures needs a controlled fault mechanism and an explicit expectation. Likewise, a declared 401 does not tell you how to create an expired session or prove revocation.

Ask a coding agent:

Read our OpenAPI document, existing API tests, and quote requirements.
For the selected read operation, propose a matrix of response-schema,
business, missing-resource, and authorization checks. Identify which
expected values come from the schema and which need a separate requirement.
List missing fixtures and unsupported schema constructs. Do not edit yet.
Do not call production or automatically exercise every operation.

2. Separate a schema assertion from a business assertion

This small executable example validates an OpenAPI 3.1-compatible response-body schema. It uses synthetic objects, not an HTTP request or a live ContextQA run. It demonstrates why a shape check alone is insufficient.

In a disposable Node.js project, install a pinned validator and save its lockfile:

npm init -y
npm install --save-dev --save-exact ajv@8.17.1

Save the following as quote-contract.test.mjs:

import assert from 'node:assert/strict';
import test from 'node:test';
import Ajv2020 from 'ajv/dist/2020.js';

const quoteSchema = {
  type: 'object',
  required: ['id', 'currency', 'totalCents'],
  additionalProperties: false,
  properties: {
    id: { type: 'string', minLength: 1 },
    currency: { type: 'string', enum: ['USD'] },
    totalCents: { type: 'integer', minimum: 0 },
  },
};
const validate = new Ajv2020({ allErrors: true }).compile(quoteSchema);
const approvedQuote = { id: 'quote-fixture-1', currency: 'USD', totalCents: 1200 };

function assertKnownQuote(body) {
  assert.equal(validate(body), true, JSON.stringify(validate.errors));
  assert.equal(body.id, 'quote-fixture-1');
  // Requirement: two 500-cent items plus 200-cent shipping.
  assert.equal(body.totalCents, 1200);
}

test('the approved fixture satisfies shape and business expectations', () => {
  assertKnownQuote(approvedQuote);
});

test('schema validation rejects a string amount', () => {
  assert.equal(validate({ ...approvedQuote, totalCents: '1200' }), false);
});

test('a schema-valid wrong total still fails the business assertion', () => {
  const wrongTotal = { ...approvedQuote, totalCents: 1201 };
  assert.equal(validate(wrongTotal), true);
  assert.throws(() => assertKnownQuote(wrongTotal), { name: 'AssertionError' });
});

Run node --test quote-contract.test.mjs. All three teaching tests should pass: the last test verifies that the wrong business result is rejected. It does not accept that result as correct.

The example uses Ajv's draft 2020-12 implementation. In a real suite, resolve the response schema from the versioned OpenAPI document with compatible tooling instead of keeping an independent copy that can drift. This example is not a validator for the entire OpenAPI document, its request parameters, response statuses, headers, or references.

additionalProperties: false is deliberate in this fixture. In an evolving API, decide whether unknown response fields should fail your contract; do not introduce a strictness policy accidentally through generated tests.

3. Exercise the actual API boundary

Adapt the assertions to your existing API runner. Send the approved request, assert its expected status and content type, validate the body schema, and compare the business values. Use the real service boundary when the test claims integration coverage; a mocked response only exercises the consumer or assertion code.

For Playwright, its API testing guide describes the request context. Supply the API's actual authentication mechanism rather than assuming browser localStorage automatically becomes a bearer header. For generated input exploration, tools such as Schemathesis can derive tests from a schema. Limit operations, targets, and generated workload before running them, and retain the reproduction details for failures.

Generated inputs complement your named business cases. They do not know the correct price or tenant owner unless those expectations are supplied and asserted.

4. Add the ContextQA MCP workflow

After connecting your coding agent, inspect its authenticated manifest and confirm the intended workspace. The documented generate_tests_from_swagger tool takes file_path_or_url; inspect the current schema and supported input before calling it. A hosted service does not automatically see your local path. Provide a supported upload or an approved reachable specification without embedding credentials in its URL.

Inspect ContextQA's available OpenAPI generation tool and its side effects.
Use only the approved specification and workspace. Search for existing cases
before creating more. Generate candidate cases for the agreed scope, then
return their IDs and steps for review without claiming they are executed.
Map each candidate to our operation-and-assertion matrix. Identify missing
business, authorization, fixture, or response checks. Do not print secrets.

The documented parameter does not expose an operation filter. To limit generation to a subset, supply a reviewed reduced specification with all required schema references resolved, or explicitly accept and review the whole import. A prompt asking for a subset is not an enforced tool boundary.

Generation can create workspace resources. Review scope first and reconcile case identifiers so repeated attempts do not leave duplicates. A tool response or generated case count does not establish that all operations or statuses were covered.

Review each saved case, configure its request and response validations, and add any missing fixture-specific expectations. Then execute the approved cases and retrieve the current run's final verdict and relevant step results. Keep repository and ContextQA results separate, tied to the same application build and fixture.

5. Prove the checks detect the intended defect

In a disposable fixture, change the returned total while preserving its valid schema. The business check should fail. Change an integer to a string and verify the schema check fails. Restore the fixture and verify the baseline again.

For state-changing operations, also inspect side effects and cleanup. A correct response body does not prove that the database was updated once, that an unauthorized mutation was prevented, or that a retry avoided duplicate creation. Use API chaining or your repository integration suite for those requirements.

Record spec revision, application build, operation, fixture, expected statuses, assertions, run IDs, unavailable checks, and cleanup. If standalone code ownership matters, use the export acceptance guide; OpenAPI import does not establish lossless code export or a particular framework's support.

Frequently asked questions

Is OpenAPI validation the same as integration testing?

No. Validating the document checks its description. Checking synthetic response objects validates your assertion behavior. Sending requests across an actual service boundary adds integration evidence. State which one you performed.

Can AI generate all API tests from Swagger?

A supported specification can seed candidate tests, but requirements, test data, access rules, undocumented behavior, and unsupported constructs still need review. Compare generated coverage with an explicit matrix rather than assuming every declared response has been tested.

Should every documented status code be forced into a test?

Prioritize meaningful, controllable outcomes. Use approved fault fixtures for failure paths and retain cases that validate important client behavior. Do not destabilize a shared environment to produce a particular status.

Next step

Set up ContextQA MCP and bring one API operation plus its business assertion matrix. For a review of generated coverage, authentication, chaining, and execution evidence, book an API testing workflow demo.

Sources and examples reviewed September 25, 2026. Verify supported specification versions, schemas, and tools in the environment you use.