Testing with Codex

Use Codex for repository-native tests and reproducible result receipts, then connect ContextQA MCP for reviewed browser tests and code export.

Quick answer

Codex can inspect your repository, write tests and run the commands available in its execution environment. Give it a behavior contract and require evidence from the test runner. ContextQA MCP adds a separate way to create, execute and inspect managed application tests. Use both when a change needs fast code-level checks and proof of a complete browser journey.

The example here is a CSV export change: a user applies filters, exports a report, and receives a file containing exactly the authorized, filtered records. The scenarios are illustrative. Replace the contract, fixtures and commands with your repository's verified equivalents.

Prerequisites

Have Codex connected to the correct checkout, dependencies and test services ready, and permission to use a disposable test environment. Identify a stable fixture with known records, an expected CSV contract and a cleanup procedure. Separate the application deployment's identity from the documentation or test branch: the agent must test the build you intend to release.

Repository-native unit tests still run through Jest, Vitest, pytest or your existing runner. ContextQA MCP is not a shell interface for those local runners.

1. Make testing instructions discoverable

Keep project commands and boundaries in your repository's AGENTS.md. Codex reads repository instructions; see the official AGENTS.md guide. Use real commands from the project instead of copying a generic npm test assumption.

A useful instruction template is:

Testing expectations
- Read the relevant package/service instructions before editing.
- Use the existing test framework and documented fixture setup.
- Identify the focused test command and required release checks.
- Do not weaken assertions, skip tests or update snapshots to hide failures.
- Record commands, environment, exit status and test results.
- Report blocked checks separately from passing checks.

This gives future tasks consistent expectations without embedding production credentials or personal account details in the repository.

2. Establish a baseline and an independent contract

Start with an inspection request:

Read the repository instructions and inspect the CSV export implementation,
existing tests and CI configuration. Identify the exact test commands and
required services. Run the current focused tests, then report the baseline.
Do not change application code or install new dependencies yet.

Define the exported data contract from product requirements. For example: headers have an approved order; commas and quotes are escaped; timestamps use a specified timezone; filtered-out records and other tenants' records are absent. Decide whether spreadsheet formula-like values must be escaped according to your application's policy; do not invent a rule after seeing the output.

Design a test matrix from this approved CSV contract. Use a fixture whose
expected rows we can calculate independently. Include an empty result, a quoted
value, a timezone boundary, an excluded record and a different-tenant record.
Do not compute expected output by calling the production export function.
Mark requirements that still need an explicit decision.

3. Implement tests at the correct layers

Use unit tests for pure serialization and integration tests for queries, authorization and export endpoints. The integration boundary needs the actual filtering/authorization behavior; mocking it away produces misleading confidence.

Implement the approved serializer cases in the existing unit runner. Add an
integration test using disposable records in two tenants. Request the export
as the permitted test user; assert headers and exact record membership, then
verify excluded records are absent. Use deterministic time and identifiers.
Run the focused tests and keep the original output for review.

When a test fails, ask Codex to explain whether the cause is a product defect, test error or environment problem before editing. For a bug fix, retain evidence that the regression test detects the original defect. Do not treat more passing assertions as a substitute for testing the missing boundary.

4. Verify the user journey and the file

Add one end-to-end test using the browser framework already in the project. The Playwright guide covers a code-owned browser workflow.

In the approved test deployment, sign in as the export fixture user, apply the
specified filters and download the CSV. Assert the UI filter state, then parse
the downloaded file and check its approved headers and exact row membership.
A successful click, toast or download event alone is insufficient.
Run the test and record the actual result plus relevant failure artifacts.

If the environment cannot download or inspect files, report that limitation. A UI-only run can prove the export control is accessible; it cannot prove the file contents. Keep those verdicts separate.

5. Configure ContextQA MCP in Codex

Add this table to your Codex configuration, normally ~/.codex/config.toml. A trusted project can use .codex/config.toml. Preserve existing settings:

[mcp_servers.contextqa]
url = "https://mcp.contextqa.com/mcp"

Then authenticate and inspect the connection:

codex mcp login contextqa
codex mcp list

In the Codex terminal interface, /mcp shows connected servers. These settings and commands follow the official Codex MCP documentation. ContextQA uses browser OAuth with your ContextQA account; finish the callback as described in ContextQA MCP setup.

A hosted MCP server cannot read your local checkout merely because Codex can. Share only the approved requirement text or a supported resource. Similarly, the remote test runner needs a reachable test URL or an explicitly configured network path.

6. Create a reviewed ContextQA browser case

Discover current tools and coverage first:

Read the connected ContextQA tool schemas. Search for existing report-export
cases, inspect their steps and list available test environments. Show which
tools can retrieve run results and step evidence. Make no changes yet.

Then request a bounded new case:

Create the missing report-export browser case at [approved test URL] using
our configured disposable user and the reviewed filter scenario. Return the
case ID and complete assertions before running it. If file-content inspection
is not available in this environment, state that explicitly and leave that
assertion to our repository test; do not substitute a download toast.

After review, execute only the agreed case. Use the returned case/execution identifiers, poll to a terminal state, and retrieve the verdict and step details. The execution reference documents execute_test_case, get_execution_status, get_test_case_results and get_execution_step_details; the connected schema determines exact parameters.

Do not launch overlapping runs of the same case and then assume “latest” refers to the intended execution. A finished job may contain failed assertions. A missing credential is a blocker, not a passing negative test.

7. Produce a reviewable result and export if useful

Ask Codex for a receipt containing:

• The acceptance criterion and test level.
• The commit/deployment and fixture identity.
• The local command or ContextQA execution/result ID.
• Passed, failed, skipped or blocked status with evidence.
• Remaining gaps and the owner of the next action.

For code ownership, inspect the available export tool and supported formats, then follow export and portability. Retrieve the actual artifact, review it, and execute it in the repository before calling the export verified. A server-side output directory is not your laptop's path.

Troubleshooting and FAQ

Codex says tests passed without output. Ask for the command, exit status and result summary. Treat an unexecuted command as a proposed check.

MCP login succeeded but tools fail. Check the selected account, workspace permissions and current tool manifest. See MCP troubleshooting.

Can the agent repair every failing test automatically? It can propose repairs; preserve the contract and inspect the diff. A changed expectation needs product justification, not simply a green run.

Use this on one release-critical journey

Connect ContextQA MCP and complete a first test. For a team evaluation spanning code, browser runs and release evidence, book a ContextQA demo with your representative workflow.