> For the complete documentation index, see [llms.txt](https://learning.contextqa.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://learning.contextqa.com/testing-with-ai-coding-agents/codex-testing.md). # Testing with Codex Use Codex for repository-native tests and reproducible result receipts, then connect ContextQA MCP for reviewed browser tests and code export. ## Quick answer Codex can inspect your repository, write tests and run the commands available in its execution environment. Give it a behavior contract and require evidence from the test runner. ContextQA MCP adds a separate way to create, execute and inspect managed application tests. Use both when a change needs fast code-level checks and proof of a complete browser journey. The example here is a CSV export change: a user applies filters, exports a report, and receives a file containing exactly the authorized, filtered records. The scenarios are illustrative. Replace the contract, fixtures and commands with your repository's verified equivalents. ## Prerequisites Have Codex connected to the correct checkout, dependencies and test services ready, and permission to use a disposable test environment. Identify a stable fixture with known records, an expected CSV contract and a cleanup procedure. Separate the application deployment's identity from the documentation or test branch: the agent must test the build you intend to release. Repository-native unit tests still run through Jest, Vitest, pytest or your existing runner. ContextQA MCP is not a shell interface for those local runners. ## 1. Make testing instructions discoverable Keep project commands and boundaries in your repository's `AGENTS.md`. Codex reads repository instructions; see the [official AGENTS.md guide](https://developers.openai.com/codex/guides/agents-md). Use real commands from the project instead of copying a generic `npm test` assumption. A useful instruction template is: ``` Testing expectations - Read the relevant package/service instructions before editing. - Use the existing test framework and documented fixture setup. - Identify the focused test command and required release checks. - Do not weaken assertions, skip tests or update snapshots to hide failures. - Record commands, environment, exit status and test results. - Report blocked checks separately from passing checks. ``` This gives future tasks consistent expectations without embedding production credentials or personal account details in the repository. ## 2. Establish a baseline and an independent contract Start with an inspection request: ``` Read the repository instructions and inspect the CSV export implementation, existing tests and CI configuration. Identify the exact test commands and required services. Run the current focused tests, then report the baseline. Do not change application code or install new dependencies yet. ``` Define the exported data contract from product requirements. For example: headers have an approved order; commas and quotes are escaped; timestamps use a specified timezone; filtered-out records and other tenants' records are absent. Decide whether spreadsheet formula-like values must be escaped according to your application's policy; do not invent a rule after seeing the output. ``` Design a test matrix from this approved CSV contract. Use a fixture whose expected rows we can calculate independently. Include an empty result, a quoted value, a timezone boundary, an excluded record and a different-tenant record. Do not compute expected output by calling the production export function. Mark requirements that still need an explicit decision. ``` ## 3. Implement tests at the correct layers Use [unit tests](/testing-with-ai-coding-agents/unit-testing.md) for pure serialization and [integration tests](/testing-with-ai-coding-agents/integration-testing.md) for queries, authorization and export endpoints. The integration boundary needs the actual filtering/authorization behavior; mocking it away produces misleading confidence. ``` Implement the approved serializer cases in the existing unit runner. Add an integration test using disposable records in two tenants. Request the export as the permitted test user; assert headers and exact record membership, then verify excluded records are absent. Use deterministic time and identifiers. Run the focused tests and keep the original output for review. ``` When a test fails, ask Codex to explain whether the cause is a product defect, test error or environment problem before editing. For a bug fix, retain evidence that the regression test detects the original defect. Do not treat more passing assertions as a substitute for testing the missing boundary. ## 4. Verify the user journey and the file Add one [end-to-end test](/testing-with-ai-coding-agents/end-to-end-testing.md) using the browser framework already in the project. The [Playwright guide](/testing-with-ai-coding-agents/playwright-testing.md) covers a code-owned browser workflow. ``` In the approved test deployment, sign in as the export fixture user, apply the specified filters and download the CSV. Assert the UI filter state, then parse the downloaded file and check its approved headers and exact row membership. A successful click, toast or download event alone is insufficient. Run the test and record the actual result plus relevant failure artifacts. ``` If the environment cannot download or inspect files, report that limitation. A UI-only run can prove the export control is accessible; it cannot prove the file contents. Keep those verdicts separate. ## 5. Configure ContextQA MCP in Codex Add this table to your Codex configuration, normally `~/.codex/config.toml`. A trusted project can use `.codex/config.toml`. Preserve existing settings: ```toml [mcp_servers.contextqa] url = "https://mcp.contextqa.com/mcp" ``` Then authenticate and inspect the connection: ```bash codex mcp login contextqa codex mcp list ``` In the Codex terminal interface, `/mcp` shows connected servers. These settings and commands follow the [official Codex MCP documentation](https://developers.openai.com/codex/mcp). ContextQA uses browser OAuth with your ContextQA account; finish the callback as described in [ContextQA MCP setup](https://mcp.contextqa.com/docs). A hosted MCP server cannot read your local checkout merely because Codex can. Share only the approved requirement text or a supported resource. Similarly, the remote test runner needs a reachable test URL or an explicitly configured network path. ## 6. Create a reviewed ContextQA browser case Discover current tools and coverage first: ``` Read the connected ContextQA tool schemas. Search for existing report-export cases, inspect their steps and list available test environments. Show which tools can retrieve run results and step evidence. Make no changes yet. ``` Then request a bounded new case: ``` Create the missing report-export browser case at [approved test URL] using our configured disposable user and the reviewed filter scenario. Return the case ID and complete assertions before running it. If file-content inspection is not available in this environment, state that explicitly and leave that assertion to our repository test; do not substitute a download toast. ``` After review, execute only the agreed case. Use the returned case/execution identifiers, poll to a terminal state, and retrieve the verdict and step details. The [execution reference](/mcp-server/tool-reference/execution-and-results.md) documents `execute_test_case`, `get_execution_status`, `get_test_case_results` and `get_execution_step_details`; the connected schema determines exact parameters. Do not launch overlapping runs of the same case and then assume “latest” refers to the intended execution. A finished job may contain failed assertions. A missing credential is a blocker, not a passing negative test. ## 7. Produce a reviewable result and export if useful Ask Codex for a receipt containing: * The acceptance criterion and test level. * The commit/deployment and fixture identity. * The local command or ContextQA execution/result ID. * Passed, failed, skipped or blocked status with evidence. * Remaining gaps and the owner of the next action. For code ownership, inspect the available export tool and supported formats, then follow [export and portability](/testing-with-ai-coding-agents/export-and-portability.md). Retrieve the actual artifact, review it, and execute it in the repository before calling the export verified. A server-side output directory is not your laptop's path. ## Troubleshooting and FAQ **Codex says tests passed without output.** Ask for the command, exit status and result summary. Treat an unexecuted command as a proposed check. **MCP login succeeded but tools fail.** Check the selected account, workspace permissions and current tool manifest. See [MCP troubleshooting](/mcp-server/troubleshooting.md). **Can the agent repair every failing test automatically?** It can propose repairs; preserve the contract and inspect the diff. A changed expectation needs product justification, not simply a green run. ## Use this on one release-critical journey [Connect ContextQA MCP and complete a first test](/mcp-server/installation-and-setup.md). For a team evaluation spanning code, browser runs and release evidence, [book a ContextQA demo](https://contextqa.com/book-a-demo/?utm_source=learning.contextqa.com\&utm_medium=referral\&utm_campaign=docs_demand_gen\&utm_content=codex-testing) with your representative workflow. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://learning.contextqa.com/testing-with-ai-coding-agents/codex-testing.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.