Tutorial: Prepare a CI/CD quality gate

Prepare a reliable ContextQA test plan, scoped automation credential, and pipeline gate without relying on unpublished REST endpoints.

Quick answer

Prepare a reliable ContextQA test plan, scoped automation credential, and pipeline gate without relying on unpublished REST endpoints. Follow the documented prerequisites and steps, then verify the expected result before using the workflow in a shared or production process.

What this page covers

This tutorial prepares the ContextQA and pipeline-side pieces of a quality gate. It intentionally does not provide a REST call copied from portal traffic: the verified production CI / CD tab describes the start and status operations but does not publish endpoint paths.

Outcome

By the end, you will have:

• A small, stable suite for release-blocking coverage.
• A test plan that supplies the browser or device, environment, and execution settings.
• A narrowly scoped API token stored in your CI system's secret manager.
• A quality-gate design ready for the public API contract supplied to your workspace.

Prerequisites

• Permission to create or edit test suites and test plans.
• Superadmin access for API token creation, or an administrator who can create the token for you.
• Permission to add protected secrets and variables to your pipeline project.

Step 1: Build a focused CI suite

1. Open Plans → Test Suites.
2. Click Create Test Suite.
3. Choose the required platform and add only deterministic, release-blocking cases.
4. Give the suite a purpose-based name such as Checkout PR gate.
5. Run each case individually and fix unstable setup before continuing.

A small suite gives developers fast, actionable feedback. Keep broad regression coverage in a separate scheduled plan.

Step 2: Create the executable test plan

1. Open Plans → Test Plans and click New Test Plan.
2. Add the CI suite.
3. Map it to the required browser or device.
4. Select the staging environment and configure timeouts, screenshots, and recovery behavior.
5. Start with one parallel node while validating the gate.
6. Create the plan, then run it manually once.

Do not connect CI until the plan passes against the same environment and data that the pipeline will use.

Step 3: Review the production CI/CD surface

Open the plan and select CI / CD. The current page lists supported integration targets and two REST operation concepts:

• Start Test Plan — returns a run ID for polling.
• Check Status — returns status, result, and suite breakdown information.

Test Plan CI/CD tab with supported pipeline targets and REST operation cards

The verified production tab does not publish endpoint paths. Older examples using /auth/login, /testplans/{id}/execute, or /executions/{id}/status are obsolete. Do not substitute internal portal request URLs.

Step 4: Create a pipeline credential

1. Open Settings → API Tokens.
2. Click Create token.
3. Name it for the caller, such as GitHub — checkout PR gate.
4. Choose the shortest practical expiry.
5. Grant only the scopes required by the public contract supplied to your workspace.
6. Copy the value when it appears and save it immediately in the pipeline's secret manager.

Use a dedicated token per repository or pipeline so one integration can be rotated or revoked without disrupting the others.

Step 5: Prepare pipeline variables

Use your CI system's protected secret store for the token and ordinary configuration variables for non-secret identifiers.

Value	Storage	Example name
API token	Protected secret	CONTEXTQA_API_TOKEN
Test plan ID	Variable	CONTEXTQA_TEST_PLAN_ID
Public API base URL, when supplied	Variable	CONTEXTQA_API_BASE_URL

Never store an account password, token value, or generated authorization header in the workflow file.

Step 6: Design the gate

When the public contract is available, configure the pipeline job to:

1. Start the configured test plan.
2. Record the returned run ID without logging credentials.
3. Poll using the documented status operation at a bounded interval.
4. Stop at the documented terminal state.
5. Pass only for the documented successful result.
6. Surface the run ID and direct reviewers to ContextQA for the report.

Do not construct a report URL unless the public response explicitly returns one.

Step 7: Validate before enforcing

1. Run the job on a disposable branch.
2. Confirm a passing plan produces a successful job.
3. Use an intentionally failing disposable test to confirm the job blocks as expected.
4. Confirm logs contain no token, password, request header, or sensitive test data.
5. Only then make the job required in branch or deployment protection.

Troubleshooting

The portal shows operation cards but no paths

That is the verified production behavior. Obtain the current public contract from the official ContextQA reference available to your workspace. Do not derive it from browser developer tools.

The token is rejected

Check expiry, activation state, permission scopes, and the IP allowlist. The token authenticates calls only within its configured scope.

The plan works manually but fails in CI

Compare the environment, browser or device mapping, test data, and network access. The pipeline should trigger the same saved plan you validated manually.

Related pages

• Test Plans
• API tokens
• GitHub Actions
• CI integration acceptance runbook
• Integrations overview
• ContextQA MCP server