Jenkins

Jenkins ContextQA integration — trigger test plans from a Jenkinsfile, poll for results, and fail builds automatically when tests fail.

Who is this for? SDETs, developers, and engineering managers who use Jenkins and want to trigger ContextQA test plans from a Jenkinsfile and block builds when tests fail.

Jenkins ContextQA integration: A CI/CD pattern where a Jenkins pipeline triggers a ContextQA test plan via the REST API, polls for execution completion, and maps the test result to the Jenkins build status — blocking releases when tests fail.

Jenkins pipelines need to know whether the application they just built actually works. ContextQA provides a REST API and MCP server that Jenkins can call to trigger a test plan, wait for results, and take action based on pass or fail. This page provides a complete, working Jenkinsfile example and explains every configuration decision.

Prerequisites

Before configuring the Jenkins integration:

• A ContextQA account with at least one configured test plan

• The test plan ID (visible in the URL when viewing the plan: /test-plans/<plan_id>)

• A ContextQA service account email and password dedicated to CI use (do not use a personal account)

• Jenkins 2.x with the Pipeline plugin installed

• The Jenkins Credentials plugin for storing secrets

Storing credentials in Jenkins

Never hardcode ContextQA credentials in a Jenkinsfile. Store them as Jenkins credentials:

1. In Jenkins, navigate to Manage Jenkins → Credentials → System → Global credentials.

2. Click Add Credentials.

3. Select Username with password as the kind.

4. Set ID to contextqa-credentials.

5. Enter the service account email as Username and the password as Password.

6. Click Save.

In the Jenkinsfile, reference these credentials using the usernamePassword binding. The pipeline exposes them as environment variables CONTEXTQA_USERNAME and CONTEXTQA_PASSWORD.

How test plan execution works via the API

ContextQA's test plan execution endpoint uses a GET request, not POST. This is intentional — the execution is identified by the plan ID in the URL path, and the GET request initiates execution while returning the execution ID immediately. The correct endpoint pattern is:

GET https://server.contextqa.com/api/v1/testplans/<plan_id>/execute
Authorization: Bearer <token>

The response body includes an executionId. Use this ID to poll the execution status endpoint until the status transitions to PASSED, FAILED, or PARTIAL.

The status polling endpoint:

GET https://server.contextqa.com/api/v1/executions/<execution_id>/status
Authorization: Bearer <token>

Response includes a status field. Poll every 30 seconds until status is not RUNNING or PENDING.

Authentication uses a Bearer token obtained by posting credentials to the auth endpoint:

POST https://server.contextqa.com/api/v1/auth/login
Content-Type: application/json
{ "email": "<email>", "password": "<password>" }
→ { "token": "<bearer_token>" }

Complete Jenkinsfile example

pipeline {
    agent any

    environment {
        CONTEXTQA_PLAN_ID = '<your_test_plan_id>'
        CONTEXTQA_BASE_URL = 'https://server.contextqa.com'
    }

    stages {
        stage('Build') {
            steps {
                echo 'Building application...'
                // Your build steps here
            }
        }

        stage('Deploy to Staging') {
            steps {
                echo 'Deploying to staging...'
                // Your deploy steps here
            }
        }

        stage('Trigger ContextQA Tests') {
            steps {
                withCredentials([usernamePassword(
                    credentialsId: 'contextqa-credentials',
                    usernameVariable: 'CONTEXTQA_USERNAME',
                    passwordVariable: 'CONTEXTQA_PASSWORD'
                )]) {
                    script {
                        // Step 1: Authenticate and retrieve a Bearer token
                        def authResponse = sh(
                            script: """
                                curl -s -X POST "${CONTEXTQA_BASE_URL}/api/v1/auth/login" \\
                                    -H "Content-Type: application/json" \\
                                    -d '{"email":"${CONTEXTQA_USERNAME}","password":"${CONTEXTQA_PASSWORD}"}'
                            """,
                            returnStdout: true
                        ).trim()

                        def token = readJSON(text: authResponse).token
                        env.CONTEXTQA_TOKEN = token

                        // Step 2: Trigger the test plan (GET request)
                        def triggerResponse = sh(
                            script: """
                                curl -s -X GET "${CONTEXTQA_BASE_URL}/api/v1/testplans/${CONTEXTQA_PLAN_ID}/execute" \\
                                    -H "Authorization: Bearer ${token}"
                            """,
                            returnStdout: true
                        ).trim()

                        def executionId = readJSON(text: triggerResponse).executionId
                        env.CONTEXTQA_EXECUTION_ID = executionId
                        echo "ContextQA execution started: ${executionId}"
                    }
                }
            }
        }

        stage('Poll ContextQA Result') {
            steps {
                script {
                    def status = 'RUNNING'
                    def maxAttempts = 60  // 60 x 30s = 30 minutes max
                    def attempt = 0

                    while (status in ['RUNNING', 'PENDING'] && attempt < maxAttempts) {
                        sleep(30)
                        attempt++

                        def statusResponse = sh(
                            script: """
                                curl -s -X GET "${CONTEXTQA_BASE_URL}/api/v1/executions/${env.CONTEXTQA_EXECUTION_ID}/status" \\
                                    -H "Authorization: Bearer ${env.CONTEXTQA_TOKEN}"
                            """,
                            returnStdout: true
                        ).trim()

                        status = readJSON(text: statusResponse).status
                        echo "ContextQA status (attempt ${attempt}): ${status}"
                    }

                    env.CONTEXTQA_STATUS = status

                    if (status == 'RUNNING' || status == 'PENDING') {
                        error("ContextQA execution timed out after ${maxAttempts * 30} seconds.")
                    }
                }
            }
        }
    }

    post {
        always {
            script {
                def reportUrl = "https://app.contextqa.com/executions/${env.CONTEXTQA_EXECUTION_ID}"
                currentBuild.description = "ContextQA: ${env.CONTEXTQA_STATUS} — ${reportUrl}"

                // Write the result URL to a file so it can be archived as an artifact
                writeFile(
                    file: 'contextqa-result.txt',
                    text: "Status: ${env.CONTEXTQA_STATUS}\nReport: ${reportUrl}\n"
                )
                archiveArtifacts artifacts: 'contextqa-result.txt', allowEmptyArchive: true
            }
        }
        success {
            script {
                if (env.CONTEXTQA_STATUS == 'FAILED') {
                    error("ContextQA tests FAILED. See report: https://app.contextqa.com/executions/${env.CONTEXTQA_EXECUTION_ID}")
                }
            }
        }
    }
}

How the Jenkinsfile works

Authentication stage: The pipeline calls the ContextQA login endpoint with the Jenkins-managed credentials and extracts the Bearer token. The token is stored as a pipeline environment variable for use in subsequent stages. Tokens have a limited lifetime — for long-running pipelines, implement token refresh logic or obtain the token immediately before each API call.

Trigger stage: The test plan is triggered with a GET request to the execute endpoint. ContextQA returns an executionId immediately. The pipeline does not wait for the test to complete at this point.

Polling stage: The pipeline polls the status endpoint every 30 seconds. The maxAttempts value of 60 gives a 30-minute window, which is appropriate for most regression suites. Adjust this value based on your suite's expected runtime. Add a buffer of at least 20% above your average plan duration to avoid false timeouts.

Post-build actions: Regardless of outcome, the pipeline writes the execution report URL to a text file and archives it as a Jenkins artifact. The build description is also updated with the status and URL, making it visible in the Jenkins build history without opening the full build log. If the ContextQA status is FAILED, the success post block explicitly fails the build with an error message that includes the report URL.

Publishing results as a build description

The line currentBuild.description = "ContextQA: ${env.CONTEXTQA_STATUS} — ${reportUrl}" writes to the Jenkins build description field, which appears in the build history column of the Jenkins dashboard. This gives release managers a one-line status for every build without requiring them to open the build log.

Frequently Asked Questions

Why does the test plan execute endpoint use GET instead of POST?

ContextQA's execute endpoint is designed as a GET request because the execution is fully parameterized by the plan ID — there is no request body. Some HTTP clients default to POST for "trigger" operations, but ContextQA's API contract requires GET. Using POST to the execute endpoint will return a 405 Method Not Allowed error.

What if my ContextQA test plan takes longer than 30 minutes?

Increase the maxAttempts value. For a 60-minute maximum window, set maxAttempts = 120 (120 attempts × 30 seconds = 60 minutes). Alternatively, reduce the polling interval to 15 seconds and keep maxAttempts at 60. Do not set the polling interval below 10 seconds — aggressive polling is not necessary and adds load to the ContextQA API.

Can I trigger a specific test suite rather than a full test plan?

The execution endpoint is scoped to test plans, not individual suites. If you need suite-level granularity in CI, create a dedicated test plan that contains only the suites you want to run in that pipeline stage, and use that plan's ID in the Jenkinsfile.

Related

• GitHub Actions integration

• Azure DevOps integration

• GitLab CI integration

• CircleCI integration

• Running tests and test plans

Connect ContextQA to your CI/CD pipeline in 15 minutes. Book a Demo → — See the full integration walkthrough for your existing toolchain.