# GitLab CI

{% hint style="info" %}
**Who is this for?** SDETs, developers, and engineering managers who use GitLab CI and want to trigger ContextQA test plans from `.gitlab-ci.yml` and fail pipeline stages on test failures.
{% endhint %}

> **GitLab CI ContextQA integration:** A CI/CD configuration where a `.gitlab-ci.yml` pipeline stage triggers a ContextQA test plan, polls for execution completion, and maps the test result to the GitLab job exit code — blocking merges and deployments when tests fail.

GitLab CI pipelines can call external services between build stages. ContextQA exposes a REST API that GitLab CI jobs can use to trigger a test plan, wait for results, and surface failures as a failed pipeline stage. This page provides a complete `.gitlab-ci.yml` example and covers credential storage, polling logic, and result publication.

## Prerequisites

Before configuring the GitLab CI integration:

* A ContextQA account with at least one configured test plan
* The test plan ID (visible in the URL when viewing the plan: `/test-plans/<plan_id>`)
* A ContextQA service account (dedicated email and password — do not use a personal account)
* GitLab CI/CD configured on your repository

## Storing credentials as GitLab CI/CD variables

Store ContextQA credentials as masked CI/CD variables so they are never visible in job logs.

1. In your GitLab project, navigate to **Settings → CI/CD → Variables**.
2. Click **Add variable**.
3. Set **Key** to `CONTEXTQA_USERNAME`, **Value** to your service account email. Check **Masked** and **Protected** if your pipelines run on protected branches.
4. Repeat for **Key** `CONTEXTQA_PASSWORD` with your service account password.
5. Click **Save variables**.

GitLab will mask these values in all job logs. The `Masked` flag prevents the values from appearing even if a job explicitly echoes them.

## How the ContextQA API works from GitLab CI

Authentication: POST credentials to the ContextQA login endpoint to receive a Bearer token.

Test plan execution: Send a **GET** request (not POST) to the execute endpoint. ContextQA returns an execution ID immediately.

Polling: Repeatedly GET the execution status endpoint until the status is `PASSED`, `FAILED`, or `PARTIAL`.

The base URL for all API calls is `https://server.contextqa.com`.

## Complete .gitlab-ci.yml example

```yaml
stages:
  - build
  - deploy
  - test
  - report

variables:
  CONTEXTQA_BASE_URL: "https://server.contextqa.com"
  CONTEXTQA_PLAN_ID: "<your_test_plan_id>"

build:
  stage: build
  script:
    - echo "Building application..."
    # Your build commands here

deploy_staging:
  stage: deploy
  script:
    - echo "Deploying to staging..."
    # Your deploy commands here

contextqa_tests:
  stage: test
  image: alpine:latest
  before_script:
    - apk add --no-cache curl jq
  script:
    # Step 1: Authenticate and get Bearer token
    - |
      AUTH_RESPONSE=$(curl -s -X POST "${CONTEXTQA_BASE_URL}/api/v1/auth/login" \
        -H "Content-Type: application/json" \
        -d "{\"email\":\"${CONTEXTQA_USERNAME}\",\"password\":\"${CONTEXTQA_PASSWORD}\"}")
      TOKEN=$(echo "$AUTH_RESPONSE" | jq -r '.token')
      if [ -z "$TOKEN" ] || [ "$TOKEN" = "null" ]; then
        echo "ERROR: Failed to authenticate with ContextQA."
        exit 1
      fi
      echo "Authentication successful."

    # Step 2: Trigger the test plan (GET request)
    - |
      TRIGGER_RESPONSE=$(curl -s -X GET "${CONTEXTQA_BASE_URL}/api/v1/testplans/${CONTEXTQA_PLAN_ID}/execute" \
        -H "Authorization: Bearer ${TOKEN}")
      EXECUTION_ID=$(echo "$TRIGGER_RESPONSE" | jq -r '.executionId')
      if [ -z "$EXECUTION_ID" ] || [ "$EXECUTION_ID" = "null" ]; then
        echo "ERROR: Failed to trigger ContextQA test plan."
        echo "Response: $TRIGGER_RESPONSE"
        exit 1
      fi
      echo "ContextQA execution started: ${EXECUTION_ID}"
      echo "EXECUTION_ID=${EXECUTION_ID}" >> contextqa.env
      echo "TOKEN=${TOKEN}" >> contextqa.env

    # Step 3: Poll for completion
    - |
      source contextqa.env
      STATUS="RUNNING"
      ATTEMPT=0
      MAX_ATTEMPTS=60

      while [ "$STATUS" = "RUNNING" ] || [ "$STATUS" = "PENDING" ]; do
        if [ "$ATTEMPT" -ge "$MAX_ATTEMPTS" ]; then
          echo "ERROR: ContextQA execution timed out after $((MAX_ATTEMPTS * 30)) seconds."
          exit 1
        fi
        sleep 30
        ATTEMPT=$((ATTEMPT + 1))
        STATUS_RESPONSE=$(curl -s -X GET "${CONTEXTQA_BASE_URL}/api/v1/executions/${EXECUTION_ID}/status" \
          -H "Authorization: Bearer ${TOKEN}")
        STATUS=$(echo "$STATUS_RESPONSE" | jq -r '.status')
        echo "Attempt ${ATTEMPT}: ContextQA status = ${STATUS}"
      done

      REPORT_URL="https://app.contextqa.com/executions/${EXECUTION_ID}"
      echo "ContextQA final status: ${STATUS}"
      echo "Report: ${REPORT_URL}"

      echo "CONTEXTQA_STATUS=${STATUS}" >> contextqa.env
      echo "CONTEXTQA_REPORT_URL=${REPORT_URL}" >> contextqa.env

      if [ "$STATUS" = "FAILED" ]; then
        echo "ContextQA tests FAILED. See report: ${REPORT_URL}"
        exit 1
      fi

  artifacts:
    reports:
      dotenv: contextqa.env
    paths:
      - contextqa.env
    expire_in: 7 days
    when: always

publish_result:
  stage: report
  image: alpine:latest
  before_script:
    - apk add --no-cache curl
  script:
    - echo "ContextQA Status: ${CONTEXTQA_STATUS}"
    - echo "Report URL: ${CONTEXTQA_REPORT_URL}"
    # Post result as MR comment if running in a merge request context
    - |
      if [ -n "$CI_MERGE_REQUEST_IID" ]; then
        curl -s -X POST \
          "https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/merge_requests/${CI_MERGE_REQUEST_IID}/notes" \
          -H "PRIVATE-TOKEN: ${GITLAB_API_TOKEN}" \
          -H "Content-Type: application/json" \
          -d "{\"body\":\"**ContextQA Test Result:** ${CONTEXTQA_STATUS}\\n\\nExecution report: ${CONTEXTQA_REPORT_URL}\"}"
        echo "MR comment posted."
      fi
  needs:
    - job: contextqa_tests
      artifacts: true
  when: always
```

## How the pipeline works

**Authentication:** The `before_script` installs `curl` and `jq` on the Alpine base image. The first script block POSTs credentials to the ContextQA login endpoint. If authentication fails, the job exits immediately with code 1, which fails the pipeline stage.

**Triggering the test plan:** The execute endpoint uses a GET request. ContextQA returns the `executionId` in the response body. The execution ID and token are written to `contextqa.env`, a dotenv-format file that GitLab CI can pass between jobs using the `artifacts: reports: dotenv` mechanism.

**Polling:** The polling loop runs every 30 seconds with a 60-attempt ceiling (30-minute maximum). Adjust `MAX_ATTEMPTS` based on your suite's expected duration. The loop exits when the status transitions out of `RUNNING` or `PENDING`. A `FAILED` status exits with code 1, failing the pipeline stage and blocking any downstream jobs that depend on `contextqa_tests`.

**Artifact passing:** The `contextqa.env` dotenv file is published as a job artifact. The `publish_result` job uses `needs: artifacts: true` to receive the environment variables `CONTEXTQA_STATUS` and `CONTEXTQA_REPORT_URL` from `contextqa_tests`. This pattern works even when `contextqa_tests` fails, because `when: always` is set on `publish_result`.

**MR comment:** If the pipeline runs in a merge request context (`CI_MERGE_REQUEST_IID` is set), the `publish_result` job posts a comment to the MR with the test status and report URL. This requires a `GITLAB_API_TOKEN` CI/CD variable with API scope. If you do not want MR comments, remove the conditional block.

## Storing the GitLab API token for MR comments

To post MR comments, add a fourth CI/CD variable:

1. Navigate to **Settings → CI/CD → Variables**.
2. Add a variable with **Key** `GITLAB_API_TOKEN`.
3. Set the value to a GitLab personal access token or project access token with `api` scope.
4. Mark it as **Masked**.

If you use a project access token, ensure it has the `api` role at minimum. Personal access tokens scoped to `api` also work but are tied to an individual user — prefer project access tokens for CI use.

## Failing the pipeline on test failure

The job exits with code 1 when `STATUS = "FAILED"`. In GitLab CI, a non-zero exit code from any script block fails the job and marks it as `failed` in the pipeline graph. Any downstream stage jobs that list `contextqa_tests` as a dependency will not run. This provides automatic release gating without additional configuration.

If you want the pipeline to continue after test failure (for example, to always run a cleanup stage), add `allow_failure: true` to the `contextqa_tests` job definition. The job will be marked as `failed` with a warning icon but will not block downstream jobs.

## Frequently Asked Questions

### Why does the script use GET to trigger the test plan instead of POST?

ContextQA's test plan execute endpoint is a GET request by API contract. The execution is fully identified by the plan ID in the URL path — there is no request body. Using POST will return a 405 Method Not Allowed. This is documented accurately in the ContextQA API reference.

### How do I run ContextQA only on merge requests, not on every push?

Use GitLab's `rules` keyword on the `contextqa_tests` job:

```yaml
contextqa_tests:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
```

This restricts the job to merge request pipelines only.

### Can I run different test plans for different branches?

Yes. Use GitLab CI variables with branch-specific values, or use `rules` with `variables` overrides to set `CONTEXTQA_PLAN_ID` based on `$CI_COMMIT_BRANCH`.

## Related

* [GitHub Actions integration](https://learning.contextqa.com/integrations/github-actions)
* [Jenkins integration](https://learning.contextqa.com/integrations/jenkins)
* [CircleCI integration](https://learning.contextqa.com/integrations/circleci)
* [Azure DevOps integration](https://learning.contextqa.com/integrations/azure-devops)
* [Running tests and test plans](https://learning.contextqa.com/execution/running-tests)

{% hint style="info" %}
**Connect ContextQA to your CI/CD pipeline in 15 minutes.** [**Book a Demo →**](https://contextqa.com/book-a-demo/) — See the full integration walkthrough for your existing toolchain.
{% endhint %}