Azure DevOps

How to connect ContextQA with Azure Boards to create work items from test failures and track bugs with full reproduction context.

Who is this for? SDETs, developers, and engineering managers who use Azure Boards for issue tracking and want to push test failures directly as work items.

The ContextQA Azure DevOps integration connects test failures directly to Azure Boards work items. When a test identifies a bug, you can create an Azure Boards ticket with a single click. The ticket is pre-populated with the test description, reproduction steps, and a link back to the ContextQA result page so developers have everything they need to reproduce and fix the issue.

---

Setting up the Azure DevOps integration

Step 1: Generate a Personal Access Token in Azure DevOps

1. Open your Azure DevOps dashboard.

2. Go to your profile menu, select the three-dot menu, and choose User settings.

3. Select Profile, then navigate to Personal access tokens.

4. Click New token.

5. Give the token a clear name and set the permissions required — you can use Full access or limit permissions to Work Items (read and write) depending on your security policy.

6. Click Create and copy the generated token immediately. You will not be able to view it again after closing the dialog.

Step 2: Enable the Azure DevOps plugin in ContextQA

1. In ContextQA, click the Settings icon in the left sidebar.

2. Navigate to the Plugin section.

3. Find Azure DevOps Board in the plugin list and enable it.

Step 3: Enter your integration credentials

1. Paste the Personal Access Token copied in Step 1 into the access key field.

2. Enter your Azure DevOps organization URL (copy this from the address bar of your Azure DevOps portal, for example https://dev.azure.com/your-org).

3. Click Create.

ContextQA verifies the credentials and confirms a successful connection. The integration is now active for all projects in your ContextQA workspace.

---

Creating work items from test failures

Once the integration is configured, you can report a bug to Azure Boards directly from any failing test result.

Step 1: Identify a failed test case

Open the execution result page for a failed test run. The failed step is highlighted with a red status badge, a screenshot, and an error message.

Step 2: Start reporting a bug

Click the Report Bug option in the test result toolbar.

Step 3: Select Azure Boards as the destination

When the bug reporting dialog opens, select Azure Board from the list of available integrations.

Step 4: Fill in work item details

• Issue type — select the work item type appropriate for your Azure project (e.g., Bug, Test Case, Shared Step)

• Title — enter a clear title; ContextQA automatically pre-fills the test description and reproduction steps in the description field to save time

Step 5: Save and sync

Click Save. The work item is created immediately in Azure Boards and a direct link appears in the ContextQA result page.

Step 6: Verify in Azure Boards

Follow the generated link to open Azure Boards and confirm the ticket was created. The work item includes the full bug description, test step history, and all context provided by ContextQA.

---

Managing the work item in Azure Boards

Once the ticket is in Azure Boards:

• Assign it to the responsible developer using the standard Azure Boards assignment workflow

• Use Azure Boards sprint and backlog management to prioritize the fix

• Update the ticket status as the fix progresses — the link in ContextQA continues pointing to the same ticket so QA engineers can monitor resolution

---

Configuring project and area path

When creating work items, ContextQA uses the organization and credentials provided during setup. If your Azure DevOps organization contains multiple projects, select the appropriate project in the work item creation dialog each time you report a bug. Area path and iteration configuration are managed within Azure Boards according to your existing project structure.

---

Azure Pipelines CI/CD integration

To run ContextQA test plans as part of an Azure Pipelines build or release pipeline, use the ContextQA MCP server or REST API from a pipeline script step.

Example pipeline task using a script step:

- task: Bash@3
  displayName: 'Run ContextQA Test Plan'
  inputs:
    targetType: 'inline'
    script: |
      # Trigger the test plan and capture the execution ID
      EXEC_ID=$(curl -s -X POST "https://api.contextqa.com/v1/test-plans/<plan-id>/execute" \
        -H "Authorization: Bearer $(CONTEXTQA_API_KEY)" \
        -H "Content-Type: application/json" | jq -r '.execution_id')

      # Poll for completion
      STATUS="running"
      while [ "$STATUS" = "running" ]; do
        sleep 30
        STATUS=$(curl -s "https://api.contextqa.com/v1/executions/$EXEC_ID" \
          -H "Authorization: Bearer $(CONTEXTQA_API_KEY)" | jq -r '.status')
      done

      echo "Execution completed with status: $STATUS"
      [ "$STATUS" = "passed" ] || exit 1

Store your CONTEXTQA_API_KEY as a secret pipeline variable in Azure DevOps. Set the pipeline variable in Pipelines → Library → Variable Groups and reference it as $(CONTEXTQA_API_KEY).

The script returns a non-zero exit code if the test plan fails, which causes the Azure Pipelines stage to fail and blocks deployment — the standard pattern for quality gates in a CI/CD pipeline.

---

Traceability

With both the Jira and Azure DevOps integrations in use, ContextQA can push bugs to whichever tracker your team uses. Each bug includes:

• A link to the ContextQA result page with screenshots, video, and step logs

• The reproduction steps generated by the AI

• The test case name and ID for lookup in ContextQA

This ensures that developers have enough context to reproduce the issue without needing access to ContextQA directly.

Connect ContextQA to your CI/CD pipeline in 15 minutes. Book a Demo → — See the full integration walkthrough for your existing toolchain.