Skip to content
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Azure DevOps

Connect Azure DevOps to ContextQA, generate tests from Azure Boards work items, report failed runs as bugs, and link execution evidence.

Documentation owner: Deep Barot · Last updated: October 22, 2026

Quick answer

Connect Azure DevOps under Integrations & Plugins. You can then use Assets → Requirements → New Requirement → Generate from Integrations to select Azure Boards work items for AI test generation, or select Bug on a failed run and choose Azure DevOps as the defect destination. ContextQA keeps a link between the work item and the requirement or run evidence.

Key takeaways

  • Connect Azure DevOps under Integrations & Plugins.

  • Open your Azure DevOps dashboard.

  • Open Assets → Requirements and select New Requirement.

What this page covers

Who is this for? SDETs, developers, and engineering managers who use Azure Boards for issue tracking and want to push test failures directly as work items.

The ContextQA Azure DevOps integration supports two user-facing loops:

  • Requirement to coverage — select Azure Boards work items and let ContextQA generate reviewable tests from their content.

  • Failure to defect — create a work item from a failed run with reproduction context and a link back to the evidence.

ContextQA integrations catalog showing Jira, Slack, Linear, Azure DevOps, GitHub, Figma, test lab, and CI/CD options

Setting up the Azure DevOps integration

Step 1: Generate a Personal Access Token in Azure DevOps

  1. Open your Azure DevOps dashboard.

  2. Go to your profile menu, select the three-dot menu, and choose User settings.

  3. Select Profile, then navigate to Personal access tokens.

  4. Click New token.

  5. Give the token a clear name and grant the narrowest Azure Boards work-item permissions the ContextQA workflow requires. Avoid Full access unless your administrator and ContextQA support confirm that it is necessary.

  6. Click Create and copy the generated token immediately. You will not be able to view it again after closing the dialog.

Step 2: Enable the Azure DevOps plugin in ContextQA

  1. In ContextQA, click Settings in the left sidebar.

  2. Open Integrations & Plugins.

  3. Find Azure DevOps Board in the plugin list and enable it.

Step 3: Enter your integration credentials

  1. Paste the Personal Access Token copied in Step 1 into the access key field.

  2. Enter your Azure DevOps organization URL (copy this from the address bar of your Azure DevOps portal, for example https://dev.azure.com/your-org).

  3. Click Create.

ContextQA verifies the credentials and confirms a successful connection. The integration is now active for all projects in your ContextQA workspace.


Generate tests from Azure Boards work items

  1. Open Assets → Requirements and select New Requirement.

  2. Choose Generate from Integrations. During a staged rollout, the first card may still read Generate from Jira before the integration chooser appears.

  3. Choose Azure DevOps. If it is not connected, follow the link back to Integrations & Plugins.

  4. Search by work-item ID or title. Narrow the list by Project and Work Item Type when needed.

  5. Select the work items and click Continue.

  6. Choose the platform, publish mode, environment, test-case types, and optional context.

  7. Start analysis. Answer clarification questions when ContextQA needs more information.

  8. Open the resulting requirement and review Test Cases, Questionnaire, Requirement Data, Metadata, and Gaps before approval.

The picker supports multiple selections and reports unreadable items before generation. Remove failed items rather than publishing a partial selection accidentally.


Creating work items from test failures

Once the integration is configured, you can report a bug to Azure Boards directly from any failing test result.

Step 1: Identify a failed test case

Open the execution result page for a failed test run. The failed step is highlighted with a red status badge, a screenshot, and an error message.

Step 2: Start reporting a bug

Click Bug in the test result toolbar.

Step 3: Select Azure DevOps as the destination

When the destination chooser opens, select Azure DevOps. The same action can also offer Jira when both integrations are configured.

Step 4: Fill in work item details

  • Issue type — select the work item type appropriate for your Azure project (e.g., Bug, Test Case, Shared Step)

  • Title — enter a clear title; ContextQA automatically pre-fills the test description and reproduction steps in the description field to save time

Step 5: Save and sync

Click Save. The work item is created immediately in Azure Boards and a direct link appears in the ContextQA result page.

Step 6: Verify in Azure Boards

Follow the generated link to open Azure Boards and confirm the ticket was created. The work item includes the full bug description, test step history, and all context provided by ContextQA.


Managing the work item in Azure Boards

Once the ticket is in Azure Boards:

  • Assign it to the responsible developer using the standard Azure Boards assignment workflow

  • Use Azure Boards sprint and backlog management to prioritize the fix

  • Update the ticket status as the fix progresses — the link in ContextQA continues pointing to the same ticket so QA engineers can monitor resolution


Configuring project and area path

When creating work items, ContextQA uses the organization and credentials provided during setup. If your Azure DevOps organization contains multiple projects, select the appropriate project in the work item creation dialog each time you report a bug. Azure Boards manages area path and iteration configuration according to your existing project structure.


Azure Pipelines CI/CD integration

Use an Azure Pipelines stage to coordinate a ContextQA test-plan run and require its result before deployment.

Before configuring the stage:

  1. Create and manually validate the test plan.

  2. Open its CI / CD tab and confirm Azure DevOps is listed.

  3. Obtain the current public start and status contract supplied for your workspace.

  4. Create a dedicated API token with the narrowest required scopes.

In Pipelines → Library → Variable Groups, add the API token as a secret variable named CONTEXTQA_API_TOKEN. Store the test plan ID and the public base URL, when supplied, as non-secret variables. Authorize the variable group only for the pipelines that need it.

When the public contract is available, the stage should:

  1. Start the plan with the published method, path, and payload.

  2. Capture the returned run ID.

  3. Poll the published status operation with a bounded timeout.

  4. Return a non-zero exit code for every documented non-success result.

  5. Include the run ID in the job output so reviewers can find the run in ContextQA.

Place the stage after the target environment is ready and before the protected deployment stage. Use environment approvals and checks when production requires an additional human gate.


Traceability

With both the Jira and Azure DevOps integrations in use, ContextQA can push bugs to whichever tracker your team uses. Each bug includes:

  • A link to the ContextQA result page with screenshots, video, and step logs

  • The reproduction steps generated by the AI

  • The test case name and ID for lookup in ContextQA

  • A direct execution or result link when the work item was created from an external Azure DevOps workflow

This gives developers enough context to begin triage from Azure Boards. Opening the linked screenshots, video, logs, or live result still requires authorized access to the corresponding ContextQA organization and workspace.

Connect ContextQA to your CI/CD pipeline in 15 minutes. Book a Demo → — See the full integration walkthrough for your existing toolchain.

Last updated

Was this helpful?