Playwright & Azure DevOps Pipeline

In this article we will discuss about how to run Playwright test from Azure DevOps Pipeline.

Prerequisites,

· Knowledge on NodeJS, Azure DevOps and Playwright

· Register into Azure Devops account

· Playwright test scripts in GitHub Repo

Once registered into Azure DevOps, create an organization (a2ztestautomation).

Create a Project(Playwright integration test) inside an organization.

Import (from Github) :

Before this step, push your playwright test scripts into Github repo. Now to make use of this repo here,

Repo -> Import a repository -> Enter GitHub repo url.

Now the project is available in Azure Repo.

Create a pipeline :

Select ‘Use the classic editor’ & Select ‘Empty Job’.

Add tasks to job:

We need to add 4 tasks namely,

a. NodeJS tool installer
b. Command line script
c. Npm
d. Publish reports,
  Playwright will generate reports in various formats by providing 
  configuration in playwright.config.ts file. Here we can publish
  either .html report or jUnit .xml report.
    i.  Publish .html reports using Publish Pipeline Artifact task
    ii. Publish .xml reports using Publish Test Results task

To add a new task, click ‘+’ button on the side of ‘Agent Job 1’ and search the required tool in search bar as below,

a. Add NodeJS tool Installer :

b. Add Command line script:

Link working directory:

c. Add npm :

d. Publish Results

i. Publish .html reports using Publish Pipeline Artifact task

To generate .html report, we need to configure playwright.config.ts file as,

  reporter: 'html',

Under Agent Job 1,

Add a task -> Publish Pipeline Artifact and configure as below,

Note: Already once run your playwright test in local, which would have created a report folder ‘playwright-report’ under the project folder. The same folder is selected here for ‘File or directory path’.

Run a pipeline :

On running the pipeline, we may get an error — ‘No hosted parallelism has been purchased or granted’. To resolve this, we need to register Azure Devops parallelism request at https://aka.ms/azpipelines-parallelism-request.

This request may take 2 to 3 days for approval. Once it is approved we would be able to run the Pipeline job.

Once pipeline got executed successfully and will display the status as below,

Now to view the results, click on ‘Agent job 1’ -> will display the detail log of each executed task as below,

Now to view the published results under /playwright-report, click on ‘Agent job 1’ :

Now on click of artifact from the displayed link will navigate to ‘Artifacts’ page. This page will display the list of generated artifacts — in our case .html report under ‘playwright-report’ folder as,

A report can be downloaded to our local on click of ‘index.html’ which will include,

ii. Publish .xml reports using Publish Test Results task :

To generate .xml report, we need to configure playwright.config.ts file as,

  reporter: [['junit', { outputFile: 'results.xml' }]],

Under Agent Job 1,

Add a task -> Publish Test Results and configure as below,

Now, on running a pipeline,

On clicking ‘Test’ tab will display the test summary as,

So, depends on playwright report types, we need to configure pipeline tasks.

Reference :

Continuous Integration | Playwright

Publish Pipeline Artifact — Azure Pipelines | Microsoft Learn

Conclusion:

As a result we can run our Playwright test from Azure DevOps pipeline . Similar to this we can create a release pipeline also.