Performance Testing in Playwright

In this article we cover how do we capture performance metrics from the page during test run using CDP’s(Chrome DevTool Protocol) Performance domain in Playwright.

Chrome DevTool Protocol

The Chrome DevTools Protocol allows for tools to instrument, inspect, debug and profile Chromium, Chrome and other Blink-based browsers. Many existing projects currently use the protocol.

Instrumentation is divided into a number of domains (DOM, Debugger, Network etc.). Each domain defines a number of commands it supports and events it generates. Both commands and events are serialized JSON objects of a fixed structure.

Protocol Domains

Chrome DevTools Protocol is divided into domains. Each domain has a set of commands and events that it supports.

For example, the Network domain contains APIs for accessing the HTTP requests and responses made when rendering a page. The DOM (Document Object Model) domain. It exposes APIs for reading from and writing to the DOM. You can access query selectors, get element attributes, manipulate nodes and even scroll to selected nodes. Another useful domain is Performance domain.

Apart from powering Developer Tools in Chrome, Chrome DevTools Protocol provides some of the underlying functionality used in popular testing libraries like Playwright, Puppeteer, and Selenium.

Playwright also uses Chrome DevTools Protocol to interact with Chromium-based browsers. One exciting feature in Playwright is BrowserContexts. BrowserContexts lets you operate many independent browser sessions. If a page opens another window, that page gets added to the parent context. A browser context can have multiple pages(tabs).

If you don’t want to use the high-level methods provided by Playwright, you can use CDPSession to directly interact with Chrome DevTools Protocol.

Performance Domain :

The main purpose of the Performance domain is to get metrics from the page. We will be focusing on the getMetrics method which allow us to extract runtime metrics during the test execution. This method will return the below measurement as key/value pair.

· Timestamp : The timestamp when the metrics sample was taken.

· Documents : Number of documents in the page.

· Frames : Number of frames in the page.

· JSEventListeners : Number of events in the page.

· Nodes : Number of DOM nodes in the page.

· LayoutCount : Total number of full or partial page layout.

· RecalcStyleCount : Total number of page style recalculations.

· LayoutDuration : Combined durations of all page layouts.

· RecalcStyleDuration : Combined duration of all page style recalculations.

· ScriptDuration : Combined duration of JavaScript execution.

· TaskDuration : Combined duration of all tasks performed by the browser.

· JSHeapUsedSize : Used JavaScript heap size.

· JSHeapTotalSize : Total JavaScript heap size.

Before start scripting, let we see how to view these metrics in browser.

1. Open Google Chrome
2. Launch Google page
3. Press F12 and navigate to ‘Performance’ tab.

Now, click the record button and do some action in the web page for example search ‘Playwright’.

Once the action is completed, stop the recording which will display the metrics as below,

The above window has various measurements of Network, Main, Frame, GPU, etc..

For example, we can see duration details under ‘Timings’. Timings will give individual time points performed in each frame and also it gives performance marks i.e start and end points of each actions. Each give detailed trace as below,

All of these can be made available as artifactory like .json file.

Now we see how to get these Chrome and web measurements automatically using Playwright scripts.

Playwright Implementation :

1. Create a separate folder and open it in VS code.

2. Install Playwright and its dependencies,

npm install playwright

npm install @playwright/test

3. Create a performance.test.ts file under tests folder. In this file we create two tests,

Test 1 : To get the list of Performance metrics

Test 2 : To get the duration measurement of particular action by marking start and end positions.

These two can be implemented by using Performance API methods. These API methods can be invoked as Javascript commands using evaluate method.

Test 1 : To get the list of Performance metrics

test.describe('CDP Demo', () => {
    test('Get performance metrics', async ({ page, browser }) => {
        //Create a new connection to an existing CDP session to enable performance Metrics
        const session = await page.context().newCDPSession(page)
        //To tell the CDPsession to record performance metrics.
        await session.send("Performance.enable")
        await page.goto("https://www.google.com/")
        await page.getByTitle("Search").click()
        await page.getByTitle("Search").fill("Playwright")
        await page.press('[title="Search"]', 'Enter')
        console.log("=============CDP Performance Metrics===============")
        let performanceMetrics = await session.send("Performance.getMetrics")
        console.log(performanceMetrics.metrics)
    })

To run the script from terminal,

npm run test performance.test.ts

The output will be,

Test 2 : To get the duration measurement of particular action by marking start and end positions

test("Capture performance traces by marking actions using Performance API", async ({ page, browser }) => {
        console.log("========== Start Tracing Perf ===========")
        await browser.startTracing(page, { path: './perfTraces.json', screenshots: true })
        await page.goto("https://www.google.com/")
        //Using Performanc.mark API
        await page.evaluate(() => (window.performance.mark('Perf:Started')))
        await page.getByTitle("Search").click()
        await page.getByTitle("Search").fill("Playwright")

        //Using performance.mark API
        await page.evaluate(() => (window.performance.mark('Perf:Ended')))

        //Performance measure
        await page.evaluate(() => (window.performance.measure("overall", "Perf:Started", "Perf:Ended")))

        //To get all performance marks
        const getAllMarksJson = await page.evaluate(() => (JSON.stringify(window.performance.getEntriesByType("mark"))))
        const getAllMarks = await JSON.parse(getAllMarksJson)
        console.log('window.performance.getEntriesByType("mark")', getAllMarks)

        //To get all performance measures of Google
        const getAllMeasuresJson = await page.evaluate(() => (JSON.stringify(window.performance.getEntriesByType("measure"))))
        const getAllMeasures = await JSON.parse(getAllMeasuresJson)
        console.log('window.performance.getEntriesByType("measure")', getAllMeasures)
        await page.press('[title="Search"]', 'Enter')
        console.log("======= Stop Tracing ============")
        await browser.stopTracing()
    })

In the above code we mark the start and end positions of particular action, get measurements of it and store its traces in .json file.

The output will be,

After running the script, this will generate a perfTraces.json file under project folder. These measurements can be viewed from browser by uploading this file as below,

Now from the browser we can view the details as,

Reference :

Chrome DevTools Protocol — Performance domain

Page | Playwright

User Timing Level 3 (w3.org)

Conclusion:

By having these metrics, we can measure each test step duration. This domain will help to evaluate metrics from the page which can be used for necessary assertions in Test Automation using the benchmark values returned by this API.