Cypress Retries

Cypress helps in automating end-to-end scenarios. Sometimes cypress test execution may ends in flaky test results. There are some behaviors that are hard to verify and make tests flaky (i.e., unreliable) and fail sometimes due to unpredictable conditions (eg., temporary breakouts in external dependencies, connectivity errors, etc.). Some other common race conditions that could result in unreliable tests include:

• Resource dependencies availability
• Connectivity issues
• Animations
• API calls
• Backend Server availability

With test retries, Cypress is able to retry failed tests to help reduce test flakiness. Retries are very much helpful to in reducing continuous integration (CI) build failures.

How does it work?

By default, tests will not retry when they fail. We need to enable test retries in configuration to use this feature.

Once test retries are enabled, tests can be configured to have X number of retry attempts. For example, if test retries has been configured with 2 retry attempts, Cypress will retry tests up to 2 additional times (for a total of 3 attempts) before potentially being marked as a failed tests.

Step-by-step how the test retries works ,

Assuming we have configured test retries with 2 retry attempts (for a total of 3 attempts), here is how the tests might run:

• A test runs for the first time. If the test passes, will proceed with remaining tests.
• If the test fails, Cypress will attempt to run the test a second time.
• If the test passes after the second attempt, will proceed with remaining tests.
• If the test fails a second time, Cypress will make the final third attempt to re-run the test.
• If the test fails a third time, Cypress will mark the test as failed and then will proceed with remaining tests.

Configuring Test Retries :

Test retries can be configured in two ways :

Global configuration

Custom configuration

Global Configuration :

This can be done at cyperss.json, using two different options :

runMode — allows to define the number of test retries when running cypress run.

openMode — allows to difine the number of test retries when running cypress open.

In cypress.json

"retries" : {
     "runMode" : 2,
     "openMode" : 2
  }

To configure the retry attempts for all tests run in both cypress run and cypress open :

{
        "retries": 2
}

After running test spec in open mode ,

Custom configuration :

Can be set either at a Test Suite level or at an Individual test level.

To configure retry attempts on a Test Suite level,

describe("Testing login functionality", {
  "retries": {
    runMode: 2,
    openMode : 2
  }
},() => {
  it.only("Login success", () => {
    cy.visit("https://the-internet.herokuapp.com/login")
    cy.get("#username").type("tomsmith")
    cy.get("#password").type("SuperSecretPassword!")
    cy.get(".radius").click()
    //cy.url().should("include", "secure")
    //To make it fail
    cy.url().should("include", "success")
  })

To configure retry attempts on a Test level,

it.only("Login Failure-Username", {
    "retries": {
      runMode: 2,
      openMode : 2
    }
  }, () => {
    cy.visit("https://the-internet.herokuapp.com/login")
    cy.get("#username").type("tom")
    cy.get("#password").type("SuperSecretPassword!")
    cy.get(".radius").click()
    //cy.get(".flash").should("contain", "Your username is invalid!")
    //To make it fail
    cy.get(".flash").should("contain", "Your user is invalid!")
  })

While running in run mode, by default cypress will take the screenshots and screenshot names will include attempt numbers as below,

Results displayed in Dashboard :

Information related to test retries is displayed on the Test Results tab for a run which will demonstrate the number of failed attempts.

References :

Test Retries | Cypress Documentation