Cucumber Reports

This article is about how to generate various reports in Cucumber and in case of failure how to attach screenshots with reports. Also about generating reports in CI environment.

Cucumber.io has developed a free cloud-based service for sharing reports. Cucumber reporting service allows to configure cucumber to publish results to the cloud which can be accessed from browser throughout the organization. This feature is greatly helpful while testing in CI environment.

Cucumber uses reporter plugins to produce reports that contain information about which scenarios have passed or failed.

Publishing to the Cucumber Reports service is currently supported in:

- Cucumber-JVM `6.7.0` and above

- Cucumber-Ruby `5.1.1` and above

- Cucumber-JS `7.0.0` and above

Built-in reporter plugins:

If you don’t want to publish your reports to the [Cucumber Reports](https://reports.cucumber.io/) service, you can generate local reports using one of the following built-in reporter plugins (also known as “formatters”):

​ message

​ progress

​ pretty

​ html

​ rerun

​ junit

​ testng

Generate Reports :

Let we see how to generate,

Default Cloud Report

HTML Report using built-in plugin

Default Cloud Report :

Cucumber Cloud Report does not require any extra plugins but we need to include standard cucumber maven dependencies and cucumber.properties file.

Create a maven project and update pom.xml,

<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-java</artifactId>
    <version>6.8.0</version>
 </dependency>
 <dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-core</artifactId>
    <version>6.8.0</version>
  </dependency>
  <dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-junit</artifactId>
    <version>6.8.0</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.8.0</version>
  </dependency>

Create a cucumber.properties file under /src/test/resource/ with below content,

cucumber.publish.enabled = true
cucumber.publish.quiet = true

This will publish cucumber reports in cloud and will be available for 24hrs. These reports should be saved for later use.

Create a sample feature file, step definition file and runner class. And in runner class, under CucumberOptions include plugin details as below.

plugin = {
    "pretty"
}

‘Pretty’ plugin will give verbose console output.

Run this as ‘Junit Runner’ which will provide cucumber cloud report link in console as below,

Eclipse console output

The above displayed url will display a report with detailed log,

Cucumber Cloud Report

HTML Report :

To generate custom report in html format include below plugin in Runner class,

plugin = {
    "pretty",
    “html:target/html-reports/report.html”
}

After execution will generate HTML report under target/html-reports/ folder as,

HTML Report

Attaching screenshots :

In case of any failed scenario the screenshots could be attached as image with a report. This has to be handled in Step Definition file as,

@After
public void includeScreenshot(Scenario scenario) throws IOException {
if(scenario.isFailed()) {
File screenshot = ((TakesScreenshot)         driver).getScreenshotAs(OutputType.FILE);
byte[] fileContent = FileUtils.readFileToByteArray(screenshot);
scenario.attach(fileContent, “image/png”, “image1”);
}}

After execution if any scenario got failed, cloud report gives a detailed log with screenshot as below,

Report attached with screenshot

And html report also have the same details as,

Other built-in plugins :

In case of using pretty, message and progress plugins, either any one of these report can be generated at a time on console.

In case of using junit and testng plugins, either of these reports can be generated in a separate file depends on runner and also by providing below plugin details in runner class,

plugin = { “message”,
 “html:target/html-reports/report.html”,
 “junit:target/junit-reports/”
 })

Generating Reports in CI Environment :

To run our test in CI environment, I am going to use Jenkins server.

Before running in CI environment we first move our maven project into GitHub https://github.com/anandhi-k/MvnCucumberReports.git.

Create a job in Jenkins as,

Step 1: Create a freestyle project in Jenkins with below config,

Setting GitHub URL

Run as Maven Test

Step 2: Build a project

Step 3: View the reports after successful run.

Build console output in Jenkins

Conclusion :

Cucumber Reports generate Cloud reports and also custom reports in given format which provides easy way to share our test reports with stakeholders.

References:

Share reports and living documentation with the whole team | Cucumber Blog

Reporting — Cucumber Documentation