WebdriverIO is one of the most popular test frameworks and an excellent Web integration tool.

In fact, it's one of my favourites ❤️

But, to write truly great acceptance tests you need more than that:

• You need business-friendly abstractions that capture the language of your domain and make even the most sophisticated, multi-actor, and cross-system workflows easy to design and adapt as the requirements change.
• You need in-depth reporting that tells you not only what tests were executed, but also what requirements and business capabilities have (and have not!) been tested.
• And on top of that, you need to be able to interact with all the interfaces of your system, so that the slower UI-based interactions are just one tool in your testing toolbox, rather than your only option.

Of course, all the above is way outside of the scope of what WebdriverIO is trying to accomplish.

Typically, you and your team would need to figure it out all by yourselves.

But, what if I told you that there was a better way? That there was another framework that's perfectly compatible with WebdriverIO and optimised to help you write world-class, full-stack acceptance tests following the Screenplay Pattern and SOLID design principles, even if not everyone on your team is an experienced test engineer?

What if I told you that this framework also covers business-friendly reporting and helps you write high-quality test code that's easy to understand, maintain, and reuse across projects and teams?

What if I told you that you could add it to your existing WebdriverIO test suites, today?

Please allow me to introduce, Serenity/JS!

About Serenity/JS​

Serenity/JS is an open-source acceptance testing and integration framework, designed to make writing truly great acceptance tests easier, more collaborative, and fun! 🚀

While you can use Serenity/JS to test systems of any complexity, it works particularly well in complex, workflow-based, multi-actor contexts.

At a high level, Serenity/JS is a modular framework that provides adapters that make it easy to integrate your tests with Web apps, REST APIs, Node.js servers, and pretty much anything a Node.js program can talk to.

Thanks to Serenity/JS Screenplay Pattern, you and your team will also have a simple, consistent, and async-friendly programming model across all those interfaces.

Apart from integrating with your system under test, Serenity/JS can also integrate with popular test runners such as Cucumber, Jasmine, Mocha, Protractor, and now also WebdriverIO!

Better yet, Serenity/JS provides a unique reporting system to help you generate consistent test execution and feature coverage reports across all the interfaces of your system and across all your test suites. Serenity/JS reporting services can work together with your existing WebdriverIO reporters too!

Thinking in Serenity/JS​

The best way to get started with Serenity/JS is to follow our brand-new series of tutorials, where you'll learn how to build full-stack test automation frameworks from scratch.

Check out "Thinking in Serenity/JS" 🤓

Examples and project templates​

If you prefer to kick the tires and jump straight into the code, you'll find over a dozen example projects in the Serenity/JS GitHub repository and plenty of code samples in our API docs.

We've also created WebdriverIO + Serenity/JS project templates to help you get started:

• WebdriverIO, Serenity/JS and Cucumber
• WebdriverIO, Serenity/JS and Mocha

All the above templates are configured to produce Serenity BDD HTML reports, automatically capture screenshots upon test failure, and run in a Continuous Integration environment.

Adding Serenity/JS to an existing project​

If you're using WebdriverIO with Mocha, run the following command in your computer terminal to add the relevant Serenity/JS modules to your project:

npm install --save-dev @serenity-js/{code,console-reporter,mocha,webdriverio}

If you're using Cucumber or Jasmine instead, replace mocha with the name of your preferred test runner, so cucumber or jasmine, respectively.

Next, tell WebdriverIO to use Serenity/JS instead of the default framework adapter:

import { ConsoleReporter } from '@serenity-js/console-reporter';
import { WebdriverIOConfig } from '@serenity-js/webdriverio';

export const config: WebdriverIOConfig = {

    // Enable Serenity/JS framework adapter
    // see: https://serenity-js.org/modules/webdriverio/
    framework: '@serenity-js/webdriverio',

    serenity: {
        // Use Serenity/JS test runner adapter for Mocha
        runner: 'mocha',        // see: https://serenity-js.org/modules/mocha/
        // runner: 'jasmine',   // see: https://serenity-js.org/modules/jasmine/
        // runner: 'cucumber',  // see: https://serenity-js.org/modules/cucumber/

        // Configure reporting services
        // see: https://serenity-js.org/handbook/reporting/
        crew: [
            ConsoleReporter.forDarkTerminals(),
        ],
    },

    // ... other WebdriverIO configuration
}

And that's it!

The above configuration enables Serenity/JS Console Reporter, which produces output similar to the below and plays nicely with any existing WebdriverIO reporters you might have already configured:

To enable Serenity BDD HTML Reporter, please follow the instructions.

To learn about Serenity/JS Screenplay Pattern, follow the tutorial.

Questions? Feedback? Ideas?​

If you have questions about Serenity/JS or need guidance in getting started, join our friendly Serenity/JS Community Chat channel.

For project news and updates, follow @SerenityJS on Twitter.

And if you like Serenity/JS and would like to see more articles on how to use it with WebdriverIO, remember to give us a ⭐ on Serenity/JS GitHub! 😊

Enjoy Serenity!

Jan

For many years one of the selling features of the WebdriverIO framework was its synchronous API. Especially for folks coming from more synchronous oriented languages such as Java or Ruby, it has helped to avoid race conditions when executing commands. But also people that are more familiar with Promises tend to prefer synchronous execution as it made the code easier to read and handle.

Running asynchronous commands in a synchronous way was possible with the help of the @wdio/sync package. If installed, WebdriverIO would automatically wrap the commands with utility methods that were using the fibers package to create a synchronous execution environment. It uses some internal V8 APIs to allow to jump between multiple call stacks from within a single thread. This approach also has been popular among other projects, e.g. Meteor, where most of the code is written using asynchronous APIs which causes developers to constantly start the line of code with await.

Last year the author of the Fibers package announced that he would no longer continue to maintain the project anymore. He built Fibers when JavaScript did not have any proper mechanism to handle asynchronous code other than using callbacks. With JavaScript evolving and adding APIs like Promises or Generators there is technically no reason anymore for Fibers to exist other than preference of code style. Now with the release of Node.js v16 and the update to V8 v9 Fibers stopped working due to a change in V8 that would remove some internal interfaces Fibers was using. Given that a fix for this is non trivial and the maintainer already stepped down from the project it is unlikely that we will see support for Fibers in Node.js v16 and on.

After the WebdriverIO team discovered this we immediately took action and evaluated our options. We opened an RFC to discuss with the community in which direction the project should go. I would like to thank everyone who chimed in and provided their opinion. We experimented with some alternative options, e.g. using Babel to transpile synchronous code into asynchronous but they all failed due to various reasons. The ultimate decision was made to accept the fact that synchronous command execution won't be possible moving on and rather embrace asynchronicity.

With the release of WebdriverIO v7.9 we are happy to announce that we improved our asynchronous API and matched it with the synchronous one. When chaining element command calls users had to write aweful code like this before:

await (await (await $('#foo')).$$('.bar'))[42].click()

now this got simiplified to this:

await $('#foo').$$('.bar')[42].click()

Thanks to the enormous power of the Proxy Object the API is now much more streamlined and less verbose. This will also help users migrating a project that uses the synchronous API to become asynchronous. The team will work on a codemod to help make this process as automated and easy as possible.

At this point the WebdriverIO team wants to thank Marcel Laverdet (@laverdet on GitHub) for building Fibers and maintaining it for so many years. It is time to move on and embrace all the great JavaScript language feature many people have worked hard on. While we have updated the code examples in our docs, @wdio/sync will continue to be supported until we drop support for Node.js v14 and earlier which won't happen before April 2023. You will have enough time to slowly migrate your tests using async/await.

If you have any questions on this or the migration from a framework writing with synchronous commands to asynchronous code, feel free to drop us a line in our discussion forum or on our community Discord support server.

Choosing WebdriverIO​

JW Player is an embeddable, online video player which generates over a billion unique views every day. In order to sustain and grow this scale, the player needs to be able to function on a multitude of different web and mobile platforms. This increases the importance of automated testing to improve confidence in our releases when deploying to so many different targets. After a lengthy project of converting our legacy test framework, which comprised over 6,000 tests, the Test Engineering team at JW Player has been able to deliver more timely releases with fewer regressions. We have experienced no major rollbacks, and increased the confidence we have in the quality of our own product, thanks to WebdriverIO.

Before our migration to WebdriverIO, we had been using an open sourced Ruby framework on top of Cucumber. JW Player is officially supported on seven desktop and mobile web browsers, as well as iOS and Android versions dating back to 10 and 4.4, respectively. For coverage of these platforms, we run approximately 25,000 UI acceptance tests on a nightly basis. The legacy implementation created two problems. First, we encountered performance limitations in Ruby, as a single test run across all platforms could take up to 9 hours. Second, as the Player is implemented in JavaScript, product engineers were less likely to embrace and contribute to the Ruby-based framework. Moving to a JavaScript-native framework addressed both of these problems.

Selenium Webdriver has long been the go-to standard for web automation. Around 2018, our team began to explore several new emerging testing technologies. Cypress had limited browser support, Microsoft Playwright had not yet even been released, and Puppeteer would only execute on Chrome. A Webdriver-based solution, with its broad and dedicated support amongst browser vendors, was the clear winner.

What initially attracted us to WebdriverIO was its straightforward API and complete support for all browsers and devices we needed to test, compared to Cypress and Puppeteer which lack support for one or more of the necessary platforms. More importantly, though, was its rich plugin system and active, engaged developer community. Sponsorship from Sauce Labs, which had already made a name for itself within the testing space, gave us confidence that WebdriverIO would continue to grow and not become abandonware.

Out of the box, WebdriverIO had support for several of our existing and desired toolsets. Tools such as Allure reporting, which we use to quickly comb through product defects, as well as Report Portal, which we use to monitor test health and track trends over time, were easy for us to integrate. The granular pre and post execution hooks gave the test engineers an unprecedented ability to shape how and where tests executed.

Webdriver.IO Practical Approach​

As more features keep being added to WebdriverIO, we are continually able to simplify our codebase by removing open source dependencies and messy mixin code. We have even been able to decommission services that our old framework relied on altogether.

• The Network Primitives feature, released last year, allowed us to remove our dependency on Browsermob Proxy, a proxy tool commonly used in Selenium Webdriver applications to intercept and manipulate HTTP requests. We now call browser.mock(), specify a substring or regex of the request we want to capture and supply a simple mock object to replace the asset. The ability to delay a response allowed us to automate several complicated tests which had required manual execution. We were then able to validate the player’s behavior when particular requests, such as ads, are delayed due to network or other external conditions:

  // mock.js
  export function delayResponse3seconds() {
      return new Promise((resolve) => setTimeout(() => {
          return resolve('{ "foo": bar }');
      }, 3000));
  }
  
  // test.js
  import { delayResponse3seconds } from './mock';
  
  function rewritePattern(pattern, replacement) {
     console.log(`Attempting to rewrite: ${pattern} with: ${replacement}`);
     const toRewrite = browser.mock(`**/${pattern}`);
     toRewrite.respond(delayResponse3seconds);
     console.log(`Successfully rewrote ${pattern} to ${replacement}`);
  }
  

• The Chrome Devtools Protocol support that comes out of the box has also enabled us to automate some tests that were a manual chore. Being able to call browser.throttle(“Good3G”) after our initial page load has allowed us to more accurately verify how the video player behaves under more real world conditions for our mobile users.

• Thanks to WebdriverIO’s CDP mapping, we have been able to create and maintain a suite of performance tests. Calling browser.getMetrics() after a page load and interacting with the player has enabled us to verify that once a player is set up and embedded onto a customer’s website, it will not cause any undue Cumulative Layout Shifts for the end user, which create a disruptive page loading experience.

Summary​

Overall, JW Player’s migration to WebdriverIO was nothing short of a huge success. Between the performance and “quality of life” improvements over our old framework, WebdriverIO’s feature set has allowed us to automate a couple of hundred manual test cases. We've been able to greatly cut down the length of our regression cycles from approximately 1 week to just a couple of days. Most importantly, however, these improvements have allowed us to find a record number of defects, leading to a better quality product and delivering more customer value.