Over the last few years a lot of new testing frameworks have evolved that provide different capabilities but also come with certain limitations. While tools like Selenium are often seen as slow and flaky and others like Cypress.io or Puppeteer are hyped as the future in the testing space. In reality all tools have their own use cases, have different levels of support and are based on a completely different architecture. Comparing them is rather useless as it ends up comparing apples with oranges. In larger test suites flakiness and race conditions can happen using all of these tools so that we can safely say that up to this point none of them (including WebdriverIO) has found the ultimate solution for world /(domin|autom)ation/.
At the end of the day there are more or less two different approaches to how you can automate a browser. One is using an official W3C web standard called WebDriver and the other is the native browser interface that some of the browsers expose these days. The WebDriver protocol is the de-facto standard automation technique that allows you to not only automate all desktop browsers but also run automation on arbitrary user agents that include mobile devices, desktop applications or even Smart TVs. If you don't use WebDriver your automation framework most likely uses the native browser interfaces to run its automation on. While in the past every browser had its own (often not documented) protocol, these days a lot of browsers, including Chrome, Edge and soon even Firefox, come with a somewhat unified interface that is close to what is called the Chrome DevTools Protocol. While WebDriver provides true cross-browser support and allows you to run tests on a large scale in the cloud using vendors like Sauce Labs, native browser interfaces often allow many more automation capabilities like listening and interacting with network or DOM events while often being limited to a single browser only.
With the release of WebdriverIO v5.13 we now introduce a new option that allows you to specify the automation protocol for your test and leverage the capabilities of both worlds. With that you can now decide whether to run your tests using WebDriver or Chrome DevTools (via Puppeteer). Nothing actually changes for your tests, just the automation happens using different technologies. We've created a new NPM package that we call devtools which has all WebDriver commands implemented but executes them using Puppeteer. It is a new type of plugin that even allows you to build your own WebDriver based automation package to automate an arbitrary device with WebdriverIO. The new option, called automationProtocol, expects a string and is by default set to "webdriver" to run automation using WebdriverIOs own webdriver bindings. However if you install the new devtools package via:
$ npm install --save-dev devtools webdriverio
and set automationProtocol: 'devtools' in your options, all the automation happens via Chrome DevTools (more specifically via Puppeteer) while using the same WebdriverIO command interface:
const { remote } = require('webdriverio')
let client;
(async function () {
client = await remote({
automationProtocol: 'devtools', // CHANGE PROTOCOL HERE
capabilities: { browserName: 'chrome' } // or `browserName: 'firefox'`
})
await client.url('https://webdriver.io')
console.log(await client.getTitle())
await client.deleteSession()
})().catch(async (e) => {
console.error(e.stack)
await client.deleteSession()
})
This is especially great for local testing since you are no longer required to download a browser driver which often causes confusion as to where to download and how to run them. In addition to that, tests will run much quicker since WebdriverIO is directly connected to the browser.
As mentioned before, you can now leverage the advantages of running true cross-browser tests at scale with WebDriver, as well as leveraging all the additional automation capabilities of DevTools in one single tool. That said, given the fact that it is not possible to switch the protocols during a session, you might want to have one set of tests that require Puppeteer involvement, while also having a second set that you run cross-browser in the cloud.
In order to access Puppeteer in your tests, we have introduced a command called getPuppeteer(), which returns the browser class of the Puppeteer framework. From there on you can access the Puppeteer interface and all of its other classes. Note that these interfaces provide commands that are solely promise-based and WebdriverIO does not wrap them as you might know it using the WDIO testrunner. In this case we recommend wrapping all Puppeteer command calls manually within the call command that ensures that all promises are resolved before it moves on with other commands. For example, the following script shows how you can use WebdriverIO using the devtools automation protocol via Puppeteer in the WDIO testrunner world:
describe('my e2e tests', () => {
// ...
it('replaces the WebdriverIO logo with the Puppeteer logo', () => {
browser.url('https://webdriver.io')
/**
* run Puppeteer code with promises to intercept network requests
* and replace the WebdriverIO logo in the docs with the Puppeteer logo
*/
browser.call(async () => {
const puppeteerBrowser = browser.getPuppeteer()
const page = (await puppeteerBrowser.pages())[0]
await page.setRequestInterception(true)
page.on('request', interceptedRequest => {
if (interceptedRequest.url().endsWith('webdriverio.png')) {
return interceptedRequest.continue({
url: 'https://user-images.githubusercontent.com/10379601/29446482-04f7036a-841f-11e7-9872-91d1fc2ea683.png'
})
}
interceptedRequest.continue()
})
})
// continue with sync WebdriverIO commands
browser.refresh()
browser.pause(2000)
})
// ...
})
We also made sure that you can use the devtools automation protocol with WDIO testrunner services like @wdio/devtools-service to make the experience as seamless as possible. You can find the complete script example to run in standalone mode in the WebdriverIO example directory.
We recommend only running tests on the devtools protocol if your functional test requires some sort of automation capability that is not provided by WebDriver. It also makes sense to switch to devtools whenever running tests locally as the test execution will be much faster. If you follow our best practices you should split up your wdio config files per environment (e.g. local testing vs. run tests against Sauce Labs or in your grid). You can now have a config that defines a set of tests that require some Puppeteer interaction:
// wdio.devtools.conf.js
const { config } = require('./wdio.conf.js')
exports.config = Object.assign(config, {
/**
* set automation protocol to devtools
*/
automationProtocol: 'devtools',
/**
* define a set of tests that require Puppeteer involvement
*/
specs: [
'tests/e2e/devtools/**'
],
/**
* run tests only on Firefox and Chrome in headless mode
*/
capabilities: [{
browserName: 'chrome',
'goog:chromeOptions': {
headless: true
}
}, {
browserName: 'firefox',
'moz:firefoxOptions': {
headless: true
}
}]
// other specific options ...
})
With the new automationProtocol option, we've opened the project up to automation technologies beyond WebDriver. There are advantages and disadvantages using both approaches and with this feature we allow you to use both in one single tool. The devtools package is still work in progress and we are finalizing all of its features within the upcoming weeks. If you have any feedback, bugs or comments on this please reach out via Twitter or on our community Discord server.
Thanks!
