When you are automating with Protractor on different browsers you can use a cloudservice like for example Sauce Labs. I also use Sauce Labs for my own image comparison module and it gave me a lot of insights about how it works. Especially the browser matrix, which can show the teststatus browser/platform/operating system combination after each test, is a very useful feature for users.

Browser matrix

With the browser matrix you can keep track of the test status of your project for various browser/platform/operating system combinations, see an example below.

It’s a simple badge that can be placed in your README.MD file of your project. The way how it can be implementation can be found here. The most importand part of the browser matrix is that you need to update each browser/platform/operating system combination after each test.

When you use Protractor in combination with Sauce Labs you will get a lot of features, but it will NOT update the running instance in Sauce Labs out of the box (at least not what I could find). After some experimenting I found a solution how to do this.

How to update the browser matrix

To be able to update the browser matrix you need to tell the Sauce Labs instance after each test what the status is. This can be done with the following:

• 1 extra npm dev-dependency in your (test)project that is called saucelabs. The saucelabs module is a wrapper around the Sauce Labs REST API for Node.js. See here how to implement the module.
• use the onComplete and onCleanUp from Protractor with some aditional scriptin

After the module is installed the following Protractor configuration script needs to be used.

const SauceLabs = require('saucelabs');

const SAUCE_USERNAME = process.env.SAUCE_USERNAME;
const SAUCE_ACCESS_KEY = process.env.SAUCE_ACCESS_KEY;

// Create an empty variable that will be used to store the session id
let JOB_ID;

exports.config = {
    multiCapabilities: [
        {
            appiumVersion: "1.6.3",
            browserName: 'Safari',
            deviceName: "iPhone 6 Simulator",
            deviceOrientation: "portrait",
            platformName: 'iOS',
            platformVersion: '10.0',
            username: SAUCE_USERNAME,
            accessKey: SAUCE_ACCESS_KEY
        },
        {
            browserName: 'chrome',
            platform: "Windows 10",
            version: "latest",
            screenResolution: "1400x1050",
            username: SAUCE_USERNAME,
            accessKey: SAUCE_ACCESS_KEY
        }
    ],

    framework: 'jasmine',
    specs: ['example_spec.js'],
    jasmineNodeOpts: {
        defaultTimeoutInterval: 30000
    },

    // Run for each capability
    onComplete: function () {
		// Get the session id and store it in the JOB_ID
        return browser.getSession()
            .then((session) => JOB_ID = session.getId());
    },
    // Run for each capability
    onCleanUp: function (exitCode) {
        const saucelabs = new SauceLabs({
            username: SAUCE_USERNAME,
            password: SAUCE_ACCESS_KEY
        });

        return new Promise((resolve, reject) => {
			// Connect to Sauce Labs and update the 
			// job with the correct exit code status
            saucelabs.updateJob(JOB_ID, {passed: exitCode === 0},
                () => resolve(),
                (error) => reject('Error:', error));
        });
    }
};

Once you’ve update the browser matrix  with the above script, the Sauce Labs Dashboard is automatically updated with the correct status.

JOB_ID, onComplete and onCleanUp

A specific identifier is needed to update Sauce Labs, in the script above it is called the JOB_ID. The JOB_ID is attached to Protractors browser.session()-method. This needs to be done for each running capability (in the above example an iPhone and a Chrome browser).

onComplete: function () {
	// Get the session id and store it in the JOB_ID
    return browser.getSession()
        .then((session) => JOB_ID = session.getId());
}

The onComplete of Protractor is run for each capability that is used. It can be used to get the JOB_ID of the running capability that is used to update Sauce Labs and will look something like this 7dd209268fe84708aadc4b85bd97d401.

To update the session in Sauce Labs, the exit code of each running instance needs to be retrieved. This can be done in the onCleanUp. The onCleanUp is run for each capability.

onCleanUp: function (exitCode) {
    const saucelabs = new SauceLabs({
        username: SAUCE_USERNAME,
        password: SAUCE_ACCESS_KEY
    });

    return new Promise((resolve, reject) => {
		// Connect to Sauce Labs and update the 
		// job with the correct exit code status
        saucelabs.updateJob(JOB_ID, {passed: exitCode === 0},
            () => resolve(),
            (error) => reject('Error:', error));
    });
}

After retrieving the exit code the instance in Sauce Labs needs to be updated. This is done with the npm-module saucelabs, that is required in the first line of the script, and with the code below updated

	saucelabs.updateJob('7dd209268fe84708aadc4b85bd97d401', {
		// Exit code translated to tre / false
		passed: true
	});

Conclusion

So with the code above Sauce Labs can be updated for each browser/platform/operating system combination that has been run from the (multi)capabilities. This will make the browser matrix and the dashboard from Sauce Labs show the correct status for your project.

I hope you like it, if you have any questions feel free to ask. You know where to find me ;-).