rerun flakey protractor tests

During my work at a large bank in the Netherlands I encountered that our test environment wasn’t that stable as we wanted. Sometimes

  • services had a hick-up
  • app installs didn’t finish in the expected time
  • testdata wasn’t correctly loaded
  • and some other stuff..

To eliminate those dependencies I needed something to retry a testcase / testsuite. Because we worked with Protractor it was very hard to find a good solution, until I found protractor-flake of Nick Tomlin.

protractor-flake is a wrapper around Protractor that will read the exit code of all tests. If some tests fail, protractor-flake will read the log and retry the failed tests-files before they actually fail.

At the time I tried protractor-flake it didn’t match our needs because protractor-flake supported Jasmine / Mocha and we were using CucumberJS in combination with Protractor. But then the advantage of Open Source Software kicks in. I’ve created a feature request and the pull request was merged. Since september 2016 CucumberJS is supported by protractor-flake, hurray!

How to use

protractor-flake is easy to use. First use it as a (dev)-dependency in your project

npm install protractor-flake --save-dev

Then run it with this command from the root of you project

protractor-flake --protractor-path=/path/to/protractor --parser cucumberMulti --node-bin node --max-attempts=3 -- path/to/protractor.conf.js

See the documentation of protractor-flake for more information about how to use CucumberJS in combination with protractor-flake.

Credits go to Nick Tomlin for creating this wonderful module!

Credits:

← Previous post

Next post →

4 Comments

  1. Sridhar

    I intentionally failed one test case by marking ‘expect’ as ‘false’ . This is shown in console as one test case fail out of 15

    Then how would able to run this failed test script

    my project structure is
    — Protractor Example
    —–node-modules
    —–data
    —–pages
    —-Specs
    —-flake
    —-conf.js
    —package.json

    Output:

    C:\Users\XXXXXXXX\Documents\protractor-example-master\protractor-example-master>protractor conf.js
    [07:21:39] I/launcher – Running 1 instances of WebDriver
    [07:21:39] I/direct – Using ChromeDriver directly…
    Spec started

    angular app
    x should add a new friend
    – Expected true to be false.
    √ should delete an existing friend
    √ should not display non-found search terms
    √ should display found search terms
    √ should display no rows when all friends deleted
    √ should display actual count before saving new friend

    non-angular login
    √ should display message for invalid credentials
    √ should display message for empty credentials
    √ should goto friend pages on successful login

    Quality Shepherd blog
    √ should display 5 posts per page
    √ should return search results
    √ unfound search term should return no results
    switching to window 1
    switching to window 0
    A Jasmine spec timed out. Resetting the WebDriver Control Flow.
    x should open social media link in new window
    – Error: Timeout – Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.
    √ sidebar should have a set width
    √ should find an older post by paging

    **************************************************
    * Failures *
    **************************************************

    1) angular app should add a new friend
    – Expected true to be false.

    2) Quality Shepherd blog should open social media link in new window
    – Error: Timeout – Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.

    Executed 15 of 15 specs (2 FAILED) in 1 min 53 secs.
    [07:23:36] I/launcher – 0 instance(s) of WebDriver still running
    [07:23:36] I/launcher – chrome #01 failed 2 test(s)
    [07:23:36] I/launcher – overall: 2 failed spec(s)
    [07:23:36] E/launcher – Process exited with error code 1

    How can I run my failed test scripts . Please help me on this

  2. Sridhar

    I got the solution it works fine .

    My failed test scripts are running 2 times but one question it states no specs defined so rerunnig all specs again .

  3. Sridhar

    All specs are running how can i run only failed specs. Please help me on that

Leave a Reply