In my previous post I addressed the concepts of the World context and Background scenario. I am going to pause in actually working with CucumberJS in delivering code in a BDD manner for this article and address another vital part of the development process: Automation.

> code

Supported files related to this and any subsequent posts on this topic will be available at:
https://github.com/bustardcelly/cucumberjs-examples

Why Automate?

The benefit of automation is in time saved. If we can pinpoint tasks that we do repetitively during development and automate them, we can save large amounts of time that could be put back into doing more coding, thinking, learning, laughing, beering - the list goes on.

While going through the basics of CucumberJS in the previous articles, I was demonstrating how I use the tool to TDD from the outside-in and constantly hopping back and forth from my code editor to the terminal to run this simple command over and over:

$ node node_modules/.bin/cucumber-js

Now, I am pretty good at CMD+TAB - not to brag - but I would much prefer to get feedback on my tests instantly as I describe my asserts and modify my code. If we automate the running of Cucumber on file change, we can start seeing results quicker and we can take that time removed from manually switching between editor and terminal and running commands and put that time back into thinking how to design our code more cleanly :)

This automation and instant feedback really shows its worth in the Refactor phase in which you have previously implemented code to pass the criteria in a Feature.

Task Automation and JavaScript

When talking about build tools and task automation for JavaScript projects, there are roughly 3 types of task runners that are brought up:

1. Grunt
2. Gulp
3. npm run

There are additional tools that have been around for some time - particularly, in the past I have used Ant and Make for my build process - but, generally speaking, these are the prevelant tools of the trade in JavaScript as it stands today.

Some people think that each oppose each other, but I lean toward utilizing one or the others based on its merits and the requirements of a project. I have several on-going projects, both professional and personal, and each project actually incorporates one of each of these. I will discuss my thoughts on the value of each tool within each section in which I set up automation for our Cucumber tests.

Npm run

I'll start off with the basics. By basics, I mean that there is no additonal tool required to install and get up and running with our test automation; we have everything already installed that we need: node & npm.

Benifits

James Halliday had previously written a great post addressing the use of npm run in task automation, and I agree in so far as keeping things simple. The benefits of defining tasks in the scripts tag of your package.json file are a) simplicity and b) no additional tooling. I do have the reservations that the "simplicity" vanishes if you have to eventually maintain dozens of tasks - some that may be similar with different arguments - and continually want to run series and parallels.

When your build process becomes more complex, James Halliday does mention moving it to a bash file, but I would argue to perhaps look at a more robust solution with the tools described previously (and in more detail later in this article).

Usage

If we were to simply take the process of running our tests as we have in previous articles, we can modify our package.json file by adding the following scripts tag and task:

package.json

{
  "name": "cucumberjs-examples",
  ...
  "scripts": {
      "test": "node node_modules/.bin/cucumber-js"
  }
  ...
}

Plain and simple. To run it, we'd hop over to our terminal:

$ npm run test

> cucumberjs-examples@0.1.0 test /Users/toddanderson/Documents/workspace/custardbelly/cucumberjs-example
> node node_modules/.bin/cucumber-js
........

2 scenarios (2 passed)
8 steps (8 passed)

Same result. We get to type less now, so that's good. However, we are not really automating the process for running our tests. As mentioned previously, we are looking for the benefit of automating the running of our tests on change to Step Definitions and code.

As such, we create a simple node script that will watch our script and feature/step_definitions and run the CucumberJS CLI tool on each change:

cuke-watcher.js

#!/usr/bin/env node
var watch = require('node-watch');
var child_process = require('child_process');
var running = false;
var cucumber;

var JS_EXT = /^.*\.js/i;
var options = ['node_modules/.bin/cucumber-js', 
               'features', 
               '-r', 'features/step_definitions',
               '-f', 'pretty'];

watch(['./features/step_definitions', 'script'], {recursive:true}, function(filename) {

  if(!running && filename.match(JS_EXT)) {

    running = true;

    cucumber = child_process.spawn('node', options)
                    .on('exit', function() {
                      running = false;
                    });

    cucumber.stdout.on('data', function(d) {
      console.log(String(d));
    });

    cucumber.stderr.on('data', function(d) {
      console.error(String(d));
    });

  }

});

We are using the wonderful node-watch module that is a convenient wrapper to fs.watch, and upon change, we spawn the Cucumber tool as a child process. Running Cucumber as a child process allows it to exit its own process without having to exit the watch process, which would kill all automation.

Modify our package.json to invoke this script:

package.json

{
  "name": "cucumberjs-examples",
  ...
  "scripts": {
      "test": "node node_modules/.bin/cucumber-js",
      "watch": "node cuke-watcher.js"
  }
  ...
}

And run it in the terminal:

$ npm run watch

And you will see the cursor flash with the watch process running. If you were to modify any of the Step Definition or source files of our current Grocery List example project, you would see CucumberJS run and produce the same results as before - the only difference is that it is now waiting for you to modify them again so it can run again.

It will continue to watch and run tests until you exit the process, most commonly done by focusing on the terminal and hitting CTRL+C.

Grunt

Up until a few years ago, I stuck with what I knew best and preferred to maintain builds for my JavaScript projects using Make. After having the fortunate opportunity to hear Ben Alman speak and demonstrate Grunt at TXJS in 2012, I decided to give Grunt a real try - and I have, for the most part, not looked back.

Grunt is my go-to task automation tool for large projects that involve various complex tasks for developing, testing and deployment. For the most part, if my tasks are not solely based on files, I will incorporate Grunt into my project.

Along with a solid history and great documentation, there is also a very active community that creates task plugins for Grunt: http://gruntjs.com/plugins. You can find virtually anything you need, and if for some odd reason you can't, you can create one: Creating Grunt Plugins Docs.

It is too much for this article to get into a discussion about Grunt and its concepts, so please read the documentation on their site for clarity. In this section I intend to address how to use Grunt to automate the tests we have been developing in this series.

Usage

Just as we setup a watcher for our Step Definitions and scripts for our npm run example, we will be using 2 Grunt plugin tasks provided by the wonderful development community:

• grunt-contrib-watch - maintained by the contributors for gruntjs
• grunt-cucumber - maintained by venerable developer Omar Gonzalez

I will assume you have Grunt installed properly and start with the Gruntfile:

Gruntfile.js

module.exports = function(grunt) {
  'use strict';

  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    watch: {
      cucumber: {
        files: ['features/**/*.js', 'script/**/*.js'],
        tasks: ['cucumberjs']
      }
    },
    cucumberjs: {
      src: 'features',
      options: {
        steps: 'features/step_definitions',
        format: 'pretty'
      }
    }
  });

  grunt.loadNpmTasks('grunt-contrib-watch');
  grunt.loadNpmTasks('grunt-cucumber');

  grunt.registerTask('watch-tests', 'Starts a watch for test automation.', ['watch:cucumber']);

};

Within the Gruntfile we have configured cucumberjs task with the same arguments we have been using in previous examples and configured a watch task to listen for changes to JavaScript files in the features and script directories.

Additionally, we have defined a watch-tests task which we can invoke using Grunt from the command line:

$ grunt watch-tests

Running that will do, essentially, what we have done using npm run in the previous section: the watch process will be active and execute the Cucumber specs upon a change to JavaScript files in the target directories. To stop the task from running, focus on the terminal and CTRL+C to exit the process.

Gulp

Gulp is a (relatively) new-comer to the JavaScript-based task runner ecosystem. Like Grunt, it is a task-based build tool that has good documentation and a lively community contributing plugins. All good things.

Gulp's build system uses streams which allows you to pipe multiple tasks together. As well, the tasks defined in the gulpfile are the code itself - as opposed to Grunt in which you provide a configuration for your tasks that are consumed by Grunt and provided to the targetted plugin through a task registry.

There are plenty of other articles contrasting the two, so I won't rehash them here. I will say that I choose Gulp over Grunt in projects where the build requirements are strictly focused on files - ie, take these files, do something to them (min, concat) and put them in this directory.

As a side note: I highly recommend James Halliday's excellent stream-handbook to get a better understanding of streams in node.

Usage

The plugin community for Gulp is fairly active, but as I mentioned previously Gulp's system is really based on streams. As such, I don't intend to see a plugin for running CucumberJS tests specifically, as we have with grunt-cucumber - nor do intend to make one or continue to look for one: I don't think such a plugin is well suited to Gulp.

That's said, we can certainly set up a watch task on our Step Definitions and scripts just as we have in the previous examples!

gulpfile.js

var gulp = require('gulp');
var watch = require('gulp-watch');
var child_process = require('child_process');

var cucumber;
var running = false;
var options = ['node_modules/.bin/cucumber-js',
               'features', 
               '-r', 'features/step_definitions',
               '-f', 'pretty'];

gulp.task('cucumber', function() {
  if(!running) {
    running = true;
    cucumber = child_process.spawn('node', options)
                    .on('exit', function() {
                      running = false;
                    });
    cucumber.stdout.on('data', function(d) {
      console.log(String(d));
    });

    cucumber.stderr.on('data', function(d) {
      console.error(String(d));
    });
  }
});

gulp.task('watch-tests', function() {
  gulp.src(['features/**/*.js', 'script/**/*.js'])
      .pipe(watch(function() {
        gulp.run('cucumber');
      }));
});

Running that is very similar to how we ran our Grunt task:

$ gulp watch-tests

As you may notice, this example fairly similar to the npm run example shown previously. The main difference is that we pipe the streams through the gulp-watch plugin which triggers CucumberJS as a child process upon change to targetted files.

Conclusion

So, which one should you use? Whichever! They each have their merits and it should be discussed with your team in alignment with the requirements for the current project.

It was my intent to showcase how to incoporate automated testing with CucumberJS using the three most popular task runner / build tools so we can devote more time to thinking about design instead of hopping from our code editor to the terminal for each change to our Step Definitions.

I didn't introduce any new concepts related to CucumberJS, itself, but felt it necessary to address automation at this stage within the series as I feel it is a vital part to Test-Driven Development and the Agile process and something that should be addressed in the early stages of a project.

That said, I look forward to the next article where we get to write more tests and which I intend to introduce running tests in a browser environment :)

Source for examples related to this post can be found in the 0.3.0.post tag on my Github account.