I have previously written about TDD in JavaScript, most notably using the BDD-style library Jasmine in a series on building a Test-Driven Grocery List Application. In that posts series I went through thinking of User Stories for Features and Scenarios as actual development tasks, and - reading back on it - it's all very green (no pun intended) in my finding a way to deliver test-driven code. Nothing wrong with that and I will most likely look upon this and subsequent posts in the same manner. That said, I still hold true that TDD is the best way to deliver concise, tested and well thought-out code.

Since that time, however, I have incorporated a different tool into my TDD workflow for JavaScript-based projects that affords me the integration of Feature specs more closely to my development and truly encompasses my current ideal of Behaviour Driven Development: CucumberJS. Essentially, it allows me to truly adhere to TDD while developing from the outside in - running automated tests that fail until I have written code that supports a feature.

> assumptions and notes

For the examples in this post, it is assumed that you are familiar with NodeJS, npm, developing node modules and common unit testing practices as these topics too large to discuss in this post.

Supported files related to this and any subsequent posts on this topic will be available at:
https://github.com/bustardcelly/cucumberjs-examples

CucumberJS

CucumberJS is a JavaScript port of the popular BDD tool Cucumber (which itself was a rewrite of RSpec). It allows you to define Feature Specs in a Domain-Specific-Language (DSL) - called Gherkin - and run your specs using a command line tool which will report the passing and/or failing of scenarios and the steps they are comprised of.

It is important to note that Cucumber itself does not provide a default assertion library. It is a testing framework providing a command line tool that consumes defined Features and validates Scenarios by running Steps that are written in JavaScript. It is the developers choice to include the desired assertion library used in order to make those steps pass or fail. It is my intent to clarify the process by example through a single Feature with multiple Scenarios in this post.

Installation

You can install CucumberJS in your project using npm:

$ npm install cucumber --save-dev

Gherkin

If you had followed along in the previous TDD Series, you will find the specs defined in that series similar to Gherkin. In fact, I will be re-hashing a feature spec from that series to demonstrate working through your first cuke (aka, passing feature spec).

If we were to remake the Grocery List application under TDD/BDD using Cucumber, we would first start with a feature using the Gherkin syntax:

/features/add-item.feature

Feature: Shopper can add an item to their Grocery List
  As a grocery shopper
  I want to add an item to my grocery list
  So that I can remember to buy that item at the grocery store

  Scenario: Item added to grocery list
    Given I have an empty grocery list
    When I add an item to the list
    Then The grocery list contains a single item

  Scenario: Item accessible from grocery list
    Given I have an empty grocery list
    When I add an item to the list
    Then I can access that item from the grocery list

The Feature defines a business value, while the Scenarios define the steps that provides that value. Most often, in the software development world, it is from these Scenarios that development tasks are taken on and QA tests are defined.

I stopped at two Scenarios, but we could very easily add more scenarios to this feature; immediately what comes to mind are item insertion rules and validation of properties that allow for an item to be added or rejected. In hindsight, it could make more sense in creating seperate feature specs for such details. We could spend a whole post on such topics, though... let's get back to the feature already defined.

Within each Scenario is a list of sequential Steps: Given, When and Then. It is these steps that CucumberJS will execute after having consume this feature spec. After each of those, you can optionally have And and But, however - though necessary and unavoidable at times - I try to stay away from such additional step clauses.

Running it

Having saved that down to a file in a /features direcory, we can then run it under Cucumber:

$ node_modules/.bin/cucumber-js

By default, CucumberJS will consume all feature specs found in the relative /features directory.

The current console output will look something like the following which essentially means that all the steps have not been located or defined:

UUUUUU

2 scenarios (2 undefined)
6 steps (6 undefined)

You can implement step definitions for undefined steps with these snippets:

this.Given(/^I have an empty grocery list$/, function(callback) {
  // express the regexp above with the code you wish you had
  callback.pending();
});

this.When(/^I add an item to the list$/, function(callback) {
  // express the regexp above with the code you wish you had
  callback.pending();
});

this.Then(/^The grocery list contains a single item$/, function(callback) {
  // express the regexp above with the code you wish you had
  callback.pending();
});

this.Then(/^I can access that item from the grocery list$/, function(callback) {
  // express the regexp above with the code you wish you had
  callback.pending();
});

So we have 6 undefined Steps that make up 2 Scenarios and the CucumberJS ci tool even provides examples of defining them!

An important part of that snippet to understand is that there are only 4 steps to implement. In our Feature we have 2 Scenerios each with 3 Steps. There are a total of 6 steps, but we only need to define 4. The reason being that each Scenario shares the same Given and When step; these only need to be defined once and will be run separately for each Scenario. Essentially, if you define similar Steps using the same context, it will reuse the "setup" for a single Step within each Scenario.

I use "setup" in quotes because I mean it more in a role of defining context for When and Then steps.

I don't want to get it confused with the setup/teardown methods of other unit testing practices - which are known as Before/After support tasks in CucumberJS - and carry more of a context of setting up an environment in which tests are then executed (such as filling a DB of users) and then tearing down that set up.

Step Definitions

In the previous section, we saw that running CucumberJS against our Add Item Feature alerted us that we have undefined (and, though not printed in red, failing) scenarios to support the feature. By default CucumberJS reads in all features from the /features directory relative to where the command was run, but it could not locate the supported step files in which these methods are defined.

As mentioned previously, CucumberJS does not provide an assertion library. The only assumption at this point - since the CucumberJS tool is run under NodeJS - is that the supported steps will be loaded as node modules with an exported function to be executed. As we start implementing the steps, we will need to decide on the assertion library to use in validating our logic. We'll put that decision on the shelf at the moment and get the barebones setup to fail :)

To start, let's take those step definitions provided by the CucumberJS ci tool and drop them into a node module:

_/features/stepdefinitions/add-item.steps.js

'use strict';

module.exports = function() {

  this.Given(/^I have an empty grocery list$/, function(callback) {
    callback.pending();
  });

  this.When(/^I add an item to the list$/, function(callback) {
    callback.pending();
  });

  this.Then(/^The grocery list contains a single item$/, function(callback) {
    callback.pending();
  });

  this.Then(/^I can access that item from the grocery list$/, function(callback) {
    callback.pending();
  });

};

By default, CucumberJS will look for steps to be loaded within a folder titled step_definitions under the /features directory relative to where you issue the command. You can optionally use the -r option to have CucumberJS load steps from another location. Running the default is the same as setting the following step definition directory option:

./node_modules/.bin/cucumber-js -r features/step_definitions

The console output will now look like the following:

P--P--

2 scenarios (2 pending)
6 steps (2 pending, 4 skipped)

Not too suprising seeing as we notify the callback of a pending state. CucumberJS enters the first step (Given) and is immediately returned with a pending notification. As such, it doesn't bother with entering any subsequent steps and marks them as skipped.

Note: It is too much to get into a discussion about client-side modules and AMD vs CommonJS. For the purposes of this example I will be using CommonJS, as I my current interests lie in utilizing Browserify for client-side development. For a long time, I was a proponent of RequireJS and AMD. Again, a whole other discussion :)

Given

_/features/stepdefinitions/add-item.step.js

'use strict';

var GroceryList = require(process.cwd() + '/script/model/grocery-list');

module.exports = function() {

  var myList;

  this.Given(/^I have an empty grocery list$/, function(callback) {
    myList = GroceryList.create();
    callback();
  });
  ...

};

If we were to run that again, we'd get an exception right away:

$ ./node_modules/.bin/cucumber-js

module.js:340
    throw err;
          ^
Error: Cannot find module './script/model/grocery-list'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous> (/Users/toddanderson/Documents/workspace/custardbelly/cucumberjs-example/features/step_definitions/add-item.steps.js:3:19)

Which is understandable, we haven't any other code but this step definition module and are trying to require a module that doesn't exist. In sticking with TDD, this is a good thing - we know why it's failing and we expect it; I would be pulling my hair out if it didn't throw an exception!

In order to get this to pass, we'll create a node module in the specified directory and which exports an object with a create method:

/script/model/grocery-list.js

'use strict';

module.exports = {
  create: function() {
    return Object.create(null);
  }
};

We have provided the minimal requirement to get our Given step to pass. We'll worry about the details as we approach the latter steps.

Run that again, and CucumberJS enters in to the When step of each Scenario and aborts due to pending return.

$ ./node_modules/.bin/cucumber-js

.P-.P-

2 scenarios (2 pending)
6 steps (2 pending, 2 skipped, 2 passed)

When

In reviewing the When step for the Scenarios:

When I add an item to the list

we need to provide a way in which to add an item to the Grocery List instance created in the Given - and do so in as little code to make the step pass...

First, we'll define our expectation of the make up and add signature of the Grocery List in the step definitions:

_/features/stepdefinitions/add-item.step.js

...
module.exports = function() {

  var myList,
      listItem = 'apple';

  this.Given(/^I have an empty grocery list$/, function(callback) {
    myList = GroceryList.create();
    callback();
  });

  this.When(/^I add an item to the list$/, function(callback) {
    myList.add(listItem);
    callback();
  });
  ...

};

If we run that again:

$ ./node_modules/.bin/cucumber-js

.F-.F-

(::) failed steps (::)

TypeError: Object object has no method 'add'

Oooo-weee! Now we're talking. Big, bright red F's. :)

To make that get back to passing, we'll modify grocery-list with as little code as possible:

/script/model/grocery-list.js

'use strict';

var groceryList = {
  add: function(item) {
    //
  }
};

module.exports = {
  create: function() {
    return Object.create(groceryList);
  }
};

Run again, and CucumberJS has progressed to the Then steps which are reporting a pending state.

$ ./node_modules/.bin/cucumber-js

..P..P

2 scenarios (2 pending)
6 steps (2 pending, 4 passed)

Then

Note: This section will be a little example heavy as we quickly switch from modifying our code and run the spec runner frequently.

First Scenario

In reviewing the first Scenario's Then step:

Then The grocery list contains a single item

we will need to prove that the Grocery List instance grows by a factor of 1 for each new item added.

Update the step to define how we expect that specification to be validated:

_/feature/stepdefinitions/add-item.step.js

...
var assert = require('assert');
...
module.exports = function() {
...

  this.Then(/^The grocery list contains a single item$/, function(callback) {
    assert.equal(myList.getAll().length, 1, 'Grocery List should grow by one item.');
    callback();
  });

...
};
...

We've pulled in the assert module and attempt to validate that the length of the Grocery List has grown by a value of 1 after having run the previous step - When - in adding the item.

Run that and we'll get an exception:

$ ./node_modules/.bin/cucumber-js

..F..P

(::) failed steps (::)

TypeError: Object #<Object> has no method 'getAll'

Let's add that method to our Grocery List model:

/script/model/grocery-list.js

'use strict';

var groceryList = {
  add: function(item) {
    //
  },
  getAll: function() {
    //
  }
};

module.exports = {
  create: function() {
    return Object.create(groceryList);
  }
};

And back to running our specs:

$ ./node_modules/.bin/cucumber-js

..F..P

(::) failed steps (::)

TypeError: Cannot read property 'length' of undefined

Seeing as the code is not returning anything from getAll(), we can't access a length property for our assertion test.

If we modify the code to return an Array:

_/feature/stepdefinitions/add-item.step.js

...
getAll: function() {
  return [];
}
...

And run the specs again, we'll get the assertion error message we provided:

$ ./node_modules/.bin/cucumber-js

..F..P

(::) failed steps (::)

AssertionError: Grocery List should grow by one item.

Now, we have a proper Fail being reported to us from an assertion that causes the step to not pass. Hooray!

-- take a breather --

Let's pause here for a second before adding more code to get this step to pass. The issue at hand is not actually adding an item to the array being returned, it is more about ensuring that an item is added through the add method and the result from getAll being a list extended with that item.

Implementation details that are involved in making this test pass is where your team uses their architecture experience, but care is required that only the most essential code is added and not to go overboard in thinking about the internals of the Grocery List collection model. It's a slippery tight-rope that could easily fall down a rabbit hole - just like that poorly-worded metaphor :)

-- get back to work! --

For the purposes of this examples, we'll use the propertiesObject argument of Object.create to define a list getter that will serve as a mutable array for our grocery list items:

/script/model/grocery-list.js

'use strict';

var groceryList = {
  add: function(item) {
    this.list.push(item);
  },
  getAll: function() {
    return this.list;
  }
};

module.exports = {
  create: function() {
    return Object.create(groceryList, {
      'list': {
        value: [],
        writable: false,
        enumerable: true  
      }
    });
  }
};

If we run that, we'll find that the first Scenario is now passing!

$ ./node_modules/.bin/cucumber-js

.....P

2 scenarios (1 pending, 1 passed)
6 steps (1 pending, 5 passed)

Second Scenario

In reviewing the final step of our 2nd Scenario, the pending implementation is accessing the added item:

Then I can access that item from the grocery list

To make this step pass we need to verify that we can access the item appended to the Grocery List by invoking add() with an item.

As with the implementation of accessing the length of the Grocery List, there are several ways in which we could make this test pass in the code. Again, I feel this is where software development experience and taste comes into play with regards to architecture, but I also do prefer trying to produce the least amount of code possible; and I will be the first to admit that sometimes I go a little absent-minded and create more code than is necessary... hence, trying :)

That said, we also have to take into account language and environment specifications in how we address making the assertion pass - and the browser, with its history, has many to consider. That is not a slight, it is just a forethought in setting expectations for requirements.

Specifically: suppose we were to say that the step can be verified using the Array.indexOf() method on the collection returned from 'getAll()' on the Grocery List object? Without a polyfill, then we are limiting ourselves to passing assertions on IE 9 and older. Such considerations are just the tip of the iceberg when deciding about what to introduce into your codebase in order to have your tests pass, and really should be left up to a team discussion on what is considered necessary to get the product to production.

I could go on and on, but let's just assume we want to cover all bases when it comes to browsers (IE 6 and up, shudder). In my opinion, to make this second Scenario turn green, we will add a getItemIndex() method with the following signature:

+ getItemIndex(itemValue):int

We'll first modify the step to fail:

_/feature/stepdefinitions/add-item.step.js

this.Then(/^I can access that item from the grocery list$/, function(callback) {
  assert.notEqual(myList.getItemIndex(listItem), -1, 'Added item should be found at non-negative index.');
  callback();
});

The acceptance in order for this test to pass is that the index at which the added item resides in the collection is non-negative. For this scenariom we are not trying to validate a specification as to where new item is added in a list (eg, prepended or appended), but simply that it is accessible.

Running that will produce an exception:

$ ./node_modules/.bin/cucumber-js

.....F

(::) failed steps (::)

TypeError: Object #<Object> has no method 'getItemIndex'

Let's modify our Grocery List object to support the getItemIndex method:

/script/model/grocery-list.js

'use strict';

var groceryList = {
  add: function(item) {
    this.list.push(item);
  },
  getAll: function() {
    return this.list;
  },
  getItemIndex: function(value) {
    var index = this.list.length;
    while(--index > -1) {
      if(this.list[index] === value) {
        return index;
      }
    }
    return -1;
  }
};

module.exports = {
  create: function() {
    return Object.create(groceryList, {
      'list': {
        value: [],
        writable: false,
        enumerable: true  
      }
    });
  }
};

In our implementation of getItemIndex, the list is traversed and, if item is found, the index is returned. Otherwise, a value of -1 is returned. Essentially, how the Array.indexOf method works of ECMAScript 5.

Note: I know it seems silly to use Object.create from ECMAScript 5, but not Array.indexOf. The reason - mostly - being that I normally always include a polyfill for Object.create and not for Array.indexOf. I suppose habit.

Now if we run the specs again under CucumberJS:

$ ./node_modules/.bin/cucumber-js

......

2 scenarios (2 passed)
6 steps (6 passed)

Our cukes are GREEN! (This is the point you wipe your brow and slow clap).

Conclusion

In this post, I introduce how I use the BDD tool CucumberJS in order to adhere to Test Driven Development in JavaScript. I went through using an example of a single Feature with two Scenarios and turning failing Steps to green cukes. If you are unfamiliar with the process with making tests fail first only to produce code to make the test pass, I hope you followed along; I may be wordy and the process could appear to take a lot of time, but development under such practices does start to move smoothly once you get in the groove. Additionally, I think there is a huge reward in having your code under a test harness when it comes to refactoring and bug fixing - both in developer health and business.

The Future

I was going to cover the following topics in this post but have decided to exclude with the hopes of re-addressing in a later post:

• The World support utility
• Build integration (Grunt and Gulp) and automation
• Report generation for Continuous Integration

Additionally, in a following post I want to address how I use CucumberJS to run tests that rely on browser integration - ie, window and document access.

In the past I have used ZombieJS to much success, but Omar Gonzalez has tipped me to him Karma solution that I am excited to test-drive.