If you have been following along on this blog, you may be aware of the massroute_js project i have up on github. I am testing out various JavaScript toolkits, libraries and frameworks and have pushed a few up on the repo. As I was finishing up playing around with the last framework, it dawned on me that I didn’t have an example with NO framework; NO not being the latest the JS framework. NO being no… but with more face-palm. Perhaps it is telling of the state of JavaScript development these days that I am missing a plain vanilla JS example of my MassRoute application, and it is all true. I still don’t have one committed to the repo. However, the reasoning for that is a lengthy one – I got obsessed with a proper development environment and custom build system By obsessed, I mean I researched a fair bit of tools and libraries involved in the following areas that I thought fit into my workflow:

• AMD
• Code Quality
• Unit Testing
• Minification / Concatenation
• Build / Deploy

I intend to address each of these topics in this article and the tools/libraries I found and those I had chosen for my workflow. It should be noted that I was not considered with development and deployment of the other two piece of the webstack – HTML and CSS. It is vital to have a good workflow when working with those technologies as well, and especially if all three are part of your job. There are some great tools and libraries out there for developing and shipping HTML and CSS and perhaps I will dive into that at a later date.

For now, I have created a common library in the massroute_js repo to develop what was turning out to be common pieces of the application I was developing against various JavaScript libraries out there. In there are the tools and libraries I will discuss in this post along with a simple build script: https://github.com/bustardcelly/massroute-js/tree/master/massroute-examples/common

I will do my best to explain the history and usage behind each tool and library, but will try to not get too in depth as I may get lost in the actual meaning of this post – which is to highlight the tools and the workflow in which they can be used together.

AMD

The Asynchronous Module Definition – commonly referred to as AMD – is a specification that defines how modules and their dependencies can be defined and loaded asynchronously. The asynchronous part of AMD fits nicely in a browser environment so as to not block rendering and script execution while loading modules, but the bigger take away for my development purposes is really the modularization of code and dependency management.

There is a larger history behind AMD and its fruition from CommonJS which I will not go into, not only as I do not have enough personal involvement to speak intelligently about such matters, but that James Burke and Addy Osmani already have some extremely insightful articles already out there on the webs:

James Burke: Simplicity and JavaScript modules
Addy Osmani: Writing Modular JavaScript with AMD, CommonJS and ES Harmony

I started dabbling and really getting interested in the idea of modular JavaScript when I began using the Dojo toolkit. That was back at 1.6.1 release and the use of dojo.provide, dojo.require and dojo.declare. Now at 1.7, Dojo is fully compliant with AMD. Here is an article discussing Dojo’s move to AMD compatibility: http://dojotoolkit.org/features/1.6/async-modules. Just as an aside – you really should check out Dojo, particularly if you come from a Flex background and have happened upon this post.

So, with some familiarity and a strong interest to incorporate modular development, I set out to find a library that would fit nicely in my workflow. There are a handful of AMD-compatible libraries, most notably curl from John Hann, Backdraft, and RequireJS from James Burke. I settled on the last RequireJS as, not only because of its ease of use and the cleanest, most concise documentation, but because of its author and his history and active role in the community; not to mention that the related optimization tool – r.js – was also a good selling point.

RequireJS

The RequireJS API defines how to define a module as well as how to load modules with dependencies. Along with other niceties, it also provides the ability to define dependency load order and loading of text files (including CSS).
Just taking some stripped down and generalized examples from the common lib of my massroute_js repo, a module is defined as such:

/script/com/custardbelly/js/RequestToken.js

define( function() {

    var RequestToken = (function() {
        this.then = function( handler ) {
            ...
        }
    });
    return RequestToken;

})l

> and a module with dependencies are defined as so:

/script/com/custardbelly/js/Request.js

define( ["com/custardbelly/js/RequestToken"], function( RequestToken ) {

    var Request = (function( url ) {
        this.send = function( variables ) {
            var token = new RequestToken();
            ...
            return token;
        }
    });
    return Request;

});

So with these two examples we see how to define modules with and without dependencies, hopefully demonstrating the benefit of modularization but also the beauty of composition through dependency that AMD libraries like RequireJS provide.

Another great benefit in using RequireJS is that it gets rid of you having to manage your code in namespaces. In other words, this is no longer necessary:

(function( window ) {
    var massroute = getNamspace( 'com.custardbelly.massroute' );

    function getNamespace( value ) {
        var parts = value.split( '.' ),
            i = 0,
            length = parts.length,
            package, parent = window;

        for( i; i < length; i++ ) {
            package = parts[i];
            parent[package] = parent[package] || {};
            parent = package;
        }
     };

    var Something = function() {
        ....
    };

    massroute.Something = Something;

})( this );

What is happening under the hood, loading and reference-wise, is that RequireJS (now require on the window Object) holds a list of file references in its own contexts.urlMap property – the key being the normalized string of the module based on configuration and the value being the actual url for that module file. As dependency requests are made, a lookup is made on require’s contexts.loaded property which maps the normalized module string to a flag of already loaded and available. A script tag is actually written and attached to the DOM to begin loading of the script just as most script loaders do. What sets it apart is the use of async and datasets. If we take a look at what is appended for the Request module from a previous example:

<script type="text/javascript" charset="utf-8" async="" data-requirecontext="_" data-requiremodule="com/custardbelly/js/Request" src="./../script/com/custardbelly/js/Request.js"></script>

We can see that the values on the datasets directly correspond to those of the key/value pairs in require.context.urlMap for your application. So as these are loaded, the flag in require.context.loaded is flipped. Pretty elegant, and if you are interested in more about RequireJS’s design and the requirements it adheres to, this is a great article: http://requirejs.org/docs/requirements.html. Now… back to looking at code.

When it came time to employ these modules, I would require() where my application is responsible for making a request. Let’s just take on start-up in a main file as an example:

/app/main.js

(function( require ) {
    require.config({
        baseUrl: ".",
        paths: {
            "com": "./script/com"
        }
    });

    require( ['com/custardbelly/js/Request'], function( Request ) {

        var request = new Request( 'http://somewhere.fun/go'),
            token = request.send({person:'Todd'});

        token.then( relax );

        function relax() {
            console.log( 'ahhh' );
        }
    });
})( requirejs );

It should be noted that anything (ie. objects, functions, native objects) can be returned from a module definition. Typically, though, and which is seen from these examples, i tend to return constructors probably due to my class-based language background; in other words, x requires y as x is going to create at least one new instance of it. But you could very well return an object or a function, and the real power comes in when you consider composition and module dependencies… and for all you DI fans out there like me, the cogs might start spinning up in that noggin. I have yet to do a real test-drive of IoC containers for JavaScript but there must be or could be something that is a perfect match for RequireJS. I am aware of wire, but as I said, have yet to give it a go – Santa failed on delivering more hours in the day If you know of any, please leave a comment.

So that is where my AMD loader choice stands. I have been using it for some time and have been pretty happy. Keep in mind that all these modules are separated to their own files. Depending on the size of the project, that can tally up to a lot of requests. And if we compound that with a slow network and a limited caching capabilities, we’re talking about trade-offs in even using AMD… unless we can optimize our development workspace down to a reasonable production environment that will go live. We’ll address those concerns a little later in the article…

AMD reading link dump:

AMD specification on GitHub
(UMD) Universal Module Definition
John Hann: AMD vs CommonJS
DailyJS: The How and Why of AMD
Addy Osmani: Writing Modular JavaScript with AMD, CommonJS and ES Harmony
James Burke: Simplicity and JavaScript modules
Tom Dale: AMD is Not the Answer
ES Harmony modules proposal
RequireJS

Code Quality

What makes JavaScript so fun is you can get away with a lot of shi… pped code that is littered with syntactical errors, misspellings, declared and unused variables and not to mention code that is improperly formatted. Such things can cause your application to fail silently and unsuspectingly to a user – no flag is explicitly raised that the code is failing unless an end-user cares about opening the debugger tools of a browser and submitting tickets for you. Such things you can’t mitigate and handle properly with more code because it is the code itself… well, i guess you could wrap everything in a little try… catch’s, but that would be silly.

If you are coming to JavaScript from a language that gets compiled, finding such mistakes during development might be a bit of a challenge in as far as the IDE department goes. There are a handful of excellent IDEs out there targeting web development (I enjoy Sublime Text 2), but the nature of an interpreted language leaves the ability to analyze and catch possible syntax and runtime errors before deploying code a challenge; unlike SDKs and IDEs that employ compiler tools that can determine errors in live edit or through a pre-compilation build.

That’s not to say, by any means, that a program has less bugs in a compiled environment than in an interpreted one. It just means you have to be a little more diligent in analyzing and testing your code – more on testing later. This type of analysis without actually executing or running the code for a program is commonly referred to as linting. When it comes to JavaScript, you can’t go far into searching for a linter without finding JSLint, nor JSHint and their connection.

I won’t go over the benefits of one or the other, nor their history as there are plenty of articles out there on such. For the benefit of my own development environment and workflow, I have chosen JSHint for linting code – mainly because of the ease of set-up especially when it comes to disregarding the formatting of my code. I am much more concerned with syntactical errors and do not consider such things as indentation to be detrimental to the performance of the program, especially as it will later be run through minification. That is in no way stating that proper formatting is un-important, especially, especially, especially when working with a team. I do strive to keep a consistent format to my code, and certain IDEs help in some respect with coding standards and conventions, but at this time JSLint is a little too strict and feels a little more like prodding the code into a certain style – like a master’s apprentice. Not necessarily a bad thing and I may change my mind at some point, but for now JSHint it is. Plus, there is a sweet JSHint package for Sublime Text 2 that I can hook into a hotkey to check for errors on the current file I have open.

Code Quality reading link dump:

JSLint
JSHint
Anton Kovalyov: Why I Forked JSLint to JSHint
JSHint Sublime Plugin: uipoet/sublime-jshint

Unit Testing

This topic is too large to really go into discussing methodologies, extolling the virtues of and defining best practices for in this article. As such, I will simply state that I am guilty of not doing enough testing during the design of applications. For a time, when I was getting more familiar with languages and programming in general, creating unit tests for my code seemed more as a distraction from getting down to brass tacks and delivering a product. I’ll admit that, and the hugely naive stance that it takes That is in no way to say that now I practice good Test Driven Development (TDD). Far from it; really far from it. But I am trying to get better. More importantly, my thinking has changed in that I now believe unit testing is actually a proper way of testing the design of my application rather than finding errors in the code. I think that was a big leap for me. And, for whatever it may mean, coming to an interpreted language like JavaScript, designing and developing upon unit tests becomes even more important to me. Test-Driven JavaScript Development by Christian Johansen has definitely gone a long way in swaying my development practices for JavaScript, and I highly recommend picking up this book.

Like I said, it is too much to get into a discussion of TDD and unit testing, I wanted more to highlight the choices I have made for my development and deployment workflow in unit testing my code. I looked at and tried out a couple different unit testing frameworks for JavaScript – with Jasmine, Hiro and QUnit catching my eye.

QUnit

I ended up settling with QUnit for the following reasons:

1. Familiarity and ease of use.
2. Easy integration with RequireJS.
3. Easy and already available source/docs for integration with headless test runners (JSTestDriver, PhantomJS – more on this later).

All of those are important. That list is really more representative of the order the contact states of a 3-way bulb where it eventually all came together. It was imperative that I could keep developing without choosing a Unit Testing framework that influenced the AMD library I chose and (as I will go into in a bit) it was necessary for my deployment needs to run the tests in a headless environment; during development, i want to write my tests (hopefully using “TDD-like-you-mean-it“) and run them quickly and visually, but I also was looking forward to take that work in unit testing unmodified and provide it to a headless test runner when it came to run a full deployment. So QUnit fit in nicely… for now at least. The familiarity and ease of use is a nice initial draw, but the other two are really the weighing factors and I would not mind giving Jasmine more of a go at a later date if i can ensure those two points.

In any event, using RequireJS and QUnit together is rather straight-forward, especially after finding this nice gist from drewwells: https://gist.github.com/920405. The main rule to remember is to set autostart to false on QUnit, and only invoke :start() once you have required all tests:

/test/index.html

<script src="../lib/require-1.0.4.js"></script>
<script src="lib/qunit.js"></script>
<script>
    QUnit.config.autostart = false;

    require.config({
        baseUrl: ".",
        paths: {
            "com": "../script/com"
        }
    });

    requirejs(['script/RequestTest.js', 'script/RequestTokenTest.js', 'script/RouteStopTest.js'], function() {
        QUnit.start(); // Tests loaded, run tests
    });
</script>

In basic terms, you have replaced adding <script> tags for each test – as you normally would when working with QUnit – with loading them as modules through RequireJS. The tests utilize RequireJS as well, but do not define a module; rather require necessary dependencies through RequireJS and define tests that QUnit will find:

/test/script/RequestTest.js

require( ['com/custardbelly/js/RequestToken', 'com/custardbelly/js/Request'], function( RequestToken, Request ) {

    module( "Request Test" );

    test( 'send returns RequestToken', function() {
        var token,
            xhr = new Request( 'http://webservices.nextbus.com/service/publicXMLFeed?command=routeList&a=mbta' );

        token = xhr.send();
        equals( ( token !== 'undefined' ), true, 'Request.send() returns RequestToken' );
    });
    ...
});

The source library dependencies are defined in the first argument for require(), while the second argument is a handler with there reference supplied as arguments. As for the module() call, there’s nothing there now accept the name that will be associated and printed along with the tests but the QUnit modules can also be used to define your setUp and tearDown fixtures. Check out the docs for more information.

PhantomJS

As I mentioned in the previous part of this section, it is a requirement for my deployment process to be able to run my unit tests in a headless environment. As well, it is preferable to not modify the set-up for unit testing in the development environment to run the tests later in deployment. I originally checked out JsTestDriver for my headless environment as it is undoubtably the most widely known – and for good reason. It even has an adapter for QUnit, and I spent a great deal of time to get it working with my QUnit + RequireJS setup. To no available, however, as this discussion reveals in which script loading is not really supported by JsTestDriver, which is required (no pun) as we saw earlier in this article that RequireJS writes async <script> tags to the DOM.

That discovery led me to PhantomJS which I am pretty happy out. As the site states:

“PhantomJS is a headless WebKit with JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.
PhantomJS is created by Ariya Hidayat.”

What that basically means is that it is nothing short of awesome, and by executing the QUnit adapter script provided at http://code.google.com/p/phantomjs/wiki/TestFrameworkIntegration I could point to my local main file that I use for running unit tests during development and run PhantomJS during deployment to stop and report back any failures encountered. Here is an example of that command:

./bin/phantomjs run-qunit.js file://localhost/Users/todd/massroute_js/massroute-examples/common/test/index.html

I actually played around with PhantomJS farther beyond just trying to integrate with my RequireJS and QUnit set up for deployment, and the API it provides is exceptional and proves its use in a variety of automated tasks: http://code.google.com/p/phantomjs/wiki/Interface.

* If you are interested in how you can hook up additional logging/output for PhantomJS+QUnit, @gavincoop tweeted this gist to me that incorporates output of JUnit XML, which I hadn’t incorporated into my workflow: https://gist.github.com/1588423

Unit Testing link dump:

Test Driven Development
Jasmine
Hiro
QUnit
JsTestDriver
PhantomJS
Test-Driven JavaScript Development by Christian Johansen.

Minification & Concatenation (min-concat)

As I was researching and introducing unit testing for my development workflow, the choice of unit testing framework began to hinge on its integration in a headless testing environment that could be used in a deployment scheme. When it comes to deployment and production-level sources, it is recommended to minify and concatenate scripts. Concatenation has the benefit of minimizing HTTP requests (with a trade off of size of course), and should be done with respect to dependencies; meaning, I would concatenate Service and Token modules together, but not with interactivity utility scripts – the user may not require one or the other during their session, so there is no need to bundle them together and bloat file size and request response time.

Typically I would save the min-concat for deployment. I would develop and test (both with browser developer tools and unit test) with fully spaced-up, line-break-containing, long-variable-name-using scripts and only min-concat when it came time to push to staging and production. Some say that this may present false positives as I would not be testing with what is to be live code. I would tend to agree and I am looking for a reliable way to fit this into the dev/deploy workflow, but for now it is much easier to test code with the full source using the browser developer tools (though i have heard that there may be new advancements in file swapping a’la Charles or Service Capture for dev tools).

When it comes to minification and concatenation of JavaScript files, there are a handful. When researching the right one to fit into my current development and deployment workflow, I also discovered and kick the tires on some fuller build systems that incorporated min-concat to see if they were something I could use. I highlight a few in the links section of this part as I think they may be beneficial to others, but I stuck with rolling my one build/deployment script mainly due to the dependency (happily and no pun intended ) of RequireJS in my development.

Minification

Here is a quick rundown of the most popular minifier/optimization scripts out there that I see:

• YUI Compressor
• Google Closure Compiler
• UglifyJS

In one way or another, most open-source build systems I have come across use one or a coupling of these. Each have their own strengths and compiler options, but I can not speak at length on the benefits outweighing one or the other. The reason that I didn’t delve into which incorporated better into my workflow was due to my want of modular development and RequireJS. So for obvious reasons that lead me to the supported optimizer tool from RequireJS – r.js.

r.js is an optimizer for JavaScript source files that use the AMD API. It can be configured to use either UglifyJS(default) or the Closure Compiler and can be run under node.js and Rhino. It is a minifier and concatenator rolled into one and does so with respect to AMD loaders.

When using r.js, you provide a build file that defines your desired configuration in optimization. There is a good base example up in github: https://github.com/jrburke/r.js/blob/master/build/example.build.js, and just as a quick example of what I have for a build file for the custom script in massroute-js/common:

({
    baseUrl: '../../script',
    name: 'com/custardbelly/js/Request',
    include: ['com/custardbelly/js/RequestToken'],
    out: '../../dist/script/custardbelly.min.js',
    optimize: "closure",
    wrap: true
})

That is the defining the base URL for modules to be used by the AMD loader (baseUrl), the main module (name) and its dependencies (include[]), the output filename (out), preferred optimizer (optimize), and whether or not to wrap the optimized file in (function() { … }()); to encapsulate the code.

Now, for my “commons” library that is being developed for the massroute-js examples, I have source split up into two packages – one that may be considered common to my domain, and one that is considered common to the application. Unfortunately, a single build file for r.js doesn’t seem to be compatible to define multiple output modules with their own separate dependencies. In remembering the configuration file from previous example, I had thought this would be achievable by placing the name, include and out properties into each element of the modules list available from the r.js build API, eg.:

({
    baseUrl: '../../script',
    dir: '../../dist/script',
    modules: [
        {
            name: 'com/custardbelly/js/Request',
            include: ['com/custardbelly/js/RequestToken'],
            out: 'custardbelly.min.js'
        },
        {
            name: 'com/custardbelly/massroute/model/InflatableModel',
            include: ['com/custardbelly/massroute/model/Route', 'com/custardbelly/massroute/model/RouteStop'],
            out: 'massroute.min.js'
        }
    ],
    optimize: "closure",
    wrap: true
})

In this build file example, I attempted to output two optimized modules – one each for the custardbelly and massroute namespace, as I figured with the previous example representing a single module output with its configuration, defining them in the list in modules would perform multiple module outputs; which is not the case. That is when i stumbled upon this ticket in requirejs’ github and gave the solution of using multiple “single file” builds, which is what I wound up using in my deployment.

It should be noted that James Burke has also released almond.js, which is a light-weight AMD loader. With this, it is possible to run r.js and compile in almond.js to remove the dependency of loading the require.js library at runtime. I attempted to fold this solution into my deployment build, but ended up not using it mainly because of how Rhino looks up resources based on the name property in the r.js configuration file; the only way to reliably fold almond.js into my optimized library was to include in the same directory as my custom scripts. I generally like to separate my scripts out into folders that rightly define their role and particularly do not intermix third-party libraries with my own scripts. But whatever. I am being a stickler and if it comes a benefit, I will probably incorporate a script that will temporarily copy the /external/lib/almond.js file into my /scripts directory only to remove if after running optimization with r.js.

Min/Concat link dump:

Yahoo!: Best Practices For Speeding Up Your Website
YUI Compressor
Google Closure Compiler
UglifyJS
r.js
almond.js

Other min-concat build systems

juicer: A command line tool for JavaScript and CSS developers.(Incredible work and will probably use again at a later date)
smoosh: something like a himalountain. (I personally liked using this)
grunt: A command line build tool for JavaScript projects.
jaguarandi: Ant build script to optimize and version Javascript, CSS, and HTML files
modulr-node: Resolves and concatenates CommonJS module dependencies for use in the browser.

Deployment

The last couple sections of research – headless test runners and min-concat optimization – were leading up to deployment. These two areas are outside of my development workflow and are part of my deployment workflow. Truth be told, I will probably incorporate running headless test on the optimized scripts as well to make sure that the compiler didn’t ming anything once in production, but for now I am putting trust in the fact that it does not – and any runtime errors that occur are due to my source code (it happens!).

In any event, here we stand with being tasked to fold in linting, testing, optimization and deployment. In the last section, I highlighted a few build systems I had come across as I was researching minification and concatenation of JavaScript, but I wanted to roll my own deployment script just so I can tailor it to my needs. I am not intending to provide it as an open source project as I feel it is very specific, but it is available and being continually developed from the massroute-js github if you are interested.

When it came to creating the build script(s), I sat for moment to think about the environment I wanted to use. I took under consideration what I thought should be considered of a client-side developer to have installed on her machine. Is it too much to ask to install node.js or ruby for the sole purpose of running a development/deployment script? Should I stick with old reliable ANT as it is pretty much accepted as a requirement for current and past projects? I answered “No” to both, though I do think the first question is a good discussion for another time; and I have written so many ANT build scripts that I wanted to do something different…

I settled on a simple Makefile, as it is quick and easy to understand and fulfills its current purpose:

# Lint - JSHint
LINT = ./build/lint/jshint
SRC_DIR = ./script

# Tests - PhantomJS
PHANTOM = ./build/phantom/phantomjs
PHANTOM_QUNIT_RUNNER = ./build/phantom/run-qunit.js
TEST_DIR = ./test
TEST_INDEX_URL = file://localhost/Users/todd/massroute_js/massroute-examples/common/test/index.html

# Min/Concat - r.js
R_JS = java -classpath ./build/rhino/js.jar:./build/closure/compiler.jar org.mozilla.javascript.tools.shell.Main
R_DIR = ./build/require

all: lint phantom optimize

lint:
	@echo '==> JSHint $<'
	@@for file in `find ${SRC_DIR} -name "*.js"`; \
		do echo ===Linting $$file...===; ${LINT} $$file; done;
	@echo

phantom:
	@echo '==> Phantom $<'
	${PHANTOM} ${PHANTOM_QUNIT_RUNNER} ${TEST_INDEX_URL};
	@echo

optimize:
	@echo '==> r.js $<'
	@@for file in `find ${R_DIR} -name "*.build.js"`; \
	   do echo ===r.js: $$file...===; ${R_JS} ${R_DIR}/r.js -o $$file; done;
	@echo

Certainly, this sort of Makefile is intended for the lead developer or automated system to run at deployment, rather than being shared across multiple developers during development as the constants are pretty system specific. If this development/deployment workflow were to introduced into a company workflow, I would probably opt for ANT or Rake – which I intend to add to the repository at some point. As well, this deployment is simply just for JavaScript. I may find down the line that a more complete system is required which can optimize HTML, CSS, images, etc. and actually deploy files to a server. The playing field then widens and sticking with Make is probably not going to be the optimal solution, and I may rethink requiring node.js or ruby environments for deployment.

Conclusion

So there we have it. My current development and deployment workflow. Subject to change, as always. Decisions towards testing and optimization were definitely weighted with respect to my choice in AMD, but I feel RequireJS presents a strong enough case for modular development, so I am pretty confortable and happy with it at the moment. It was a great learning experience researching and testing all the various frameworks, libraries and systems in finding what worked and I appreciate the openness of the JavaScript community in releasing such quality and well-documented code; I hope to return the favor some day.

A quick round-up from the original listing of what i wanted to achieve, marked with the decision:

• AMD – RequireJS
• Code Quality – JSHint
• Unit Testing – QUnit & PhantomJS
• Minification / Concatenation – r.js (+Closure Compiler)
• Build / Deploy – make

cheers!

It should also be noted that some explanations may be heavily influenced by my experience in developing for the Flash Platform and particularly their relation to the ActionScript and the Flex mark-up language.

The massroute-js example using Knockout can be found at: http://github.com/bustardcelly/massroute-js/tree/master/massroute-examples/

The initial draw to test out the Knockout library was its basis on the Model-View-View Model (MVVM) architectural pattern. I won’t go into to much detail about what MVVM is and what it can provide, as there are already many good explanations out there; I will just quickly point you to wikipedia and its explanation on the Knockout site directly. If you are not up for clicking those links, MVVM is an architectural design pattern – like Model-View-Controller (MVC) – but is similar to the Model-View-Presenter (MVP) pattern. In basic terms, the ViewModel of MVVM is similar to the Presenter of MVP in so much as the View is responding to changes on an abstraction of the Model rather than directly on the Model. However, they differ in that the ViewModel is an abstraction of not only the properties of the Model but also provides an API that reflects the actions taken by a View that in turn affect the state of the Model; the Presenter from MVP has more of a role in being knowledgable about the View directly – subscribing to events and accessing the View’s API. These are both markedly different from the MVC approach in which the Controller is only responsible for updating the Model in response to user actions on the View – providing an indirect relationship of the View to the Controller in which the state of the View is directly related to the state of the Model.

Data Binding and Observables

At the heart of Knockout’s MVVM implementation is data binding. If you stumbled upon here and are coming from the Flex world, the concept of data binding is probably all too familiar. In basic terms, data binding is a process in which one party, bound to a change on another party, is automatically updated to reflect that change when it occurs. Most notably, data binding is utilized in updating UI that is bound to a model representing state.

In broader terms, data binding utilizes the Observer Pattern. If you are familiar with subscribing to events and assigning handlers that respond to change in UI, then you are already aware of using an Observer. Within the implementation of the data binding concept, the subscription and publishing mechanisms are hidden away and as a change to the subject (the one subscribed to) occurs, they change is published to a dependent (the subscriber). What i mean by “hidden away” is that you as a developer do not directly establish this relationship by setting up event listeners and implementing the logic in response to a change in state. Libraries that allow for data binding provide a way in which you can establish this relationship which is then wired behind the scenes. Again, if you are coming from the Flex world, this is familiar in the use of curly brackets ({}) in-line in your MXML or (my preference) the methods of the BindingUtils class.

Using Knockout, you define observables which dispatch notifications when changes to a value occur. Properties on a ViewModel are defined with observables to which an element can be bound, creating a dependency that is reflected in the rendering of the UI. Let’s get away from wordy word-words and see an implementation of a ViewModel and how it relates to declarations of elements on the DOM. At the core you will have your model object that defines – among other things – the observables:

var model = {
    title: ko.observable('Hello World'),
    items: ko.observableArray()
};

If your dependency is on value change to one object – whether it be a string, number, boolean, object, etc. – you use ko.observable as defined in this example for the title property on the model object. If you want to detect and notify of changes to a collection of objects, you use ko.observableArray. You can also define dependent observables which are properties that are updated based on the value of other observables. Just a quick example of dependent observables:

model.greeting = ko.dependentObservable( function() {
    return this.title + ', how are ya?';
}, model );

These observables and their notifications are then managed once you register the model:

ko.applyBindings( model );

On the HTML side, you use the datasets to define the binding definitions. Knockout recognizes the data-bind attribute on elements of the DOM and – in broad, general terms – evaluates the attribute value as a binding expression. As a quick example of how this would look in using the model from the previous examples, the mark-up would look something like the following:

<header>
    <hgroup>
        <h1 data-bind="text: title"></h1>
        <h2 data-bind="text: greeting"></h2>
    </hgroup>
</header>
<section>
    <nav>
        <ul data-bind='foreach: items'>
            <li class='list-item'>
            	<a data-bind="attr: {href:url}, text: label"></p>
            </li>
        </ul>
    </nav>
</section>

As changes to the title property on the model occur, both items in the header group will be updated on the DOM (greeting because of its dependency on title), and as the item collection changes so will the un-order list, with the declared list item serving as the item renderer to be used in adding child elements to the list (more on that in the next section).

Properties are updated by invoking the observable. That may seem a little strange at first, as you are probably more familiar with updating object properties using the = expression. But the properties, once declared as ko.observables, are actually not the primitive objects they represent and are functions themselves. So if we were to update the properties on the model in the previous examples, that would be done as in the following:

model.title( 'Hello Again' );
model.items( [
    {label:'foo', url:'http://foo.com'},
    {label:'bar', url:'http://bar.com'}
] );

And that is important to remember, as the same goes for accessing the property value. Accessing model.title will actually return the observable function, not the string value. To get the underlying value, you use ko.utils.unwrapObservable, as in the following example:

var titleValue = ko.utils.unwrapObservable( model.title() );

You can also subscribe to changes explicitly in JavaScript, and if you are coming from the Flex world, this is pretty much like the BindingUtils:

var itemSubscription = model.items.subscribe( function( collection ) {
    // do something really cool that will win you friends and influence others.
});
...
// No longer need to win friends and influence others.
itemSubscription.dispose()

That is just a birds-eye-view quick run-down of how the view model plays a part in knockout, and i really encourage you to check out the document on their site to see all of what is capable: http://knockoutjs.com/documentation/introduction.html.

Control Flow

If you’ve looked at those examples or are familiar with Knockout and have checked out the massroute-js/knockout example I put on github, you might see that things are set up a bit different. Indeed they are; I had begun my example using knockout-1.2.1.js, but had found this post toward the end of my finishing the example: http://blog.stevensanderson.com/2011/08/31/knockout-1-3-0-beta-available/. I was so taken with the changes to Control Flow Bindings, that I moved my example to that. If this is all new to you and that last sentence was complete gibberish, I’ll explain; I just wanted to give a heads up if any of you were curious enough to check out their current examples and look at my project on github.

What I take away from the concept of Control Flow Binding, is that you can define (and Knockout with manage) the existence of DOM elements based on a ViewModel condition. If you are coming from the Flex world, you can think of it somewhat in terms of the State API and state-based declarations for elements, such as includeIn and excludeFrom. But it goes a little farther in that, in so much as it is also allows for and is used in rendering whole graphs of DOM elements based on a condition and model data. A common example is lists, and the UI elements you define are the item renderers – depending on the existence of a collection, graphs of defined HTML elements are rendered, added to the DOM and handed data.

Prior to the current work on Knockout that is in beta and available from the projects github, there was a dependency on an external templating engine – for example, jquery-tmpl which was commonly the go-to engine since there is already a dependency on jQuery -that would allow you to define UI elements to use in deferred rendering within the Control Flow and state of the target ViewModel. In the latest beta for Knockout, you can now declare those elements inline within the document and the library will handle the existence and rendering based on the defined condition.

To give a quick example, here is the whole section in the massroute-js/knockout project that displays the list of routes available from the real-time MBTA feed:

<section id="routesection" data-bind="visible: routes.visible">
    <h1 data-bind="text: routes.title"></h1>
    <p data-bind="if: routes.list().length === 0">loading...</p>
    <nav data-bind="if: routes.list().length > 0">
        <ul data-bind='foreach: routes.list'>
            <li class='route-item list-item icon-list-item' style='cursor: pointer;'>
                <figure>
                    <img src='style/image/bus_icon.png' />
                    <figcaption data-bind="text: title"></figcaption>
                </figure>
            </li>
        </ul>
    </nav>
</section>

If this is example you are seeing that utilizes the Knockout library, i will just quickly mention that Knockout recognizes the data-bind attribute and interprets the value as an expression allowing you to associated DOM elements with a ViewModel declaratively. It is too much of a discussion to go into the dataset attribute and their usage, but wanted to give a brief explanation of how it is relevant to this snippet and will now defer you to this documentation: http://dev.w3.org/html5/spec/Overview.html#embedding-custom-non-visible-data

To get a clearer picture of Control Flow and element templating, take a look at the <ul> and <li> fragments of this example. A foreach is declared on the binding for the ul element, which in turn will render a new instance of the list item declared within the ul for this example. In simpler terms, this allows us to define the item renderer for the list declaratively inline within the control flow definition. A pretty cool addition to the library, and one in which, even though i am just test-driving Knockout, warranted me using Knockout 1.3.0 beta to do away with using another dependency on a template engine.

There are also some other additions to the library which I am excited to try out, most notably Binding Providers and Throttling. Check out this post from Steven Sanders about Knockout 1.3.0 Beta: http://blog.stevensanderson.com/2011/08/31/knockout-1-3-0-beta-available/.

Links

I encourage you to check out the Knockout home page at http://knockoutjs.com/ and the awesome interactive tutorials they have available at http://learn.knockoutjs.com. You can also checkout the latest developments from github – https://github.com/SteveSanderson/knockout – and, again, check out what is coming in the next version from this nice article: http://blog.stevensanderson.com/2011/08/31/knockout-1-3-0-beta-available/.

You can checkout the rest of my massroute-js/knockout example on github here: https://github.com/bustardcelly/massroute-js/tree/master/massroute-examples/knockout. Questions, comments and constructive berating are welcome (though not so much the last one).

If there are numerous children within a container and their style definitions are rather complex, there may be some lag within the rendering cycle. Though it may not cause the Flash Player to time-out or even invoke the beachball/hourglass of the user’s system, any perceived lag in rendering that is a considerable amount (to be discussed at a later date) could send the user packing if not in the very least curious as to the reliability of the application; needless to say, most UI heavy applications are those in which a user interacts heavily with the application in order to get something accomplished.

Setting UX/design decisions aside and trusting that optimizations – such as absolute positioning – are being used where appropriate, there still may come a time where the task at hand calls for a lengthly child list within a container, or even a rather small list with a heavy render cycle due to customization. In such a situation, having the ability to defer rendering is a great improvement to the perceived “snappiness” of your application.

Back in the Flex 3 days – well, i shouldn’t say “back” as the MX containers are still available in the SDK and i am sure there are still people working on Flex 3 whether grudgingly or not – the MX containers had a property called creationPolicy. This allowed a developer to define the how the container was to render its children – basically, whether it should build its display tree upon creation of the container or defer. If you deferred it by setting the property to ContainerCreationPolicy.NONE, you could at anytime within the life of the application call createComponentsFromDescriptors() on that container to have its children be created.

Now it should be noted that this functionality is at parity in Flex 4.x, yet only available to SkinnableContainers – and more accurately those that implement IDeferredContentOwner. Those containers within the Flex 4 SDK which are considered lighter-weight, such as Group and DataGroup, do not play host to skins or define style properties for graphical backgrounds but they also do not support deferred content inherently either.

I should stop here and say that even though Group and DataGroup are in the same boat as not being privy to a creationPolicy, the example code should really only be translated to a Group container. Deferred content for a DataGroup should be left to its defined layout and its properties. That said, there is a way to defer the child element creation of a Group rather simply; that is by utilizing the [DefaultProperty] metadata.

You can see the full gist of the moving pieces on my github, but just to give a quick example and rundown:

package
{
    import flash.events.Event;

    import mx.events.FlexEvent;

    import spark.components.Group;

    /**
     * Use the deferredElements property as the modifier that is handed the list of IVisualElements defined in MXML.
     */
    [DefaultProperty("deferredElements")]

    /**
     * Group container extension in order to do deferred child element attachment.
     */
    public class DeferredGroup extends Group
    {
        protected var _elementAddQuota:int = 10;

        /* IVisualElement[] */
        protected var _deferredElements:Array;
        protected var _deferredElementsChanged:Boolean;

        protected var _deferredElementAttacher:IDeferredElementAttacher;

        public function DeferredGroup()
        {
            super();
        }

        /**
         * @inheritDoc
         */
        override protected function commitProperties():void
        {
            super.commitProperties();
            if( _deferredElementsChanged )
            {
                queueDeferredElements();
                _deferredElementsChanged = false;
            }
        }

        /**
         * Defines and executes the IDeferredElementAttacher implementation used in queue-ing up "packs"
         * of IVisualElement instances declared in MXML and assigned as deferredElements.
         */
        protected function queueDeferredElements():void
        {
            // Define IDeferredElementAttacher and listen for completion on attach of all elements within deferredElements.
            _deferredElementAttacher = new FrameDeferredElementAttacher( this );
            _deferredElementAttacher.addEventListener( Event.COMPLETE, handleDeferredAttachmentComplete, false, 0, true );

            // "Chunk" up lists of IVisualElements based on limit quota.
            var i:int;
            var startIndex:int;
            var endIndex:int;
            var length:int = _deferredElements.length;
            var count:int = Math.ceil(length / _elementAddQuota);
            for( i = 0; i < count; i++ )
            {
                startIndex = i * _elementAddQuota;
                endIndex = startIndex + _elementAddQuota;
                endIndex = ( endIndex > length ) ? length : endIndex;
                _deferredElementAttacher.addElementPack( _deferredElements.slice( startIndex, endIndex ) );
            }
            _deferredElementAttacher.start();
        }

        /**
         * Event handler for completion from the IDeferredElementAttacher instance.
         * @param evt Event
         */
        protected function handleDeferredAttachmentComplete( evt:Event ):void
        {
            // Kill reference.
            _deferredElementAttacher.removeEventListener( Event.COMPLETE, handleDeferredAttachmentComplete, false );
            _deferredElementAttacher.dispose();
            _deferredElementAttacher = null;

            // -> Do whatever you normally would do in a handler for CREATION_COMPLETE.
        }

        /**
         * The amount of IVisualElements to "chunk" into packets for deferred queue.
         * @return int
         */
        public function get elementAddQuota():int
        {
            return _elementAddQuota;
        }
        public function set elementAddQuota( value:int ):void
        {
            _elementAddQuota = value;
        }

        /**
         * The DefaultProperty array of IVisualElements declared in MXML markup.
         * @return Array Required to be IVisualElement[]
         */
        public function get deferredElements():Array
        {
            return _deferredElements;
        }
        public function set deferredElements( value:Array ):void
        {
            if( _deferredElements == value ) return;

            _deferredElements = value;
            _deferredElementsChanged = true;
            invalidateProperties();
        }
    }
}

Essentially, I have defined the [DefaultProperty] to be the deferredElements property. By default, any Flex container’s [DefaultProperty] is the mxmlContent property. If left to default, anything declared in MXML within the container declaration is handed to the mxmlContent property which in turn is used to add the children to the container using ActionScript at a later time within its instantiation cycle. Just as an example, the Label and Button of the Group container defined in this MXML can be considered the array of IVisualElements known to the parent Group as mxmlContent.

<s:Group>
    <s:Label text="Hello, World" />
    <s:Button label="foo" />
</s:Group>

… and same goes for Flex 3 containers. Not much has changed, aside form non-parity in creationPolicy on Flex 4 containers that don’t implement IDeferredContentOwner. However, in the example provided above and in the gist, the declared children in MXML aren’t handed to the mxmlChildren, but rather the deferredChildren. Because of this, we can then check if we have deferred content upon a pass to invalidationProperties() and act accordingly – as is done by invoking queueDeferredElements() in the example:

/**
 * Defines and executes the IDeferredElementAttacher implementation used in queue-ing up "packs" of IVisualElement instances declared in MXML and assigned as deferredElements.
 */
protected function queueDeferredElements():void
{
    // Define IDeferredElementAttacher and listen for completion on attach of all elements within deferredElements.
    _deferredElementAttacher = new FrameDeferredElementAttacher( this );
    _deferredElementAttacher.addEventListener( Event.COMPLETE, handleDeferredAttachmentComplete, false, 0, true );

    // "Chunk" up lists of IVisualElements based on limit quota.
    var i:int;
    var startIndex:int;
    var endIndex:int;
    var length:int = _deferredElements.length;
    var count:int = Math.ceil(length / _elementAddQuota);
    for( i = 0; i < count; i++ )
    {
        startIndex = i * _elementAddQuota;
        endIndex = startIndex + _elementAddQuota;
        endIndex = ( endIndex > length ) ? length : endIndex;
        _deferredElementAttacher.addElementPack( _deferredElements.slice( startIndex, endIndex ) );
     }
     _deferredElementAttacher.start();
}

And in that method, an IDeferredElementAttacher (non-SDK, part of the gist) implementation is used to queue up smaller arrays of child elements and dequeued to start adding children to the target container. In this example that implementation is a FrameDeferredElementAttacher instance which waits a frame to move along in its queue. Included in the gist is also one for timer-based element addition.

This post got rather long as far as just wanting to show off how to defer child content of a Group container in Flex 4, but I just wanted to bring light my solution as there is no parity of creationPolicy on non-IDeferredContentOwner containers in the SDK.

See the whole gist here: https://gist.github.com/1340472.

It should also be noted that some explanations may be heavily influenced by my experience in developing for the Flash Platform and particularly their relation to the ActionScript and the Flex mark-up language.

As far as JavaScript libraries that go beyond providing an abstraction layer/decorator to the DOM and a facade for AJAX requests – such as one would find with jQuery – I have been interested in digging into Dojo for some time. I should state that I am in no way knocking jQuery. It works extremely well at what it purports to do. I do, however, have an interest in how to “maintain” an application written in JavaScript in as much as not only adhering to architectural patterns but code organization and dependency management. That is not to say you could not work with another library that incorporates or works laterally with jQuery for such support; I am also interested in those libraries that provide more of a unified application structure alongside AJAX and DOM access. Enough defending jQuery…

For many reasons Dojo caught my eye because not only does it provide DOM access/manipulation through abstraction (as well as using Sizzle as the selector engine, as jQuery does) but dependency management through a modular package system, as well. What that really means is that I can organize my scripts in a directory tree structure, and define classes and their dependencies using the provide/declare/require API of Dojo. This allows for a common “package structure” and resolves to namespaced objects at runtime (or build time- maybe more on that later), providing a modular-approach to building a JavaScript application.

The example was built against Dojo 1.6.1, so any further explanations in this post are dependent on my knowledge working with the Dojo library at that version.

Modules

Now, truth be told, I utilized the Dojo module dependency system as I would with declaring and importing classes. This is mainly because of my experience with languages that are class-based (or provide a class “structure”) and that Dojo provides an elegant and easy way for class declaration and inheritance on top of the prototypical nature of JavaScript. So my modules really became more like classes or groupings of common classes – which is perfectly acceptable to me and how I would prefer to work – but it should be noted (if you were to look at the example in the github repo) that modules are not restricted to only declaring classes.

Just to provide a quick example of what i mean by treating modules as independent classes or a grouping of common classes, let’s look at an abbreviated version of the main context module (script file) for the application:

dojo.provide( "context.massroute" );
dojo.require( "model.session" );

dojo.declare( "context.massroute.MassRouteContext", null, {
    constructor: function() {
        this.session = new model.session.Session();
    }
}

The value within a dojo.provide() invocation creates (if not previously existant) a namespaced object on global (window) scope and caches the reference within dojo._loadedModules with the property value of the string argument (”context.massroute” in this example). Essentially this sets up a “package” structure for your declarations with access to classes once a module has been requested and loaded.

Modules are loaded using dojo.require(). The dojo.require() states a dependency on another module. For this example, Dojo is requested to load the “model.session” module. More on the association between files and package declarations in a bit, but for now know that if “model.session” is not found on the dojo._loadedModules cache, a request for /model/session.js will be made. Once loaded, it would go through much of what is currently being covered in this example: defining namespace, loaded dependencies and declaring classes. If we were to open the /model/session.js file found the application script source, we would find a dojo.declare() for the model.session.Session class.

With each dojo.declare() within the provided namespace, the objects on the direct global scope and within dojo._loadedModules are updated to hold reference to the class, which essentially allows you to create a new instance of a class as one normally would:

var context = new context.massroute.MassRouteContext();

// declare explanation
The dojo.declare() takes three arguments:

• className: defines the associated name for the class within the package/namespace.
• inheritance: defines the classes to “mixin” to your class. This can be null, a single class reference or an array of class references. The class reference(s) can be thought of as base or multiple inheritance if defined.
• properties: defines the object that encapsulates all the logic particular to the class

The inheritance I found of particular note as dojo provides an easy way to call super methods. Essentially, within the scope of the override method of a subclass you can call this.inherited(arguments).

To get a better understanding of how this all works and what is modified as modules are loaded and classes declared, if we suppose that this module is requested to be loaded in the current domain, the global object will be modified to reflect the following:

[window] {
    dojo: {
        _loadedModules: {
            "context.massroute": {
                "MassRouteContext": function() { ... };
            }
            "model.session": {
                "Session": function() { ... };
            }
        }
    }
    context: {
        massroute: {
            MassRouteContext: function() { ... };
        }
    }
    model: {
        session: {
            Session: function() { ... };
        }
    }
}

That gives a little insight into Dojo’s nested namespace pattern and how access to classes is provided, as well as how Dojo manages caching of previously requested dependencies.

Just for kicks, if we translated this example over to ActionScript it would be read as this:

package context.massroute {
    import model.session.Session;

    class MassRouteContext {
        public var session:Session;

        function MassRouteContext() {
            session = new Session();
        }
    }
}

I touched on it briefly before, but you may be wondering how dojo knows how to associated these namespaces with modules to load within its package system. By default, Dojo will assume that a file with the name of the ending namespace resides in a relative location of its request – meaning, if my main index.html file resides at the root of the directory, when I dojo.require(”context.massroute”), dojo will make a request for ./context/massroute.js if the path to that module is not defined. If you have taken a look at the example in my github repo, you may notice that /content/massroute.js is not located at the root but in a parenting directory describing the scripts for my application.

You can define the path to your modules by declaring a global dojoConfig object prior to load of the Dojo library. An abbreviated version of what that would be for the example above:

<script>
    dojoConfig = {
      baseUrl: './',
      modulePaths : {
            app : "js/app",
            context: "js/app/context",
            model: "js/app/model"
        }
    };
</script>
<script src="http://ajax.googleapis.com/ajax/libs/dojo/1.6.1/dojo/dojo.xd.js"></script>

This defines the base URL for the directories defined in modulePaths. Now when you dojo.require( “context.massroute” ), dojo will look up the associated directory for the context namespace and request to load massroute.js. Once loaded, it is important to note that the file isn’t inserted into a script tag which is a common practice for script loaders. It is actually run through dojo.eval() which creates namespaced objects on global and dojo scopes as described above.

Some Caveats

There are some things that I didn’t immediately realize or think of off-hand that may be of interest to those of you whether or not you come from a class-based language:

• Anything you think is defined privately to the “class” in the constructor using dojo.declare() is actually set at global scope.
• No such things as private members to the class. Prepending members with “_” is the convention to “convey” to developers that a member is private.

There is a way around this whole private member business, however. If you are in dire need to keep something private, you can use an Immediately Invoked Function Expression (IIFE) to define your private members as well as the class, such as:

dojo.provide( "context.massroute" );
dojo.require( "model.session" );

(function() {
    var _session = new model.session.Session();
    dojo.declare( "context.massroute.MassRouteContext", null, {
        constructor: function() {
            console.info( _session );
        }
    }
}
)();

It should also be noted that you don’t have to use dojo.declare() at all to get the same structure. Essentially – from what I gather – under the hood, the library does the following (in as far as constructing the namespace) which you are perfectly welcome to do as well in a module and can help with your private requirements using a mix of IIFE and the Constructor Pattern:

dojo.require( "model.session");

(function() {
    if( !dojo._hasResource["context"] ) {
        dojo._hasResource["context"] = true;
        dojo.context = {};
    }
    if( !dojo._hasResource["context.massroute"] ) {
        dojo._hasResource["context.massroute"] = true;
        dojo.provide( "context.massroute" );

        context.massroute.MassRouteContext = function() {
            var _session = new model.session.Session();
            return {
                getSession: function() {
                    return _session;
                }
            }
        }
    }
})();

Obviously, going that route, you will miss the niceties that Dojo provides in class inheritance but just throwing it out there.

Events

Dojo’s event system was another part of the framework i enjoyed. Using dojo.connect() and dojo.disconnect() you could easily assign and remove event handling (delegate functions) to DOM events on a target scope, respectively. And, because dojo.connect() returned a connect object, it conveniently made it possible to disconnect all connection I may have set up in a view controller that was going to be trashed:

var connections = [];
connections.push( dojo.connect( dojo.byId('submitButton'), 'onclick', handleSubmitRequest ) );
connections.push( dojo.connect( dojo.byId('cancelButton'), 'onclick', handleCancelRequest ) );
...
dojo.forEach( connections, dojo.disconnect );

It should be noted that when using dojo.connect() you are not limited to just DOM events. You can connect to “custom” events on non-DOM objects. To do so, the second argument is actually the function on the target object that you wish to “listen” for invocation of. As a quick example:

var test = function() {
    return {
        run: function() {
            setTimeout( this.onTimeout, 1000 );
        },
        onTimeout: function() {
            console.info( "test.onTimeout" );
        }
    }
};

var myTest = new tester();
dojo.connect( myTest, "onTimeout", function() {
    console.info( "dojo.connect() > onTimeout" );
});
myTest.run();

Along with dojo.connect() and dojo.disconnect(), using dojo.publish() and dojo.subscribe()/dojo.unsubscribe() you can make use of the global event bus to broadcast and subscribe to custom events – similar to, say, the NSNotificationCenter from Objective-C. I used this heavily in reporting what i deemed application-level state related events from the view controllers in the Dojo example in the massroute-js github repo, and thought it was a pleasure to work with.

Other

Just quick notes on some other things that i found interesting from the Dojo framework that I either used or want to use in the future using Dojo:

dojo.hitch

Keeping scope was a pleasure with dojo.hitch(). Due to developer requirements from MassDOT in using their API for real-time transportation data, you cannot make API calls more frequently than every 10 seconds. This meant that I had to implement a request queue in which I had to route the calls: immediate invoke if timer not running, else your next in line when timer runs out. Keeping scope of what was to be called when inside this queue manager was necessary as the queue manager itself was not supposed to have any logic about the service, only knowledge of state between orders coming and going. So it was important that when it was time to process the order, the scope was tied to the one who placed the order.
dojo reference: dojo.hitch()

dojo.behavior

Though I went with my own custom view controllers and their logic encapsulated in respective classes, Dojo provides the ability to assign “behaviors” which i really want to look into further as it looks to be a nice design in separation of logic from view using mediation that can be swapped out quite easily at runtime.
dojo toolkit: Using dojo.behaviour

Wish I Coulda

The following are a couple things I struggled with when working with the Dojo toolkit.

Development/Debugging

I love breakpoints. Being able to set a breakpoint and traverse a call stack and inspect properties is the pleasurable way for me to debug any problems. However, due to the design of Dojo’s modular architecture I found myself relying more on console for debugging purposes as modules are essentially read from a URL and evaluated as an expression. This means i couldn’t locate the file resource in development tools and set a breakpoint in any of my modules.

I am sure there is some trick or project set-up that would provide a more debugging-friendly environment when developing an application with Dojo, but I couldn’t find it. If you know of one, please leave a comment.

Build

From the looks of it, Dojo has a pretty powerful Build System. Intended to produce minified and concatenated scripts, using the build system allows you to package only what your app required from the Dojo framework. I frustratingly tried to get it to work a couple times, but it always ended up with the full dojo and my scripts were never minified or concatenated and it created this weird directory structure that made no sense. I am probably doing it wrong, but there is little out there in documentation if anything goes wrong.

There is also the online Build Tool, but i didn’t know the whole breath of source I needed for my build; I am assuming from the docs that there is where the power of the Build System and profiles comes into play.. Something I would definitely like to check out further because, next to the modular design, this *in theory* makes Dojo a real winner.

Conclusion

All in all, I was pretty impressed by Dojo and would recommend trying it out and may bring it up as a framework of choice for any larger projects in the future (especially if i could get Build to work for me). There was a slight learning curve mainly in the class declaration and inheritance design, but nothing over-daunting and actually quite revealing. The curve probably would have been higher if I was not already familiar with popular JavaScript patterns.

Dojo Toolkit
massroute-js/dojo

Centered around the theme of developing a web-based application to access real-time MBTA transportation data (made available by MassDOT – more info here), I wanted to explore various libraries and micro-application frameworks that are out in-the-wild for JavaScript; some more well-known than others, but still hold my interest.

The initial commit included the work i had done with jQuery Mobile, which i have been actively working with for almost a year and have given a couple presentations on. Most recently, I committed an example of my exploration into dojo. I rather enjoyed working with the dojo framework and toolkit and hope to have another post that spotlights my experience working with it – actually i hope to post on each library/framework that i dig into with the repo, so we’ll see how that goes. Next up on my curiosity list is Knockout…

I won’t list all those that have caught my eye, so if you have any suggestions please feel free to leave them in the comments.

[Disclaimer]
If you look at the source of the examples in the repo, I don’t claim to be a master of the library or frameworks I am exploring, needless to say the JavaScript language itself. So, if you see something that is considered bad practice or all together wrong, please let me know.

The most exciting new feature coming to Adobe AIR to me is the ability to compile against a library that can communicate with a device natively. Though some of the Stage* API opens a whole new world of rendering improvements, the inclusion of Native extensions for Adobe AIR (NE) relieves the wishing and waiting of what will be exposed at the device level in future AIR SDK releases; we can now extend the Runtime ourselves. With the arrival of Native extensions, the possibilities of what can be achieved with a mobile AIR application grow larger; and certainly with that, so does complexity in design and the requirement to develop with mobile performance in mind.

Keeping the complexities and performance in check (and to calm myself down from excitement of the endless possibilities) I decided to give it a quick test-run by creating a Native Extension that I thought was sorely missing from the Adobe AIR SDK – battery information on iOS. In all seriousness, getting up and running and accessing the battery information of my iPhone from an application AIR application was rather painless and – dare i say it – easier than i expected. I intend to give a quick walk through of how I went about creating the iBattery Native Extension and how to use it in an AIR application.

Disclaimer

I am going to assume you are familiar with setting up an Apple Developer account and downloading the SDK and won’t even go into the headache of provisioning profiles and the deployment process for an iOS application. I am also going to assume that you are familiar with using the recent Flex 4.5 SDK to create mobile Flex applications. The application in this example is in no way complex, but I am not going to cover how the pieces of the application work – only how you can compile against and include an Native extension for Adobe AIR.

Source

If you are curious or just want to jump to code, I created a github repo with all the source as well as deployment builds or if you just came here for the Native Extension and know your way around you can download ibatteryextension.ane directly.

Requirements

• Flex 4.5.1 SDK
• AIR 3 Runtime and SDK (Release Candidate)
• iOS SDK

Also, take a look at these excellent articles:

• Extending Adobe AIR by Oliver Goldman
• Why Native Extensions for AIR
• Developing ActionScript Extensions for Adobe AIR
• Native Extensions for Adobe AIR in the AIR Developer Center

I am not going to cover how to get this going with any IDEs (eg. Flash Builder 4.5). I will show you how to do all this from the command line. There are some niceties with IDEs that will separate you from invoking the command line tools of the SDK directly, but sometimes it is nice to see what is going on behind the scenes.

I went about downloading the Flex 4.5 SDK and unzipping it somewhere on my local disk, then downloaded the Adobe AIR SDK and overlaid it on the Flex SDK using the instructions here: How To Overlay the AIR SDK. For the purposes of the examples in this article let’s just say that SDK is located on my machine at: /Users/todd/SDKs/flex_4.5_air_3rc1.

The whole iOS & Xcode set up, you are on your own. Sorry, not to be mean. There is some great information already out there and i want to keep this article more focused on Native Extensions for Adobe AIR and not to prevent hair-loss developing for iOS.

Moving Pieces

There are three major portions to create an AIR application utilizing the Native Extension for Adobe AIR:

1. Native code compiled for target platform.
2. Flex library project to deploy the Native Extension for Adobe AIR.
3. Mobile AIR application.

Depending on your target device/platform, the language and generated library on the native-side can vary and really is covered well in Oliver Goldman’s article. For the purposes of this example, the native part involved me writing some C/Obj-C and creating a static library (.a file). The Flex library project is actually two-fold; I created a library that interfaced with the flash.external.ExtensionContext, then generated an .ane file from the SWC and static library. The Mobile AIR application is then compiled against this Native Extension and the .ane library file is included in the generated .ipa file.

Native

I am not that much of an C/Objective-C developer. I have developed and deployed a handful of iOS applications in the past (some reaching the AppStore), but honestly have not touched it in quite some time. So I have some history, but cannot speak at length on how things work and why. Let’s just say, for the example in this article, that I knew enough to get in and get out and carry on on the ActionScript side of things.

When creating a native extension targeting the iOS platform, you’ll write some C code (which can call Obj-C) and deploy a static library with a .a extension. What i did was create a new Library project in Xcode, imported the header file included with the AIR SDK (found at /include/FlashRuntimeExtensions.h) and added a .m (Obj-C implementation) file to the project that will serve as the implementation of the native context and expose the API for accessing the battery life and information of the iOS device:

// Access battery life.
FREObject GetBatteryLife(FREContext ctx, void* funcData, uint32_t argc, FREObject argv[]) {
    UIDevice *device = [UIDevice currentDevice];
    [device setBatteryMonitoringEnabled:YES];
    float life = [device batteryLevel];

    FREObject retVal;
    FRENewObjectFromDouble( life, &retVal );
    return retVal;
}

// Access info about battery
FREObject GetBatteryInfo(FREContext ctx, void* funcData, uint32_t argc, FREObject argv[]) {
    UIDevice *device = [UIDevice currentDevice];
    [device setBatteryMonitoringEnabled:YES];
    int info = [device batteryState];

    FREObject retVal;
    FRENewObjectFromInt32( info, &retVal );
    return retVal;
}

Boiled down and abstracted in thinking on the ActionScript side of things, the API of this native library exposes two methods: GetBatteryLife and GetBatteryInfo. Each return a numerical value related to their context: a Number representing the percent of battery life left on the device and an Integer representing battery state, respectively. The relation of the Number value to percent is fairly straight-forward. The Integer returned from [[UIDevice currentDevice] batteryState] relates to the various “states” that the battery can be in, and they are:

• 0 : Unknown
• 1 : Unplugged
• 2 : Charging
• 3 : Full

The FREObject is covered more properly by Oliver Goldman in his article, and I just blindly assume that the ExtensionContext of the AIR SDK knows how to interpret this value for me so I can cast as an ActionScript type and move on. We’ll see how that happens in the next section.

The Native Extension

The previous section lightly covered the native side of things and generated a static library file that will be used to compile an Native Extension (NE) library that will be used by an AIR application to access battery information of an iOS device it is deployed to. Creating the Native Extension file (.ane) is actually a two step process:

1. Create a Flash library project compiled against AIR libraries and expose an API that interfaces with the native library through ExtensionContext.
2. Use the ADT command line tool with to generate an .ane file compiled against the library SWC and the native library.

The flash.external.ExtensionContext from the AIR SDK is your main access point to the native library. Essentially, you create a new instance of an ExtensionContext using an ID defined in an extension descriptor file compiled into the Native Extension. For the iBattery Native Extension, this is what my extension descriptor file looks like:

<extension xmlns="http://ns.adobe.com/air/extension/2.5">
  <id>com.custardbelly.ibattery</id>
  <versionNumber>1</versionNumber>
  <platforms>
    <platform name="iPhone-ARM">
            <applicationDeployment>
                <nativeLibrary>libAIRExtensionC.a</nativeLibrary>
                <initializer>ExtInitializer</initializer>
                <finalizer>ExtFinalizer</finalizer>
            </applicationDeployment>
        </platform>
  </platforms>
</extension>

Highlighted in that snippet are some important bits. The id node value will be used to create a new ExtensionContext instance. The nativeLibrary node value is the native library created in the previous section. For this example, it is also of note that we have defined the iPhone-ARM platform as a target, as well. This extension descriptor file describes the association of id to native library that the ExtensionContext will look up upon instantiation. To create an ExtensionContext:

com.cusardbelly.air.extensions.battery.ios.Battery

_extensionContext = ExtensionContext.createExtensionContext( "com.custardbelly.ibattery", "main" );

… and then use the call() method to invoke the corresponding method exposed in the native library. For the purposes of the example in this article, and for the iBattery Native Extension, I basically exposed the same API that was on the native side:

com.cusardbelly.air.extensions.battery.ios.Battery

public function getBatteryLife():Number
{
    return _extensionContext.call( 'GetBatteryLife' ) as Number;
}

public function getBatteryState():int
{
    return _extensionContext.call( 'GetBatteryInfo' ) as int;
}

With my API done, I compiled the library into a SWC:

> /Users/todd/SDKs/flex_4.5_air_3rc1/bin/compc -output build/iBattery.swc -load-config+=ibattery_lib.config +configname=airmobile

If you are unfamiliar with using the command line tools of the SDK, I used the compc tool which is used to generate SWC files. The iBattery.swc file is generated and placed in a build directory and is compiled against an additional custom config (which just defines source location) and the +configname=airmobile directive. That’s actually a little fun tidbit. If you want to generate a SWC or SWF that uses the AIR mobile libraries, just add +configname=airmobile and they’ll be compiled against for you without defining them in an additional config file.

I then took the iBattery.swc and unzipped it to get the library.swf file. This is necessary for generating a Native Extension file (.ane) using the ADT command line tool:

> /Users/todd/SDKs/flex_4.5_air_3rc1/bin/adt -package -target ane ../release/ibatteryextension.ane extension.xml -swc iBattery.swc -platform iPhone-ARM library.swf libAIRExtensionC.a

That generates an ibatteryextension.ane in a release directory and defines the target SWC library and platform as well as compiling in the descriptor, SWF library and native library.

That Native extension is then used as one would as SWC library in an AIR Mobile application to interface with the native library. You need to compile against the .ane and include it in an extension directory within the AIR application.

The AIR Mobile Application

The application I created to showcase the iBattery Native Extension is pretty dead simple. Again, I am assuming you have some knowledge of a mobile AIR application and its pieces. There are a ton of great articles out there that can help in developing an AIR application targeting mobile, so I won’t provide any more explanations in this article, I just wanted to show you quickly how the communication works and the requirements for compilation of the application.

To being I just have a main ViewNavigatorApplication that defines a single view:

iBatteryExample.mxml

<?xml version="1.0" encoding="utf-8"?>
<s:ViewNavigatorApplication xmlns:fx="http://ns.adobe.com/mxml/2009"
        xmlns:s="library://ns.adobe.com/flex/spark"
        firstView="BatteryTestView">

</s:ViewNavigatorApplication>

… and the BatteryTestView provides a UI to request the battery information on the device and does all the communication through the ActionScript side of the Native Extension library generated previously:

BatteryTestView.mxml

<?xml version="1.0" encoding="utf-8"?>
<s:View xmlns:fx="http://ns.adobe.com/mxml/2009"
        xmlns:s="library://ns.adobe.com/flex/spark"
        title="BatteryTestView" creationComplete="handleCreationComplete();">

    <fx:Script>
        <![CDATA[
            import com.custardbelly.air.extensions.battery.ios.Battery;

            protected var _batteryExtension:Battery;

            protected function handleCreationComplete():void
            {
                _batteryExtension = new Battery();

                lifeButton.addEventListener( MouseEvent.CLICK, handleLifeRequest, false, 0, true );
                infoButton.addEventListener( MouseEvent.CLICK, handleInfoRequest, false, 0, true );
            }

            protected function handleLifeRequest( evt:Event ):void
            {
                try {
                    console.appendText( "Battery Life Percentage: " );
                    console.appendText( ( _batteryExtension.getBatteryLife() * 100 ).toString() + "%\n" );
                }
                catch( e:Error )
                {
                    console.appendText( "Error: " + e.message );
                }
            }

            protected function handleInfoRequest( evt:Event ):void
            {
                console.appendText( "Battery State: " + ( _batteryExtension.getBatteryState() ).toString() + "\n" );
            }
        ]]>
    </fx:Script>

    <s:layout>
        <s:VerticalLayout paddingLeft="10" paddingRight="10" paddingTop="10" paddingBottom="10" />
    </s:layout>

    <s:TextArea id="console" width="100%" height="100%" editable="false" text="Hello World!" />
    <s:HGroup width="100%" height="24" verticalAlign="middle">
        <s:Button id="lifeButton" label="get life" />
        <s:Button id="infoButton" label="get info" />
    </s:HGroup>

</s:View>

Access to the ActionScript API from the Native Extension is available just as if you were developing against an external SWC library, and I created a new instance of a Battery to request life and information of the device which is then just printed out in a text area:

BatteryTestView.mxml

_batteryExtension = new Battery();

console.appendText( "Battery State: " + ( _batteryExtension.getBatteryState() ).toString() + "\n" );

To generate my AIR application for iOS, was a two step process. First, I generated the application SWF targeting AIR mobile:

> /Users/todd/SDKs/flex_4.5_air_3rc1/bin/mxmlc +configname=airmobile -output build/iBatteryExample.swf src/iBatteryExample.mxml -load-config+=battery_app.config

Again i used the +configname=airmobile directive to include the mobile SWC from the Adobe AIR SDK without defining the dependencies in an additional config file. I did, however, have an additional config file that defined the Native Extension (.ane file) to compile against as an external library:

battery_app.config

<?xml version="1.0"?>
<flex-config xmlns="http://www.adobe.com/2006/flex-config">
    <compiler>
        <external-library-path append="true">
            <path-element>ext/ibatteryextension.ane</path-element>
        </external-library-path>
    </compiler>
</flex-config>

That generated an iBatteryExample.swf which was then used alongside an application descriptor file to compile and generate the .ipa file (application installer file for iOS). We can create that using the ADT command line tool:

> /Users/todd/SDKs/flex_4.5_air_3rc1/bin/adt -package -target ipa-test-interpreter -provisioning-profile {path.to}.mobileprovision -storetype pkcs12 -keystore {path.to}.developer_identity.p12 -storepass {pass} ../release/iBatteryExample.ipa iBatteryExample-app.xml iBatteryExample.swf -extdir ../ext/

There are two things in this command that you should note. First off, you will need to replace the {path.to} and {pass} tokens to point to your iOS developer files and password, respectively. Second is the -extdir option. This defines where the application can locate the Native Extension (.ane file).

And that was pretty much it for the application. Pretty basic, but a rather quick way to get up and running to allow someone to find information about their battery on an iOS device.

Conclusion

The addition of Native Extensions for Adobe AIR to the AIR SDK opens a lot of doors not only for native device integration but also to the experience you can provide in a mobile AIR application – with the biggest takeaway (as a developer) being that we no longer have to wait and see what gets exposed to us at the device-level in future AIR SDK releases. We can just write our own native extension now.

Hopefully this article provided some insight on how to quickly get up and running in creating your own Native Extensions and how to incorporate them into your mobile AIR application. And it should be noted that though this example was iOS specific, the native library for an Native Extension for Adobe AIR is in no way restricted to that platform and you should really check out Oliver Goldman’s excellent article, Extending Adobe AIR on the Adobe DevNet.

The source discussed in this article can be found on github at http://github.com/bustardcelly/iBattery and the ibatteryextension.ane itself (if you’d like to use it in your application) can be downloaded from this link.

They are:

1. s:Image
2. ContentCache
3. LinkedList

read on to find out more about them…

1. s:Image

Finally, a Spark equivalent of mx:Image! And with it comes its own skin – ImageSkin – that allows the ability to show loading progress (with its own skin!). I can’t tell you how many times i have made these for client projects. Many – let’s keep it at that. However, the skin contract I would create for these custom “images-with-loading-indicator-components” (as I would call them) defined an mx:Image as the graphics container. The reason being a security issue with trying to manipulate bitmap data added to a BitmapImage.

Fortunately it looks like some updates to BitmapImage have been added as well in Flex 4.5, of which i am assuming clears up the security issues seeing as the skin contract for s:Image defines a BitmapImage as its graphic layer. Maybe I will dig into it later and come up with more info (or if someone reading knows, please tell), but what immediately pops out are the new load events and properties such as trustedSource, contentLoader and bitmapData (which returns a clone).

Usage

<s:Image source="http://upload.wikimedia.org/wikipedia/commons/archive/4/4e/20090913162821!Pleiades_large.jpg"
                width="800" height="600"
                enableLoadingState="true"
                />

Remember how i mentioned contentLoader as a new property for BitmapImage (and the decorating s:Image)? That is typed to an IContentLoader interface, of which ContentCache is an implementation.

2. ContentCache

Many 3rd party libraries have been written for this. I’ve created some in AS2, some in AS3. Basically just a lookup on file access either remote or embedded so as not to load or generate new content – some with load queues, some with instant request. And that is what ContentCache provides – a queueable, cacheable loader for files on a remote resource. Another cool feature is being able to assign a grouping for queued requests and priority in loading – (another property on s:Image and BitmapImage not addressed previously).

If we look at the load() signature on ContentCache:

public function load(source:Object, contentLoaderGrouping:String=null):ContentRequest

we can see the grouping designation associated with the load request as the second argument. The first argument can be either a URLRequest object or a String (well technically you can supply anything there, but it will either be resolved to a URLRequest or String within the load function). We also see that load() returns an instance of ContentRequest. That can either be an active request in the queue or currently running or filled and considered complete from cache.

The content property on ContentRequest is typed as an Object and the docs suggest it can be anything. In the instance of the one returned from ContentCache it looks as though it is always typed to LoaderInfo (the target loader of the request). Pretty cool. So basically, you request ContentCache to load your image file, check for complete on the returned ContentRequest, if false, assign event handlers for complete. When using ContentCache, the content value on the ContentRequest (from what i see) will always be LoaderInfo. Obviously it is flexible enough to create your own IContentLoader implementation to return content of a different type.

Usage

public var cache:ContentCache;
public var requests:Vector.<ContentRequest>;

protected function requestImages():void
{
    cache = new ContentCache();
    cache.prioritize( "walls" );
    cache.enableCaching = true;
    cache.enableQueueing = true;

    requests = new Vector.<ContentRequest>();
    requests.push( cache.load( "http://30.media.tumblr.com/tumblr_loc6v1EmWE1qzpsuoo1_500.jpg", "superheros" ) );
    requests.push( cache.load( "http://26.media.tumblr.com/tumblr_locs8oFznL1qzpsuoo1_400.jpg", "walls" ) );
    requests.push( cache.load( "http://25.media.tumblr.com/tumblr_loc2ysUcfw1qzpsuoo1_500.jpg", "superheros" ) )
    requests.push( cache.load( "http://24.media.tumblr.com/tumblr_loarags4Du1qzpsuoo1_500.jpg", "walls" ) )

    var request:ContentRequest;
    var i:int = requests.length;
    while( --i > -1 )
    {
        request = requests[i];
        if( request.complete )
        {
            requests.splice( i, 1 );
            addImage( (request.content as LoaderInfo).content as Bitmap );
        }
        else
        {
            addRequestHandlers( request );
        }
    }
}		

protected function addRequestHandlers( request:ContentRequest ):void
{
    request.addEventListener( Event.COMPLETE, handleRequestComplete );
}
protected function removeRequestHandlers( request:ContentRequest ):void
{
    request.removeEventListener( Event.COMPLETE, handleRequestComplete );
}

protected function handleRequestComplete( evt:Event ):void
{
    var request:ContentRequest = ( evt.target as ContentRequest );
    var index:int = requests.indexOf( request );
    requests.splice( index, 1 );
    removeRequestHandlers( request );

    var info:LoaderInfo = request.content as LoaderInfo;
    addImage( info.content as Bitmap );
}

protected function addImage( source:Bitmap ):void
{
    var img:BitmapImage = new BitmapImage();
    img.width = source.width;
    img.height = source.height;
    img.source = source;
    // imageHolder is just some container on the display list.
    imageHolder.addElement( img );
}

The Good?

Clean implementation. Remember that there is a contentLoader property on s:Image and BitmapImage now. You can use ContentLoader if it suits your needs, but you can also roll your own implementation of IContentLoader! I like.

The Bad?

Not necessarily bad implementations – we can certainly extend ContentLoader and make any modifications – these are more of things to consider when you are using it:

There is no add() and run() on the API, or at least an autostart argument to load(). Invoking load() immediately starts loading. I may want to build a queue then kick it off. Plus it kind of kills the prioritize, because say your first two calls are #1) without priority association #2) with priority association. The first one is already in the loading process, so #2 does not take priority.

Even with prioritizing, do not rely on the order in which you call load() to be the order in which you will receive complete on the requests. This is mainly due to varying load times, priority, and cacheing. So if you are using ContentLoader as a queue loader, maintain the order outside of the ContentLoader and as requests come in as complete, fill that order accordingly. In essence, ContentLoader should be thought of – in my opinion – more of a cache of requests, rather than a queue loader per se. The enableQueue property does not pertain to “order in, order out” but rather “wait your turn”. The above example utilizes the priority API of ContentLoader to just show an example. If you run that a couple times, you will see what i mean about the order of the queue and priority being not what you would expect.

…

Now what caught my eye as I was checking out ContentCache is that is used linked lists, and more over that LinkedList was now available in the SDK! I’ll be it, in the mx.utils package (why?!) but still.

3. LinkedList

If you are unfamiliar with the concept of linked lists, they are – in real simple terms – a great way to traverse a set of data using a node structure; unlike an array that stores data accessible from element index, a linked list really is more of an access point to nodes that have the knowledge of the next node and – depending on the type of linked list – at times, the previous node.

In the case of the LinkedList from the Flex SDK – which is a doubly linked list – each item in the list points to the previous and next item if existant. So you can imagine, if you want to traverse the list from the first element to the last, you just point the next node from the current node. Not only does each node hold a reference to the next and previous node, but also the data which you are storing. So basically when you add data to a linked list, you are requesting the data be wrapped in a node and hold reference to the previous and next node depending on where you insert it.

There have been some great implementations out there – such as ones found in polygonal’s data structures – and i have built a couple in my day for clients with varying functionality based on requirements.

I should say that it is a great exercise to create your own linked list and I recommend that you should do so. The one included in the SDK is good, but in my opinion very limited (or maybe i should say “lightweight”) and its implementation is a little different than how I would have handled. But, implementation aside, there is a LinkedList in the SDK now. It’s bare bones. No iterator, no traversal API on the list itself, and you traverse by accessing the nodes directly through the list.

Usage:

var list:LinkedList = new LinkedList();
list.push( "foo" );
list.push( "bar" );
list.push( "baz" );

var node:LinkedListNode = list.head;
while( node.next )
{
    trace( "Node value: " + node.value );
    node = node.next;
}

// <<outputs>>
// Node value: foo
// Node value: bar
// Node value: baz

The Good?

If i need a simple linked list, i dont have to create a new one again… yay!

The Bad?

Again, not necessarily bad in implementation but things i may have done differently:

One thing i would prefer is that you don’t have access the LinkedListNode directly. I think that the node wrapper for your data should be hidden or only accessible from the linked list (or iterator). This means that access to the data is provided through another layer of API and when you call things like head(), next(), or previous() you would actually be returned the data you sent to store – not the node. And typically this would be resolved by exposing access to an iterator that provides an API to traverse the linked list. So, the linked list has an API on how to store the data (ie. push(), insertAt(), remove(), etc.) and the iterator has the traversal API (ie. next(), previous(), etc.). But that would just be my way of working with a linked list and I see nothing stopping me from adding that layer myself to this lightweight doubly linked list from the SDK.

Is it fast? I’ll leave that to Jackson Dunstan to find out… I am guessing no – or rather not as fast as it could be if it were part of the player globals like Array and Vector. And why oh why is it stuck in the mx package?!? I don’t mean to complain, but it seems like there is no reason to have LinkedList stuck in the Flex framework. There’s no binding. Its not even in MX collections. Whatever. Maybe it was just a quick implementation to use in request of icon images for speed on Flex mobile.

So in any event, I would say it is a great exercise to roll your own linked list so you have it for any ActionScript-only AND Flex projects. But you can also find some really useful implementations out there.

Conclusion

Nothing really to conclude. I just found these while digging around and became intrigued as they were fairly common things that i implemented over and over with varying functionality on projects based on requirements and now they are available in the Flex 4.5 SDK. If you have found some that you really like, let me know.

Download Source + Presentation

The talk was about the jQuery Mobile framework and was sort of a nuts and bolts overview of what the framework has to offer and its foundation on the principal of Progressive Enhancement. Thanks to all who made it out. I really appreciate it and hopefully it was worth taking you away from the beautiful city for a little bit.

You can also check out a loose transcript of what i *may* have said here: jQuery Mobile: Progressive Enhancement with HTML5 [FATC 2011] transcript. [Warning - its a PDF]

There were more slides than the time allotted and I didn’t want to keep those who were nice enough to attend from exploring the city by watching me ramble on over my time limit. So i can’t guarantee that the transcript was true to the presentation; it was more of a rough draft of my thoughts on the slides.

All the demo source code and a PDF of the slide-deck can be downloaded here as well:
Download Source + Presentation

Thanks again to everyone who came out and to Elad and Jose for putting on another great Flash And The City conference!

Link dump of what i may have mentioned and what is in the slides:

Flash And The City
jQuery Mobile
jQuery Mobile A-Grade browser support
Progressive Enhancement
Selleck Waterfall Sandwich
jQuery Mobile: 960
FamFamFam icons
jQuery UI Theme Roller
WAI-ARIA
Introduction to WAI ARIA by Gez Lemon
Presentation by Scott Jehl of the Filament Group (specifically the Screen Reader demo)
LabJS
yepnope
Modernizr
Web-App Meta
WebClips
HTML5 cache.manifest
PhoneGap
Adobe AIR
Making the most of StageWebView by Sean Voisen
NimbleKit
QuickConnectFamily
Titanium
Rhodes
Infrared5
me on twitter

In this article I am going to address showing a Log In/Sign Up dialog in our jQuery Mobile application. Instead of forcing a user to log in upon first landing on our application (as we saw when setting up Security on our database in the past article), we are going to present the dialog when a user tries to perform an operation that requires session and user validation. The way we have set up the albums database in CouchDB is that everyone can view all the album documents, but only users of the albums database (those assigned with a user-role of “albums-user“) are allowed to add new album documents and only those associated users of a document are allowed to edit and delete their album document. So, from a client-side perspective, the dialog will be shown on Add, Edit and Delete if a previous session for the user has not been established.

Actually, it is probably a little misleading to say we will be doing all this in jQuery Mobile. I am actually going to incorporate jQuery concepts into our jQuery Mobile application. We are going to present the dialog as a jQuery widget and manage the communication through a jQuery plugin for our application. So as far as the jQuery Mobile framework is concerned, we won’t be learning anything new per se – we’ll be learning how to incorporate jQuery widgets and plugins into our application. In previous articles, the main bulk of jQuery Mobile application. We have also gotten familiar with the jquery.couch plugin that comes with CouchDB and handles most (if not all) the communication points we need for our application. So working with the jQuery library is not all that unfamiliar.

Right. Didn’t i say there will be more coding in this article? What am i doing yammering on? Its time to put your jQuery hats on, because its about to get a whole lot more fun*!

*guarantees not included.

Template

We are going to create a login dialog pretty much as your standard form with the option to either “log in” or “sign up” (if i ever don’t use the space in those and it bothers you, i apologize. its become a little habit). In our planning, we currently want the dialog to appear in at least 3 places based on an operation that requires session and user validation – Add, Edit and Delete. Down the road, there could be more. We could add a login button to some or every page to allow to login at any time within the application. We’ll stick to the 3 operations for now, but in knowing that the same visual piece will be used in multiple places throughout the application, it makes for a good case to use a template for the UI of our login dialog.

We learned about templates on the CouchDB side previously – using Mustache and Partials to create our views. Same concept. We want to be able to declare some mark-up in one place that will be rendered in the DOM upon request from anywhere in the application. To do that, we’ll use the jquery.tmpl plugin available at: http://github.com/jquery/jquery-tmpl. Why jquery.tmpl? It is simple to use and – though still in beta – is considered an official jQuery plugin, so documentation can be found on the jQuery site at: http://api.jquery.com/jquery.tmpl/. In any event, make sure to download the jquery.tmpl plugin from http://github.com/jquery/jquery-tmpl as we are going to use it for our login dialog.

With jquery.tmpl downloaded and added to the /vendor/couchapp/_attachments/ folder of our albums coucapp directory (in following the examples within this series, for me that directory is /Documents/workspace/custardbelly/couchdb/albums), open up the loader.js file from that same directory and save the following modification to include the jquery.tmpl:

/vendor/couchapp/_attachments/loader.js

function couchapp_load(scripts) {
  for (var i=0; i < scripts.length; i++) {
    document.write('<script src="'+scripts[i]+'"><\/script>')
  };
};

couchapp_load([
  "/_utils/script/sha1.js",
  "/_utils/script/json2.js",
  "vendor/couchapp/jquery-1.4.4.js",
  "/_utils/script/jquery.couch.js",
  "vendor/couchapp/jquery.couch.app.js",
  "vendor/couchapp/jquery.couch.app.util.js",
  "vendor/couchapp/jquery.mustache.js",
  "vendor/couchapp/jquery.evently.js",
  "vendor/couchapp/jquery.mobile-1.0a2.js",
  "vendor/couchapp/jquery.tmpl.js"
]);

Pretty straight forward; just adding the jquery.tmpl plugin to be loaded into the DOM. Now we need to declare the mark-up for our login dialog. We’ll just add it to our main page and its usage will be revealed later. For now, open up the /_attachments/index.html file and save the following script tag and content between the previously declared script tags for the loader.js and our application script:

<script src="vendor/couchapp/loader.js"></script>
<script id="loginSignupDialog" type="text/x-jquery-tmpl">
    <div role="dialog" data-backbtn="false" class="ui-dialog ui-body-a">
      <div class="ui-header ui-bar-a ui-corner-top ui-overlay-shadow">
	<h1 class="ui-title"></h1>
	<a id="dialogCloseButton" href="#" data-icon="delete" data-iconpos="notext" style="left: 15px; top: .4em; position: absolute;" />
      </div>
      <div data-role="content" class="ui-body-c ui-corner-bottom ui-overlay-shadow">
        <p>You need to be logged in to do that!</p>
        <form action="#" method="post">
          <label for="username">Username:</label>
          <input type="text" name="username" id="username" value=""  />
          <label for="password">Password:</label>
          <input type="password" name="password" id="password" value="" />
          <a id="submitButton" href="#" type="submit" data-role="button" data-theme="b">Submit</a>
          <hr/>
          <a id="optionLinkButton" href="#" />
        </form>
      </div>
      <div data-role="footer" />
    </div>
</script>
<script type="text/javascript" charset="utf-8">
     $db = $.couch.db("albums");

The loginSignupDialog template looks pretty familiar if you have been following along in this series. It is just a hacked up jQuery Mobile page dialog with a form – looks pretty much what our albumdelete.html template we created before but with a form in it. You may notice that the header and optionLinkButton have no textual content. That will be updated based on the state of the dialog (login or signup). We’ll get to that later. What is important is to know that this template declaration can be rendered using the following:

$("#loginSignupDialog").tmpl()

Once it has been rendered it can be added to the DOM as you normally would with jQuery (eg, appendTo()), but since we are developing in the context of a jQuery Mobile application, we will treating it as a page in the application. Things to remember and will act upon later.

Also important to note is the type attribute of the template declaration – text/x-jquery-tmpl. That is so the HTML parser knows to treat that markup as a string and not try to parse and add to the DOM. Without that type, you likely will get a parse error when viewing your application in a browser. The plugin does not search for the explicit type value of text/x-jquery-tmpl in order to due its rendering, it just needs to have some denotation for the browser engine to recognize not to render the content to the DOM – text/x-jquery-tmpl is just easier to recognize what is being reserved as a jQuery template by human readers.

Organization and Re-use

Up to this point in the series, if you have been following along, the extent of our working with jQuery has been in accessing and modifying the DOM. We’ve used the jQuery Mobile API mainly to listen for events and change pages. The extent of working with the jQuery Mobile framework has been more under the hood – which i believe is the intent – though we have employed some hacks to get things to work as we would like. So, the JavaScript we have done up until this point has been to manipulate the DOM associated with a particular view – we created our Partials as quasi-view controllers and we have some script related to the jQuery Mobile pages we have defined in index.html.

That’s fine. I might do some things a little different if this was going to get my seal of approval, but we were having fun. I’m gonna get a little more organized now, however, and use some great functionality and API available to us just by loading jQuery and jQuery Mobile that we have been foolishly ignoring. Not really ignoring… we just never had a really strong use case to address it until now.

We are going to employ two architectural concepts of jQuery in order to present our login dialog and authenticate session with our CouchDB instance: widgets and plugins.

Before we dive into some code, perhaps I should clarify how I interpret each concept as really they can be somewhat interchangeable and used in the same fashion. For instance they both can target an element like so:

$("#myElement").something();

It is typically the something() invocation, for me, that sets them apart (aside from the code behind the something()). If it is a directive to manipulate the element or its ancestry in the DOM, then it is a plugin. If it is a call to decorate the element in such a way that it looks and behaves in a manner usually not associated with that element, then i consider it a widget (in fact, as we will discuss later, there is more to this in terms of a factory/builder plugin for widgets). Say for instance: $(”#myElement”).slider() would manipulate the contents of #myElement to behave like a slider control.

And then of course, plugins can declare a global access variable and define an API, as we have seen with $.mobile (jquery-mobile-1.0a2.js) and $.couch (jquery.couch.js), which takes the plugin concept from being more of a “utility” method on element(s) to that of a library.

Of course, I could be totally off-base in how i interpret these concepts and am open to discussion. If you have different interpretations or more insight please leave a comment.

Widget

Truthfully, a widget is more tied with the jQuery UI library. Included in the development-bundle source for jQuery UI is a jquery.ui.widget script that defines the widget factory/builder and the defined UI widget elements with the jquery.ui.* namespaced file name. We are not going to load the jQuery Mobile basically contains that script from jquery.ui.widget and extends it to mobile.widget to preform some string manipulation on the data attribute. In fact, most of the controls (and even page!) that are “mobilized” are widgets.

So we have $.widget at our disposal (thanks to jQuery Mobile). We’ll use the $.widget factory to create a login dialog widget to provide functional logic to our login template create previously in this article. In this sense, you can think of the login dialog widget, itself, as a sort of presenter. We can go about mucking around with the layout and such of our template and wire up our widget to respond to events from elements in our view.

Before we dive right in to some code, I wanted to touch on some finer points so when we take a look at how our login dialog widget is created we might have a clearer understanding of its construction. $.widget is considered a factory method to create custom widgets and is an extension of $.Widget (capital W). $.Widget itself acts as sort of a builder. Upon widget-ization of an element, _init() and _create() hook methods are invoked and available for override in our custom widget. There are also some methods for options, enablement, and for triggering events (_trigger()) to name a few. But most importantly, there is a destroy() method. To squash any memory leaks, we must tidy up our mess.

So, to break it down, $.Widget provides a lifecycle method template and is the builder of our widget. $.widget is a factory method for creating custom widgets and allows us to widget-ize elements based on the name of our widget. What the hell does that mean? If we create a widget using the $.widget factory and provide it a name of “albums.loginDialog“, we can widget-ize our template as such:

$("#loginSignupDialog").tmpl().loginDialog()

In fact, that is essentially what we are going to do… but we first lets cover how we’re going to do it. For some extra reading, this is a great article form jQuery UI explaining the widget factory: http://jqueryui.com/docs/Developer_Guide. There is also this excellent write up i found by Eric Hynds: http://www.erichynds.com/jquery/tips-for-developing-jquery-ui-widgets/.

Less talk. More code. Open up your favorite editor and save the following snippet as jquery.albums.loginDialog.js in the /_attachments/script directory of the albums couchapp (for me that directory is /Documents/workspace/custardbelly/couchdb/albums):

/_attachments/script/jquery.albums.loginDialog.js

(function( $ ) {

  $.widget( "albums.loginDialog", {

    options: {
      state: 0, // 0 - log in state. 1 - sign up state.
      loginText: "Log In",
      signUpText: "Sign Up"
    },

    _create: function() {
      var ops = this.options;
      var $element = this.element;

      // Current page reference.
      var currentPage = $.mobile.activePage;
      var pageLink = currentPage.attr( "id" );
      // It is an internal page link.
      if( pageLink.indexOf( ".html" ) == -1 ) pageLink = "#" + pageLink;
      var closeButton = $element.find( "div[class*='ui-header'] a#dialogCloseButton" )
                                .attr( "href", pageLink );
      // Hold reference in custom data expando.
      $element.data("previous", pageLink );

      // Wrap the content in a dialog page.
      var wrapper = this._wrapDialog( $element );
      // Wire interactions.
      this._wire();
      this._changeState( ops.state );
      // For some reason, i have to add it to the DOM in order to changePage() to it.
      $("body[class*=\"ui-mobile-viewport\"]").append(wrapper);
      $.mobile.changePage( [currentPage, wrapper], "pop", false );
    },

    _setOption: function( key, value ) {
      this._changeState( ( key == "state" ) ? value : this.options.state );
      jQuery.Widget.prototype._setOption.apply( this, arguments );
    },

    _wrapDialog: function( dialogElement ) {
      // Page wrapper usually created on external page.
      var dialogPage = $("<div data-role=\"page\">");
      dialogPage.append( dialogElement );
      dialogPage.page();

      dialogPage.bind( "pagebeforeshow", function() {
          dialogPage.unbind( "pagebeforeshow" );
          var h = parseFloat(dialogPage.innerHeight());
          h -= ( parseFloat(dialogElement.css("border-top-width")) + parseFloat(dialogElement.css("border-bottom-width")) );
          // define the height based on innerHeight of wrapping parent page and the border styles applied to a dialog.
          dialogElement.css( "height", h + "px" );
      });
      dialogPage.bind( "pagehide", function() {
          dialogPage.unbind( "pagehide" );
          dialogPage.empty();
          dialogPage.remove();
      });
      return dialogPage;
    },

    _changeState: function( state ) {
      var mainText = ( state == 0 ) ? this.options.loginText : this.options.signUpText;
      var optionText = ( state == 0 ) ? this.options.signUpText : this.options.loginText;
      var $element = this.element;
      var header = $element.find( "div[class*='ui-header']" );
      var page = $element.find( "div[data-role='content']" );
      var title = header.find( "[class*='ui-title']" );
      var optionLinkButton =  page.find( "#optionLinkButton" );
      var submitButton = page.find( "a[aria-label='submit']" );
      title.html( mainText );
      optionLinkButton.html( optionText + "?" );
      submitButton.html( mainText );
      submitButton.buttonMarkup();
    },

    _wire: function() {
      var ref = this;
      var $element = ref.element;
      var ops = ref.options;
      var page = $element.find("div[data-role='content']");
      var optionLinkButton = page.find( "#optionLinkButton" );

      optionLinkButton.bind( "click", function(event) {
        event.preventDefault();
        // toggle state.
        ref._setOption( "state", ( ops.state == 1 ) ? 0 : 1 );
        return false;
      });

      $element.bind( "submit", function(event) {
        event.preventDefault();
        var username = $element.find( "input#username" ).val();
        var password = $element.find( "input#password" ).val();
        var uievent = ( ops.state == 0 ) ? "login" : "signup";
        var ui = {name:username, password:password};
        ref._trigger( uievent, {type:uievent}, ui );
        return false;
      });
    },

    _unwire: function() {
      var $element = this.element;
      var page = $element.find( "div[data-role='content']" );
      var optionLinkButton =  page.find( "#optionLinkButton" );
      optionLinkButton.unbind( "click" );
      $element.unbind( "submit" );
    },

    close: function() {
      var $element = this.element;
      this._trigger( "close", {type:"close"} );
      $.mobile.changePage( $element.data("previous"), undefined, false );
    },

    destroy: function() {
      this._unwire();
      // jQuery Mobile keeps adding a Submit button substitue to the template upon show,
      // Lets remove it here.
      var $element = this.element;
      var page = $element.find( "div[data-role='content']" );
      var submitButton = page.find( "a[aria-label='submit']" );
      submitButton.remove();
      // super destroy.
      jQuery.Widget.prototype.destroy.call( this );
      this.element = null;
    }

  });

})(jQuery)

You wanted code? You got it I am not going to go into explaining the under-workings of $.widget or $.Widget but more the work we have here that relates to jQuery Mobile and our application. Again, if you are curious, jQuery UI has some great documentation at http://jqueryui.com/docs/Developer_Guide and you can also look at the JavaScript source for jQuery Mobile. That is chock full of widgets. And then there is this awesome write up by Eric Hynds: http://www.erichynds.com/jquery/tips-for-developing-jquery-ui-widgets/.

Now, let’s step through it… If you are being introduced to jQuery by this article series: a) i hope i have not mislead you in my explanations and b) this might be the first time you have seen this anonymous function declaration:

(function( $ ) {
...
})(jQuery)

There is more going on here (conflict resolution) than i will explain, but in essence this is a self-executing method that has a reference to jQuery. Basically, upon load any code within this anonymous function will be run and have access to jQuery using the $ token; once jquery.albums.loginDialog.js is loaded, we invoke the $.widget factory method to create a widget accessible via loginDialog() on an element reference:

/_attachments/script/jquery.albums.loginDialog.js

$.widget( "albums.loginDialog", {
...
} );

The second argument to $.widget – the whole big object – is basically the meat of our widget; its got some declared default options, overidden inherited methods and some other private and public methods to make our loginDialog work. Our options are just some default values to set the initial state (either to login or signup) and the textual content associated with the state. Our login dialog isn’t going to be very pretty or contain much content other than a form and a way to switch between login and signup.

/_attachments/script/jquery.albums.loginDialog.js

options: {
  state: 0, // 0 - log in state. 1 - sign up state.
  loginText: "Log In",
  signUpText: "Sign Up"
},

When it enters _create() (inheritance call from $.Widget), is when we do some real magic:

/_attachments/script/jquery.albums.loginDialog.js

var $element = this.element;

// Current page reference.
var currentPage = $.mobile.activePage;
var pageLink = currentPage.attr( "id" );
// It is an internal page link.
if( pageLink.indexOf( ".html" ) == -1 ) pageLink = "#" + pageLink;
var closeButton = $element.find( "div[class*='ui-header'] a#dialogCloseButton" )
                          .attr( "href", pageLink );
// Hold reference in custom data expando.
$element.data("previous", pageLink );

The target element (the one widget-ized by calling $(”#myElement”).loginDialog()) is accessed using the this keyword and is available through the life of the widget. In order to get a reference to the current page prior to opening the loginDialog, we access the id value of $.mobile.activePage and determine if it is an external or internal page. We then pass that as the href value for the close button on the loginDialog and add a data-previous attribute to the target element.

After the previous page has been determined to return to after login or signup, we then wrap the dialog template element in a page div listen for events, change the state to the default defined in options and change the page to the loginDialog:

/_attachments/script/jquery.albums.loginDialog.js

// Wrap the content in a dialog page.
var wrapper = this._wrapDialog( $element );
// Wire interactions.
this._wire();
this._changeState( ops.state );
// For some reason, i have to add it to the DOM in order to changePage() to it.
$("body[class*=\"ui-mobile-viewport\"]").append(wrapper);
$.mobile.changePage( [currentPage, wrapper], "pop", false );

You see that funky append before $.mobile.changePage()?

/_attachments/script/jquery.albums.loginDialog.js

$("body[class*=\"ui-mobile-viewport\"]").append(wrapper);

The comment above that line explains it, but for some reason i couldn’t just switch to this dynamically created page. Instead i had to add it quickly at the end of the body (accessed by the ui-mobile-viewport class) and then was able to switch from the previous page to show a loginDialog. No big deal, just something to look out for.

Most of what is in jquery.albums.loginDialog, if you have been following along, may be pretty familiar to you. We do the same sizing on pagebeforeshow and emptying on pagehide, and change values based on state using jQuery. Your basic fanfare. I just wanted to touch on two more methods in jquery.albums.loginDialog: close and destroy:

/_attachments/script/jquery.albums.loginDialog.js

close: function() {
var $element = this.element;
this._trigger( "close", {type:"close"} );
$.mobile.changePage( $element.data("previous"), undefined, false );
},

destroy: function() {
  this._unwire();
  // jQuery Mobile keeps adding a Submit button substitue to the template upon show,
  // Lets remove it here.
  var $element = this.element;
  var page = $element.find( "div[data-role='content']" );
  var submitButton = page.find( "a[aria-label='submit']" );
  submitButton.remove();
  // super destroy.
  jQuery.Widget.prototype.destroy.call( this );
  this.element = null;
}

The close() method we have exposed and is not part of $.Widget. That just allows access to anyone with a reference to the created loginDialog to directly close it (as opposed to explicitly closing it within the dialog). You can see that we change the page by using the previous data value set on the element within _create(). We also dispatch a “close” event using _trigger(). The _trigger() method is part of $.Widget and can be bound to like other events, yet the type is slightly different within the context of a widget. The type value is actually appended to the widget name, so if you were to listen for close on this:

$("#loginSignupDialog").tmpl().loginDialog().bind( "logindialogclose", handleLoginDialogClose );

That’s one way to do it, though i often assign handlers by passing in a callback object like so:

$("#loginSignupDialog").tmpl().loginDialog( {
        close: function( evt, ui ) {
            ...
        }
});

We actually trigger two other events within our loginDialog – “login” and “signup” – which are dispatched upon submit based on current state. We’ll see how all this is handled a little later on in more code, but i quickly wanted to touch on something that looks odd in the destroy() override:

/_attachments/script/jquery.albums.loginDialog.js

var $element = this.element;
var page = $element.find( "div[data-role='content']" );
var submitButton = page.find( "a[aria-label='submit']" );
submitButton.remove();

When a form is added top the DOM through jQuery Mobile, it actually duplicates and hides the originally declared submit button and decorates a that duplicate for display. In that decoration, it is given the aria-label attribute for accessibility. Not entirely a huge issue, but if you were to show the template form more than once, you would start to see n+ times the amount of submit buttons! So we just remove it in our destroy override and move on.

Alright. Hopefully you are still with me. You may or may not have created your first jQuery widget. Isn’t it glorious or not glorious, respectively? With our template and widget defined, now begs the question: When do we show this thing?

Plugin

So you thought introducing two new concepts (jQuery templates and widgets) was enough for one article? Well, in the words of television’s Emeril, ‘We’re going to kick it up a notch!‘ That’s actually probably a paraphrase. I don’t know the exact words he said and i don’t feel like googling it. I was just pandering to the developer crowd who enjoys watching cooking shows… Its a low point in this article and i am not proud of it. Let’s keep going.

If we step back and remember why we started doing all this business in this mini-series, we wanted to use validation on operations to our database documents. In the previous article we spoke of user contexts, and updated our album document to require a user field. So, our main intent with this loginDialog is to verify a current session and then either login or signup a user. If the session is valid (user previously logged in) then, the user reference is stored and – dependent on the action – used in further transactions.

Now, we could just open the loginDialog all willy-nilly, here and there, and try to track down the persisted user across actions, but a saner approach (for me at least) is to encapsulate this logic in a plugin. You may be familiar with jQuery plugins… actually we have been using them throughout this whole series. We just haven’t directly looked at how they are made or work. Like I did with our widget example, I am not going to go into a discussion of jQuery plugins per se – I am going to try and focus on the task at hand and provide explanations as to how it pertains to our application as a whole. That said, to read more about jQuery plugins, this is always a great place to start: http://docs.jquery.com/Plugins/Authoring.

Alright, we are going to take a little piecemeal approach to fleshing out our plugin, explaining as we go along, and I’ll provide it in its entirety afterward. To start, open up your favorite editor, create a file named jquery.albums.app.js and save it in the /_attachments/script folder of our albums couchapp directory. Add the following:

/_attachments/script/jquery.albums.app.js

(function($) {

  $.albums = $.albums || {};
  $.extend( $.albums, {
  });

})(jQuery)

That is the start of our albums plugin which will be accessible by $.albums. We’re extending our (hopefully) newly created $.albums object to flesh out its public API. $.albums will serve as a facade/adaptor to the $.couch plugin and handle the display and event from the loginDialog. So there are roughly five main functions of out $.albums plugin:

• Log In – Direct log in request.
• Log Out – Direct log out request
• Save Document – Request to Create or Update a document.
• Delete Document – Request to Delete a document.

The Log In and Log Out methods mainly interface directly with $.couch and won’t deal with any UI. They are just facades to track the logged in user so we can properly send user data when creating or updating documents. The Save Document and Delete Document methods are more adaptors for $.couch communication – checking session and presenting the loginDialog if necessary. Update the jquery.albums.app.js with the following:

/_attachments/script/jquery.albums.app.js

(function($) {

  $.albums = $.albums || {};
  $.extend( $.albums, {

    dialog: undefined,
    database: undefined,

    logIn: function( name, password, options ) {
      doLogIn( name, password, options );
    },

    logOut: function( options ) {
      doLogOut( options );
    },

    saveDocument: function( document, options ) {
      checkSession( {
        available: function( userCtx ) {
          document.user = user;
          $.albums.database.saveDoc( document, options );
        },
        unavailable: function() {
          showDialog( options );
        }
      });
    },

    deleteDocument: function( document, options ) {
      checkSession( {
        available: function( userCtx ) {
          $.albums.database.removeDoc( document, options );
        },
        unavailable: function() {
          showDialog( options );
        }
      });
    }

  });

})(jQuery)

So there we have our 4 public methods of $.albums. We have also added to public properties: dialog and database. These two will be the targets to the loginDialog template and a reference to our albums database, respectively. This is so we don’t rely so heavily on globally declared variables to do what we need in our nice little encapsulated plugin. In any event, if you have looked at the contents of these methods, they all call other functions that we haven’t declared yet. Lets start with checkSession() in saveDocument and deleteDocument. Add the following after the $.extend declaration and prior to close of function():

/_attachments/script/jquery.albums.app.js

function checkSession( options ) {
    options = options || {};
    $.couch.session( {
      success: function( response ) {
        var context = response.userCtx;
        if( context.name == null ) {
          if( options.unavailable ) options.unavailable();
        }
        else{
          if( options.available ) options.available( context );
        }
      },
      error: function( status, error, reason ) {
        if( options.unavailable ) options.unavailable();
      }
    });
}

The checkSession() does just that: request the current session established by our client through the $.couch plugin. If the userCtx comes back as null, there is no session and no logged in user on our end. We can try this out using curl on the command line:

> curl -vX GET http://127.0.0.1:5984/_session
< {"ok":true,"userCtx":{"name":null,"roles":[]},"info":{"authentication_db":"_users","authentication_handlers":["oauth","cookie","default"]}}

Depending on the value of userCtx, checkSession() will invoke an available() or unavailable() callback method on the anonymous options argument. The available responder on saveDocument() and deleteDocument() is different – one saves, the other deletes – but if unavailable is entered, they both call showDialog(). Append the following script to jquery.albums.app.js after the checkSession() declaration:

/_attachments/script/jquery.albums.app.js

function showDialog( options ) {
    $.albums.dialog.loginDialog( {
      login: function( evt, ui ) {
        doLogIn( ui.name, ui.password, {
          success: function( response ) {
            $.albums.dialog.loginDialog( "close" );
          },
          error: function( status, error, reason ) {
            alert( "Error: " + error + ", " + reason );
          }
        });
      },
      signup: function( evt, ui ) {
        doSignUp( ui.name, ui.password, {
          success: function( response ) {
            $.albums.dialog.loginDialog( "close" );
          },
          error: function( status, error, reason ) {
            alert( "Error: " + error + ", " + reason );
          }
        });
      }
    });
}

Hey! That’s using our widget! I don’t know why i seriously get excited when i see that… In any event, there it is in all its glory. The login template will be handed to $.albums and we’ll widget-ize it within the showDialog() invocation:

$.albums.dialog.loginDialog( {
...
});

If you remember back when we created the widget, it will dispatch two events when submitted based on state: login and signup. Here we have added the callbacks to those events which in turn call either doLogIn() or doSignUp(). We’ll get to that in a bit, but i wanted to point out the success responders in the anonymous callback objects we pass to those methods:

success: function( response ) {
    $.albums.dialog.loginDialog( "close" );
}

If we have passed being logged in/signed up, we close the loginDialog. We exposed the close() method in jquery.albums.loginDialog.js, but its important to note that you cannot call close() using dot-notation like a property. The reason is that we are not holding a reference to the loginDialog. Calling $(”#myElement”).loginDialog() just decorates the target element on the DOM. We can still interface with $(”#myElement”) but those methods associated with the widget are not then accessible on the target element. So we call a public method through loginDialog().

OK. So doLogin(), doLogout() and doSignUp() are yet to be covered. For some reason i have this convention where if it is considered an internal response to a public invocation i prepend ‘do’ to the public method name. Some people prefer “_“. To each his own. Just if you are wondering… Let’s tackle the login and logout first, as they are easy. Add the following script to jquery.albums.app.js somewhere after the $.extend declaration and prior to the end of the function() block:

/_attachments/script/jquery.albums.app.js

var user; /* _user > document.name */

function doLogIn( name, password, options ) {
    options = options || {};
    var loginObj = {
      name:name,
      password:password,
      success: function( response ) {
        user = response.name;
        if( options.success ) options.success( response );
      },
      error: function( status, error, reason ) {
        if( options.error ) options.error( status, error, reason );
      }
    };
    $.couch.login( loginObj );
}

function doLogOut( options ) {
    options = options || {};
    $.couch.logout( {
      success: function( response ) {
        user = undefined;
        if( options.success ) options.success( response );
      },
      error: function( status, error, reason ) {
        if( options.error ) options.error( status, error, reason );
      }
    })
}

Again, we are just interfacing with the $.couch plugin to perform the login() and logout(). The only reason we go through $.albums instead of $.couch directly is to maintain the user, which we have privately declared. That user value is the one assigned to the user field of a document when a new album document will be created and saved through $.albums using saveDocument().

Alright, Sign Up is where the magic happens. Signing a user up through $.couch is a little trickier. In Futon it is all easy-peasy as we saw in the last article in this mini-series in a series. Now that we have taken our CouchDB instance out of Admin Party mode, the $.couch.signUp() method won’t work unless we are logged in as an admin. We don’t want to take the approach of Futon for the User Experience for our little application here. Our application shouldn’t act as an admin console to the CouchDB database as well as its current function of just adding, updating and deleting album documents.

We going to secretly log in as an admin behind the scenes and set our new user up for gold and start adding documents. So, Sign Up is actually going to be a series of commands just to sign someone up, assign their proper user-role and create their session to allow them to start working with album documents. That sequence in readable terms is:

1. Log In as Admin
2. Sign Up new User
3. Open new User document
4. Add “albums-user” User Role to new User
5. Log Out as Admin
6. Log In as new User

Maybe the jquery.couch plugin has changed since the last revision i checked out, but that is the order of things to signup and login a new user as far as i understand. The signature for $.couch.signUp() method has no arguments for administration to do this more streamlined, so we are going to create a chain of commands to signup a new user a little more efficiently. Now, in actuality, i would turn to a server-side developer and ask pretty-please that they implement a proxy in whatever language to do this and not handle it all in JavaScript (especially since we are going to expose our admin credentials – HIDE YOUR CHILDREN!), but we’re having fun and not going to production with this, right? We’re taking some liberties to explore what we have…

Now, if you have not figured out yet in following these articles, i can get a little anal. But it isn’t about everything. I seem to get focused on one thing that really irks me. And one of those things is deeply nested anonymous callbacks. Sure i am guilty of doing it and when i do it i feel dirty, but i usually let it go if its not nesting too deep. This sequence of actions for Sign Up, however, i can foresee being really ugly if we just went with anonymous callback objects down the line. So what i did was create Queue and Command “classes” to keep my sanity. The following is the script for that. Open a new document in your favorite editor (we’ll get back to jquery.albums.app.js in a bit) and save the following script as command_queue.js in /_attachments/script:

/_attachments/script/command_queue.js

(function(window) {

    function Commandable() {}
    Commandable.prototype.execute = function( data ){};

    function Queue( ops ) {
        this.options = ops || {};
        this.list = [];
        this.command;

        this.addCommand = function( command ) {
            var length = this.list.length;
            if( length > 0 ) {
                this.list[length-1].nextCommand = this;
            }
            this.list.push( command );
        }
    }
    var q = Queue.prototype = new Commandable();
    q.options = undefined;
    q.list = undefined;
    q.command = undefined;
    q.constructor = Queue;
    q.execute = function( data ) {
        if( this.list.length > 0 ) {
            this.command = this.list.shift();
            this.command.execute( data );
        }
        else {
            this.command = undefined;
            if( this.options.complete ) {
                this.options.complete();
            }
        }
    }

    function Command( target, args, ops ) {
        this.target = target;
        this.args = args || [];
        this.options = ops || {};
        this.nextCommand = undefined;
    }
    var c = Command.prototype = new Commandable();
    c.constructor = Command;
    c.getCommandOptions = function( options, nextCommand ) {
        var responder = {
            ops: options,
            success: function( response ) {
                if( options && options.success ) options.success( response );
                if( nextCommand ) {
                    nextCommand.execute( response );
                }
            },
            error: function( status, error, reason ) {
                if( options && options.error ) options.error( status, error, reason );
            }
        };
        return $.extend( {}, this.options, responder );
    }
    c.execute = function( data ) {
        this.args.push( this.getCommandOptions( this.options, this.nextCommand ) );
        this.target.apply( this, this.args );
    }

window.Queue = Queue;
window.Command = Command;

}(window));

I’m not going to say it’s sophisticated by any means, but this is a simple implementation to allow us to chain commands together. That is done by adding Commands to the Queue using addCommand(), which then appends a command using the nextCommand property of the previous (if available) Command. Both Queue and Command have a method called execute() (as extensions of Commandable). Executing a Queue will start the chain of commands; executing a Command will invoke whatever target method supplied in the constructor. If this needs more explanation, please leave a comment. We’ll see how we use this by implementing our doSignUp() method in jquery.albums.app.js. Open up jquery.albums.app.js in your favorite editor and add the following script somewhere after the $.execute declaration and before the end of the function() block:

/_attachments/script/jquery.albums.app.js

function doSignUp( name, password, options ) {
    options = options || {};
    var queueOptions = {
      complete: function() {
        if( options.success ) options.success();
      }
    }
    var queue = new Queue( queueOptions );
    queue.addCommand( new Command( $.albums.logIn, ["toddanderson", "admin123"] ) );
    queue.addCommand( new Command( $.couch.signup, [{name:name}, password] ) );
    queue.addCommand( new OpenUserDocCommand() );
    queue.addCommand( new AssignRoleCommand() );
    queue.addCommand( new Command( $.albums.logOut ) );
    queue.addCommand( new Command( $.albums.logIn, [name, password], options ) );
    queue.execute();
}

There we have our sequence of command described earlier. You can see how we pass the target method and arguments to the Command to be executed and the queue of commands is assembled using Queue:addCommand(). Again, our admin credentials are exposed and available to all – I do not recommend this practice in real life. Its just us here. So, there are two commands there in the middle that we haven’t discussed and aren’t declared in command_queue.js. They are specific to our albums application, so we will define them in jquery.albums.app.js. With the file still open, add the following script after (or before) the doSignUp() declaration:

/_attachments/script/jquery.albums.app.js

function OpenUserDocCommand( target, args, options ) {}
OpenUserDocCommand.prototype = new Command();
OpenUserDocCommand.prototype.execute = function( data ) {
      var $userDB = $.couch.db("_users");
      $userDB.openDoc( data.id, this.getCommandOptions( this.options, this.nextCommand ) );
}

function AssignRoleCommand( target, args, options ) {}
AssignRoleCommand.prototype = new Command();
AssignRoleCommand.prototype.execute = function( data ) {
      data.roles = ["albums-user"];
      var $userDB = $.couch.db("_users");
      $userDB.saveDoc( data, this.getCommandOptions( this.options, this.nextCommand ) );
}

Nothing too crazy going on in OpenUserDocCommand or AssignRoleCommand, just needed to do some work with the _users database of our CouchDB instance as an admin.

Guess what? That’s it! We’re done with jquery.albums.app.js. Whew! If you have stuck around, i am quite humbled. Seriously. A lot of code and rambling on. I don’t know if i would have made it. Anyway, here is jquery.albums.app.js in full:

/_attachments/script/jquery.albums.app.js

(function($) {

  $.albums = $.albums || {};
  $.extend( $.albums, {

    dialog: undefined,
    database: undefined,

    logIn: function( name, password, options ) {
      doLogIn( name, password, options );
    },

    logOut: function( options ) {
      doLogOut( options );
    },

    saveDocument: function( document, options ) {
      checkSession( {
        available: function( userCtx ) {
          document.user = user;
          $.albums.database.saveDoc( document, options );
        },
        unavailable: function() {
          showDialog( options );
        }
      });
    },

    deleteDocument: function( document, options ) {
      checkSession( {
        available: function( userCtx ) {
          $.albums.database.removeDoc( document, options );
        },
        unavailable: function() {
          showDialog( options );
        }
      });
    }

  });

  var user; /* _user > document.name */

  function OpenUserDocCommand( target, args, options ) {}
  OpenUserDocCommand.prototype = new Command();
  OpenUserDocCommand.prototype.execute = function( data ) {
      var $userDB = $.couch.db("_users");
      $userDB.openDoc( data.id, this.getCommandOptions( this.options, this.nextCommand ) );
  }

  function AssignRoleCommand( target, args, options ) {}
  AssignRoleCommand.prototype = new Command();
  AssignRoleCommand.prototype.execute = function( data ) {
      data.roles = ["albums-user"];
      var $userDB = $.couch.db("_users");
      $userDB.saveDoc( data, this.getCommandOptions( this.options, this.nextCommand ) );
  }

  function checkSession( options ) {
    options = options || {};
    $.couch.session( {
      success: function( response ) {
        var context = response.userCtx;
        if( context.name == null ) {
          if( options.unavailable ) options.unavailable();
        }
        else{
          if( options.available ) options.available( context );
        }
      },
      error: function( status, error, reason ) {
        if( options.unavailable ) options.unavailable();
      }
    });
  }

  function doLogIn( name, password, options ) {
    options = options || {};
    var loginObj = {
      name:name,
      password:password,
      success: function( response ) {
        user = response.name;
        if( options.success ) options.success( response );
      },
      error: function( status, error, reason ) {
        if( options.error ) options.error( status, error, reason );
      }
    };
    $.couch.login( loginObj );
  }

  function doLogOut( options ) {
    options = options || {};
    $.couch.logout( {
      success: function( response ) {
        user = undefined;
        if( options.success ) options.success( response );
      },
      error: function( status, error, reason ) {
        if( options.error ) options.error( status, error, reason );
      }
    })
  }

  function doSignUp( name, password, options ) {
    options = options || {};
    var queueOptions = {
      complete: function() {
        if( options.success ) options.success();
      }
    }
    var queue = new Queue( queueOptions );
    queue.addCommand( new Command( $.albums.logIn, ["toddanderson", "admin123"] ) );
    queue.addCommand( new Command( $.couch.signup, [{name:name}, password] ) );
    queue.addCommand( new OpenUserDocCommand() );
    queue.addCommand( new AssignRoleCommand() );
    queue.addCommand( new Command( $.albums.logOut ) );
    queue.addCommand( new Command( $.albums.logIn, [name, password], options ) );
    queue.execute();
  }

  function showDialog( options ) {
    $.albums.dialog.loginDialog( {
      login: function( evt, ui ) {
        doLogIn( ui.name, ui.password, {
          success: function( response ) {
            $.albums.dialog.loginDialog( "close" );
          },
          error: function( status, error, reason ) {
            alert( "Error: " + error + ", " + reason );
          }
        });
      },
      signup: function( evt, ui ) {
        doSignUp( ui.name, ui.password, {
          success: function( response ) {
            $.albums.dialog.loginDialog( "close" );
          },
          error: function( status, error, reason ) {
            alert( "Error: " + error + ", " + reason );
          }
        });
      }
    });
  }

})(jQuery)

Now before you run off, kiss your significant other and tell him/her that you will soon be billionaires, we have a little more work to do to get this up and running in our Albums application.

Loader and Index

We kind of went off the deep-end there and dove right into code that code stand on its own outside of our current design of the Albums application. That’s good, but now we gotta reel it back in and wire it up. To start, lets add our new scripts in the loader. Open up /vendor/couchapp/_attachments/loader.js and save the following modifications:

/vendor/couchapp/_attachments/loader.js

function couchapp_load(scripts) {
  for (var i=0; i < scripts.length; i++) {
    document.write('<script src="'+scripts[i]+'"><\/script>')
  };
};

couchapp_load([
  "/_utils/script/sha1.js",
  "/_utils/script/json2.js",
  "vendor/couchapp/jquery-1.4.4.js",
  "/_utils/script/jquery.couch.js",
  "vendor/couchapp/jquery.couch.app.js",
  "vendor/couchapp/jquery.couch.app.util.js",
  "vendor/couchapp/jquery.mustache.js",
  "vendor/couchapp/jquery.evently.js",
  "vendor/couchapp/jquery.mobile-1.0a2.js",
  "vendor/couchapp/jquery.tmpl.js",
  "script/command_queue.js",
  "script/jquery.albums.app.js",
  "script/jquery.albums.loginDialog.js"
]);

We’ve just added the last three files we have been working on: the command_queue script and our $.albums plugin and loginDialog widget. Now we’re going to make some modifications to the inline script on our index.html document. In a previous article we hooked up the saving of a new album document to the internal jQuery Mobile addAlbum page, and were using the jquery.couch plugin directly to save the document. Now we are just going to go through our $.albums plugin to perform that action which will open the loginDialog if a session is currently not available on our client. Open the /_attachments/index.html file in your favorite editor and save the following modifications in the handleDocumentReady() function:

/attachments/index.html

function handleDocumentReady()
{
    $("#home").bind( "pagebeforeshow", refreshAlbums );
    refreshAlbums();

    // Set database reference and dialog template on albums.
    $.extend( $.albums, {
        database: $db,
        dialog:  $("#loginSignupDialog").tmpl()
    });

    $("#addSubmitButton").live( "click", function( event ) {
        event.preventDefault();
        var document = {};
        document.artist = $("input#addArtistField").val();
        document.title = $("input#addTitleField").val();
        document.description = $("textarea#addDescriptionField").val();
        document.creation_date = ( new Date() ).getTime();
        $.albums.saveDocument( document, {
            success: function() {
                $.mobile.changePage( "#home", "slidedown", true, true );
            },
            error: function( status, error, reason ) {
                alert( "Cannot save new document.\n" + status + ", " + reason + ", " + error );
            }
        });
        return false;
    });

    $("#addAlbum").bind( "pagehide", function() {
        $("input#addArtistField").val( "" );
        $("input#addTitleField").val( "" );
        $("textarea#addDescriptionField").val( "" );
    });
}

Not that much to change. First we assign the default properties for the dialog template and database reference for our $.albums plugin and then we replace the call to save the document using jquery.couch to that of our $.albums plugin. Ta-da! Now we can add new documents as a logged in user.

There is, however, two more places that we need to interact with our $.albums plugin: the edit page and the delete page. Without modifying those and with the new authorization and validation we have introduced, we’d only be able to create and read documents…

Oh nos! There are tons of albums that i entered in late at night in a different state of mind! I am second guessing my decision on wanting Hall & Oates‘ War Babies!

These are the type of exclamations i am assuming are going through your head… because i would never…

album-delete and album-edit

If you go way back in this article series and have been following along, you may remember that we create quasi-view controllers for our couchapp template pages and included the as partials in the show function. There are two that we need to modify with our recent changes to using the $.albums plugin to verify session: /_attachments/script/album-delete-dialog.js and /_attachments/script/album-edit-page.js. They will be small changes, but once deployed will give our application a little more security on who can perform modifications on album documents.

Open up /_attachments/script/album-delete-dialog.js in your favorite editor and save the following change to the handleDelete() function:

/_attachments/script/album-delete-dialog.js

function handleDelete( event )
{
    event.preventDefault();
    var docId = $("#dialogContent").data("identity");
    // First open doc based on ID in order to get full document.
    $db.openDoc( docId, {
        success: function( document ) {
            // Then use the opened doc as reference to remove.
            $.albums.deleteDocument( document, {
                success: function() {
                    $.mobile.changePage( "#home", "slide", true, true );
                },
                error: function() {
                    alert( "Could not remove document with id: " + docId );
                }
            });
        },
        error: function() {
            alert( "Could not find document with id: " + docId );
        }
    });
    return false;
}

Open up /_attachments/script/album-edit-page.js in your favorite editor and save the following change to saveDocument():

/_attachments/script/album-edit-page.js

function saveDocument( document )
{
    $.albums.saveDocument( document, {
        success: function( response )  {
            updateEditableAlbum( document );
            navigateToAlbumPage( document._id );
        },
        error: function( status, error, reason ) {
            alert( "Cannot save document: " + document._id + "\n" + reason );
        }
    });
}

That’s all! basically moved away from actions through the reference to our albums database to using the $.albums plugin. Finally. Now let’s push all this to our CouchDB instance so we can play around with it.

Deployment

If you remember from the last article when we introduced authorization, we need to pass our credentials when we push our couchapp to the CouchDB instance:

couchapp push albums http://toddanderson:admin123@127.0.0.1:5984/albums

You’ll will have to replace to username and password, but once we have pushed we can now checkout our application at http://127.0.0.1:5984/albums/_design/albums/index.html. If you visit your application and try to either add, edit or delete a document and have not previously logged in, you will see something on the following:

[Log In]

[Sign Up]

Conclusion

We have come a long way! We now have authorization and user validation in our jQuery Mobile albums application ensuring that a user must be logged in to create a new album document and that the owner of a document is the only one that can make modifications or delete the document from the database. Fun stuff.

I think this is going to be the last article of this series. Of course if this application were to go live, it would need a lot more work. For instance, the ability to log in and log out persisted on every page and information about who is logged in, etc. – but i will leave that up to you if you want to keep going. Hope you had fun and gleaned some useful information that i hope is not misleading.

Thanks again if you have actually took the time to follow along in this series. I am truly humbled if you sat through it all.

Cheers!

[Note] This post was written against the following software versions:
CouchDB – 1.0.1
CouchApp – 0.7.2
jQuery – 1.4.4
jQuery Mobile – 1.0a2
jQuery Templates Plugin 1.0.0pre
If you have found this post and any piece has moved forward, hopefully the examples are still viable/useful. I will not be updating the examples in this post in parellel with updates to any of the previously mentioned software, unless explicitly noted.

Articles in this series:

1. Getting Started
2. Displaying a page detail of a single album.
3. Templates and Mustache
4. Displaying an editable page of an album.
5. Creating and Adding an album document.
6. Deleting an album document
7. Authorization and Validation – Part 1
8. Authorization and Validation – Part 2

Full source for albums couchapp available here

Wait… so what’s with the versioning of Part 7 in the title?! Its a multipart article of an article series. Barring i don’t break the space-time continuum for breaking up an article into sections, introducing authorization and validation will be spread over a couple posts. There is a lot of information involving set-up and development, plus we will jump into some new concepts – such as jQuery widgets and plugins – that may be a little too much information to digest in a single article.

In the first installment of this multi-part article I intend to address authorization and validation of our document operations so as to not let any old joey-bagadonuts hit the application and start adding, deleting and modifying our precious album list we have been painstakingly curating. If you have been following along, you may have noticed that we have thrown security and user-permitted actions out the window. This article won’t address the real breadth of security that your CouchDB instance should employ, but I will cover moving away from our admin party (anybody can do anything) and introduce the concept of user contexts, as well as how they can be used to validate operations requested for documents in the albums database of our CouchDB instance.

Light Reading

Before I start, there are two excellent articles out there that address authorization and validation in CouchDB that are indespensible. I read them before I started this series while working on another project, but came back to them as a refresher. I recommend you go check them out as they provide a more concise explanation of the work we are about to do (i’ll just touch on the finer points to get this application up and running). They are:

CouchOne – What’s new in CouchDB 1.0 — Part 4: Security’n stuff: Users, Authentication, Authorisation and Permissions
CouchDB The Definitive Guide – Validation Functions

While those articles are an excellent resource for the task at hand, the sites themselves have a wealth of information which I highly suggest perusing, as well. Alright, armed with a little knowledge, lets dig in.

Fix This

If you have already taken your CouchDB instance out of admin party, you can skip this section or read on.

If you have been following along in this series and have only been using the command line to interact with CouchDB, there is a utility application called Futon that ships with CouchDB. I actually browse my CouchDB instance using Futon in CouchOne (neé CouchDBX for Mac). If you have ever visited http://127.0.0.1:5984/_utils, that’s Futon. If you have a version of CouchOne running on your machine, that shows Futon within the browse window. I recommend using CouchOne for local development as it makes it easier to start and stop the service as well as pretty prints out the http calls (not as verbose as i would like, but still useful).

So if you’ve visited Futon and have been running our application that we have built along in the series under admin party (ie. all access), then you may see something of the sort on the right hand side of Futon. Notice the bottom portion:

[Futon side panel]

We intend to Fix this.

Click that link, and a dialog should appear looking like the following:

[Create Server Admin]

Make sure to read what is in that dialog as it is useful and i will not reiterate its information here… we’re just trying to get back to coding people! Enter in whatever username and password you choose. For the purposes of this article series (as the information will be used later) I entered:

u: toddanderson
p: admin123

Click Create to create the admin user and relax. What just happened is that you created an admin that can now do everything everybody used to be able to do. That username and hashed password are now saved in /etc/couchdb/local.ini of your CouchDB instance and is viewable in Futon when you go to Configuration:

[Configuration panel]

I have to admit, I am doing this all backtracking. Meaning I took my CouchDB instance off of admin party some time ago. So I am trusting this is still the way to go about it. And I am hoping that since CouchDB 1.0+ there has been a _users database. To check if there is within Futon, go to Overview and one of the first databases listed should be _users. If its there, great! It should probably even have its authentication validation view included so we are all set, and the admin you just created may or may not be automatically entered as a user (i have on mine).

If _users is not available for you… leave a comment and we can work something out.

Creating Users

Remember that rush of power you had when CouchDB was running under admin party? That light may have gone out once you created an admin… but think of it – now you possess the control to create/update/delete as many users as you want. Isn’t it glorious. Now we are going to reign it in a bit and create a new user.

The easiest way to set up a new user is through Futon. If you are still logged in as the admin you just created, log out from the right hand panel and use the Signup link:

[FutonSign Up]

That will open up the Sign Up dialog allowing you to enter a username and password:

[Create User Dialog]

I am creating a new user named custardbelly with a super-secret password that will be salted using sha1. That is all handled by the jquery.couch plugin (the same one we have been using in the examples in this series) employed by Futon. If we open up the new user from the _users database we should see the following:

[User Entry]

There are a couple things to note here. First off, I took that screen shot after i already assigned a role (“albums-user”) to it. My bad. To do that for you new user, just click on the roles value field and enter “albums-user” (with quotes). That role assignment will be used later when validating documents, so don’t think too hard about it right now.

The other things of note are the auto-generated type field with a value of “user”, and the password_sha and salt fields – auto-generated and populated with values created through the jquery.couch plugin during signup. The _id for a user also has to have the form of reverse-domain for a couchdb user: org.couchdb.user:custardbelly. It must be in that format to be a valid user id so if you ever go about creating a new User manually, either from the _user database New Document option or from the command line, keep that in mind. Last, but not least, the name field is populated with the value of the username (the one entered in the Sign Up dialog through Futon and the value at the end of the _id property). The name property is commonly used in validating documents against a user context which we will get into a little later.

That said, you could create a new user using the database controls in Futon or the command line, but some extra work would be needed to create that sha1 encrypted password. It would be neat to have a script that would do that and output a json file that can be passed in the command line creation of a User, but i won’t go into that right now. Just so you can see what that would look like, the following is an example of how to create a user using the command line:

curl -vX PUT http://toddanderson:admin123@127.0.0.1:5984/_users/org.couchdb.user:custardbelly -d '{"name":"custardbelly", "roles":["albums-user"], "type":"user", "password-sha":"39bc3d994b6a0ce19cb60726b630237d494ae928", "salt":"312b9eb84105e322eb508a559b0000d3"}'

The -d argument takes a valid json string and you could alternatively point to a file using:

-d @custardbelly_user.json

So yeah, a script would be awesome to perform the password encryption and generate that json file with the proper fields. If you have one, let me know. Otherwise it might be a little project for me at a later date. For now, I just use Futon to create new users.

Adding User to Documents

Great. Now we have to associate a user with each album in the albums database. Technically, becaaue we are working with a document-based DMS, we don’t have to go and add a user field to all the album documents we currently have entered. Not having a user field on an album document – though we will include on in all future creates and updates to albums – will not break either party: client or server. But it will be needed for validation on operations. And because we are working with a lovely document-based DMS, its not pulling teeth. For the album documents i currently have in my albums database, i have gone and added a user field with the value of the user name previously created:

[Document Update]

It is important to not that the user field value of an album document must match a name field value from a _user database document. This property will be assigned to the user attribute of a user context and be used for comparison on validation. So… human error and misspelling are high at stake in this case when done manually, unfortunately – i’ve definitely been guilty of it and spent hours cursing and pointing at mouses and monitors when only to finally say, “oh… missed the ‘t’ in there. huh“.

Alright. Now we have our album(s) associated with user(s) from the _user database. Let’s take a peek at how validation will occur when performing operations on albums with our client-side jQuery Mobile application. (ah, jQuery Mobile. not much of it in this article of the series, unfortunately. But hold on to your hats… there’s a ton of it in the next!)

Validation

When operations to a document are requested, CouchDB performs, or rather invokes, validation based on the presence of a validate_doc_update script in each /_design document of a database. If the script is not present, there is no validation and anyone (barring the security level assigned to the database) can do whatever they want. So we took CouchDB out of admin party, but that only puts restrictions on operations that require admin credentials once an admin is stored – operations like user creation, user role assignment, database creation, etc, essentially administrative tasks for your CouchDB instance. However, without a validate_doc_updates script in the /_design of our albums CouchApp, anyone can still update an album document however they see fit.

Truthfully, i spend my days as a client-side architect/developer. So i can’t speak well enough about how CouchDB invokes the validate_doc_update and why we are allowed to use JavaScript. There is some type of interpreter that intercepts a create or update (/delete) operation and invokes the validate_update_doc script. If no exception is thrown from that script, it continues along with the operation. Easy concept to grasp and a beautiful design by the CouchDB team… someday i will dig more into how all this comes to happen…

So, we can use good ol’ familiar JavaScript for our validation. When the validate_doc_update is invoked, it is passed two document objects – the “if-all-goes-well” new document and old document – and the user context. The user context object is representative of the current user logged into a session and has property values that can be used for validation on document operations.

To keep you even further from creating the validate_update_doc , let’s take a look at what is returned when you create a session:

> curl http://toddanderson:6s0jo772c0kcnwg@127.0.0.1:5984/_session
> {"ok":true,"userCtx":{"name":"toddanderson","roles":["_admin","admin"]},"info":{"authentication_db":"_users","authentication_handlers":["oauth","cookie","default"],"authenticated":"default"}}

So that userCtx object is essentially what is passed during invocation of validation_doc_update. On to the code! Open up your favorite text editor, add the following script and save the file as validate_doc_update.js in the root of your albums couchapp directory (for me, in following with this series, is: /Documents/workspace/custardbelly/couchdb/albums/):

/validate_doc_update.js

function( newDoc, oldDoc, userCtx ) {
  // Load validation script.
  var v = require("vendor/couchapp/lib/validate").init( newDoc, oldDoc, userCtx );

  // Create method to test if valid user.
  v.isAlbumsUser = function() {
    return v.isAdmin() || userCtx.roles.indexOf("albums-user") != -1;
  }

  // Ensure that a current session exists for editing.
  if( !userCtx.name ) {
    v.unauthorized( "You need to be logged in order to do that." );
  }
  else if( !v.isAlbumsUser() ) {
    v.forbidden( "You do not have proper access to edit this document." );
  }

  // Ensure that any updates need to match user.
  var isDeletingWithoutPermission = ( newDoc._deleted && ( oldDoc.user != userCtx.name ) );
  var isUpdatingWithoutPermission = ( newDoc.user != userCtx.name ) || ( oldDoc && ( newDoc.user != oldDoc.user ) );
  // If either non-permission criteria is met, checking delete first...
  if( !v.isAdmin() && isDeletingWithoutPermission ) {
    v.forbidden( "Only the creator of this document has permission to delete." );
  }
  else if( !v.isAdmin && ( !newDoc._deleted && isUpdatingWithoutPermission ) ) {
    v.forbidden( "Only the creator of this document has permission to update." );
  }
  else {
    // If it is being deleted, we are all set.
    if( newDoc._deleted ) return true;

    // Require a user field.
    v.require( "user" );
    // Ensure the assigned user is not changed.
    v.unchanged( "user" );
    // Ensure that user does not have value of undefined.
    v.assert( (newDoc.user != "undefined"), "New documents must have an associated user." );
  }
}

First off, when you create a CouchApp, you get a bunch of scripts available to you in the /vendor directory. We seen and used some of these, most notably in the loader.js from /vender/couchapp/_attachments that is loaded by the index.html document of our jQuery Mobile application. In the first line of our validate_doc_update we require another JavaScript provided through CouchApp – the validate.js. The validate.js script essentially exposes helper methods for determining user roles and validating document updates as well as convenience methods for throwing exceptions (as forbidden or unauthorized). The init() method is a utility method to access a new instance of this validation object and call these methods against the newDoc, oldDoc and userCtx objects.

We add a new method to our validation object to check if the user has a role of is an “albums-user” or is an admin:

/validate_doc_update.js

v.isAlbumsUser = function() {
  return v.isAdmin() || userCtx.roles.indexOf("albums-user") != -1;
}

That is then used to verify that we can go forward in our validation of the document based on the userCtx and documents. It is important to note that a document that is in the process of being deleted is assigned a _deleted property before being passed to the validate_doc_update. That is important to our validation as it will not necessarily be filled with a “user” property, nor is the “user” property necessary on the newDoc to validate the operation. We need to check if a delete operation is valid by comparing the oldDoc to the userCtx:

/validate_doc_update.js

var isDeletingWithoutPermission = ( newDoc._deleted && ( oldDoc.user != userCtx.name ) );
var isUpdatingWithoutPermission = ( newDoc.user != userCtx.name ) || ( oldDoc && ( newDoc.user != oldDoc.user ) );
// If either non-permission criteria is met, checking delete first...
if( !v.isAdmin() && isDeletingWithoutPermission ) {
  v.forbidden( "Only the creator of this document has permission to delete." );
}
else if( !v.isAdmin && ( !newDoc._deleted && isUpdatingWithoutPermission ) ) {
  v.forbidden( "Only the creator of this document has permission to update." );
}

If our validation passes through that, then all that is left is to make sure that we have the correct fields and their values are valid:

/validate_doc_update.js

else {
  // If it is being deleted, we are all set.
  if( newDoc._deleted ) return true;

  // Require a user field.
  v.require( "user" );
  // Ensure the assigned user is not changed.
  v.unchanged( "user" );
  // Ensure that user does not have value of undefined.
  v.assert( (newDoc.user != "undefined"), "New documents must have an associated user." );
}

If we are simply deleting the document, return true to pass the validation, else we ensure that a user field is present, that it has not changed and that it is not undefined. If none of those checks throws an exception, the validation will be complete and the update (create or update) will pass.

What’s cool about Futon is that it will use that validation script when working on documents within Futon… pretty meta. So if you have shut down validation to no-one-at-all-ever, and go and try to update a document in your database, you’ll get an alert and will have to log in as an admin and change your validate_doc_update.

Push Validation to our Albums CouchApp

Alright, with our validate_doc_update script ready to go, its time to push it live. If you have been following along in this series, you are familiar with how we push updates using couchapp. However, with our new privileges that we implemented in this article, we have to do it a little differently. Whoa whoa whoa! Calm down… its not that bad. We just have to pass our admin credentials in the command:

couchapp push albums http://toddanderson:admin123@127.0.0.1:5984/albums

All we did was use basic access authentication to push an update to our couchapp with the credentials of the admin we created previously. Now you have authority. Can you feel the power? Sometimes i think i feel the power, but it could be gas. Not this time, though. I swear.

Security

If you have been playing around with Futon and looked at your albums database, you may have seen a Security… option in the list of actions:

[Security Option]

If you log in as the administrator, click option and fill in the credentials with the admin and roles we created previously in this article, you will enforce a login prior to viewing the application:

[Security Dialog]

If you save that and logout of Futon then try to access the albums database, you will get the following alert:

[Security Alert]

Likewise, if we now browse to our application at http://127.0.0.1:5984/albums/_design/albums/index.html you’ll be presented with this page on landing:

[Application Landing]

That is the session.html shipped with CouchDB that provides an easy way to present a login or signup gateway page for your application. We don’t want that. It breaks the User Experience for our albums application. We are going to present a login/signup only when a user tries to perform an operation that requires session validation. So, if you went ahead and added security, go and roll that back be emptying the fields… just wanted to show you a little more about security.

Now Your Stuck

[Uh-oh]

LOL. You fell for it. What a ruse! If you have visited your application and tried to either create, update or delete a document, you can’t! That’s the end of this series. See you around!

No, no. Come back. Lower your fists. We’re going to add a login/signup dialog to our jQuery Mobile application so we can go about our business as we had done previously, but this time using user credentials and validation. This post has already gotten rather long, so that will come in the next installment of this mini-series. Just sit back for a bit and feel your gas power rise.

Conclusion

In this first installment of Part 7 mini-series within a series, we address user credentials, validation and security for our database and albums application. All fun stuff, and prior to CouchDB 0.11 a sore point. I won’t go into how I had done it previously, but it boiled down to giving in and keeping the admin party. Maybe i wasn’t smart enough. But thankfully the intelligent people leading CouchDB made me look smart again in later releases.

Next up in the mini-series within a series within Part 7 within this blog within these series of tubes, we will use the information gathered in this article along with our validation script and modify our jQuery Mobile application to present a login/signup dialog when an operation is requested that requires session authentication and user credentials. Hurrah!

[Note] This post was written against the following software versions:
CouchDB – 1.0.1
CouchApp – 0.7.2
jQuery – 1.4.4
jQuery Mobile – 1.0a2
If you have found this post and any piece has moved forward, hopefully the examples are still viable/useful. I will not be updating the examples in this post in parellel with updates to any of the previously mentioned software, unless explicitly noted.

Articles in this series:

1. Getting Started
2. Displaying a page detail of a single album.
3. Templates and Mustache
4. Displaying an editable page of an album.
5. Creating and Adding an album document.
6. Deleting an album document
7. Authorization and Validation – Part 1
8. Authorization and Validation – Part 2

Full source for albums couchapp here.