JavaScript Unit Testing

This is the fifth blog of a series of posts about the topics Continuous Integration and Continuous Delivery. It explains how important continuous integration is to deliver software in short iterations to the customer with a high quality standard.

Modern applications use more and more JavaScript to provide a rich and interactive user interface. Especially with HTML 5, JavaScript code is getting even more. I am wondering that JavaScript is still not taken as serious as most of the other programming languages. There is still not enough awareness that JavaScript code is an important part of applications and has to have a good code quality as well. I have seen projects which were writing a lot of server-side unit tests but had no quality assurance on the client-side.

The tools have been improved in the last couple years but are still not as intuitive as they should be. For instance, Visual Studio does not support Unit Testing of JavaScript code out-of-the-box. But at least there are already a couple of JavaScript Unit Testing Frameworks available.

In the following blog, different JavaScript Test Frameworks shall be tested and compared. It is focused on the TFS integration in order to execute the tests during the CI build.

Browser-based or Browser-less?

There are two different approaches, either the test frameworks are using a browser to execute the tests or the JavaScript code is interpreted and executed from a host application.

Browser-based Frameworks: QUnit, JS-Test-Driver, ...
Browser-less Frameworks: JSTest.NET, google-js-test, ... Crosscheck, ...

The browser-less frameworks can be executed usually pretty easy. Also the integration in CI builds is much easier because the overhead of starting and stopping a browser is not needed. But there is one big disadvantage with browser-less frameworks. The execution runs in a virtual environment and the different browser habits cannot be tested. Additionally some features are usually not supported by these frameworks. That is the reason why I prefer browser-based frameworks.

Writing JavaScript Tests can be tricky

In general, writing JavaScript Unit Tests is not as easy like testing server-side code, because usually JavaScript code is calling web services and interacting with the DOM of the browser. Of course, you can separate your JavaScript logic from the DOM interaction and service calls (and you should always do that!). But that does not change the fact that loading data and manipulating the DOM are the main tasks of your JavaScript code. If you would just test the pure JavaScript logic without DOM interaction, you would miss a big part of your code.

Mocking Ajax-Request

The first problem with AJAX service calls can be solved by using a mocking framework. If you are using JQuery, you just need to include the JQuery Mockjax library and you can easily redirect your AJAX calls to return the data you need for your test:

$.mockjax({
  url: 'testurl/test',
  responseText: 'Result from the test operation.'
});

This line hooks into the JQuery library and returns the given response text for all JQuery requests to the defined url. The response text can be simple text, JSON or any other content.

DOM Manipulation

The DOM interaction problem is more difficult. Almost in all cases, JavaScript code communicates and manipulates the browser's DOM. Asynchronously retrieved data has to be displayed in a certain way. This topic is also the most important task of a JavaScript unit testing framework (besides the test execution, of course).

There are different approaches to support the declaration of HTML markup for unit tests. The most frameworks like QUnit for example, need a real HTML document for the test execution. The unit tests are written within this document and executed by simply loading the document. The results are shown afterwards by the testing framework within the browser as HTML output.

This approach has two big disadvantages:

• All the tests have to work in the context of the HTML page. The JavaScript unit tests usually highly depend on the HTML markup. If a lot of different cases have to be tested, a new HTML page has to be created each time. These pages are usually just slightly different but cause a lot of troubles and effort in the test maintainance.
• The test results are usually shown as HTML output in the browser and cannot be automatically processed. But this is very important to fail the Continuous Integration Build and deny the check-in.

But there is JS-Test-Driver, a tool especially made for the integration of JavaScript unit tests in CI builds as well as an easy definition of HTML markup. It makes it much easier to execute JavaScript unit tests within a CI build and to reduce the effort to write tests.

JS-Test-Driver

JS-Test-Driver is a great Unit Testing framework, which supports inline definition of DOM elements and a seamless integration into the Continuous Integration build.

The HTML markup for unit tests is not written in a separate HTML page. It can be defined with a special DOC comment, e.g. /*:DOC += */. The html document is automatically created and can be used within your test case.

TestCase.prototype.testMain = function() {   
  /*:DOC += <div class="main">
</div>
*/   
  assertNotNull($('.main')[0]); 
};

That is the reason why js-test-driver is my favorite Javascript Test Framework. It scales like a charm and allows to define HTML tags within the tests. Additionally it can be easily integrated into the build process.

Configuration of JS-Test-Driver:

The following script shows how to configure JS-Test-Driver. It is quite self-explaining. The "server" declaration defines the binding for the started server. "load" defines which scripts should be available during the tests. "test" defines where are the unit tests located. Additionally plug-ins like code coverage calculation can be integrated as well.

server: http://localhost:4224

load:
 - Script/Main/*.js
 - Script/Page/*.js

test:
 - Script/UnitTests/*.js

plugin:
 - name: "coverage"   
   jar: "coverage.jar"   
   module: "com.google.jstestdriver.coverage.CoverageModule"

Integrate JS-Test-Driver into Team Foundation Server Build

JS-Test-Driver starts a server and a browser instance, runs the tests for you and posts the result to the server. The result can be evaluated during the CI Build and check-ins can be even rejected when just one test-case fails. After that JS-Test-Driver is also shutting down the server and the browser.

To integration JS-Test-Driver into the TFS Build a configuration file (like above) and a build target has to be created:

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <!-- SolutionDir is the dir, where the solution file exist -->
    <SolutionRoot>$(MSBuildStartupDirectory)\..</SolutionRoot>
  </PropertyGroup>

 <Target Name="JSTestDriver">
    <PropertyGroup>
      <JSTestDriverJar>$(SolutionRoot)\JsTestDriver\JsTestDriver-1.3.4.b.jar</JSTestDriverJar>
      <JSTestDriverConfig>$(SolutionRoot)\JsTestDriver\jsTestDriver.conf</JSTestDriverConfig>
      <BrowserPath>C:\Program Files (x86)\Internet Explorer\iexplore.exe</BrowserPath>
    </PropertyGroup>

    <Exec Command='java -jar "$(JSTestDriverJar)" --port 40000 --basePath "$(SolutionRoot)" --browser "$(BrowserPath)" --config "$(JSTestDriverConfig)" --tests all --verbose' />
  </Target>

</Project>

This target starts JSTestDriver and can be easily executed from the local or TFS build:

build JSTestDriver

The screenshots show how the JSTestDriver target can be added to the TFS build workflow XAML. The MS Build activity uses the JSTestDriver target to start the Java jar-file and executes the javascript unit tests. If one of the tests fails the MS Build activity returns an error and therefore also the build fails. If the gated check-in is enabled, the code is not committed in the code basis until the tests are fixed.