FuncUnit: Getting Funky with automated JavaScript testing

I’ve recently been attempting to make the testing of the software I’m working more agile, as the team I work with use Scrum as the development methodology. The problems could be summarised as:

• There is always a lag between development and testing: the software isn’t ready enough to test until late in the sprint, so sprints either overrun, or a “clean up” sprint is required to finish the testing, or fix bugs found very late.
• JavaScript functional testing (as opposed to unit testing) has traditionally been hard, especially with the kind of UI we typically build: a web GIS with lots of map interaction:pan, zoom, click to query, draw shapes.

The holy grail of my search has been to reduce the amount of manual regression testing we do, and increase the amount of unit testing. To make any impact this requires automation of complex UI tests in multiple browsers, linked to our Continuous Integration system (a hybrid of Cruise Control, nAnt, MSBuild and Team Foundation Server). Achieving this will allow the team to focus on more exploratory testing, and reduce sprint overruns. It should also highlight bugs at or shortly after check in time, so we don’t find them only after a huge round of manual regression tests.

This week I came across a new tool, FuncUnit. This is created by an outfit called Jupiter IT, who also produce JavaScriptMVC. I liked the look of it straight away because it ties together existing frameworks I am familiar with: jQuery, qUnit, Selenium – but offers automated functional tests of modern, whizzy JavaScript map UIs – i.e. not just unit tests of static components (no bad thing in themselves of course). You even write the tests in jQuery-like syntax – perfect for our development team who are fluent in jQuery and Dojo. Best of all it allows the specification of user interactions like drag – so we can automate scripts to pan our maps and draw zoom boxes or sketch shapes, something that has always eluded us.

I shan’t repeat too much detail on how FuncUnit works – you can go and read this for yourself. But an example might be useful to indicate how powerful and yet simple to implement this is. I had my first tests running within 15 minutes, performing an address look up using our Gazetteer client (running via a REST service against a remote web service), drilling down through a pick list of potential address matches, selecting one from the list, zooming to it on the map and kicking off a map search. The automation of this test is almost no further effort: create a batch file to call it from the command line instead of opening the test harness in a browser, and set up a scheduled task to run it after our nightly staging server deployment.

Installing FuncUnit

You can get the latest download here. Installation is just a matter of extracting the files. I put them in a web folder so I can set up a server that the whole team can access.

For my first test suite I used the template HTML file and copied the contents of the demo folder to a new folder, replacing the demo test page and JS file with my own.

Copying the demo scripts and changing to find the HTML elements in my application, I ended up with the following script:

module("wp",{
    //Set up: Open the map page
    setup : function(){
        S.open("/LocalViewWeb/Sites/LV/")
    }
})

//Main test. NB each test reloads the application being tested, so all my logic is in a single test.
test("Address Search", function(){
    //wait for the Gazetteer search box to become visible.
    S("#partialAddress").visible(function(){
        //Click to clear the hint. Type a street name.
        S("#partialAddress").click().type("Walton Way");
        //Click the "Search" button.
        S("#searchButton").click(function(){
           //Wait for the results container show. 
            S("#resultsContainer").visible(function(){ 
               //check for at least 1 result
                var items = S(".picklist_row").size();
                //use a qUnit assertion

                ok(items > 0, "pick list has items");
            }); 
        })
    });
    //Now look in the results list. 
    //use jQuery syntax to click the first in the list
    S(".picklist_row:first").click(function(){
        //When selected address is displayed, check the text
        S("#selectedAddress").visible(function(){
            var a = S("#selectedAddressText > a").text();
           //assert it's not an empty string
            ok(a!="", "Selected address: " + a);
        });
    });
    //Now choose a map search to run.
    S("#searchHeader").visible(function(){
        S("#searchHeader").click(function(){
            //this only appears when results have been found
            S(".result_row").visible(function(){
                var divs = S("#resultPanel > div").size();
                ok(divs>1, "Feed search results found");
            });
        });
    });
})

The results are displayed in the test page.

Running this is just a matter of opening the test harness in a browser. Automating it is just a matter of using EnvJs to run it from the command line, supplying the browser(s) you wish to test with. In about 30 minutes I’ve got to a state where as of tomorrow, I no longer need to regression test address searching, zooming to the address on a map or running a geographic search. In another 30 minutes I can get map zooming and panning covered off and have a stab at a map-based click-to-identify-features test. Hurrah – this saves me and the whole team days of time each sprint!

ShortcominGS

So far so good, but to get this to be as useful as possible I need a bit more:

• Alerts when the tests fail – run tests from Cruise Control server instead
• Ability to test multiple IE versions – create new VMs for IE6 and IE7

I’m really impressed at how far I’ve got so quickly though – it’s going to have an immediate impact. I’ll post more examples as I go along.