Tuesday, July 8, 2008

Where are the JavaScript unittesting frameworks?

Now that JavaScript has been patched together into a number of promising server-side frameworks (Synergy, Jaxer, and, coming soon(ish), Rhino on Rails), JavaScript is really taking off as a potentially full-stack enterprise solution.

One serious failing, however, seems to be the continued lack of decent developer support tools, the worst of which has got to be testing. Sure, there are a number of decent browser test automation frameworks out there:

Sure, these are all great platforms - for AUTOMATION. Let's be clear, however: Automation is in no way the same thing as unit testing. Automation requires a full environment, and seeks to replicate the end-user experience on the full product stack by means of emulated behaviors - mouse clicks, key presses, and so on. Usually, such tests launch a browser app, set up the environment, execute the test case steps (which can mean multiple page loads), and then close the browser app, for EVERY TEST CASE. Such tests are absolutely necessary for any non-trivial user-facing product.

At the same time, developers need more granularity than that. We need to test hundreds of individual methods and edge cases in our code that target specific functions. We need a framework that can be run at build-time, and can plow through hundreds of focused test cases in a matter of seconds. We need a browser, sans GUI.

For the purpose of headless, command-line browser code testing, it seems only the GPL Crosscheck framework offers the robustness of browser emulation required for any serious work. Built on Rhino with a Java layer of cleverly reverse-engineer browser behaviors, Crosscheck itself has numerous bugs and drawbacks in common browser features (no cookie emulation, poor DOM Level 0 support, nonexistent IFrames, incomplete CSS emulation), the greatest of which seems to be that development has stalled on the project.

It should be immediately apparent to any front-end developer maintaining a moderate-to-large codebase why command-line unit-testing is important. Also, I realize that, as a developer, I should be getting things done and building solutions. One could investigate building a command-line app for Windows (and perhaps a fork for Linux using Wine?) that would instantiate the actual browser binaries from the command line and run code without launching the browser chrome; it seems this should work at least for Internet Explorer (COM objects) and Firefox (XULRunner) and maybe Safari too. I would like to submit patches to Crosscheck, but I'm not sure if the core developers even have time to review patches, and I don't personally have the time to sink into development of a competing framework right now. My hope is that someone out there in the development community will take notice of this need and have the time and/or resources to tackle this problem, and maybe one day I'll be that person. In the meantime, we'll have to settle for running our test suites in a matter of hours and not minutes.

No comments: