Being a developer means you can get hooked on making computers work for you instead of the other way around. Our team recently figured automatic web UI test tools can make life a lot easier for testers, by taking care of repetitive tests for them. However, it is important to keep future changes in the UI in mind. These changes could make maintenance on automatic tests more time-consuming than doing the tests manually! In these first few posts, I’d like to discuss several aspects of automatic web UI tests to keep in mind. Using these criteria, we can compare a number of automatic test tools to decide which to use for automatic web UI testing. In this comparison, we have decided to require support for Selenium, a popular web browser automation tool.

Automatic tests

Future maintenance on your tests is an important aspect in selecting an automatic test tool. Just like with software development, writing a procedure (such as a login procedure) in multiple places means you have to update both when the application logic changes. This is a problem many early automatic test tools had, as each test was a script that described a path through the application. When the name of a single button changed, all tests that used this button had to be updated.
A high-level domain specific language (DSL) allows a tester to define a test case at a high level. This DSL describes scenario’s (login, navigate to the client page, open Mr. Smith’s file) which in turn are made up of steps. These individual steps can be translated to low-level actions, such as clicking a button. Using this three-level abstraction improves both maintainability and readability: The name of the button to click for logging in is stored in a single step, making it easy to maintain, but the test can be read in the high-level DSL, which is easy enough to be read and understand. A  detailed explanation of this abstraction can be found at Gojko Adzic’s blog.

Selenium

Selenium is a web browser automation tool. The first version (Selenium RC) did this using JavaScript, which wasn´t allowed by all browsers. Selenium 2 has a new back-end known as WebDriver, which supports both more and newer browsers. Use of Selenium WebDriver is recommended by the Selenium developers and support for Selenium RC will be discontinued in a future version of Selenium.
Part of the Selenium family is Selenium IDE. This is a plugin for Firefox that can record and play actions in the browser. These actions can be exported to unit test suites of several programming languages, including C#, Java, Python and Ruby. This allows your unit tests (using the Selenium libraries) to replay these tests. Part of the popularity of Selenium can be explained by its accessible and easy-to-use Selenium IDE. Therefore, use of Selenium is a given in this comparison.
If we want to use Selenium for composing our web UI tests, we need to somehow define a DSL. This could be done using Selenium: Record a test, then copy it to your favourite IDE and keep extracting methods until you’ve established a DSL. However, this doesn’t improve readability for stakeholders that much, as well as requiring testers to write code. Selenium needs to be combined with a high-level tool, which allows us to define a DSL while keeping your tests readable. In the next post, we’ll compare several of them and decide which one we’re going to examine in depth.