Acceptance tests must meet the needs of several groups, including the users and the developers. Long-lived tests must be written in the language of each group, using terms users will recognize and a programming language and style in which the developers are competent.
We create tests by modelling the purpose of a test from the user’s perspective: send a message, order a book, etc. Each test is decomposed into individual actions: to send a message, a user must be logged in, select the compose message icon, specify one or more recipients, type a minimum of either a subject or a message, then select Send. From this list of actions, create a skeleton in the programming language of choice and create a method name that reflects each action. Show these to both the users and programmers and ask them to tell you what they think each step represents. Now is a great time to refine the names and decide which methods are appropriate: before you’ve invested too much time in the work. If you wait until later, your natural protective instincts will make it harder for you to accept good suggestions and make useful changes.
Utilities such as recording tools can help reduce the effort required to discover how to interact with the web application. The open-source test automation tool Selenium (http://seleniumhq.org/) includes a simple IDE record and playback tool that runs in the Firefox browser. Recorded scripts can help bootstrap your automated tests. However, don’t be tempted to consider the recorded scripts as automated tests: they’re unlikely to be useful for long. Instead, plan to design and implement your test code properly, using good software design techniques. Read on to learn how to use the PageObject design pattern to design your test code.
Two of the tools I find most useful are Firebug (http://getfirebug.com/), a Swiss Army knife for the Web Browser, and Wireshark (http://www.wireshark.org/), a network protocol analysis tool with a distinguished pedigree. Firebug is extremely useful when learning how to interact with a web application or debug mysterious problems with your tests when they seem to be misbehaving. I encourage you to persist when learning to use these tools – it took me a while to get used to their foibles, but I wouldn’t be without either of them these days.
Several years of experience across multiple project teams have taught us that the tests are more likely to survive when they’re familiar and close to the developers. Use their programming language, put them in their codebase, use their test automation framework (and even their operating system). We need to reduce the effort of maintaining the tests to a minimum. Get the developers to review the automated tests (whether they write them or you do) and actively involve them when designing and implementing the tests.
Typically, our acceptance tests use the xUnit framework; for example, JUnit for Java projects (see http://www.junit.org/). A good source of inspiration for creating effective tests is Gerard Meszaros’ work (see http://www.xunitpatterns.com).
By using effective test designs, we can make tests easier to implement and maintain. The initial investment is minor compared to the benefits. One of my favourite designs is called Page Objects (see PageObjects on the Google Code site). A PageObject represents part or all of a page in a web application – something a user would interact with. A PageObject provides services to your test automation scripts and encapsulates the nitty-gritty details of how these services are performed. By encapsulating the nitty-gritty stuff, many changes to the web application, such as the reordering or renaming of elements, can be reflected in one place in your tests. A well-designed PageObject separates the ‘what’ from the ‘how’.
// Given I have a valid user account and am at the login page,// When I enter the account details and select the Enter button,// Then I expect the inbox to be displayed with the most recent email selected.
The previous code consists of three programming comments that are easy for users to read. The actual programming code is entered immediately below each comment. Programming concepts such as literate programming are intended to make the code almost as readable as the textual comments.
Isolate things that change from those that don’t. For example, separate user account data from your test code. The separation makes changes easier, faster, and safer to implement, compared to making updates in the code for each test.
Writing automated tests may be easy for some of you. In my case, I started with some simple example tests and tweaked them to suit my needs. I received boosts from working with more experienced practitioners who were able to correct my course and educate me in how to use various tools effectively. I recommend pairing with one of the developers of the software to be tested when you face a new testing requirement. Their intimate knowledge of the code and your understanding of the tests can form a potent combination. For instance, by working with one of the developers on a recent project, we were able to implement bi-directional injection of JSON messages and capture the responses from the server to test a key interaction between the server and client that was causing problems in production.
I encourage you to try out examples, tweak them, experiment, and plunge in to writing your first automated tests. Learn about AJAX – it underpins the web applications. And learn from more experienced practitioners – I’ve added some links at the end of the article to some of the people I respect who write great acceptance tests, including Antony Marcano and Alan Richardson.
Part 2 of this series helps you create more specialized tests (for example, to emulate mobile web browsers) and gives advice on how to increase the utility and effectiveness of your tests.Further InformationIntermediate work products
Bulletproof Ajax—An incredibly good book on how to write good AJAX code. It starts with the basics and builds reliably and clearly from good foundations. The DOM manipulation code is relevant for implementing your acceptance tests in tools such as WebDriver.
Building a web site with Ajax —Again, a book that starts simple and builds a simple application step by step.
Acceptance tests are more A+S than T+G (Antony Marcano, in his blog at testingReflections.com)A+S => Activities + SpecificT+G => Tasks + General
Alan Richardson: any and everything. For example, see:A generalised model for User Acceptance Testing andA little abstraction when testing software with Selenium-RC and Java, both at the Evil Tester blog
Nice article! But...Why use "given/when/then" as just comments when you can make them executable? :) See http://cukes.info for more info.
Hi, we use a custom Java framework on top of Selenium to do functional testing of ExtJS UIs. The framework uses some of the principles described here, particularly the PageObject pattern and a literate "actions layer". Read about it at http://www.neocoders.com/portal/articles/selenium-and-extjsComments are totally welcome.Cheers,Lindsay
I wrote a quick follow-on to this post about the $x function in Firebug. Perhaps you can mention it in part 2?
Tom: I'm not sure why you need to code your site for Selenium in a way that exposes it to attack..can you give us an example? In our suite we inject the JS hooks we need through Selenese eval commands at the start of the test suite run, into the dev version, so no test-related JS is ever deployed in the production version..
The re-usability of automated test cases is the ultimate aim for any testing framework. One of the greatest barriers to people adopting automated testing is the amount of time and effort (i.e. money) that goes into creating something that then becomes a liability and seemingly in need of endless maintenance. When, if properly planned and well implemented, Test scripts/ cases are actually an asset with value to an organisation.This is a core focus for us at JadeLiquid with our LiquidTest framework, and is a topic we have blogged about in the past here: http://www.jadeliquid.com/blogs/2009/protecting-investment-tests-that-do-not-break-with-page-changes.php Duncan ThomasJadeLiquid Software
We found that for many websites the iMacros Firefox addon is a more efficient way to record & replay acceptance, regression and performance tests. The only issue here is the lack of Safari support (no problem for us...) and the limited load test support. But I guess a Google load test is in a class of its own anyway :D
The comments you read and contribute here belong only to the person who posted them. We reserve the right to remove off-topic comments.