Testing Blog

Testing on the Toilet: Test Behaviors, Not Methods

Monday, April 14, 2014
Share on Twitter Share on Facebook
Google
Labels: Erik Kuefler , TotT

6 comments :

  1. Infant GeraldApril 14, 2014 at 5:40:00 PM PDT

    Thanks as a QAE, I was so tempted to write all my validations in a single test esp when there is a continous flow happening. Now that temptations has eased.
    Howeve this because more challenging when we write UI Tests with a lot of inputs before it even lands on the page to test. Where is the line to be drawn before going crazy and write automation tests for each test case vs combining all tests in 1 giant automation tests.

    ReplyDelete
    Replies
      Reply
  2. UnknownApril 15, 2014 at 1:28:00 AM PDT

    Nice point in the post.
    To Gerald's point, I believe that test setup is indeed a nontrivial issue. However, the sort of silver bullet is to keep the UI test at minimum. Test through user interface the things which are behavior of your UI, not the entire functionality. If you follow (and one Should follow) the concept of clean architecture (see uncle Bob martin's literature on this), then what we need to automate is behavior of the application (aka use cases of application) via application boundary.
    Having said that, there are certain things that are not the behavior of you application use case but of the UI the way it is implemted right now. There are two important poi ts here. One, there is stuff on UI that needs to be tested even when bulk of our efforts are on automating use cases through boundary. There is no denying it. Second point is that UI is what it is now, it will change without any change in business. You won't want to refactor your automated cases because your UI has changed. So only test through UI what is really a UI behavior.
    We have tried Selenium and found it to be difficult to maintain. Currently looking at Graphene. HTH!

    --Nafees

    ReplyDelete
    Replies
      Reply
  3. UnknownApril 15, 2014 at 11:32:00 PM PDT

    Great entry. Do you still suggest to keep the name of the method in the test name?? We currently do that (as you do as well in your example "processTransaction_...") but I often have the feeling this might get quite messy once you want to apply a refactoring. How do you handle that?

    ReplyDelete
    Replies
    1. UnknownApril 25, 2015 at 4:32:00 PM PDT

      I would treat too many 'processTransaction_' methods as duplication..even though it's (only) about method names. Just Search/Replacing its occurences in the test code when you rename the production code would ignore that code smell. Moving the test methods to their own class instead (and removing the 'processTransaction_') is more than just cosmetics. You get slimmer, more focussed test classes for different behaviours of your production code. Constantly applying this technique when you get the feeling of messinesa will help you cut your class accordingly once it becomes too clumsy during development and needs to be broken into several smaller classes.

      Delete
      Replies
        Reply
    2. Reply
  4. MirusApril 25, 2014 at 9:26:00 AM PDT

    You bring up very good points with this article -- namely creating more discrete tests that can be used to better cover the intended behaviour of the product under test. Tests are more modular and are easier to maintain and can be better adapted when the product under test changes.

    One issue with breaking up larger tests into "smaller tests" (like the example provided in the blog post) is that the smaller tests now become dependent on the order of execution -- the tests are no longer atomic and must be run in a specific order. For example, testProcessTransaction_sendsEmailWhenBalanceIsLow() can only be executed after testProcessTransaction() otherwise testProcessTransaction_sendsEmailWhenBalanceIsLow() will fail falsely if testProcessTransaction() isn't called first. This is an especially prickly issue if you're working with automation stacks like Java and JUnit 4.8 which do not guarantee the order of execution.

    ReplyDelete
    Replies
    1. UnknownApril 25, 2015 at 4:49:00 PM PDT

      When you break down one large test into multiple small ones you need to call your method(s) under test in each of them. That makes them independent given that your @Before provides a clean setup. In this particular example this is accomplished pretty easily. Nevertheless as soon as repositories or databases are involved it can become an art to keep run times within acceptable boundaries due to expensive setup and complex teardown mechanisms. This should not concern you in isolated unit tests though.

      Actually it's a good feedback mechanism that the order of JUnit tests differs depending on your system. Tells you whether your tests are independent or not :-)

      Delete
      Replies
        Reply
    2. Reply
Add comment
Load more...

The comments you read and contribute here belong only to the person who posted them. We reserve the right to remove off-topic comments.

  

Labels


  • TotT 77
  • GTAC 61
  • James Whittaker 42
  • Misko Hevery 32
  • Anthony Vallone 27
  • Patrick Copeland 23
  • Jobs 18
  • Code Health 13
  • C++ 11
  • Andrew Trenk 10
  • Patrik Höglund 8
  • JavaScript 7
  • Allen Hutchison 6
  • George Pirocanac 6
  • Zhanyong Wan 6
  • Harry Robinson 5
  • Java 5
  • Julian Harty 5
  • Alberto Savoia 4
  • Ben Yu 4
  • Erik Kuefler 4
  • Philip Zembrod 4
  • Shyam Seshadri 4
  • Chrome 3
  • John Thomas 3
  • Lesley Katzen 3
  • Marc Kaplan 3
  • Markus Clermont 3
  • Sonal Shah 3
  • APIs 2
  • Abhishek Arya 2
  • Adam Bender 2
  • Alan Myrvold 2
  • Alek Icev 2
  • Android 2
  • April Fools 2
  • Chaitali Narla 2
  • Chris Lewis 2
  • Chrome OS 2
  • Diego Salas 2
  • Dillon Bly 2
  • Dori Reuveni 2
  • Jason Arbon 2
  • Jochen Wuttke 2
  • Kostya Serebryany 2
  • Marc Eaddy 2
  • Marko Ivanković 2
  • Max Kanat-Alexander 2
  • Mobile 2
  • Oliver Chang 2
  • Simon Stewart 2
  • Stefan Kennedy 2
  • Test Flakiness 2
  • Tony Voellm 2
  • WebRTC 2
  • Yvette Nameth 2
  • Zuri Kemp 2
  • Aaron Jacobs 1
  • Adam Porter 1
  • Adel Saoud 1
  • Alan Faulkner 1
  • Alex Eagle 1
  • Anantha Keesara 1
  • Antoine Picard 1
  • App Engine 1
  • Ari Shamash 1
  • Arif Sukoco 1
  • Benjamin Pick 1
  • Bob Nystrom 1
  • Bruce Leban 1
  • Carlos Arguelles 1
  • Carlos Israel Ortiz García 1
  • Cathal Weakliam 1
  • Christopher Semturs 1
  • Clay Murphy 1
  • Dan Shi 1
  • Dan Willemsen 1
  • Dave Chen 1
  • Dave Gladfelter 1
  • Derek Snyder 1
  • Diego Cavalcanti 1
  • Dmitry Vyukov 1
  • Eduardo Bravo Ortiz 1
  • Ekaterina Kamenskaya 1
  • Elliott Karpilovsky 1
  • Espresso 1
  • Google+ 1
  • Goran Petrovic 1
  • Goranka Bjedov 1
  • Hank Duan 1
  • Havard Rast Blok 1
  • Hongfei Ding 1
  • Jason Elbaum 1
  • Jason Huggins 1
  • Jay Han 1
  • Jeff Listfield 1
  • Jessica Tomechak 1
  • Jim Reardon 1
  • Joe Allan Muharsky 1
  • Joel Hynoski 1
  • John Micco 1
  • John Penix 1
  • Jonathan Rockway 1
  • Jonathan Velasquez 1
  • Josh Armour 1
  • Julie Ralph 1
  • Karin Lundberg 1
  • Kaue Silveira 1
  • Kevin Bourrillion 1
  • Kevin Graney 1
  • Kirkland 1
  • Kurt Alfred Kluever 1
  • Manjusha Parvathaneni 1
  • Marek Kiszkis 1
  • Mark Ivey 1
  • Mark Striebeck 1
  • Matt Lowrie 1
  • Meredith Whittaker 1
  • Michael Bachman 1
  • Michael Klepikov 1
  • Mike Aizatsky 1
  • Mike Wacker 1
  • Mona El Mahdy 1
  • Noel Yap 1
  • Patricia Legaspi 1
  • Peter Arrenbrecht 1
  • Peter Spragins 1
  • Phil Rollet 1
  • Pooja Gupta 1
  • Project Showcase 1
  • Radoslav Vasilev 1
  • Rajat Dewan 1
  • Rajat Jain 1
  • Rich Martin 1
  • Richard Bustamante 1
  • Roshan Sembacuttiaratchy 1
  • Ruslan Khamitov 1
  • Sean Jordan 1
  • Sharon Zhou 1
  • Siddartha Janga 1
  • Simran Basi 1
  • Stephen Ng 1
  • Tejas Shah 1
  • Test Analytics 1
  • Test Engineer 1
  • Tom O'Neill 1
  • Vojta Jína 1
  • iOS 1
  • mutation testing 1


Archive


  • ►  2022 (2)
    • ►  Feb (2)
  • ►  2021 (3)
    • ►  Jun (1)
    • ►  Apr (1)
    • ►  Mar (1)
  • ►  2020 (8)
    • ►  Dec (2)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Aug (2)
    • ►  Jul (1)
    • ►  May (1)
  • ►  2019 (4)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Jul (1)
    • ►  Jan (1)
  • ►  2018 (7)
    • ►  Nov (1)
    • ►  Sep (1)
    • ►  Jul (1)
    • ►  Jun (2)
    • ►  May (1)
    • ►  Feb (1)
  • ►  2017 (17)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Sep (1)
    • ►  Aug (1)
    • ►  Jul (2)
    • ►  Jun (2)
    • ►  May (3)
    • ►  Apr (2)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2016 (15)
    • ►  Dec (1)
    • ►  Nov (2)
    • ►  Oct (1)
    • ►  Sep (2)
    • ►  Aug (1)
    • ►  Jun (2)
    • ►  May (3)
    • ►  Apr (1)
    • ►  Mar (1)
    • ►  Feb (1)
  • ►  2015 (14)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (2)
    • ►  Aug (1)
    • ►  Jun (1)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (1)
    • ►  Feb (1)
    • ►  Jan (2)
  • ▼  2014 (24)
    • ►  Dec (2)
    • ►  Nov (1)
    • ►  Oct (2)
    • ►  Sep (2)
    • ►  Aug (2)
    • ►  Jul (3)
    • ►  Jun (3)
    • ►  May (2)
    • ▼  Apr (2)
      • Testing on the Toilet: Test Behaviors, Not Methods
      • The *Real* Test Driven Development
    • ►  Mar (2)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2013 (16)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Aug (2)
    • ►  Jul (1)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (2)
    • ►  Jan (2)
  • ►  2012 (11)
    • ►  Dec (1)
    • ►  Nov (2)
    • ►  Oct (3)
    • ►  Sep (1)
    • ►  Aug (4)
  • ►  2011 (39)
    • ►  Nov (2)
    • ►  Oct (5)
    • ►  Sep (2)
    • ►  Aug (4)
    • ►  Jul (2)
    • ►  Jun (5)
    • ►  May (4)
    • ►  Apr (3)
    • ►  Mar (4)
    • ►  Feb (5)
    • ►  Jan (3)
  • ►  2010 (37)
    • ►  Dec (3)
    • ►  Nov (3)
    • ►  Oct (4)
    • ►  Sep (8)
    • ►  Aug (3)
    • ►  Jul (3)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (3)
    • ►  Mar (3)
    • ►  Feb (2)
    • ►  Jan (1)
  • ►  2009 (54)
    • ►  Dec (3)
    • ►  Nov (2)
    • ►  Oct (3)
    • ►  Sep (5)
    • ►  Aug (4)
    • ►  Jul (15)
    • ►  Jun (8)
    • ►  May (3)
    • ►  Apr (2)
    • ►  Feb (5)
    • ►  Jan (4)
  • ►  2008 (75)
    • ►  Dec (6)
    • ►  Nov (8)
    • ►  Oct (9)
    • ►  Sep (8)
    • ►  Aug (9)
    • ►  Jul (9)
    • ►  Jun (6)
    • ►  May (6)
    • ►  Apr (4)
    • ►  Mar (4)
    • ►  Feb (4)
    • ►  Jan (2)
  • ►  2007 (41)
    • ►  Oct (6)
    • ►  Sep (5)
    • ►  Aug (3)
    • ►  Jul (2)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (7)
    • ►  Mar (5)
    • ►  Feb (5)
    • ►  Jan (4)

Feed

follow us in feedly
  • Google
  • Privacy
  • Terms