Testing Blog

The Advantages of Unit Testing Early

Wednesday, July 15, 2009
Share on Twitter Share on Facebook
Google
Labels: Shyam Seshadri

14 comments :

  1. Jay D MumperJuly 16, 2009 at 12:21:00 PM PDT

    While there is no question that key functional code should have unit tests, the more pragmatic development groups I work with say, "Why should I test an obscure UI method? It will take longer to make the test than to write the six or eight lines of code for the whole method and besides, if I use Eclipse right, five of those lines are written for me before I write anything." The other line I hear is, "We're a manufacturing company, not a software vendor! Of course I expect our vendors to use cutting edge practices, but that's not us." How would you respond to these?

    ReplyDelete
    Replies
      Reply
  2. UnknownJuly 16, 2009 at 12:28:00 PM PDT

    Right on!

    I’d like to supplement your reasoning with a couple of guidelines for writing and organizing tests:

    http://www.bugfree.dk/blog/2009/06/19/my-unit-testing-guidelines

    For some reason it’s a challenge to convince developers to test. The Return on Investment isn’t obvious to many developers.

    I think one reason is that people mix the concepts of unit testing and integration testing, partly because a unit testing framework is used for both.

    People new to unit testing frameworks often start writing integration tests because they’re the easiest to write. Unfortunately they’re also the most brittle, and tend to fail for no apparent reason — when some dependency isn’t properly configured. After a few such false positives developers grow tired of testing because they find the maintenance cost too high.

    Another reason is that unit testing is hard. Getting used to think in terms of testability doesn’t come natural to many developers. The same goes for dependency injection and mocking frameworks.

    In addition many frameworks used by the application aren’t written with testability in mind. And so extra effort must go into mocking their APIs.

    Finally, in my experience, organizing tests so that a team of developers are comfortable with the structure is paramount to success. Basically, a common set of guidelines should be in place from early on and at least one team member must be passionate about testing for it to take off.

    ReplyDelete
    Replies
      Reply
  3. Flash SheridanJuly 16, 2009 at 1:05:00 PM PDT

    I agree strongly with your main points, about the value of both unit testing and test-driven development. But I do have some worries about subsidiary points, and also think you need to distinguish the two more emphatically. Unit testing when you don’t already have a framework to use can be hard, perhaps even impractical—but even in such cases test-driven development is still useful.

    And I think we all need to be wary of Beizer’s Pesticide Paradox and Whittaker’s False Sense of Thoroughness. Google, for all its devotion to unit testing, and the overall quality level of its products, is not immune to this. (Specifics on request.) You call some simple high-level testing “A complete waste of my time when I could have known the same thing by just running my unit tests.” But even though unit tests may well be better than high-level tests, they do tell you different things. I’ve seen a lot of products which passed their unit tests but had glaring high-level bugs.

    ReplyDelete
    Replies
      Reply
  4. Shyam SeshadriJuly 16, 2009 at 1:26:00 PM PDT

    @Jay Mumper, for the first one, while I don't argue that it takes more time, the six or eight lines more often than not grows into a much bigger / longer method on an active project. And before long, you have this long, delicate method where any mucking around leads to bugs. Wouldn't spending a few extra minutes writing some tests around that save you hours later down ? As for the second one, if you are the one developing a software, I don't are what company you are, you are a software engineer, and it becomes your responsibility to make sure you test your application. Now whether you want to drain thousands of dollars in manual testing or write some unit tests and reduce the strain on qa efforts down the line, thats a call you will have to make.

    @Flash, I completely agree that you do need higher levels of test. There is no way unit tests are going to catch integration issues and issues in interaction between components. What I do strongly hate is the lack of either and then just running your app and manually testing it. That is what I was complaining about in that line you quoted. Focussing just on higher level testing and ignoring unit testing is a huge glaring mistake in my opinion.

    ReplyDelete
    Replies
      Reply
  5. LiuJuly 17, 2009 at 12:27:00 AM PDT

    Can someone give an best example for these guidelines in a real opensource project? That would be a lot of help if exists. Great Thanks!

    ReplyDelete
    Replies
      Reply
  6. DaveJuly 17, 2009 at 8:13:00 AM PDT

    This is really good stuff, thanks for posting!

    Does your team have a "Top 10" list of books that discuss the art of TDD that you could recommend?

    ReplyDelete
    Replies
      Reply
  7. David KendallJuly 17, 2009 at 10:17:00 AM PDT

    Good question Liu. I'm convinced that thorough unit tests are something that are written about, but don't exist in real life.

    ReplyDelete
    Replies
      Reply
  8. Shyam SeshadriJuly 17, 2009 at 10:51:00 AM PDT

    @Liu, check out the Testability Explorer project (http://testability-explorer.googlecode.com/) which has a pretty good hygiene of writing tests along with all their code, as well as integration tests for some of the more complicated interactions. In particular, check out the core plugin under src.

    @Dave, if you are planning to work on an entirely new codebase and want to know how to tdd in such an environment, then Kent Beck's TDD book ( http://www.amazon.com/Test-Driven-Development-Addison-Wesley-Signature/dp/0321146530) comes to mind. But it doesn't talk about working with legacy code bases much and assumes you are familiar with the XUnit pattern. Most of the stuff I learnt has been from pairing with other developers and trying out different techniques and seeing what works and what doesn't. In the end, it does become as simple as, write a test, write the code to get it to pass, and iterate. And as long as you follow the base principles listed at (http://misko.hevery.com/code-reviewers-guide/), then you should be good to go.

    ReplyDelete
    Replies
      Reply
  9. MySchizoBuddyJuly 17, 2009 at 11:20:00 AM PDT

    Request for an article.
    How would you unit test a form class.
    Example code:
    $form->text('username', array('class' => 'users'));
    This creates an input text field called username
    Thankyou

    ReplyDelete
    Replies
      Reply
  10. TobyJuly 17, 2009 at 12:42:00 PM PDT

    This is all fine and great, but what do you do in the case where you have code that is inherently not machine testable, because the output requires human judgement? This is often the case in my projects as I work on Natural Language Processing - unit testing this is entirely nontrivial.

    Second, my biggest gripe with unit testing is that it can lull you into a false sense of security. You can be inclined to think your code works and does what it should, when in reality it only behaves well for the test cases covered in your unit tests. For more complex systems it is very easy to miss out on many corner cases in your unit tests, and as a developer it is very frustrating to be handed code that "works because the unit tests say so" only to find out that the unit tests didn't cover my use cases.

    ReplyDelete
    Replies
      Reply
  11. LiuJuly 20, 2009 at 9:10:00 PM PDT

    @Shyam Seshadri, Great Thanks!
    Misko Hevery is a great Coach! Thanks all your guys hard work!

    ReplyDelete
    Replies
      Reply
  12. Alec MunroJuly 21, 2009 at 6:14:00 AM PDT

    @Toby
    This is an interesting issue (the false sense of security).
    What I have found is that by adopting more traditional QA processes, such as test plans, it's much easier to see unit tests in context. This tends to help in avoiding any particular type of test becoming "god".
    Now, it can sometimes be challenging to adapt existing organizational practices to incorporate automated testing organically, because they are often tuned for manual testers. But the rewards are usually worth it, especially if there are a wide variety of tests in use.

    ReplyDelete
    Replies
      Reply
  13. UnknownJuly 22, 2009 at 11:33:00 PM PDT

    Hi Shyam,

    Good post. I've posted a "made me think" response in the Typemock Blog . We're doing a webinar next week about this very topic - how to motivate people to start unit testing in an organization.

    ReplyDelete
    Replies
      Reply
  14. Aleph Tav TechnologiesJuly 6, 2015 at 12:09:00 AM PDT

    Unit tests are basically written and executed by software developers to make sure that code meets its design and requirements and behaves as expected.
    The goal of unit testing is to segregate each part of the program and test that the individual parts are working correctly.

    ReplyDelete
    Replies
      Reply
Add comment
Load more...

The comments you read and contribute here belong only to the person who posted them. We reserve the right to remove off-topic comments.

  

Labels


  • TotT 104
  • GTAC 61
  • James Whittaker 42
  • Misko Hevery 32
  • Code Health 31
  • Anthony Vallone 27
  • Patrick Copeland 23
  • Jobs 18
  • Andrew Trenk 13
  • C++ 11
  • Patrik Höglund 8
  • JavaScript 7
  • Allen Hutchison 6
  • George Pirocanac 6
  • Zhanyong Wan 6
  • Harry Robinson 5
  • Java 5
  • Julian Harty 5
  • Adam Bender 4
  • Alberto Savoia 4
  • Ben Yu 4
  • Erik Kuefler 4
  • Philip Zembrod 4
  • Shyam Seshadri 4
  • Chrome 3
  • Dillon Bly 3
  • John Thomas 3
  • Lesley Katzen 3
  • Marc Kaplan 3
  • Markus Clermont 3
  • Max Kanat-Alexander 3
  • Sonal Shah 3
  • APIs 2
  • Abhishek Arya 2
  • Alan Myrvold 2
  • Alek Icev 2
  • Android 2
  • April Fools 2
  • Chaitali Narla 2
  • Chris Lewis 2
  • Chrome OS 2
  • Diego Salas 2
  • Dori Reuveni 2
  • Jason Arbon 2
  • Jochen Wuttke 2
  • Kostya Serebryany 2
  • Marc Eaddy 2
  • Marko Ivanković 2
  • Mobile 2
  • Oliver Chang 2
  • Simon Stewart 2
  • Stefan Kennedy 2
  • Test Flakiness 2
  • Titus Winters 2
  • Tony Voellm 2
  • WebRTC 2
  • Yiming Sun 2
  • Yvette Nameth 2
  • Zuri Kemp 2
  • Aaron Jacobs 1
  • Adam Porter 1
  • Adam Raider 1
  • Adel Saoud 1
  • Alan Faulkner 1
  • Alex Eagle 1
  • Amy Fu 1
  • Anantha Keesara 1
  • Antoine Picard 1
  • App Engine 1
  • Ari Shamash 1
  • Arif Sukoco 1
  • Benjamin Pick 1
  • Bob Nystrom 1
  • Bruce Leban 1
  • Carlos Arguelles 1
  • Carlos Israel Ortiz García 1
  • Cathal Weakliam 1
  • Christopher Semturs 1
  • Clay Murphy 1
  • Dagang Wei 1
  • Dan Maksimovich 1
  • Dan Shi 1
  • Dan Willemsen 1
  • Dave Chen 1
  • Dave Gladfelter 1
  • David Bendory 1
  • David Mandelberg 1
  • Derek Snyder 1
  • Diego Cavalcanti 1
  • Dmitry Vyukov 1
  • Eduardo Bravo Ortiz 1
  • Ekaterina Kamenskaya 1
  • Elliott Karpilovsky 1
  • Elliotte Rusty Harold 1
  • Espresso 1
  • Felipe Sodré 1
  • Francois Aube 1
  • Gene Volovich 1
  • Google+ 1
  • Goran Petrovic 1
  • Goranka Bjedov 1
  • Hank Duan 1
  • Havard Rast Blok 1
  • Hongfei Ding 1
  • Jason Elbaum 1
  • Jason Huggins 1
  • Jay Han 1
  • Jeff Hoy 1
  • Jeff Listfield 1
  • Jessica Tomechak 1
  • Jim Reardon 1
  • Joe Allan Muharsky 1
  • Joel Hynoski 1
  • John Micco 1
  • John Penix 1
  • Jonathan Rockway 1
  • Jonathan Velasquez 1
  • Josh Armour 1
  • Julie Ralph 1
  • Kai Kent 1
  • Kanu Tewary 1
  • Karin Lundberg 1
  • Kaue Silveira 1
  • Kevin Bourrillion 1
  • Kevin Graney 1
  • Kirkland 1
  • Kurt Alfred Kluever 1
  • Manjusha Parvathaneni 1
  • Marek Kiszkis 1
  • Marius Latinis 1
  • Mark Ivey 1
  • Mark Manley 1
  • Mark Striebeck 1
  • Matt Lowrie 1
  • Meredith Whittaker 1
  • Michael Bachman 1
  • Michael Klepikov 1
  • Mike Aizatsky 1
  • Mike Wacker 1
  • Mona El Mahdy 1
  • Noel Yap 1
  • Palak Bansal 1
  • Patricia Legaspi 1
  • Per Jacobsson 1
  • Peter Arrenbrecht 1
  • Peter Spragins 1
  • Phil Norman 1
  • Phil Rollet 1
  • Pooja Gupta 1
  • Project Showcase 1
  • Radoslav Vasilev 1
  • Rajat Dewan 1
  • Rajat Jain 1
  • Rich Martin 1
  • Richard Bustamante 1
  • Roshan Sembacuttiaratchy 1
  • Ruslan Khamitov 1
  • Sam Lee 1
  • Sean Jordan 1
  • Sebastian Dörner 1
  • Sharon Zhou 1
  • Shiva Garg 1
  • Siddartha Janga 1
  • Simran Basi 1
  • Stan Chan 1
  • Stephen Ng 1
  • Tejas Shah 1
  • Test Analytics 1
  • Test Engineer 1
  • Tim Lyakhovetskiy 1
  • Tom O'Neill 1
  • Vojta Jína 1
  • automation 1
  • dead code 1
  • iOS 1
  • mutation testing 1


Archive


  • ►  2025 (1)
    • ►  Jan (1)
  • ►  2024 (13)
    • ►  Dec (1)
    • ►  Oct (1)
    • ►  Sep (1)
    • ►  Aug (1)
    • ►  Jul (1)
    • ►  May (3)
    • ►  Apr (3)
    • ►  Mar (1)
    • ►  Feb (1)
  • ►  2023 (14)
    • ►  Dec (2)
    • ►  Nov (2)
    • ►  Oct (5)
    • ►  Sep (3)
    • ►  Aug (1)
    • ►  Apr (1)
  • ►  2022 (2)
    • ►  Feb (2)
  • ►  2021 (3)
    • ►  Jun (1)
    • ►  Apr (1)
    • ►  Mar (1)
  • ►  2020 (8)
    • ►  Dec (2)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Aug (2)
    • ►  Jul (1)
    • ►  May (1)
  • ►  2019 (4)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Jul (1)
    • ►  Jan (1)
  • ►  2018 (7)
    • ►  Nov (1)
    • ►  Sep (1)
    • ►  Jul (1)
    • ►  Jun (2)
    • ►  May (1)
    • ►  Feb (1)
  • ►  2017 (17)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Sep (1)
    • ►  Aug (1)
    • ►  Jul (2)
    • ►  Jun (2)
    • ►  May (3)
    • ►  Apr (2)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2016 (15)
    • ►  Dec (1)
    • ►  Nov (2)
    • ►  Oct (1)
    • ►  Sep (2)
    • ►  Aug (1)
    • ►  Jun (2)
    • ►  May (3)
    • ►  Apr (1)
    • ►  Mar (1)
    • ►  Feb (1)
  • ►  2015 (14)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (2)
    • ►  Aug (1)
    • ►  Jun (1)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (1)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2014 (24)
    • ►  Dec (2)
    • ►  Nov (1)
    • ►  Oct (2)
    • ►  Sep (2)
    • ►  Aug (2)
    • ►  Jul (3)
    • ►  Jun (3)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (2)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2013 (16)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Aug (2)
    • ►  Jul (1)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (2)
    • ►  Jan (2)
  • ►  2012 (11)
    • ►  Dec (1)
    • ►  Nov (2)
    • ►  Oct (3)
    • ►  Sep (1)
    • ►  Aug (4)
  • ►  2011 (39)
    • ►  Nov (2)
    • ►  Oct (5)
    • ►  Sep (2)
    • ►  Aug (4)
    • ►  Jul (2)
    • ►  Jun (5)
    • ►  May (4)
    • ►  Apr (3)
    • ►  Mar (4)
    • ►  Feb (5)
    • ►  Jan (3)
  • ►  2010 (37)
    • ►  Dec (3)
    • ►  Nov (3)
    • ►  Oct (4)
    • ►  Sep (8)
    • ►  Aug (3)
    • ►  Jul (3)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (3)
    • ►  Mar (3)
    • ►  Feb (2)
    • ►  Jan (1)
  • ▼  2009 (54)
    • ►  Dec (3)
    • ►  Nov (2)
    • ►  Oct (3)
    • ►  Sep (5)
    • ►  Aug (4)
    • ▼  Jul (15)
      • How to think about OO
      • Call for Attendance: Google Test Automation Confer...
      • The Plague of Blindness
      • Update! GTAC Keynote Speakers: Niklaus Wirth and A...
      • The Plague of Homelessness
      • Blog Stats
      • The Plague of Boredom
      • Blog Editor: Moderating Comments
      • The Advantages of Unit Testing Early
      • Software Testing Categorization
      • The Plague of Amnesia
      • Update! Google Test Automation Conference: Website...
      • Old habits die hard
      • Why are we embarrassed to admit that we don't know...
      • Separation anxiety?
    • ►  Jun (8)
    • ►  May (3)
    • ►  Apr (2)
    • ►  Feb (5)
    • ►  Jan (4)
  • ►  2008 (75)
    • ►  Dec (6)
    • ►  Nov (8)
    • ►  Oct (9)
    • ►  Sep (8)
    • ►  Aug (9)
    • ►  Jul (9)
    • ►  Jun (6)
    • ►  May (6)
    • ►  Apr (4)
    • ►  Mar (4)
    • ►  Feb (4)
    • ►  Jan (2)
  • ►  2007 (41)
    • ►  Oct (6)
    • ►  Sep (5)
    • ►  Aug (3)
    • ►  Jul (2)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (7)
    • ►  Mar (5)
    • ►  Feb (5)
    • ►  Jan (4)

Feed

  • Google
  • Privacy
  • Terms