Testing Blog

Testivus, Testability and Dr. Jekyll and Mr. Hyde

Thursday, July 01, 2010
Share on Twitter Share on Facebook
Google
Labels: Alberto Savoia

14 comments :

  1. UnknownJuly 1, 2010 at 5:43:00 PM PDT

    Just a note but the normally used spelling would be "Dr. Jekyll and Mr. Hyde".

    ReplyDelete
    Replies
      Reply
  2. ranfordJuly 1, 2010 at 7:41:00 PM PDT

    I think that the right answer is a healthy dose of both your positions.

    While it is true that true "test first" development processes is insanely valuable in producing better architecture, validating the tests themselves and getting full coverage, it is not sufficient.

    It is my view that one should put together a full unit test suite using TDD (100% code coverage -- produce a failing test then write the minimum code to make it pass), but your Dr. Jekill & Mr. Hyde approach is very valuable at higher level functional/workflow/blackbox/whatever-you-want-to-call-it testing

    Then you will have the best of both worlds. If we can agree that there are different types of testing necessary to adequately qualify a bit of software, it doesn't seem too unreasonable to state that these different types of testing may lend themselves more closely to different approaches.

    ReplyDelete
    Replies
      Reply
  3. Roberto LiffredoJuly 2, 2010 at 12:16:00 AM PDT

    I think all of those approaches make quite a lot of sense, because they tackle the problem of testing from different sides.
    As usual, there's no silver bullet; some approaches may be better, but there's always some limitation that blocks from adopting in all possible contexts.

    In my opinion, the main problem is that software development has evolved quite a lot, but the same does not applies to software testing. We have now tools and methodologies that enable a much higher complexity in software, but the testing is about the same.
    I think the only solution so far, is to change the way software is developed, and build the software taking testability into the design.
    Then, continue with the existing approaches as well, and then also using "Dr. Jeckill and Mr Hide" approach, in addition, to ensure an even better layer of testing.

    ReplyDelete
    Replies
      Reply
  4. UnknownJuly 5, 2010 at 1:03:00 AM PDT

    I think of unit tests as a way to fix some behavior, and in that allowing us to refactor or otherwise enhance our code, constantly making it better without the fear of breaking an already achieved goal. They're like source control for behavior :)

    This is why I find the maxim adequate for unit tests.

    For other types of tests a Mr. hide personality is definitely a must.
    Truly, the best QA's I know posses a devilish twisted mind...

    ReplyDelete
    Replies
      Reply
  5. display nameJuly 7, 2010 at 1:18:00 PM PDT

    I don't think it is always enough to just think about the test. Sometimes, writing the test is the only way to expose the limitations of the test harness and know what testability hooks are required.

    ReplyDelete
    Replies
      Reply
  6. Rickard LindbergJuly 8, 2010 at 2:42:00 AM PDT

    I think it is very difficult to create testable code if you are not actually writing tests. It is very easy to add some functionality where it does not belong because it is just "a small fix".

    ReplyDelete
    Replies
      Reply
  7. bensleyJuly 8, 2010 at 1:48:00 PM PDT

    Others think that code and tests should not be thought of as one at all, but they should be treated independently – ideally as adversaries: "I don't want code and tests to be too "friendly". Production code should not be changed or compromised to make the testing easier, and tests should not trust the hooks put in the code to make it more testable."

    ---

    I preach and practice the exact opposite. I'm a huge advocate of merging test and production code for a variety of reasons: we get maximal code reuse, our test utilities can use the exact same code the production services use to create and modify data, and it promotes ownership between test and development in both directions. The Test Engineers develop an ownership of production code and Software Engineers develop an ownership of test code.

    I regularly look at 3 continuous builds. One is owned by developers (it's almost always red), one is owned by test engineers (it's almost always red), one is co-owned (when it breaks, people fix it immediately).

    A coworker and I have recently embedded ourselves with a developer group (we're Software Engineers in Test). In two weeks I learned more about our code base than I had in the previous 12 months. Our developers also immediately started thinking about test solutions as engineering problems and not quality assurance problems.

    And we were able to deploy a fake version of a server for testing and integration before the real server was written. Because we have embraced merging test and production code rather than run screaming, the transition from fake to real will be a 100% seamless transition, and we won't lose a single bit of automation or coverage (in fact, both will increase), all because we've embedding test code in production services.

    I'm now convinced: MERGE MERGE MERGE!

    Matthew Bensley
    SET @ Google in Mountain View, CA

    ReplyDelete
    Replies
      Reply
  8. SuryabirJuly 8, 2010 at 2:38:00 PM PDT

    Certainly thinking about testing while coding would help developers avoid some of the basic pitfalls that a tester might try. But having the Dr Jekyll and Mr Hyde approach is tough for a single person, unless of course you have MPD! :)
    I don't quite think that coding and testing should be done in isolation also, because there are a number of things a tester might think of testing only if he knows the logic behind the code. Thats why I think the approach of pairing a tester with a developer combines these 2 approaches - 1. of having 2 different personalities and 2. of developing the test cases and code in the same environment.
    Although we are using 2 resources and this could be expensive but I believe its effective.

    Surya Dodd
    www.coroware.com

    ReplyDelete
    Replies
      Reply
  9. AnonymousJuly 8, 2010 at 3:58:00 PM PDT

    @Jesus Freak: Thanks for pointing out that I mispelled BOTH Dr. JekYll and Mr. HYde. I fixed it now. Nice catch.

    ReplyDelete
    Replies
      Reply
  10. Mark RoddyJuly 13, 2010 at 10:50:00 AM PDT

    What are your thoughts? Is it enough to think about testability when designing or writing the code, or must you actually write and run some tests in parallel with the code?

    1) I do not expect everyone to use TDD, but I do expect them to deliver the same results as if they had. So you don't have to 'test first', but don't bother coming to me claiming the feature is 'done' but there wasn't time to write unit tests. Incidentally, I've never had anyone not use TDD and produce code/tests of the same quality.

    2) Do not focus on TDD to the deteriment of what it hopes to accomplish. If some other approach achieves these goals better than TDD for what you're working on then abandon TDD for that work. Do not forsake your goals for a path to said goals.

    -Mark

    ReplyDelete
    Replies
      Reply
  11. FolsJuly 14, 2010 at 10:48:00 AM PDT

    regarding Bensley's post - You hit on a great point and that is that too many times the silos of development and testing (QA) are not intersecting until it's 'too late'. I worked with a developer on a project and as he coded, I actually put comments in the code on how to test the functionality. Those comments were followed during unit testing and then used as a guide for writing test cases during the integration portion of testing. It was a bit time consuming, but I believe it worth the trouble because it saved time in bug reporting and bug fixing down the line.
    Like many things, this is about people working together and if you have a good level of DEV/QA cohesion, it goes A LONG way.

    ReplyDelete
    Replies
      Reply
  12. Bruce LebanJuly 16, 2010 at 11:18:00 AM PDT

    Thanks for sharing The Way of Testivus. I think the maxims are good except for "Think of the code and test as one." I think that's a path to disaster.

    Consider @bensley's comment: "our test utilities can use the exact same code the production services use".

    This is a great way to ensure that your test code and your production code have the same bugs. Yes, tests themselves can have bugs.

    Notwithstanding this, there is a good point here. Here's my suggested replacement:

    Think of code and test as two sides of a coin

    You can't have a coin with just one side.
    When writing the code, think of the test.
    When writing the test, think of the code.

    A coin with two heads is designed to cheat.
    Don't copy from the code to the test.
    Don't copy from the test to the code.


    Also posted in my Vroospeak blog with a bit more detail.

    ReplyDelete
    Replies
      Reply
  13. UnknownJuly 16, 2010 at 7:32:00 PM PDT

    It seems to me that the approach to testability does not matter much *if* the team follows a discipline of continuous code/test review and refactoring. On one hand, some developers are very productive as long as they do not have to write tests at the same time (bad habit?), and they can usually follow some basic testability practices to help test engineers afterwards (e.g. think twice before using final or static keywords in Java). On the other hand, some developers follow a TDD-like approach and have some extent of built-in testability. In both cases, review and refactoring can help with aligning code and test. What is the most efficient path? I guess it really depends on the teams, although I have preference for TDD.

    Having wrote that, I think pushing people having a bipolar Jekyll/Hyde approach is extremely hard, even fruitless in general if they are not in this mindset by themselves. The dissonance between creative development and destructive testing is a stumbling block for many people around there. There is also a conflict of interests, where some developers will built tests *so that* the code execution lights up the green.

    ReplyDelete
    Replies
      Reply
  14. UnknownAugust 11, 2010 at 7:52:00 AM PDT

    I can be rather difficult for us Jekyll Hyde (JH) types to integrate into a TDD organization. When designing a piece of code, particularly frameworks, I try to focus on usability of that framework rather than testability. One possible fallacy is to assume that if you do not have a TDD mentality you will not write rigorous tests. Most arguments seem to imply this, that TDD is required to generate proper and full testing coverage. Type JH developers become frustrated at the thought of being spoon-fed, essentially being told "don't forget to look both ways before crossing" by the org. Ahhh but a TD should be clever enough to "fake it", from an external point of view there's no difference except for the rigor of tests themselves, then you can spot a Jekyll/Hyde type.

    ReplyDelete
    Replies
      Reply
Add comment
Load more...

The comments you read and contribute here belong only to the person who posted them. We reserve the right to remove off-topic comments.

  

Labels


  • TotT 104
  • GTAC 61
  • James Whittaker 42
  • Misko Hevery 32
  • Code Health 31
  • Anthony Vallone 27
  • Patrick Copeland 23
  • Jobs 18
  • Andrew Trenk 13
  • C++ 11
  • Patrik Höglund 8
  • JavaScript 7
  • Allen Hutchison 6
  • George Pirocanac 6
  • Zhanyong Wan 6
  • Harry Robinson 5
  • Java 5
  • Julian Harty 5
  • Adam Bender 4
  • Alberto Savoia 4
  • Ben Yu 4
  • Erik Kuefler 4
  • Philip Zembrod 4
  • Shyam Seshadri 4
  • Chrome 3
  • Dillon Bly 3
  • John Thomas 3
  • Lesley Katzen 3
  • Marc Kaplan 3
  • Markus Clermont 3
  • Max Kanat-Alexander 3
  • Sonal Shah 3
  • APIs 2
  • Abhishek Arya 2
  • Alan Myrvold 2
  • Alek Icev 2
  • Android 2
  • April Fools 2
  • Chaitali Narla 2
  • Chris Lewis 2
  • Chrome OS 2
  • Diego Salas 2
  • Dori Reuveni 2
  • Jason Arbon 2
  • Jochen Wuttke 2
  • Kostya Serebryany 2
  • Marc Eaddy 2
  • Marko Ivanković 2
  • Mobile 2
  • Oliver Chang 2
  • Simon Stewart 2
  • Stefan Kennedy 2
  • Test Flakiness 2
  • Titus Winters 2
  • Tony Voellm 2
  • WebRTC 2
  • Yiming Sun 2
  • Yvette Nameth 2
  • Zuri Kemp 2
  • Aaron Jacobs 1
  • Adam Porter 1
  • Adam Raider 1
  • Adel Saoud 1
  • Alan Faulkner 1
  • Alex Eagle 1
  • Amy Fu 1
  • Anantha Keesara 1
  • Antoine Picard 1
  • App Engine 1
  • Ari Shamash 1
  • Arif Sukoco 1
  • Benjamin Pick 1
  • Bob Nystrom 1
  • Bruce Leban 1
  • Carlos Arguelles 1
  • Carlos Israel Ortiz García 1
  • Cathal Weakliam 1
  • Christopher Semturs 1
  • Clay Murphy 1
  • Dagang Wei 1
  • Dan Maksimovich 1
  • Dan Shi 1
  • Dan Willemsen 1
  • Dave Chen 1
  • Dave Gladfelter 1
  • David Bendory 1
  • David Mandelberg 1
  • Derek Snyder 1
  • Diego Cavalcanti 1
  • Dmitry Vyukov 1
  • Eduardo Bravo Ortiz 1
  • Ekaterina Kamenskaya 1
  • Elliott Karpilovsky 1
  • Elliotte Rusty Harold 1
  • Espresso 1
  • Felipe Sodré 1
  • Francois Aube 1
  • Gene Volovich 1
  • Google+ 1
  • Goran Petrovic 1
  • Goranka Bjedov 1
  • Hank Duan 1
  • Havard Rast Blok 1
  • Hongfei Ding 1
  • Jason Elbaum 1
  • Jason Huggins 1
  • Jay Han 1
  • Jeff Hoy 1
  • Jeff Listfield 1
  • Jessica Tomechak 1
  • Jim Reardon 1
  • Joe Allan Muharsky 1
  • Joel Hynoski 1
  • John Micco 1
  • John Penix 1
  • Jonathan Rockway 1
  • Jonathan Velasquez 1
  • Josh Armour 1
  • Julie Ralph 1
  • Kai Kent 1
  • Kanu Tewary 1
  • Karin Lundberg 1
  • Kaue Silveira 1
  • Kevin Bourrillion 1
  • Kevin Graney 1
  • Kirkland 1
  • Kurt Alfred Kluever 1
  • Manjusha Parvathaneni 1
  • Marek Kiszkis 1
  • Marius Latinis 1
  • Mark Ivey 1
  • Mark Manley 1
  • Mark Striebeck 1
  • Matt Lowrie 1
  • Meredith Whittaker 1
  • Michael Bachman 1
  • Michael Klepikov 1
  • Mike Aizatsky 1
  • Mike Wacker 1
  • Mona El Mahdy 1
  • Noel Yap 1
  • Palak Bansal 1
  • Patricia Legaspi 1
  • Per Jacobsson 1
  • Peter Arrenbrecht 1
  • Peter Spragins 1
  • Phil Norman 1
  • Phil Rollet 1
  • Pooja Gupta 1
  • Project Showcase 1
  • Radoslav Vasilev 1
  • Rajat Dewan 1
  • Rajat Jain 1
  • Rich Martin 1
  • Richard Bustamante 1
  • Roshan Sembacuttiaratchy 1
  • Ruslan Khamitov 1
  • Sam Lee 1
  • Sean Jordan 1
  • Sebastian Dörner 1
  • Sharon Zhou 1
  • Shiva Garg 1
  • Siddartha Janga 1
  • Simran Basi 1
  • Stan Chan 1
  • Stephen Ng 1
  • Tejas Shah 1
  • Test Analytics 1
  • Test Engineer 1
  • Tim Lyakhovetskiy 1
  • Tom O'Neill 1
  • Vojta Jína 1
  • automation 1
  • dead code 1
  • iOS 1
  • mutation testing 1


Archive


  • ►  2025 (1)
    • ►  Jan (1)
  • ►  2024 (13)
    • ►  Dec (1)
    • ►  Oct (1)
    • ►  Sep (1)
    • ►  Aug (1)
    • ►  Jul (1)
    • ►  May (3)
    • ►  Apr (3)
    • ►  Mar (1)
    • ►  Feb (1)
  • ►  2023 (14)
    • ►  Dec (2)
    • ►  Nov (2)
    • ►  Oct (5)
    • ►  Sep (3)
    • ►  Aug (1)
    • ►  Apr (1)
  • ►  2022 (2)
    • ►  Feb (2)
  • ►  2021 (3)
    • ►  Jun (1)
    • ►  Apr (1)
    • ►  Mar (1)
  • ►  2020 (8)
    • ►  Dec (2)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Aug (2)
    • ►  Jul (1)
    • ►  May (1)
  • ►  2019 (4)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Jul (1)
    • ►  Jan (1)
  • ►  2018 (7)
    • ►  Nov (1)
    • ►  Sep (1)
    • ►  Jul (1)
    • ►  Jun (2)
    • ►  May (1)
    • ►  Feb (1)
  • ►  2017 (17)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Sep (1)
    • ►  Aug (1)
    • ►  Jul (2)
    • ►  Jun (2)
    • ►  May (3)
    • ►  Apr (2)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2016 (15)
    • ►  Dec (1)
    • ►  Nov (2)
    • ►  Oct (1)
    • ►  Sep (2)
    • ►  Aug (1)
    • ►  Jun (2)
    • ►  May (3)
    • ►  Apr (1)
    • ►  Mar (1)
    • ►  Feb (1)
  • ►  2015 (14)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (2)
    • ►  Aug (1)
    • ►  Jun (1)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (1)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2014 (24)
    • ►  Dec (2)
    • ►  Nov (1)
    • ►  Oct (2)
    • ►  Sep (2)
    • ►  Aug (2)
    • ►  Jul (3)
    • ►  Jun (3)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (2)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2013 (16)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Aug (2)
    • ►  Jul (1)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (2)
    • ►  Jan (2)
  • ►  2012 (11)
    • ►  Dec (1)
    • ►  Nov (2)
    • ►  Oct (3)
    • ►  Sep (1)
    • ►  Aug (4)
  • ►  2011 (39)
    • ►  Nov (2)
    • ►  Oct (5)
    • ►  Sep (2)
    • ►  Aug (4)
    • ►  Jul (2)
    • ►  Jun (5)
    • ►  May (4)
    • ►  Apr (3)
    • ►  Mar (4)
    • ►  Feb (5)
    • ►  Jan (3)
  • ▼  2010 (37)
    • ►  Dec (3)
    • ►  Nov (3)
    • ►  Oct (4)
    • ►  Sep (8)
    • ►  Aug (3)
    • ▼  Jul (3)
      • Code coverage goal: 80% and no less!
      • There, but for the grace of testing, go I
      • Testivus, Testability and Dr. Jekyll and Mr. Hyde
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (3)
    • ►  Mar (3)
    • ►  Feb (2)
    • ►  Jan (1)
  • ►  2009 (54)
    • ►  Dec (3)
    • ►  Nov (2)
    • ►  Oct (3)
    • ►  Sep (5)
    • ►  Aug (4)
    • ►  Jul (15)
    • ►  Jun (8)
    • ►  May (3)
    • ►  Apr (2)
    • ►  Feb (5)
    • ►  Jan (4)
  • ►  2008 (75)
    • ►  Dec (6)
    • ►  Nov (8)
    • ►  Oct (9)
    • ►  Sep (8)
    • ►  Aug (9)
    • ►  Jul (9)
    • ►  Jun (6)
    • ►  May (6)
    • ►  Apr (4)
    • ►  Mar (4)
    • ►  Feb (4)
    • ►  Jan (2)
  • ►  2007 (41)
    • ►  Oct (6)
    • ►  Sep (5)
    • ►  Aug (3)
    • ►  Jul (2)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (7)
    • ►  Mar (5)
    • ►  Feb (5)
    • ►  Jan (4)

Feed

  • Google
  • Privacy
  • Terms