## TotT: Too Many Tests

Thursday, February 21, 2008

- GTAC 58
- TotT 55
- James Whittaker 42
- Misko Hevery 32
- Anthony Vallone 26
- Patrick Copeland 23
- Jobs 17
- C++ 11
- Andrew Trenk 8
- Patrik Höglund 8
- JavaScript 7
- Allen Hutchison 6
- Zhanyong Wan 6
- Harry Robinson 5
- Java 5
- Julian Harty 5
- Alberto Savoia 4
- Philip Zembrod 4
- Shyam Seshadri 4
- Chrome 3
- Erik Kuefler 3
- John Thomas 3
- Marc Kaplan 3
- Markus Clermont 3
- Sonal Shah 3
- APIs 2
- Alek Icev 2
- April Fools 2
- Chaitali Narla 2
- Chrome OS 2
- Diego Salas 2
- George Pirocanac 2
- Jason Arbon 2
- Jochen Wuttke 2
- Lesley Katzen 2
- Mobile 2
- Simon Stewart 2
- Tony Voellm 2
- WebRTC 2
- Yvette Nameth 2
- Zuri Kemp 2
- Aaron Jacobs 1
- Abhishek Arya 1
- Adam Bender 1
- Adam Porter 1
- Alan Faulkner 1
- Alan Myrvold 1
- Alex Eagle 1
- Android 1
- Antoine Picard 1
- App Engine 1
- Ari Shamash 1
- Arif Sukoco 1
- Benjamin Pick 1
- Bruce Leban 1
- Christopher Semturs 1
- Dave Chen 1
- Dmitry Vyukov 1
- Dori Reuveni 1
- Eduardo Bravo Ortiz 1
- Ekaterina Kamenskaya 1
- Espresso 1
- Google+ 1
- Goranka Bjedov 1
- Hank Duan 1
- Havard Rast Blok 1
- Hongfei Ding 1
- Jason Elbaum 1
- Jason Huggins 1
- Jay Han 1
- Jessica Tomechak 1
- Jim Reardon 1
- Joe Allan Muharsky 1
- Joel Hynoski 1
- John Micco 1
- John Penix 1
- Jonathan Rockway 1
- Jonathan Velasquez 1
- Julie Ralph 1
- Karin Lundberg 1
- Kaue Silveira 1
- Kevin Graney 1
- Kirkland 1
- Kostya Serebryany 1
- Kurt Alfred Kluever 1
- Manjusha Parvathaneni 1
- Mark Ivey 1
- Mark Striebeck 1
- Marko Ivanković 1
- Matt Lowrie 1
- Meredith Whittaker 1
- Michael Bachman 1
- Michael Klepikov 1
- Mike Aizatsky 1
- Mike Wacker 1
- Mona El Mahdy 1
- Noel Yap 1
- Oliver Chang 1
- Patricia Legaspi 1
- Peter Arrenbrecht 1
- Phil Rollet 1
- Pooja Gupta 1
- Radoslav Vasilev 1
- Rajat Dewan 1
- Rajat Jain 1
- Rich Martin 1
- Richard Bustamante 1
- Roshan Sembacuttiaratchy 1
- Ruslan Khamitov 1
- Sean Jordan 1
- Sharon Zhou 1
- Siddartha Janga 1
- Stephen Ng 1
- Tejas Shah 1
- Test Analytics 1
- Vojta Jína 1
- iOS 1

Great example! A suggestion if I may?

ReplyDeleteThe 27 test example doesn't take advantage of the independence provided by OR. One doesn't need to test every combination of pairs, as there is no conditional logic between them. Consider:

A = B, C = D, E = F : DoAnother()

A < B, C < D, E < F : DoAnother()

A > B, C < D, E < F : DoOneThing()

A < B, C > D, E < F : DoOneThing()

A < B, C < D, E > F : DoOneThing()

That's five tests, but it tests equivalence, greater than, and less than for each pair.

This provides the same confidence as the 27 test example, with half the tests of the decreased encapsulation example.

-Ben

And if one of the 'OR's changed to an 'XOR', the tests would still pass, but the behaviour would change.

ReplyDeleteTaking advantage of the logical independence of OR produces a gap due to its reliance on the use of OR.

Mind you whilst the level confidence may not be the same as with the 27 test example, the drop may not be particularly significant, thus illustrating the original point I suppose.

You mention, "Limit the exposure by controlling scope appropriately, as we did in the Java code above.".

ReplyDeleteI'm not a java guy but it appears to me that you've made the extracted functions private and not testable from outside this class.

What am I missing?

Gary, the first comment under the first extracted conditional indicates the visibility which is

ReplyDeletepackage.Thanks Tim.

ReplyDeleteJust not paying close enough attention.

yes... the format of this blog isn't as code friendly as an 8.5x11 page. I've been playing with the code format. Small font, or break the lines... sometimes with resulting oddness. Comments on that appreciated.

ReplyDeleteIt is also interesting to look at the NPath complexity metric for a function or method as it gives you a pretty good estimate of how many tests you need.

ReplyDeleteGood example of equivalence class partioning.

ReplyDelete