Google Testing Blog: My Unified Theory of Bugs

My Unified Theory of Bugs

Tuesday, November 18, 2008

Miško Hevery

Logical: Logical bug is the most common and classical "bug." This is your "if"s, "loop"s, and other logic in your code. It is by far the most common kind of bug in an application. (Think: it does the wrong thing)

Wiring: Wiring bug is when two different objects are miswired. For example wiring the first-name to the last-name field. It could also mean that the output of one object is not what the input of the next object expects. (Think: Data gets clobbered in process to where it is needed.)

Rendering: Rendering bug is when the output (typical some UI or a report) does not look right. The key here is that it takes a human to determine what "right" is. (Think: it "looks" wrong)

NOTETypical Application Distribution (without Testability in Mind)

Cost of Finding the BugCost of Fixing the Bug

Logical

Wiring

Rendering

Probability of Occurrence

High

Medium

Low

Difficulty of Discovering

Difficult

Easy

Trivial

Cost of Fixing

High Cost

Medium

Low

How does testability change the distribution?

Clear separation between classes (Testable Seams) --> clear separation between classes makes it less likely that a wiring problem is introduced. Also, less code per class lowers the probability of logical bug.

Dependency Injection --> makes wiring explicit (unlike singletons, globals or service locators).

Clear separation of Logic from Wiring --> by having wiring in a single place it is easier to verify.

We Like Writing Unit-Tests

Unit-tests focus on the logical bugs. Unit tests focus on your "if"s and "loop"s, a Focused unit-test does not directly check the wiring. (and certainly not rendering)

Unit-test are focused on a single CUT (class-under-test). This is important, since you want to make sure that unit-tests will not get in the way of future refactoring. Unit-tests should HELP refactoring not PREVENT refactorings. (Again, when I hear people say that tests prevent refactorings, I know that they have not understood what unit-tests are)

Unit-tests do not directly prove that wiring is OK. They do so only indirectly by forcing you to write more testable code.

Functional tests verify wiring, however there is a trade-off. You "may" have hard time refactoring if you have too many functional test OR, if you mix functional and logical tests.

Managing Your Bugs

6 comments :

zdsbsNovember 18, 2008 at 3:45:00 PM PST
Hey Misko great post, and I enjoy the blog! Is it too presumptuous of me to suggest another category of bugs? How about promiscuous references, where you accidentally share a reference and some where else someone modifies your state.
ReplyDelete
Eric JainNovember 19, 2008 at 11:56:00 AM PST
Does the "wiring" category include bugs that result from wrong assumptions about libraries or frameworks you are using? This (along with failing to take into account some of the things that can happen) is perhaps my most common kind of bug.
ReplyDelete
영회November 19, 2008 at 8:13:00 PM PST
Can you temll me how to figure out the number of bugs?
Anyway, I think this is a very good post... :)
ReplyDelete
vvagellNovember 20, 2008 at 9:24:00 AM PST
> they don't really understand testable-code

I'd love to see a blog post with a top-10 (or top-5, or top-3, whatever) guidelines to follow to produce testable code.

I realize many of your blog posts touch on various aspects of this, but it'd be helpful to see a snapshot of the most important points.
ReplyDelete
MiskoNovember 21, 2008 at 11:16:00 AM PST
Just do opposite of this:
http://misko.hevery.com/2008/07/24/how-to-write-3v1l-untestable-code/
ReplyDelete
Kim GräsmanNovember 23, 2008 at 2:10:00 AM PST
I too think there's a category missing: accidental [1] bugs.

Race conditions, timing issues, crashes, performance problems, for example. Things that aren't essential to your program, but affect it deeply when they don't work.

I didn't manage to classify them into either of the other buckets, but maybe you have more thoughts on this?

Thanks,
- Kim

[1] In the sense of the second definition here: http://en.wiktionary.org/wiki/accidental
ReplyDelete