Google Testing Blog: TotT: The Invisible Branch

TotT: The Invisible Branch

Wednesday, May 28, 2008

Full statement coverage may be necessary for good testing coverage, but it isn't sufficient. Two places where statement coverage will be inadequate are branches and loops. In this episode, we'll look at branches, and specifically the differences between statement coverage and branch coverage.Let's consider a case where branch coverage and statement coverage aren't the same. Suppose we test the following snippet. We can get complete statement coverage with a single test by using a berserk EvilOverLord:bool DeathRay::ShouldFire(EvilOverLord& o, Target& t) {
  double accumulated_rage = 0.0;

  if (o.IsBerserk())
    accumulated_rage += kEvilOverlordBerserkRage;

  accumulated_rage += o.RageFeltTowards(t);
  return (accumulated_rage > kDeathRayRageThreshold);
}
But what if DeathRay should fire at this Target even with a non-berserk Overlord? Well, we need another test for that. What should the test be? Let's rewrite the code a little bit. We would never see code like this in the real world, but it'll help us clarify an important point.bool DeathRay::ShouldFire(EvilOverLord& o, Target& t) {
  double accumulated_rage = 0.0;

  if (o.IsBerserk()) {
    accumulated_rage += kEvilOverlordBerserkRage;
  } else {
  }

  accumulated_rage += o.RageFeltTowards(t);
  return (accumulated_rage > kDeathRayRageThreshold);
}
Why do we add an else clause if it doesn't actually do anything? If you were to draw a flowchart of both snippets (left as an exercise – and we recommend against using the paper provided), the flowcharts would be identical. The fact that the else isn't there in the first snippet is simply a convenience for us as coders – we generally don't want to write code to do nothing special – but the branch still exists... put another way, every if has an else. Some of them just happen to be invisible.When you're testing, then, it isn't enough to cover all the statements – you should cover all the the edges in the control flow graph – which can be even more complicated with loops and nested ifs. In fact, part of the art of large-scale white-box testing is finding the minimum number of tests to cover the maximum number of paths. So the lesson here is, just because you can't see a branch doesn't mean it isn't there – or that you shouldn't test it.

3 comments :

pteMay 28, 2008 at 11:19:00 PM PDT
Hey Dave,

Your urinating friends might be interested in a handy tool for measuring coverage. In Java land Emma, Corbutura and Clover are by far the most common coverage tools.

Clover is the only tool that gives you per test coverage. That is, it tells you which test hit which class/method/statement/conditional. So you can see that the right test covered your if, rather than just any test. Check out the covererage report for Google-Guice.

Apologies for the blatant plug boys. now focus on your aim.

Cheers,
Pete.
ReplyDelete
Replies
wlievensMay 29, 2008 at 2:15:00 AM PDT
Check out mutation testing, it's a technique for testing the quality of your test suite. The principle is simple:

a) in the current state, all tests pass
b) make an arbitrary modification to the code under test that breaks functionality
c) if your tests still pass, they are lacking
ReplyDelete
Replies
tat-tvam-asiJune 25, 2008 at 4:41:00 AM PDT
This comment has been removed by the author.
ReplyDelete
Replies

Add comment

The comments you read and contribute here belong only to the person who posted them. We reserve the right to remove off-topic comments.

Testing Blog

TotT: The Invisible Branch

3 comments :

Labels

Archive

Feed