Testing on the Toilet: Keep Tests Focused
Monday, June 11, 2018
This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.
By Ben Yu
What scenario does the following code test?
By Ben Yu
What scenario does the following code test?
TEST_F(BankAccountTest, WithdrawFromAccount) {
Transaction transaction = account_.Deposit(Usd(5));
clock_.AdvanceTime(MIN_TIME_TO_SETTLE);
account_.Settle(transaction);
EXPECT_THAT(account_.Withdraw(Usd(5)), IsOk());
EXPECT_THAT(account_.Withdraw(Usd(1)), IsRejected());
account_.SetOverdraftLimit(Usd(1));
EXPECT_THAT(account_.Withdraw(Usd(1)), IsOk());
}
|
Translated to English: “(1) I had $5 and was able to withdraw $5; (2) then got rejected when overdrawing $1; (3) but if I enable overdraft with a $1 limit, I can withdraw $1.” If that sounds a little hard to track, it is: it is testing three scenarios, not one.
A better approach is to exercise each scenario in its own test:
TEST_F(BankAccountTest, CanWithdrawWithinBalance) {
DepositAndSettle(Usd(5)); // Common setup code is extracted into a helper method.
EXPECT_THAT(account_.Withdraw(Usd(5)), IsOk());
}
TEST_F(BankAccountTest, CannotOverdraw) {
DepositAndSettle(Usd(5));
EXPECT_THAT(account_.Withdraw(Usd(6)), IsRejected());
}
TEST_F(BankAccountTest, CanOverdrawUpToOverdraftLimit) {
DepositAndSettle(Usd(5));
account_.SetOverdraftLimit(Usd(1));
EXPECT_THAT(account_.Withdraw(Usd(6)), IsOk());
}
|
Writing tests this way provides many benefits:
- Logic is easier to understand because there is less code to read in each test method.
- Setup code in each test is simpler because it only needs to serve a single scenario.
- Side effects of one scenario will not accidentally invalidate or mask a later scenario’s assumptions.
- If a scenario in one test fails, other scenarios will still run since they are unaffected by the failure.
- Test names clearly describe each scenario, which makes it easier to learn which scenarios exist.
One sign that you might be testing more than one scenario: after asserting the output of one call to the system under test, the test makes another call to the system under test.
While a scenario for a unit test often consists of a single call to the system under test, its scope can be larger for integration and end-to-end tests. For example, a test that a web UI can send email might open the inbox, click the compose button, write some text, and press the send button.
More benefits:
ReplyDelete* You can pick and choose specific scenarios and test EXACTLY what you wanna test at a given moment.
* You get the shortest testing time since you don't have to "pre-run" the first scenario before running the overdraft scenario or You don't have to wait for the overdraft scenario to finish for to get the PASS from the first scenario.
Nice post, and I agree in general, but sometimes the issue is in chain of things, like in this case maybe after a failed attempt setting overdraft limit and re-try will not work.
ReplyDeleteI would say check it in isolation and also have basic scenarios checked as well.
Thanks well done, took me some time for me to understand Usd(5); might be better to use Java Lib. improve readability:
ReplyDeletejava.util.Currency usd = java.util.Currency.getInstance("USD");
Nice post. Just the kind of information I was searching for.
ReplyDeleteNitpicky but missing "a" in "account" in the first code example.
EXPECT_THAT(ccount_.Withdraw(Usd(1)), IsOk());
Fixed. Thanks!
ReplyDeleteAs a general rule, Helper methods should be avoided because it scatters context.
ReplyDeleteDuplicating the helper method body inside the test cases is Ok as it gives all the context necessary to understand the test case.