Isn't this counterintuitive to the testing pyramid as it encourages larger tests? I'm sort of playing devils advocate here since I agree that it's often better to test to the API.
The important thing is that you're getting the proper coverage from your tests. Often this might mean you need to pull in additional classes that your code uses in order to properly test your class.
I don't think this advice necessarily conflicts with the testing pyramid idea of focusing on smaller tests since you can still test small pieces of code this way. But having them exercise several classes instead of one class shouldn't be a problem.
UserInfoValidator requires unit test (userInfo should be mocked), simple code requires only simple tests. UserInfoService#save also requires unit testing, mock userInfo and the validator, also write an integration test for UserInfoService#save.
Testing indirectly causes bad defect localization and stronger coupling between tests and seemingly unrelated classes, thus making the tests less useful by not being able to pinpoint the defect line of code as a result of a broken test and also the test is more fragile.. changes in 3 classes could cause an non-isolated test to fail that only tests UserInfoService directly and the UserInfoValidator and UserInfo classes indirectly.
Mocks are good for some specific cases but they shouldn't be used as a general tool to remove dependencies on every single class you happen to be using. These two posts have some more details about this:
Defect localization isn't an issue here as long as you're still testing each public API. If an implementation detail class is complex enough that you wouldn't be able to figure out the problem from testing the public API then the implementation detail class should have tests too.
A public method is a public API irrespective of whether it is used by one other method or not. The whole discussion over implementation detail made no sense.
A public method should be tested irrespective of how it is implemented. The mistake the author is makes is that he/she has looked at the source code of the method and gone:
• “[oh it’s using some implementation method, I can’t be bothered testing it]” • “[oh it’s only being used by one other method, I will adjust my testing accordingly]”
A class to be tested should be treated as a black box; make no assumptions over it; and just test it as potentially toxic. You should never look at the source code for something you are intending to test.
The only time you would do the latter if you are attempting to break the code to uncover a weakness or security flaw.
You're right that a class should be tested as a black box. But not all types of classes are equal. Just because a developer decides to put some logic in its own class doesn't automatically mean the logic needs to be tested in depth.
The decision on how much testing a class needs isn't based on its usage, not its implementation: if a class is only used in one place and no one else can access it then it's not a public API. So this has nothing to do with its implementation.
Isn't this counterintuitive to the testing pyramid as it encourages larger tests? I'm sort of playing devils advocate here since I agree that it's often better to test to the API.
ReplyDeleteThe important thing is that you're getting the proper coverage from your tests. Often this might mean you need to pull in additional classes that your code uses in order to properly test your class.
DeleteI don't think this advice necessarily conflicts with the testing pyramid idea of focusing on smaller tests since you can still test small pieces of code this way. But having them exercise several classes instead of one class shouldn't be a problem.
I disagree with the conclusion of the article.
ReplyDeleteUserInfoValidator requires unit test (userInfo should be mocked), simple code requires only simple tests.
UserInfoService#save also requires unit testing, mock userInfo and the validator, also write an integration test for UserInfoService#save.
Testing indirectly causes bad defect localization and stronger coupling between tests and seemingly unrelated classes, thus making the tests less useful by not being able to pinpoint the defect line of code as a result of a broken test and also the test is more fragile.. changes in 3 classes could cause an non-isolated test to fail that only tests UserInfoService directly and the UserInfoValidator and UserInfo classes indirectly.
Mocks are good for some specific cases but they shouldn't be used as a general tool to remove dependencies on every single class you happen to be using. These two posts have some more details about this:
Deletehttp://googletesting.blogspot.com/2013/05/testing-on-toilet-dont-overuse-mocks.html
http://googletesting.blogspot.com/2015/01/testing-on-toilet-change-detector-tests.html
Defect localization isn't an issue here as long as you're still testing each public API. If an implementation detail class is complex enough that you wouldn't be able to figure out the problem from testing the public API then the implementation detail class should have tests too.
A public method is a public API irrespective of whether it is used by one other method or not. The whole discussion over implementation detail made no sense.
ReplyDeleteA public method should be tested irrespective of how it is implemented. The mistake the author is makes is that he/she has looked at the source code of the method and gone:
• “[oh it’s using some implementation method, I can’t be bothered testing it]”
• “[oh it’s only being used by one other method, I will adjust my testing accordingly]”
A class to be tested should be treated as a black box; make no assumptions over it; and just test it as potentially toxic. You should never look at the source code for something you are intending to test.
The only time you would do the latter if you are attempting to break the code to uncover a weakness or security flaw.
You're right that a class should be tested as a black box. But not all types of classes are equal. Just because a developer decides to put some logic in its own class doesn't automatically mean the logic needs to be tested in depth.
ReplyDeleteThe decision on how much testing a class needs isn't based on its usage, not its implementation: if a class is only used in one place and no one else can access it then it's not a public API. So this has nothing to do with its implementation.