This could also be extended to UI level tests (Selenium, mobile), where UI element changes should not affect tests unless the user/system flow changes (drastically).
Of course, in order to measure whether you have achieved the promised "several orders of magnitude speedup" you'd also need to have performance tests & a benchmark of your current implementation.
In the last paragraph, you mentioned: "Note that test setup may need to change if the implementation changes (e.g. if you change your class to take a new dependency in its constructor, the test needs to pass in this dependency when it creates the class), but the actual test itself typically shouldn't need to change if the code's user-facing behavior doesn't change".
Isn't this the case for the example above? Calculator takes additional AdderFactor in the constructor and so the existing tests just need to change the setup code. But I belive the actual call to add() doesn't need to be changed in the tests. So the example above is actually acceptable and not demonstrating your point
This actually fits nice to a posting by Jakub Holy "Principles for Creating Maintainable and Evolvable Tests" (2011-11-30): http://bit.ly/17wHKyK
ReplyDeleteAnd my post on Testing Pyramid, Code Coverage and using acceptance tests - even on Unit-Test-Level: http://bit.ly/17wHSOZ
This could also be extended to UI level tests (Selenium, mobile), where UI element changes should not affect tests unless the user/system flow changes (drastically).
ReplyDeleteOf course, in order to measure whether you have achieved the promised "several orders of magnitude speedup" you'd also need to have performance tests & a benchmark of your current implementation.
ReplyDeleteIn the last paragraph, you mentioned: "Note that test setup may need to change if the implementation changes (e.g. if you change your class to take a new dependency in its constructor, the test needs to pass in this dependency when it creates the class), but the actual test itself typically shouldn't need to change if the code's user-facing behavior doesn't change".
ReplyDeleteIsn't this the case for the example above? Calculator takes additional AdderFactor in the constructor and so the existing tests just need to change the setup code. But I belive the actual call to add() doesn't need to be changed in the tests. So the example above is actually acceptable and not demonstrating your point