Although I am a big fan of these patterns you can run into problems with this stuff: Lots of unit tests with mocks etc on a dao layer can turn an agile code base into a paralysed code base; on a simple db change you can't just fix abstractions you have to re-work tests. If an application is heavily data driven the value of the tests is dependent on the quality of your faked data; which is too onerous to maintain. Mocking another system is clearly beneficial but mocking out a layer (which the db can be regarded as) is a lot of high cost effort. I guess you have to be pragmatic; consider: testing with a cut down database, having a fast unit test run and a slower integration test run, removing test code that hampers agility.
Martyn is dead-on. Choices like these have trade-offs to consider. One good test quality which I have difficulty getting people to understand is "obviousness" - ideally a test should look obviously correct. Mocks and stubs may make your tests less obviously correct (thus more difficult to understand and maintain) because they hide part of the test implementation in the stub rather than in the test itself.
For example, in the example given above, the test stub has hardcoded:
return "hello world"
Then further down later in the test is:
IdGetter getter = new IdGetter(new StubDbThatReturnsId()); assertEquals("hello", getter.getIdPrefix());
The test itself is not "obviously" correct without studying the stub. Really, you want the test to be as obviously correct as possible within the test itself. In this case, "obviousness" can be increased with a simple change (the corresponding change to the stub is left as an exercise for the reader):
IdGetter getter = new IdGetter(new StubDbThatReturnsId("hello world")); assertEquals("hello", getter.getIdPrefix());
However, its not always that easy to improve "obviousness" when you're using stubs, especially if you have a lot of them or are reusing stubs across tests and packages.
bert, you're right that sometimes these tests are less obvious. One thing I'd consider is using some sort of mocking framework to create the doubles - they take a small amount of time to learn, but provide much more powerful testing abilities than can easily be had be hand-coding the doubles. The return is well worth the effort expended.
For example: DynamicMock db = new DynamicMock(typeof Database); db.ExpectAndReturn("selectString", "hello world"); IdGetter getter = new IdGetter((Database)db.Object); assertEquals("hello", getter.getIdPrefix());
This way the obviousness is preserved, and the pattern is similar between tests; having a stub throw an exception is just a matter of calling ExpectAndThrow instead of coding up a whole new class.
"If an application is heavily data driven the value of the tests is dependent on the quality of your faked data; which is too onerous to maintain."
That is a problem with state based testing, not caused by mocks/stubs. The whole point of mocks/stubs is to get away from testing state and start testing behavior and interactions.
Essential reading is "Mock Roles Not Objects": http://www.mockobjects.com/files/mockrolesnotobjects.pdf
Although I am a big fan of these patterns you can run into problems with this stuff:
ReplyDeleteLots of unit tests with mocks etc on a dao layer can turn an agile code base into a paralysed code base; on a simple db change you can't just fix abstractions you have to re-work tests.
If an application is heavily data driven the value of the tests is dependent on the quality of your faked data; which is too onerous to maintain.
Mocking another system is clearly beneficial but mocking out a layer (which the db can be regarded as) is a lot of high cost effort.
I guess you have to be pragmatic; consider: testing with a cut down database, having a fast unit test run and a slower integration test run, removing test code that hampers agility.
Martyn is dead-on. Choices like these have trade-offs to consider. One good test quality which I have difficulty getting people to understand is "obviousness" - ideally a test should look obviously correct. Mocks and stubs may make your tests less obviously correct (thus more difficult to understand and maintain) because they hide part of the test implementation in the stub rather than in the test itself.
ReplyDeleteFor example, in the example given above, the test stub has hardcoded:
return "hello world"
Then further down later in the test is:
IdGetter getter = new IdGetter(new StubDbThatReturnsId());
assertEquals("hello", getter.getIdPrefix());
The test itself is not "obviously" correct without studying the stub. Really, you want the test to be as obviously correct as possible within the test itself. In this case, "obviousness" can be increased with a simple change (the corresponding change to the stub is left as an exercise for the reader):
IdGetter getter = new IdGetter(new StubDbThatReturnsId("hello world"));
assertEquals("hello", getter.getIdPrefix());
However, its not always that easy to improve "obviousness" when you're using stubs, especially if you have a lot of them or are reusing stubs across tests and packages.
bert, you're right that sometimes these tests are less obvious. One thing I'd consider is using some sort of mocking framework to create the doubles - they take a small amount of time to learn, but provide much more powerful testing abilities than can easily be had be hand-coding the doubles. The return is well worth the effort expended.
ReplyDeleteFor example:
DynamicMock db = new DynamicMock(typeof Database);
db.ExpectAndReturn("selectString", "hello world");
IdGetter getter = new IdGetter((Database)db.Object);
assertEquals("hello", getter.getIdPrefix());
This way the obviousness is preserved, and the pattern is similar between tests; having a stub throw an exception is just a matter of calling ExpectAndThrow instead of coding up a whole new class.
Ruby's Rspec makes this kind of thing really elegant and readable. I'd post some example text here, but it looks like it'll just get mangled.
ReplyDeleteCheck it some examples here: http://rspec.rubyforge.org/examples.html
"If an application is heavily data driven the value of the tests is dependent on the quality of your faked data; which is too onerous to maintain."
ReplyDeleteThat is a problem with state based testing, not caused by mocks/stubs. The whole point of mocks/stubs is to get away from testing state and start testing behavior and interactions.
Essential reading is "Mock Roles Not Objects":
http://www.mockobjects.com/files/mockrolesnotobjects.pdf