Google Testing Blog: Testing on the Toilet: Don’t Mock Types You Don’t Own

Testing on the Toilet: Don’t Mock Types You Don’t Own

Thursday, July 16, 2020

This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.
By Stefan Kennedy and Andrew Trenk
The code below mocks a third-party library. What problems can arise when doing this?

// Mock a salary payment library
@Mock SalaryProcessor mockSalaryProcessor;
@Mock TransactionStrategy mockTransactionStrategy;
...
when(mockSalaryProcessor.addStrategy()).thenReturn(mockTransactionStrategy);
when(mockSalaryProcessor.paySalary()).thenReturn(TransactionStrategy.SUCCESS);
MyPaymentService myPaymentService = new MyPaymentService(mockSalaryProcessor);
assertThat(myPaymentService.sendPayment()).isEqualTo(PaymentStatus.SUCCESS);

Mocking types you don’t own can make maintenance more difficult:

It can make it harder to upgrade the library to a new version: The expectations of an API hardcoded in a mock can be wrong or get out of date. This may require time-consuming work to manually update your tests when upgrading the library version. In the above example, an update that changes addStrategy() to return a new type derived from TransactionStrategy (e.g. SalaryStrategy) requires the mock to be updated to return this type, even though the code under test doesn’t need to be changed since it can still reference TransactionStrategy.

It can make it harder to know whether a library update introduced a bug in your code: The assumptions built into mocks may get out of date as changes are made to the library, resulting in tests that pass even when the code under test has a bug. In the above example, if a library update changes paySalary() to instead return TransactionStrategy.SCHEDULED, a bug could potentially be introduced due to the code under test not handling this return value properly. However, the maintainer wouldn’t know because the mock would not return this value so the test would continue to pass.

Instead of using a mock, use the real implementation, or if that’s not feasible, use a fake implementation that is ideally provided by the library owner. This reduces the maintenance burden since the issues with mocks listed above don’t occur when using a real or fake implementation. For example:

FakeSalaryProcessor fakeProcessor = new FakeSalaryProcessor(); // Designed for tests
MyPaymentService myPaymentService = new MyPaymentService(fakeProcessor);
assertThat(myPaymentService.sendPayment()).isEqualTo(PaymentStatus.SUCCESS);

If you can’t use the real implementation and a fake implementation doesn’t exist (and library owners aren’t able to create one), create a wrapper class that calls the type, and mock this instead. This reduces the maintenance burden by avoiding mocks that rely on the signatures of the library API. For example:

@Mock MySalaryProcessor mockMySalaryProcessor; // Wraps the SalaryProcessor library
...
// Mock the wrapper class rather than the library itself
when(mockMySalaryProcessor.sendSalary()).thenReturn(PaymentStatus.SUCCESS);

MyPaymentService myPaymentService = new MyPaymentService(mockMySalaryProcessor);
assertThat(myPaymentService.sendPayment()).isEqualTo(PaymentStatus.SUCCESS);

To avoid the problems listed above, prefer to test the wrapper class with calls to the real implementation. The downsides of testing with the real implementation (e.g. tests taking longer to run) are limited only to the tests for this wrapper class rather than tests throughout your codebase.
“Don’t mock types you don’t own” is also described by Steve Freeman and Nat Pryce in their book, Growing Object Oriented Software, Guided by Tests. For more details about the downsides of overusing mocks (even for types you do own), see this Google Testing Blog post.

1 comment :

StuAugust 6, 2020 at 6:06:00 PM PDT
Curious, in this scenario:

>If you can’t use the real implementation and a fake implementation doesn’t exist (and library owners aren’t able to create one), create a wrapper class that calls the type, and mock this instead.

How do you then unit test this wrapper?

And for this example:
>FakeSalaryProcessor fakeProcessor = new FakeSalaryProcessor();

Isn't this just a mock that's created by hand, instead of generated at runtime by a mocking library?
What's the real difference here? Either way you have an object that's not the live, production type, and just implements the same interface, correct?

You could very reasonably argue that how the mocks are created matter - but they're both mocks (in definition used by Mockito, etc., JS libraries have more granular defs between mocks/fakes/stubs/etc).

Also, you've essentially shifted the problems around upgrading a library from the test class to the wrapper class, you haven't removed it (but you have isolated it - possibly from many tests to one single wrapper, so, sort of a win).

Do you apply this prescription outside of the tests?

One thing that annoys me about this approach is then you see bizarre wrappers where you have someone deciding that "they know best" and completely change the interface to a well documented API provided by a third party.

At the very least, you usually end up replacing a well-documented, maintained third-party API with less well-documented less well-maintained API.

While the intentions are good, here, I think it should be clarified where & how those wrappers are created.

Would you be fine with code, that, wherever possible, wraps third-party code in tests, but passes the real objects in the implementation?

Also - what about objects you (as in a team) do own? I've seen folks wrap those, because "Mocks are bad".

Fundamentally, based on the examples, it doesn't look like you're differentiating between mock/not mock, it looks like you're
differentiating between how the mocks are created, maintained, and used.

If you're not using the same instance type as the one that live code uses, you're using a mock, even if you wrote it yourself, or a third-party maintainer wrote it, and decided to call it a "Fake" instead.

I think people could be wiser about when they reach for reflection (via Mockito, etc), to
create mocks for them.. but mocks themselves .. seem inevitable in unit tests.
Otherwise, you're writing integration tests.
ReplyDelete
Replies

Add comment

The comments you read and contribute here belong only to the person who posted them. We reserve the right to remove off-topic comments.

Testing Blog

Testing on the Toilet: Don’t Mock Types You Don’t Own

1 comment :

Labels

Archive

Feed