Posted by Zhanyong Wan, Software Engineer
Life is unfair. You work every bit as hard as Joe the Java programmer next to you. Yet as a C++ programmer, you don't get to play with all the fancy programming tools Joe takes for granted.
In particular, without a good mocking framework, mock objects in C++ have to be rolled by hand. Boy, is that tedious! (Not to mention how error-prone it is.) Why should you endure this?
Dread no more. Google Mock is finally here to help! It's a Google-originated open-source framework for creating and using C++ mocks. Inspired by jMock and EasyMock, Google Mock is easy to use, yet flexible and extensible. All you need to get started is the ability to count from 0 to 10 and use an editor.
Think you can do it? Let's try this simple example: you have a ShoppingCart class that gets the tax rate from a server, and you want to test that it remembers to disconnect from the server even when the server has generated an error. It's easy to write the test using a mock tax server, which implements this interface:
class TaxServer { // Returns the tax rate of a location // (by postal code) or -1 on error. virtual double FetchTaxRate( const string& postal_code) = 0; virtual void CloseConnection() = 0; };
Here's how you mock it and use the mock server to verify the expected behavior of ShoppingCart:
class MockTaxServer : public TaxServer { // #1 MOCK_METHOD1(FetchTaxRate, double(const string&)); MOCK_METHOD0(CloseConnection, void()); }; TEST(ShoppingCartTest, StillCallsCloseIfServerErrorOccurs) { MockTaxServer mock_taxserver; // #2 EXPECT_CALL(mock_taxserver, FetchTaxRate(_)) .WillOnce(Return(-1)); // #3 EXPECT_CALL(mock_taxserver, CloseConnection()); ShoppingCart cart(&mock_taxserver); // #4 cart.CalculateTax(); // Calls FetchTaxRate() // and CloseConnection(). } // #5
Derive the mock class from the interface. For each virtual method, count how many arguments it has, name the result n, and define it using MOCK_METHODn, whose arguments are the name and type of the method.
Create an instance of the mock class. It will be used where you would normally use a real object.
Set expectations on the mock object (How will it be used? What will it do?). For example, the first EXPECT_CALL says that FetchTaxRate() will be called and will return an error. The underscore (_) is a matcher that says the argument can be anything. Google Mock has many matchers you can use to precisely specify what the argument should be like. You can also define your own matcher or use an exact value.
Exercise code that uses the mock object. You'll get an error immediately if a mock method is called more times than expected or with the wrong arguments.
When the mock object is destroyed, it checks that all expectations on it have been satisfied.
You can also use Google Mock for rapid prototyping – and get a better design. To find out more, visit the project homepage at http://code.google.com/p/googlemock/. Now, be the first one on your block to use Google Mock and prepare to be envied. Did I say life is unfair?
To keep our code at Google in the best possible shape we provided our software engineers with these constant reminders. Now, we are happy to share them with the world.
Many thanks to these folks for inspiration and hours of hard work getting this guide done:
Flaw #1: Constructor does Real Work
Warning Signs
Flaw #2: Digging into Collaborators
Flaw #3: Brittle Global State & Singletons
Flaw #4: Class Does Too Much
4 int bottles_of_beer = 0; 5 void Intern1() { bottles_of_beer++; } // Intern1 forgot to use Mutex. 6 void Intern2() { bottles_of_beer++; } // Intern2 copied from Intern1. 7 int main() { 8 // Folks, bring me one bottle of beer each, please. 9 ClosureThread intern1(NewPermanentCallback(Intern1)),10 intern2(NewPermanentCallback(Intern2));11 intern1.SetJoinable(true); intern2.SetJoinable(true);12 intern1.Start(); intern2.Start();13 intern1.Join(); intern2.Join();14 CHECK_EQ(2, bottles_of_beer) << "Who didn't bring me my beer!?";15 }
$ helgrind path/to/your/programPossible data race during read of size 4 at 0x5429C8 at 0x400523: Intern2() tott.cc:6 by 0x400913: _FunctionResultCallback_0_0::Run() ... by 0x4026BB: ClosureThread::Run() ... ... Location 0x5429C8 has never been protected by any lock Location 0x5429C8 is 0 bytes inside global var "bottles_of_beer" declared at tott.cc:4
// Deletes files that are no longer reachable via our storage system's// metadata.class FileCleaner {public: class Env { public: virtual bool MatchFiles(const char* pattern, vector* filenames) = 0; virtual bool BulkDelete(const vector& filenames) = 0; virtual MetadataReader* NewMetadataReader() = 0; virtual ~Env(); }; // Constructs a FileCleaner. Uses “env” to access files and metadata. FileCleaner(Env* env, QuotaManager* qm); // Deletes files that are not reachable via metadata. // Returns true on success. bool CleanOnce();};
class NoFileSystemEnv : public FileCleaner::Env { virtual bool MatchFiles(const char* pattern, vector* filenames) { match_files_called_ = true; return false; } ...};TEST(FileCleanerTest, FileCleaningFailsWhenFileSystemFails) { NoFileSystemEnv* env = new NoFileSystemEnv(); FileCleaner cleaner(env, new MockQuotaManager()); ASSERT_FALSE(cleaner.CleanOnce()); ASSERT_TRUE(env->match_files_called_);}
Testability Explorer: Using Byte-Code Analysis to Engineer Lasting Social Changes in an Organization’s Software Development Process. (Or How to Get Developers to Write Testable Code)
Presented at 2008 OOPSLA by Miško Hevery a Best Practices Coach @ Google
Abstract
Testability Explorer is an open-source tool that identifies hard-to-test Java code. Testability Explorer provides a repeatable objective metric of “testability.” This metric becomes a key component of engineering a social change within an organization of developers. The Testability Explorer report provides actionable information to developers which can be used as (1) measure of progress towards a goal and (2) a guide to refactoring towards a more testable code-base.Keywords: unit-testing; testability; refactoring; byte-code analysis; social engineering.
1. Testability Explorer Overview
In order to unit-test a class, it is important that the class can be instantiated in isolation as part of a unit-test. The most common pitfalls of testing are (1) mixing object-graph instantiation with application-logic and (2) relying on global state. The Testability Explorer can point out both of these pitfalls.
1.1 Non-Mockable Cyclomatic Complexity
Cyclomatic complexity is a count of all possible paths through code-base. For example: a main method will have a large cyclomatic complexity since it is a sum of all of the conditionals in the application. To limit the size of the cyclomatic complexity in a test, a common practice is to replace the collaborators of class-under-test with mocks, stubs, or other test doubles.
Let’s define “non-mockable cyclomatic complexity” as what is left when the class-under-test has all of its accessible collaborators replaced with mocks. A code-base where the responsibility of object-creation and application-logic is separated (using Dependency Injection) will have high degree of accessible collaborators; as a result most of its collaborators will easily be replaceable with mocks, leaving only the cyclomatic complexity of the class-under-test behind.
In applications, where the class-under-test is responsible for instantiating its own collaborators, these collaborators will not be accessible to the test and as a result will not be replaceable for mocks. (There is no place to inject test doubles.) In such classes the cyclomatic complexity will be the sum of the class-under-test and its non-mockable collaborators.
The higher the non-mockable cyclomatic complexity the harder it will be to write a unit-test. Each non-mockable conditional translates to a single unit of cost on the Testability Explorer report. The cost of static class initialization and class construction is automatically included for each method, since a class needs to be instantiated before it can be exercised in a test.
1.2 Transitive Global-State
Good unit-tests can be run in parallel and in any order. To achieve this, the tests need to be well isolated. This implies that only the stimulus from the test has an effect on the code execution, in other words, there is no global-state.
Global-state has a transitive property. If a global variable refers to a class than all of the references of that class (and all of its references) are globally accessible as well. Each globally accessible variable, that is not final, results in a cost of ten units on the Testability Explorer.
2. Testability Explorer Report
A chain is only as strong as its weakest link. Therefore the cost of testing a class is equal to the cost of the class’ costliest method. In the same spirit the application’s overall testability is de-fined in terms of a few un-testable classes rather than a large number of testable ones. For this reason when computing the overall score of a project the un-testable classes are weighted heavier than the testable ones.
3. How to Interpret the Report
By default the classes are categorized into three categories: “Excellent” (green) for classes whose cost is below 50; “Good” (yellow) for classes whose cost is below 100; and “Needs work” (red) for all other classes. For convenience the data is presented as both a pie chart and histogram distribution and overall (weighted average) cost shown on a dial.
[-]ClassRepository [ 323 ] [-]ClassInfo getClass(String) [ 323 ] line 51: ClassInfo parseClass(InputStream) [318] InputStream inputStreamForClass(String) [2] [-]ClassInfo parseClass(InputStream) [318] line 77: void accept(ClassVisitor, int) [302] line 75: ClassReader(InputStream) [15]
Clicking on the class ClassRepository allows one to drill down into the classes to get more information. For example the above report shows that ClassRepository has a high cost of 318 due to the parseClass(InputStream) method. Looking in closer we see that the cost comes from line 77 and an invocation of the accept() method.
73:ClassInfo parseClass(InputStream is) {74: try {75: ClassReader reader = new ClassReader(is);76: ClassBuilder v = new ClassBuilder (this);77: reader.accept(v, 0);78: return visitor.getClassInfo();79: } catch (IOException e) {80: throw new RuntimeException(e);81: }82:}
As you can see from the code the ClassReader can never be replaced for a mock and as a result the cyclomatic complexity of the accept method can not be avoided in a test — resulting in a high testability cost. (Injecting the ClassReader would solve this problem and make the class more test-able.)
4. Social Engineering
In order to produce a lasting change in the behavior of developers it helps to have a measurable number to answer where the project is and where it should be. Such information can provide in-sight into whether or not the project is getting closer or farther from its testability goal.
People respond to what is measured. Integrating the Testability Explorer with the project’s continuous build and publishing the report together with build artifacts communicate the importance of testability to the team. Publishing a graph of overall score over time allows the team to see changes on per check-in basis.
If Testability Explorer is used to identify the areas of code that need to be refactored, than compute the rate of improvement and project expected date of refactoring completion and create a sense of competition among the team members.
It is even possible to set up a unit test for Testability Explorer that will only allow the class whose testability cost is better than some predetermined cost.
by Miško Hevery
After reading the article on Singletons (the design anti-pattern) and how they are really global variables and dependency injection suggestion to simply pass in the reference to the singleton in a constructor (instead of looking them up in global state), many people incorrectly concluded that now they will have to pass the singleton all over the place. Let me demonstrate the myth with the following example.
Let's say that you have a LoginPage which uses the UserRepository for authentication. The UserRepository in turn uses Database Singleton to get a hold of the global reference to the database connection, like this:
class UserRepository { private static final BY_USERNAME_SQL = "Select ..."; User loadUser(String user) { Database db = Database.getInstance(); return db.query(BY_USERNAME_SQL, user); }}class LoginPage { UserRepository repo = new UserRepository(); login(String user, String pwd) { User user = repo.loadUser(user); if (user == null || user.checkPassword(pwd)) { throw new IllegalLoginException(); } }}
The first thought is that if you follow the advice of dependency injection you will have to pass in the Database into the LoginPage just so you can pass it to the UserRepository. The argument goes that this kind of coding will make the code hard to maintain, and understand. Let's see what it would look like after we get rid of the global variable Singleton Database look up.
First, lets have a look at the UserRepository.
class UserRepository { private static final BY_USERNAME_SQL = "Select ..."; private final Database db; UserRepository(Database db) { this.db = db; } User loadUser(String user) { return db.query(BY_USERNAME_SQL, user); }}
Notice how the removal of Singleton global look up has cleaned up the code. This code is now easy to test since in a test we can instantiate a new UserRepository and pass in a fake database connection into the constructor. This improves testability. Before, we had no way to intercept the calls to the Database and hence could never test against a Database fake. Not only did we have no way of intercepting the calls to Database, we did not even know by looking at the API that Database is involved. (see Singletons are Pathological Lairs) I hope everyone would agree that this change of explicitly passing in a Database reference greatly improves the code.
Now lets look what happens to the LoginPage...
class LoginPage { UserRepository repo; LoginPage(Database db) { repo = new UserRepository(db); } login(String user, String pwd) { User user = repo.loadUser(user); if (user == null || user.checkPassword(pwd)) { throw new IllegalLoginException(); } }}
Since UserRepository can no longer do a global look-up to get a hold of the Database it musk ask for it in the constructor. Since LoginPage is doing the construction it now needs to ask for the Databse so that it can pass it to the constructor of the UserRepository. The myth we are describing here says that this makes code hard to understand and maintain. Guess what?! The myth is correct! The code as it stands now is hard to maintain and understand. Does that mean that dependency injection is wrong? NO! it means that you only did half of the work! In how to think about the new operator we go into the details why it is important to separate your business logic from the new operators. Notice how the LoginPage violates this. It calls a new on UserRepository. The issue here is that LoginPage is breaking a the Law of Demeter. LoginPage is asking for the Database even though it itself has no need for the Database (This greatly hinders testability as explained here). You can tell since LoginPage does not invoke any method on the Database. This code, like the myth suggest, is bad! So how do we fix that?
We fix it by doing more Dependency Injection.
class LoginPage { UserRepository repo; LoginPage(UserRepository repo) { this.repo = repo; } login(String user, String pwd) { User user = repo.loadUser(user); if (user == null || user.checkPassword(pwd)) { throw new IllegalLoginException(); } }}
LoginPage needs UserRepository. So instead of trying to construct the UserRepository itself, it should simply ask for the UserRepository in the constructor. The fact that UserRepository needs a reference to Database is not a concern of the LoginPage. Neither is it a concern of LoginPage how to construct a UserRepository. Notice how this LoginPage is now cleaner and easier to test. To test we can simply instantiate a LoginPage and pass in a fake UserRepository with which we can emulate what happens on successful login as well as on unsuccessful login and or exceptions. It also nicely takes care of the concern of this myth. Notice that every object simply knows about the objects it directly interacts with. There is no passing of objects reference just to get them into the right location where they are needed. If you get yourself into this myth then all it means is that you have not fully applied dependency injection.
So here are the two rules of Dependency Injection:
So where have all the new operators gone you ask? Well we have already answered that question here. And with that I hope we have put the myth to rest!
BTW, for those of you which are wondering why this is a common misconception the reason is that people incorrectly assume that the constructor dependency graph and the call graph are inherently identical (see this post). If you construct your objects in-line (as most developers do, see thinking about the new operator) then yes the two graphs are very similar. However, if you separate the object graph instantiation from the its execution, than the two graphs are independent. This independence is what allows us to inject the dependencies directly where they are needed without passing the reference through the intermediary collaborators.
ASSERT_FLOAT_EQ(expected, actual);ASSERT_DOUBLE_EQ(expected, actual);EXPECT_FLOAT_EQ(expected, actual);EXPECT_DOUBLE_EQ(expected, actual);
assertEquals(float expected, float actual, float delta);assertEquals(double expected, double actual, double delta);
TEST(SquareRootTest, CorrectResultForPositiveNumbers) { EXPECT_FLOAT_EQ(2.0f, FloatSquareRoot(4.0f)); EXPECT_FLOAT_EQ(23.3333f, FloatSquareRoot(544.44444f)); EXPECT_DOUBLE_EQ(2.0, DoubleSquareRoot(4.0)); EXPECT_DOUBLE_EQ(23.33333333333333, DoubleSquareRoot(544.44444444444444)); // the above succeeds EXPECT_EQ(2.0, DoubleSquareRoot(4.0)); // the above fails EXPECT_EQ(23.33333333333333, DoubleSquareRoot(544.44444444444444));}
