Misko, This is a great writeup. I've been looking for a good explanation of why statics almost always lead to "badness".
The slow evolution of "simple statics" to "bad statics" is hard for people to spot. It's sort of like a glacier moving. But eventually you blink and take a fresh look and realize a LOT has changed.
You can unit test procedural code. Take a file of functions... add it to a new project. Mock up a file with all the dependant methods and fill them in.
Instead of wiring up objects in the usual way you do for OO, for procedural, you have to recompile the file with its mocked dependencies. You recompile instead of building mocks on the fly and injecting them.
So.. if procedural code is unit-testable... lots of people do it! Then so are static methods, it's just the process is a little different.
However, the advice in this post does sound very useful if you want to avoid recompiling for each unit test... which is a bit more time consuming, and takes a few more steps to automate.
Except when you need to change that MockFile for another test then you'd have to create a new MockFile for each test. This causes a lot of singular use code and ends up being like the author said "scenario testing".
Except when you need to change that MockFile for another test then you'd have to create a new MockFile for each test. This causes a lot of singular use code and ends up being like the author said "scenario testing".
As Scott says you can unit test procedural code if you try hard enough. Michael Feathers has some things to say about this in his book "Working Effectively with Legacy Code" (which incidentally should really be entitled something like "Refactoring for Testing"). You can instantiate part of your application for testing with clever linking or with a preprocessor.
But of course life is much nicer in an OO environment such as Java or Ruby.
Btw Misko I'm really impressed with your blog. Keep up the good work - it is always worth reading!
After reading your explanation, it seems to me that the testing of a static method is not the actual problem. The problem is to test code that calls static methods, because the call graph is hardwired ("there is no way to override"). This is also a reason to avoid static methods even as leaves, since the code that calls it becomes more difficult to test.
By the way, in Python it is possible to override static methods:
class A(object): __@staticmethod __def m(): ____print 'A.m'
__def f(self): ____print 'A.f' ____self.m()
class B(A): __@staticmethod __def m(): ____print 'B.m'
b = B() b.f()
This will print: A.f B.m
In Java you always call a static method on a class: although you can write "variable.staticMethod()", the compiler takes the static type of the variable to determine which method to call. In Python the runtime type of the variable is used, so if the static method is overridden, the overridden version will be called.
Therefore I think that when unit testing Python code, static methods are not a problem, as long as you call them on instances or classes passed to you instead of hardcoded class names.
It seems the OO world is in conflict over this very subject, and you have done an extremely good job of addressing it. I really enjoy your views on testability.
However, I do have some thought's I'd like to share. First, what are we as developers supposed to do when faced with API's that don't "comply" to a "no-static-methods-allowed" philosophy? I can't really rewrite Java to change Integers to have a sqrt() method. As developers we are largely stuck with what we're given in that aspect. I don't think you addressed this forgivingly enough in your post :). I think something to the point of "though your language may make heavy use of static methods, under no circumstances create your own static methods" would have come across well.
Also, I understand the significant difference between a singleton and a single shared reference of an object, for testing purposes, but isn't a container-managed (eg. Spring-configured) object nearly the same as a singleton object? I mean, if I inject a single shared instance of an object to all dependent classes via Spring, I could easily achieve the same thing with a singleton. I'm not advocating the use of a singleton, because I know that they are truly death to OOP, I agree with you on that one. But I want to understand why you argue that even factory methods are bad! The factory method pattern hides a lot of ugly details from developers and exemplifies OOP encapsulation. I really want to understand where you're coming from.
Anyway, these are just my thoughts on this matter. Honestly, thank you for a very conclusive post on this matter :)
Misko, so your Java guru Joshua Bloch got it all wrong when he recommends replacing constructors with static factories?
You are using Math.abs() in your example, which is just mathematical function. So, if abs() was an instance, non final method, that you could override with some goofy implementation to create a mock that returns negative values ( yes you can do it on ruby ), would you wire this mock object as a dependancy in your client objects?
I think static methods are fine as long as they do not modify any state, basically utility methods, that will never change. I think Math.abs() and all other utility methods in Math class are the perfect example of a valid usage of static methods. No one should ever change them. And it is not that hard to test them. assertTrue(Math.abs(Integer.MIN_VALUE) > 0)
I certainly agree, that static methods that modify a state should never be used.
Did you deliberately not account for functional programming style? In that case, a lot of methods could be thought of as "static", but could actually be very testable: * Much less reliance on state (so very little work for tests to set up state) * Functions accepting other functions as parameters gives you a perfect hook for testability.
I would _guess_ that functional programmes are perhaps some of the easiest to test?
Having said all that, true functional programming seems to be used quite rarely outside academia, so I agree wholeheartedly with what you've said, when put in the context of every codebase I've ever worked on! :)
Chances are very good that you can move the method as an instance method to one of the method's arguments. (As in method(a,b) becomes a.method(b).) Once you move it you realized that that is where the method should have been to begin with.
What about when a and b are not of object types that you created? In a web app I have a utility class with a method:
public String htmlHighlightSearchTerm(String resultTxt, String searchTxt)
This method returns a copy of resultTxt with <b> and </b> inserted around occurrences of searchTxt. This seems to me the perfect case for a static helper method, and it should be easily testable.
I do agree that using static methods when object methods make more sense is a good coding practice that will lead to testable code, but when the need arises for static helper methods, it's important to make sure they have a clear mapping of inputs to expected outputs.
If you change the title to "Static State is Death to Testability", then I agree with you. However, as others have pointed out, static methods that are pure functions are examples of functional, not procedural code, and are thus (typically) easy to test. Math.abs() falls into this category, and using it as your primary example really weakens your argument.
I'm quite sure that you know how to test an implementation of Math.abs(), and this have nothing to do with it's algorithmic simplicity. It is easy to test because it does not modify any static state. Having nothing to mock is exactly the point!
Turning everything into an instantiable object is also a bad idea. It leads to confusion, is error-prone, and is cognitively wasteful.
The real solution is for languages to allow for interfaces containing just static methods, and for the language to allow you to parametrize over different implementations of those interfaces.
The ML-style module system is EXACTLY this. A signature is an interface which is just a bag of functions and values. The implementation of this signature is called a structure. A functor is a structure that takes other structures of a given signature as arguments. More generally, the module system also includes something called abstract data types... which allows you to perform polymorphism to do EXACTLY what you talk about in your getting-rid-of-ifs talks.
Haskell's typeclasses is another approach to this.
Please please please... read about this. Please please please.
Stanimir, you are right, it should be assertFalse to pass the test, because of the numeric overflow. I should had thought about it before posting, or run the test :)
"If your application has no global state than all of the input for your static method must come from its arguments. Chances are very good that you can move the method as an instance method to one of the method's arguments. (As in method(a,b) becomes a.method(b).) Once you move it you realized that that is where the method should have been to begin with."
That's the money quote right there. Excellent point!
I don't know that I fully understand what you mean by not "wireing" something up. What if you have a simple static append function (fair warning I use groovy)...
The point I am trying to make here is that the value you are passing can be any value, and needs no relation to the class itself, the class is just the logical place to store it.
this seems like a simple enough method to test(and maybe I even want to limit override access). I do get what your saying when it comes to maybe limiting extending of the class, however, I am just not convinced that you shouldn't ever use statics except purely procedural classes.
It seems the main argument against static methods is that they are too easy to violate good programming practice with. I think the same can be said with a purely object oriented approach. Just replace "if the static method calls another static method" with "if the instance method creates another object using 'new' and calls its instance method" and you've got the same problem.
More importantly, banning static methods goes against the trend towards adding functional programming features to the language. An example would be filtering values in an Iterable. Mandating instance methods means either adding the filter method to all instances of Iterable or creating a wrapper object with a filter method. Neither are optimal solutions, hence the move towards higher order programming. Higher order programming requires functions, which in the Java world translate to static methods. Extension methods are essentially syntactic sugar for static functions. Furthermore, the closer you get to funcational programming, the more sure you can be of your tests. So how does this trend fit with the no statics mandate?
They ended up adding filter() to Stream and adding stream() to all instances of *Collection*, which is close enough to adding it to Iterable. (There are arguments about whether it should also be on Iterable.)
"Functions" don't have to be implemented with static methods either. You can if you want.
I really don't agree, statics are far from the death of testability. Many times working with legacy code, I rely on static methods paired with dependency injection seams for isolating functionality that in the past relied too much on global variable states. A interface seam or data proxy can be introduced into the parameters and be loaded with the implementation of calls to the problematic areas you mention. And that implementation can be different for your tests. Maybe it's not for everyone, but for retrofitting unit tests, I find the characteristics of statics to be extremely helpful when decoupling logic from a presentation layer in old code, prior to promoting it into a business logic layer.
Now, ten years after, Java8 is long out of the door and Functional Programming has reached even Java Developers.
With the now better understand of how static methods should be designed in a functional way (and not being "procedural" except in a very technical point of view), this article has become way outdated.
Misko,
ReplyDeleteThis is a great writeup. I've been looking for a good explanation of why statics almost always lead to "badness".
The slow evolution of "simple statics" to "bad statics" is hard for people to spot. It's sort of like a glacier moving. But eventually you blink and take a fresh look and realize a LOT has changed.
Thanks!
Patrick
You can unit test procedural code. Take a file of functions... add it to a new project. Mock up a file with all the dependant methods and fill them in.
ReplyDeleteInstead of wiring up objects in the usual way you do for OO, for procedural, you have to recompile the file with its mocked dependencies. You recompile instead of building mocks on the fly and injecting them.
So.. if procedural code is unit-testable... lots of people do it! Then so are static methods, it's just the process is a little different.
However, the advice in this post does sound very useful if you want to avoid recompiling for each unit test... which is a bit more time consuming, and takes a few more steps to automate.
Except when you need to change that MockFile for another test then you'd have to create a new MockFile for each test. This causes a lot of singular use code and ends up being like the author said "scenario testing".
DeleteExcept when you need to change that MockFile for another test then you'd have to create a new MockFile for each test. This causes a lot of singular use code and ends up being like the author said "scenario testing".
DeleteAs Scott says you can unit test procedural code if you try hard enough. Michael Feathers has some things to say about this in his book "Working Effectively with Legacy Code" (which incidentally should really be entitled something like "Refactoring for Testing"). You can instantiate part of your application for testing with clever linking or with a preprocessor.
ReplyDeleteBut of course life is much nicer in an OO environment such as Java or Ruby.
Btw Misko I'm really impressed with your blog. Keep up the good work - it is always worth reading!
After reading your explanation, it seems to me that the testing of a static method is not the actual problem. The problem is to test code that calls static methods, because the call graph is hardwired ("there is no way to override"). This is also a reason to avoid static methods even as leaves, since the code that calls it becomes more difficult to test.
ReplyDeleteBy the way, in Python it is possible to override static methods:
ReplyDeleteclass A(object):
__@staticmethod
__def m():
____print 'A.m'
__def f(self):
____print 'A.f'
____self.m()
class B(A):
__@staticmethod
__def m():
____print 'B.m'
b = B()
b.f()
This will print:
A.f
B.m
In Java you always call a static method on a class: although you can write "variable.staticMethod()", the compiler takes the static type of the variable to determine which method to call. In Python the runtime type of the variable is used, so if the static method is overridden, the overridden version will be called.
Therefore I think that when unit testing Python code, static methods are not a problem, as long as you call them on instances or classes passed to you instead of hardcoded class names.
It seems the OO world is in conflict over this very subject, and you have done an extremely good job of addressing it. I really enjoy your views on testability.
ReplyDeleteHowever, I do have some thought's I'd like to share. First, what are we as developers supposed to do when faced with API's that don't "comply" to a "no-static-methods-allowed" philosophy? I can't really rewrite Java to change Integers to have a sqrt() method. As developers we are largely stuck with what we're given in that aspect. I don't think you addressed this forgivingly enough in your post :). I think something to the point of "though your language may make heavy use of static methods, under no circumstances create your own static methods" would have come across well.
Also, I understand the significant difference between a singleton and a single shared reference of an object, for testing purposes, but isn't a container-managed (eg. Spring-configured) object nearly the same as a singleton object? I mean, if I inject a single shared instance of an object to all dependent classes via Spring, I could easily achieve the same thing with a singleton. I'm not advocating the use of a singleton, because I know that they are truly death to OOP, I agree with you on that one. But I want to understand why you argue that even factory methods are bad! The factory method pattern hides a lot of ugly details from developers and exemplifies OOP encapsulation. I really want to understand where you're coming from.
Anyway, these are just my thoughts on this matter. Honestly, thank you for a very conclusive post on this matter :)
Misko, so your Java guru Joshua Bloch got it all wrong when he recommends replacing constructors with static factories?
ReplyDeleteYou are using Math.abs() in your example, which is just mathematical function. So, if abs() was an instance, non final method, that you could override with some goofy implementation to create a mock that returns negative values ( yes you can do it on ruby ), would you wire this mock object as a dependancy in your client objects?
I think static methods are fine as long as they do not modify any state, basically utility methods, that will never change. I think Math.abs() and all other utility methods in Math class are the perfect example of a valid usage of static methods. No one should ever change them. And it is not that hard to test them.
assertTrue(Math.abs(Integer.MIN_VALUE) > 0)
I certainly agree, that static methods that modify a state should never be used.
Static methods and constructors can be mocked in Java, using PowerMock. http://code.google.com/p/powermock/
ReplyDeleteI've tried it; it works, much to my surprise.
Great article.
ReplyDeleteDid you deliberately not account for functional programming style? In that case, a lot of methods could be thought of as "static", but could actually be very testable:
* Much less reliance on state (so very little work for tests to set up state)
* Functions accepting other functions as parameters gives you a perfect hook for testability.
I would _guess_ that functional programmes are perhaps some of the easiest to test?
Having said all that, true functional programming seems to be used quite rarely outside academia, so I agree wholeheartedly with what you've said, when put in the context of every codebase I've ever worked on! :)
One problem I have with this generalization:
ReplyDeleteChances are very good that you can move the method as an instance method to one of the method's arguments. (As in method(a,b) becomes a.method(b).) Once you move it you realized that that is where the method should have been to begin with.
What about when a and b are not of object types that you created? In a web app I have a utility class with a method:
public String htmlHighlightSearchTerm(String resultTxt, String searchTxt)
This method returns a copy of resultTxt with <b> and </b> inserted around occurrences of searchTxt. This seems to me the perfect case for a static helper method, and it should be easily testable.
I do agree that using static methods when object methods make more sense is a good coding practice that will lead to testable code, but when the need arises for static helper methods, it's important to make sure they have a clear mapping of inputs to expected outputs.
Grammar Nazi -
ReplyDeleteDiff of corrections here.
If you change the title to "Static State is Death to Testability", then I agree with you. However, as others have pointed out, static methods that are pure functions are examples of functional, not procedural code, and are thus (typically) easy to test. Math.abs() falls into this category, and using it as your primary example really weakens your argument.
ReplyDeleteI'm quite sure that you know how to test an implementation of Math.abs(), and this have nothing to do with it's algorithmic simplicity. It is easy to test because it does not modify any static state. Having nothing to mock is exactly the point!
Turning everything into an instantiable object is also a bad idea. It leads to confusion, is error-prone, and is cognitively wasteful.
ReplyDeleteThe real solution is for languages to allow for interfaces containing just static methods, and for the language to allow you to parametrize over different implementations of those interfaces.
The ML-style module system is EXACTLY this. A signature is an interface which is just a bag of functions and values. The implementation of this signature is called a structure. A functor is a structure that takes other structures of a given signature as arguments. More generally, the module system also includes something called abstract data types... which allows you to perform polymorphism to do EXACTLY what you talk about in your getting-rid-of-ifs talks.
Haskell's typeclasses is another approach to this.
Please please please... read about this. Please please please.
On the algorithm side we are seeing intensive semantic methods emerge, which may in time challenge statistical methods like pagerank.
ReplyDeleteso you can't test anything that has no internal state.
ReplyDeletefor tests you can probably wrap around any object and delegate to the static methods.
btw - the line below's the best in the entire article :)
assertTrue(Math.abs(Integer.MIN_VALUE) > 0)
[should be asserFalse]. Kudos!
Stanimir,
ReplyDeleteyou are right, it should be assertFalse to pass the test, because of the numeric overflow. I should had thought about it before posting, or run the test :)
too bad x.x
ReplyDeleteI thought 'twas a satiric moment. (indeed it was the best one)
Will google give us any audio hosting. sunil kumar lalwani sksoftmind
ReplyDelete"If your application has no global state than all of the input for your static method must come from its arguments. Chances are very good that you can move the method as an instance method to one of the method's arguments. (As in method(a,b) becomes a.method(b).) Once you move it you realized that that is where the method should have been to begin with."
ReplyDeleteThat's the money quote right there. Excellent point!
I don't know that I fully understand what you mean by not "wireing" something up. What if you have a simple static append function (fair warning I use groovy)...
ReplyDeleteclass TestClass{
String foo
void someMethod(){
String extractedInfo = extract(foo)
println testMethod(extractedInfo)
}
static String testMethod(String test){
return "${test}end"
}
}
The point I am trying to make here is that the value you are passing can be any value, and needs no relation to the class itself, the class is just the logical place to store it.
this seems like a simple enough method to test(and maybe I even want to limit override access). I do get what your saying when it comes to maybe limiting extending of the class, however, I am just not convinced that you shouldn't ever use statics except purely procedural classes.
Nicely written. My sentiments exactly!
ReplyDeleteIt seems the main argument against static methods is that they are too easy to violate good programming practice with. I think the same can be said with a purely object oriented approach. Just replace "if the static method calls another static method" with "if the instance method creates another object using 'new' and calls its instance method" and you've got the same problem.
ReplyDeleteMore importantly, banning static methods goes against the trend towards adding functional programming features to the language. An example would be filtering values in an Iterable. Mandating instance methods means either adding the filter method to all instances of Iterable or creating a wrapper object with a filter method. Neither are optimal solutions, hence the move towards higher order programming. Higher order programming requires functions, which in the Java world translate to static methods. Extension methods are essentially syntactic sugar for static functions. Furthermore, the closer you get to funcational programming, the more sure you can be of your tests. So how does this trend fit with the no statics mandate?
They ended up adding filter() to Stream and adding stream() to all instances of *Collection*, which is close enough to adding it to Iterable. (There are arguments about whether it should also be on Iterable.)
ReplyDelete"Functions" don't have to be implemented with static methods either. You can if you want.
I really don't agree, statics are far from the death of testability. Many times working with legacy code, I rely on static methods paired with dependency injection seams for isolating functionality that in the past relied too much on global variable states. A interface seam or data proxy can be introduced into the parameters and be loaded with the implementation of calls to the problematic areas you mention. And that implementation can be different for your tests. Maybe it's not for everyone, but for retrofitting unit tests, I find the characteristics of statics to be extremely helpful when decoupling logic from a presentation layer in old code, prior to promoting it into a business logic layer.
ReplyDeleteNow, ten years after, Java8 is long out of the door and Functional Programming has reached even Java Developers.
ReplyDeleteWith the now better understand of how static methods should be designed in a functional way (and not being "procedural" except in a very technical point of view), this article has become way outdated.
Ruby got it right as it took it from Smalltalk: -5 abs
ReplyDelete