Thanks for the insight. I can fully understand how testing is a very tricky area for a behemoth as large as Google, especially with the diversity of products you have.
Our company, Market Dojo, finds testing challenging enough, even though we have just the one piece of software!
Enjoy reading these posts. Keen to hear more about how Google tests, particularly, what methods/processes are used in estimations.
With dev's doing the lions share of the testing, how are estimates gathered for testing? Is there a particular process used for estimation (or one you could recommend)?
1. Looks like there is a de-facto matrix structure for the testers here with dual reporting to product manager and core Eng Prod Mgr. Works well if the tester also has a long term career option linked to the Eng Prod team - basically some one to take care of the career, skill set development etc.
2. There could also be problems if the product line suffers budget constraints - who insists on minimum quality then? Is there a minimum RMI that each product team has to sign up for before release?
3. As we split test engineering and testing roles - eagerly looking forward to this explanation in the next post - it might be worthwhile to look at engaging testing service vendors as a lower cost option for the testing portion. Might be a good bridge between dog fooding and crowd testing, especially if you can throw in few SLA adherence requirements.
Very interesting post. Looking forward to reading the rest of the series.
It would be interesting to see your Test Case and defect management systems and see how they stack up against ones that are commonly used throughout the industry.
I bet they're slicker and sexier than the majority of the ones available on the market!
Exactly the reason I have always felt that IT organisation structure are disfunctional in nature.Google is pretty much the same structure. So basically governance lies with product teams and testing belongs to service lines.Huge pie of success if it comes by goes to product teams and small pie comes to service lines whereas in real sense the real execution of those products is done by service lines.Just because product team conceives a great product idea does not assure you a success, it needs that greats ideas needs to have strong implementation strategies and to do this we need to ensure that we are making everyone accountable for their role and share the burden equally. I do not have right solution now but problem statement is pretty much clear, you have operational issue to deal with in your plate. Your structure is one of the reasons why I feel google lacks quality now a days.
Hope you are not reading my comments in wrong spirit,sometimes I often research on the ways organisations are structured.
I'm reading the book "How Google Tests Software" and I'm going to share the information in this book with my friends in a workshop.
So could you please confirm if the information in that book (like roles, testing process, testing philosophy) is still correct and is till using at Google now?
Thanks for the interesting post.
SvarSlet2 questions about : "So this means that testers report to Eng Prod managers but identify themselves with a product team"
1) does this also apply to developpers ?
2) Where are Eng Prod Managers sitting ? Meaning : are they embeded in projects also or they stay outsite ?
Thanks,
Laurent
Surreal!
SvarSletThanks for the insight. I can fully understand how testing is a very tricky area for a behemoth as large as Google, especially with the diversity of products you have.
SvarSletOur company, Market Dojo, finds testing challenging enough, even though we have just the one piece of software!
Hi James,
SvarSletEnjoy reading these posts. Keen to hear more about how Google tests, particularly, what methods/processes are used in estimations.
With dev's doing the lions share of the testing, how are estimates gathered for testing? Is there a particular process used for estimation (or one you could recommend)?
Thanks.
Hi James,
SvarSletThis setup is very similar to how our company works and like at Google it works very well for us.
One line though I'd have probably have been more careful about is this:
"The better they are at testing the more they outnumber us. Product teams should be proud of a high ratio!"
I think you'll find you'll have upset a lot of your testers at Google with that.
Cem Kaner wrote a very good paper around metrics, I’d say this falls into an unwritten management metric which is probably best left unspoken.
http://www.kaner.com/pdfs/measurement_segue.pdf
Cheers,
Darren
Would like to add few comments.
SvarSlet1. Looks like there is a de-facto matrix structure for the testers here with dual reporting to product manager and core Eng Prod Mgr. Works well if the tester also has a long term career option linked to the Eng Prod team - basically some one to take care of the career, skill set development etc.
2. There could also be problems if the product line suffers budget constraints - who insists on minimum quality then? Is there a minimum RMI that each product team has to sign up for before release?
3. As we split test engineering and testing roles - eagerly looking forward to this explanation in the next post - it might be worthwhile to look at engaging testing service vendors as a lower cost option for the testing portion. Might be a good bridge between dog fooding and crowd testing, especially if you can throw in few SLA adherence requirements.
Hi James,
SvarSletVery interesting post. Looking forward to reading the rest of the series.
It would be interesting to see your Test Case and defect management systems and see how they stack up against ones that are commonly used throughout the industry.
I bet they're slicker and sexier than the majority of the ones available on the market!
Regards,
Adam Brown
http://www.gobanana.co.uk
Great post, already anxious for the others...
SvarSletReally interested to know whether Google promotes TDD/BDD or whether they use a test-after approach. If TDD/BDD, how is it enforced/maintained?
SvarSletExactly the reason I have always felt that IT organisation structure are disfunctional in nature.Google is pretty much the same structure.
SvarSletSo basically governance lies with product teams and testing belongs to service lines.Huge pie of success if it comes by goes to product teams and small pie comes to service lines whereas in real sense the real execution of those products is done by service lines.Just because product team conceives a great product idea does not assure you a success, it needs that greats ideas needs to have strong implementation strategies and to do this we need to ensure that we are making everyone accountable for their role and share the burden equally.
I do not have right solution now but problem statement is pretty much clear, you have operational issue to deal with in your plate.
Your structure is one of the reasons why I feel google lacks quality now a days.
Hope you are not reading my comments in wrong spirit,sometimes I often research on the ways organisations are structured.
Thanks for the interesting post
SvarSlet100 points
Hello James,
SvarSletI'm reading the book "How Google Tests Software" and I'm going to share the information in this book with my friends in a workshop.
So could you please confirm if the information in that book (like roles, testing process, testing philosophy) is still correct and is till using at Google now?
Thanks in advance,
Long Lee