Testing Blog

How Google Tests Software - Part One

Tuesday, January 25, 2011
Share on Twitter Share on Facebook
Google
Labels: James Whittaker

12 comments :

  1. LaurentJanuary 26, 2011 at 1:15:00 AM PST

    Thanks for the interesting post.

    2 questions about : "So this means that testers report to Eng Prod managers but identify themselves with a product team"

    1) does this also apply to developpers ?

    2) Where are Eng Prod Managers sitting ? Meaning : are they embeded in projects also or they stay outsite ?

    Thanks,
    Laurent

    ReplyDelete
    Replies
      Reply
  2. Idd Salim (Cdr)January 26, 2011 at 5:33:00 AM PST

    Surreal!

    ReplyDelete
    Replies
      Reply
  3. Nick DreweJanuary 26, 2011 at 5:41:00 AM PST

    Thanks for the insight. I can fully understand how testing is a very tricky area for a behemoth as large as Google, especially with the diversity of products you have.

    Our company, Market Dojo, finds testing challenging enough, even though we have just the one piece of software!

    ReplyDelete
    Replies
      Reply
  4. AnonymousJanuary 27, 2011 at 2:20:00 AM PST

    Hi James,

    Enjoy reading these posts. Keen to hear more about how Google tests, particularly, what methods/processes are used in estimations.

    With dev's doing the lions share of the testing, how are estimates gathered for testing? Is there a particular process used for estimation (or one you could recommend)?

    Thanks.

    ReplyDelete
    Replies
      Reply
  5. Darren McMillanJanuary 27, 2011 at 2:30:00 AM PST

    Hi James,

    This setup is very similar to how our company works and like at Google it works very well for us.

    One line though I'd have probably have been more careful about is this:

    "The better they are at testing the more they outnumber us. Product teams should be proud of a high ratio!"

    I think you'll find you'll have upset a lot of your testers at Google with that.

    Cem Kaner wrote a very good paper around metrics, I’d say this falls into an unwritten management metric which is probably best left unspoken.

    http://www.kaner.com/pdfs/measurement_segue.pdf

    Cheers,

    Darren

    ReplyDelete
    Replies
      Reply
  6. Manoj NarayananJanuary 27, 2011 at 7:11:00 AM PST

    Would like to add few comments.

    1. Looks like there is a de-facto matrix structure for the testers here with dual reporting to product manager and core Eng Prod Mgr. Works well if the tester also has a long term career option linked to the Eng Prod team - basically some one to take care of the career, skill set development etc.

    2. There could also be problems if the product line suffers budget constraints - who insists on minimum quality then? Is there a minimum RMI that each product team has to sign up for before release?

    3. As we split test engineering and testing roles - eagerly looking forward to this explanation in the next post - it might be worthwhile to look at engaging testing service vendors as a lower cost option for the testing portion. Might be a good bridge between dog fooding and crowd testing, especially if you can throw in few SLA adherence requirements.

    ReplyDelete
    Replies
      Reply
  7. UnknownJanuary 27, 2011 at 8:29:00 AM PST

    Hi James,

    Very interesting post. Looking forward to reading the rest of the series.

    It would be interesting to see your Test Case and defect management systems and see how they stack up against ones that are commonly used throughout the industry.

    I bet they're slicker and sexier than the majority of the ones available on the market!

    Regards,

    Adam Brown
    http://www.gobanana.co.uk

    ReplyDelete
    Replies
      Reply
  8. zecarreraJanuary 28, 2011 at 6:17:00 AM PST

    Great post, already anxious for the others...

    ReplyDelete
    Replies
      Reply
  9. MikeJanuary 28, 2011 at 6:19:00 AM PST

    Really interested to know whether Google promotes TDD/BDD or whether they use a test-after approach. If TDD/BDD, how is it enforced/maintained?

    ReplyDelete
    Replies
      Reply
  10. Kiran BadiFebruary 9, 2011 at 11:25:00 PM PST

    Exactly the reason I have always felt that IT organisation structure are disfunctional in nature.Google is pretty much the same structure.
    So basically governance lies with product teams and testing belongs to service lines.Huge pie of success if it comes by goes to product teams and small pie comes to service lines whereas in real sense the real execution of those products is done by service lines.Just because product team conceives a great product idea does not assure you a success, it needs that greats ideas needs to have strong implementation strategies and to do this we need to ensure that we are making everyone accountable for their role and share the burden equally.
    I do not have right solution now but problem statement is pretty much clear, you have operational issue to deal with in your plate.
    Your structure is one of the reasons why I feel google lacks quality now a days.

    Hope you are not reading my comments in wrong spirit,sometimes I often research on the ways organisations are structured.

    ReplyDelete
    Replies
      Reply
  11. UnknownMarch 7, 2012 at 7:29:00 AM PST

    Thanks for the interesting post
    100 points

    ReplyDelete
    Replies
      Reply
  12. UnknownMay 22, 2017 at 5:29:00 PM PDT

    Hello James,

    I'm reading the book "How Google Tests Software" and I'm going to share the information in this book with my friends in a workshop.

    So could you please confirm if the information in that book (like roles, testing process, testing philosophy) is still correct and is till using at Google now?

    Thanks in advance,
    Long Lee

    ReplyDelete
    Replies
      Reply
Add comment
Load more...

The comments you read and contribute here belong only to the person who posted them. We reserve the right to remove off-topic comments.

  

Labels


  • TotT 77
  • GTAC 61
  • James Whittaker 42
  • Misko Hevery 32
  • Anthony Vallone 27
  • Patrick Copeland 23
  • Jobs 18
  • Code Health 13
  • C++ 11
  • Andrew Trenk 10
  • Patrik Höglund 8
  • JavaScript 7
  • Allen Hutchison 6
  • George Pirocanac 6
  • Zhanyong Wan 6
  • Harry Robinson 5
  • Java 5
  • Julian Harty 5
  • Alberto Savoia 4
  • Ben Yu 4
  • Erik Kuefler 4
  • Philip Zembrod 4
  • Shyam Seshadri 4
  • Chrome 3
  • John Thomas 3
  • Lesley Katzen 3
  • Marc Kaplan 3
  • Markus Clermont 3
  • Sonal Shah 3
  • APIs 2
  • Abhishek Arya 2
  • Adam Bender 2
  • Alan Myrvold 2
  • Alek Icev 2
  • Android 2
  • April Fools 2
  • Chaitali Narla 2
  • Chris Lewis 2
  • Chrome OS 2
  • Diego Salas 2
  • Dillon Bly 2
  • Dori Reuveni 2
  • Jason Arbon 2
  • Jochen Wuttke 2
  • Kostya Serebryany 2
  • Marc Eaddy 2
  • Marko Ivanković 2
  • Max Kanat-Alexander 2
  • Mobile 2
  • Oliver Chang 2
  • Simon Stewart 2
  • Stefan Kennedy 2
  • Test Flakiness 2
  • Tony Voellm 2
  • WebRTC 2
  • Yvette Nameth 2
  • Zuri Kemp 2
  • Aaron Jacobs 1
  • Adam Porter 1
  • Adel Saoud 1
  • Alan Faulkner 1
  • Alex Eagle 1
  • Anantha Keesara 1
  • Antoine Picard 1
  • App Engine 1
  • Ari Shamash 1
  • Arif Sukoco 1
  • Benjamin Pick 1
  • Bob Nystrom 1
  • Bruce Leban 1
  • Carlos Arguelles 1
  • Carlos Israel Ortiz García 1
  • Cathal Weakliam 1
  • Christopher Semturs 1
  • Clay Murphy 1
  • Dan Shi 1
  • Dan Willemsen 1
  • Dave Chen 1
  • Dave Gladfelter 1
  • Derek Snyder 1
  • Diego Cavalcanti 1
  • Dmitry Vyukov 1
  • Eduardo Bravo Ortiz 1
  • Ekaterina Kamenskaya 1
  • Elliott Karpilovsky 1
  • Espresso 1
  • Google+ 1
  • Goran Petrovic 1
  • Goranka Bjedov 1
  • Hank Duan 1
  • Havard Rast Blok 1
  • Hongfei Ding 1
  • Jason Elbaum 1
  • Jason Huggins 1
  • Jay Han 1
  • Jeff Listfield 1
  • Jessica Tomechak 1
  • Jim Reardon 1
  • Joe Allan Muharsky 1
  • Joel Hynoski 1
  • John Micco 1
  • John Penix 1
  • Jonathan Rockway 1
  • Jonathan Velasquez 1
  • Josh Armour 1
  • Julie Ralph 1
  • Karin Lundberg 1
  • Kaue Silveira 1
  • Kevin Bourrillion 1
  • Kevin Graney 1
  • Kirkland 1
  • Kurt Alfred Kluever 1
  • Manjusha Parvathaneni 1
  • Marek Kiszkis 1
  • Mark Ivey 1
  • Mark Striebeck 1
  • Matt Lowrie 1
  • Meredith Whittaker 1
  • Michael Bachman 1
  • Michael Klepikov 1
  • Mike Aizatsky 1
  • Mike Wacker 1
  • Mona El Mahdy 1
  • Noel Yap 1
  • Patricia Legaspi 1
  • Peter Arrenbrecht 1
  • Peter Spragins 1
  • Phil Rollet 1
  • Pooja Gupta 1
  • Project Showcase 1
  • Radoslav Vasilev 1
  • Rajat Dewan 1
  • Rajat Jain 1
  • Rich Martin 1
  • Richard Bustamante 1
  • Roshan Sembacuttiaratchy 1
  • Ruslan Khamitov 1
  • Sean Jordan 1
  • Sharon Zhou 1
  • Siddartha Janga 1
  • Simran Basi 1
  • Stephen Ng 1
  • Tejas Shah 1
  • Test Analytics 1
  • Test Engineer 1
  • Tom O'Neill 1
  • Vojta Jína 1
  • iOS 1
  • mutation testing 1


Archive


  • ►  2022 (2)
    • ►  Feb (2)
  • ►  2021 (3)
    • ►  Jun (1)
    • ►  Apr (1)
    • ►  Mar (1)
  • ►  2020 (8)
    • ►  Dec (2)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Aug (2)
    • ►  Jul (1)
    • ►  May (1)
  • ►  2019 (4)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Jul (1)
    • ►  Jan (1)
  • ►  2018 (7)
    • ►  Nov (1)
    • ►  Sep (1)
    • ►  Jul (1)
    • ►  Jun (2)
    • ►  May (1)
    • ►  Feb (1)
  • ►  2017 (17)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Sep (1)
    • ►  Aug (1)
    • ►  Jul (2)
    • ►  Jun (2)
    • ►  May (3)
    • ►  Apr (2)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2016 (15)
    • ►  Dec (1)
    • ►  Nov (2)
    • ►  Oct (1)
    • ►  Sep (2)
    • ►  Aug (1)
    • ►  Jun (2)
    • ►  May (3)
    • ►  Apr (1)
    • ►  Mar (1)
    • ►  Feb (1)
  • ►  2015 (14)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (2)
    • ►  Aug (1)
    • ►  Jun (1)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (1)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2014 (24)
    • ►  Dec (2)
    • ►  Nov (1)
    • ►  Oct (2)
    • ►  Sep (2)
    • ►  Aug (2)
    • ►  Jul (3)
    • ►  Jun (3)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (2)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2013 (16)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Aug (2)
    • ►  Jul (1)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (2)
    • ►  Jan (2)
  • ►  2012 (11)
    • ►  Dec (1)
    • ►  Nov (2)
    • ►  Oct (3)
    • ►  Sep (1)
    • ►  Aug (4)
  • ▼  2011 (39)
    • ►  Nov (2)
    • ►  Oct (5)
    • ►  Sep (2)
    • ►  Aug (4)
    • ►  Jul (2)
    • ►  Jun (5)
    • ►  May (4)
    • ►  Apr (3)
    • ►  Mar (4)
    • ►  Feb (5)
    • ▼  Jan (3)
      • How Google Tests Software - Part One
      • Google Innovation and The Pretotyping Manifesto
      • New Year's Resolutions
  • ►  2010 (37)
    • ►  Dec (3)
    • ►  Nov (3)
    • ►  Oct (4)
    • ►  Sep (8)
    • ►  Aug (3)
    • ►  Jul (3)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (3)
    • ►  Mar (3)
    • ►  Feb (2)
    • ►  Jan (1)
  • ►  2009 (54)
    • ►  Dec (3)
    • ►  Nov (2)
    • ►  Oct (3)
    • ►  Sep (5)
    • ►  Aug (4)
    • ►  Jul (15)
    • ►  Jun (8)
    • ►  May (3)
    • ►  Apr (2)
    • ►  Feb (5)
    • ►  Jan (4)
  • ►  2008 (75)
    • ►  Dec (6)
    • ►  Nov (8)
    • ►  Oct (9)
    • ►  Sep (8)
    • ►  Aug (9)
    • ►  Jul (9)
    • ►  Jun (6)
    • ►  May (6)
    • ►  Apr (4)
    • ►  Mar (4)
    • ►  Feb (4)
    • ►  Jan (2)
  • ►  2007 (41)
    • ►  Oct (6)
    • ►  Sep (5)
    • ►  Aug (3)
    • ►  Jul (2)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (7)
    • ►  Mar (5)
    • ►  Feb (5)
    • ►  Jan (4)

Feed

follow us in feedly
  • Google
  • Privacy
  • Terms