Testing Blog

The Plague of Homelessness

Thursday, July 23, 2009
Share on Twitter Share on Facebook
Google
Labels: James Whittaker

8 comments :

  1. JoeJuly 23, 2009 at 2:35:00 PM PDT

    "These are some number of bugs that simply cannot be found until the house is lived in and software is no different. It needs to be in the hands of real users doing real work with real data in real environments."


    And hence - multi-year Betas?

    ReplyDelete
    Replies
      Reply
  2. Mark RoddyJuly 23, 2009 at 9:00:00 PM PDT

    James,
    Is your dissertation available? That sounds like an interesting read.

    -Mark

    ReplyDelete
    Replies
      Reply
  3. UnknownJuly 24, 2009 at 5:43:00 AM PDT

    When I was managing a QA Group, there were two things I would do to get my Analysts “on the floor” with the Users;

    1) I would have them physically sit with the Business (in this case- Sales, Cust Service, Financial Reporting, Print Room, etc.) for a few hours or a whole day and observe how the Users actually used the system. Inevitably, they would also get an earful from the Users about how things could be improved and why was the network so slow and why can’t they get to Gmail, anyway?! This did a few things: it gave the QA people a point of contact in the Business, it showed them actual usage, it gave them some pain points the Users were facing which they could later keep tucked in mind while testing, and it helped foster good will with the Users for IT showing that we were actually listening to them and not just sitting on the other side of the wall, throwing software over whenever we felt like screwing up their day.

    2) A little while after a project was released, I liked to have my folks go out to the Business and interview the project sponsor and/or whoever conducted the UAT. If it was a 1-day to 1-week effort, check back in a week; while projects bigger than that would have drop-ins or email follow-ups at one week and one month, just to see if things were meeting their expectations. Certainly, this isn't the job of the QA/STE group, but because we worked so closely with the end-users, I/we found it really useful to have that direct interaction with them to get our finger on the pulse, at the source and not through some sterile after-action report, post-mortem, or project retrospective (though those are all useful tools, too). This relationship also gave us first-line information on any concerns that might not be defects, per se, but were pieces of functionality that the end user just didn’t think were quite right. Again, this first-hand feedback, cobbled together over the many, many projects, made us as a group that much better able to emulate the actions, flows, and true usage patterns of the end user.

    In the sense of Homelessness, these things made us frequent visitors to these home-owners, so much so that we could drop in and borrow a cup of sugar and a bagel, or ask to run a print job to see how jobs were queuing up. If you don’t have a home (and if you are a cost center), you better become good friends with those folks with the fancy place on the hill, (the ones who actually pay the bills).

    ReplyDelete
    Replies
      Reply
  4. Júlio Esslinger ViégasJuly 26, 2009 at 8:44:00 AM PDT

    Hi,

    I've been reading your posts and found them to be very interesting. I'm from Brazil and there are a number of coleagues of mine that aren't confortable reading english. May I unofficially "translate" your posts and post them in portuguese? Of course I would reference the original post.

    Thanks.

    ReplyDelete
    Replies
      Reply
  5. Martin HJuly 27, 2009 at 3:46:00 AM PDT

    As already mentioned UAT and blind user tests will fine some of these issues.

    However there are many that will only come around over time.

    Long Beta's and where this can't be done then offering discounts to select early "adopters" in return for reports on living with the software over time are also methods where you can keep the user community with you on our journey while not keeping User faith and making sure the company image isn't damaged.

    ReplyDelete
    Replies
      Reply
  6. James WhittakerJuly 27, 2009 at 12:10:00 PM PDT

    Julio: yes you may. I am honored to be asked. Just tell your soccer team to stop beating ours!

    ReplyDelete
    Replies
      Reply
  7. James WhittakerJuly 27, 2009 at 12:17:00 PM PDT

    Well Joe, better late than too soon I suppose?!? But well aimed my friend. I think that the word beta is an honest label for the break-in period. How long should the break in be? I am still understanding the ins and outs of this labeling scheme at Google. Before I joined it seemed too conservative. I'll keep you posted about how my feelings toward this change.

    ReplyDelete
    Replies
      Reply
  8. James WhittakerJuly 27, 2009 at 12:22:00 PM PDT

    Mark, my dissertation was published in 1992 from the University of Tennessee. You can search their site for it as I no longer have an electronic copy. It may also reside in various places that such documents hang out. There is also a paper that summarizes the key result in IEEE Trans. on Software Engineering in 1994 available through the IEEE Digital Library. I hope you do find it interesting. It was a large part of my existence for many years, so much so that I'm glad it's in my rear-view mirror!

    ReplyDelete
    Replies
      Reply
Add comment
Load more...

The comments you read and contribute here belong only to the person who posted them. We reserve the right to remove off-topic comments.

  

Labels


  • TotT 104
  • GTAC 61
  • James Whittaker 42
  • Misko Hevery 32
  • Code Health 31
  • Anthony Vallone 27
  • Patrick Copeland 23
  • Jobs 18
  • Andrew Trenk 13
  • C++ 11
  • Patrik Höglund 8
  • JavaScript 7
  • Allen Hutchison 6
  • George Pirocanac 6
  • Zhanyong Wan 6
  • Harry Robinson 5
  • Java 5
  • Julian Harty 5
  • Adam Bender 4
  • Alberto Savoia 4
  • Ben Yu 4
  • Erik Kuefler 4
  • Philip Zembrod 4
  • Shyam Seshadri 4
  • Chrome 3
  • Dillon Bly 3
  • John Thomas 3
  • Lesley Katzen 3
  • Marc Kaplan 3
  • Markus Clermont 3
  • Max Kanat-Alexander 3
  • Sonal Shah 3
  • APIs 2
  • Abhishek Arya 2
  • Alan Myrvold 2
  • Alek Icev 2
  • Android 2
  • April Fools 2
  • Chaitali Narla 2
  • Chris Lewis 2
  • Chrome OS 2
  • Diego Salas 2
  • Dori Reuveni 2
  • Jason Arbon 2
  • Jochen Wuttke 2
  • Kostya Serebryany 2
  • Marc Eaddy 2
  • Marko Ivanković 2
  • Mobile 2
  • Oliver Chang 2
  • Simon Stewart 2
  • Stefan Kennedy 2
  • Test Flakiness 2
  • Titus Winters 2
  • Tony Voellm 2
  • WebRTC 2
  • Yiming Sun 2
  • Yvette Nameth 2
  • Zuri Kemp 2
  • Aaron Jacobs 1
  • Adam Porter 1
  • Adam Raider 1
  • Adel Saoud 1
  • Alan Faulkner 1
  • Alex Eagle 1
  • Amy Fu 1
  • Anantha Keesara 1
  • Antoine Picard 1
  • App Engine 1
  • Ari Shamash 1
  • Arif Sukoco 1
  • Benjamin Pick 1
  • Bob Nystrom 1
  • Bruce Leban 1
  • Carlos Arguelles 1
  • Carlos Israel Ortiz García 1
  • Cathal Weakliam 1
  • Christopher Semturs 1
  • Clay Murphy 1
  • Dagang Wei 1
  • Dan Maksimovich 1
  • Dan Shi 1
  • Dan Willemsen 1
  • Dave Chen 1
  • Dave Gladfelter 1
  • David Bendory 1
  • David Mandelberg 1
  • Derek Snyder 1
  • Diego Cavalcanti 1
  • Dmitry Vyukov 1
  • Eduardo Bravo Ortiz 1
  • Ekaterina Kamenskaya 1
  • Elliott Karpilovsky 1
  • Elliotte Rusty Harold 1
  • Espresso 1
  • Felipe Sodré 1
  • Francois Aube 1
  • Gene Volovich 1
  • Google+ 1
  • Goran Petrovic 1
  • Goranka Bjedov 1
  • Hank Duan 1
  • Havard Rast Blok 1
  • Hongfei Ding 1
  • Jason Elbaum 1
  • Jason Huggins 1
  • Jay Han 1
  • Jeff Hoy 1
  • Jeff Listfield 1
  • Jessica Tomechak 1
  • Jim Reardon 1
  • Joe Allan Muharsky 1
  • Joel Hynoski 1
  • John Micco 1
  • John Penix 1
  • Jonathan Rockway 1
  • Jonathan Velasquez 1
  • Josh Armour 1
  • Julie Ralph 1
  • Kai Kent 1
  • Kanu Tewary 1
  • Karin Lundberg 1
  • Kaue Silveira 1
  • Kevin Bourrillion 1
  • Kevin Graney 1
  • Kirkland 1
  • Kurt Alfred Kluever 1
  • Manjusha Parvathaneni 1
  • Marek Kiszkis 1
  • Marius Latinis 1
  • Mark Ivey 1
  • Mark Manley 1
  • Mark Striebeck 1
  • Matt Lowrie 1
  • Meredith Whittaker 1
  • Michael Bachman 1
  • Michael Klepikov 1
  • Mike Aizatsky 1
  • Mike Wacker 1
  • Mona El Mahdy 1
  • Noel Yap 1
  • Palak Bansal 1
  • Patricia Legaspi 1
  • Per Jacobsson 1
  • Peter Arrenbrecht 1
  • Peter Spragins 1
  • Phil Norman 1
  • Phil Rollet 1
  • Pooja Gupta 1
  • Project Showcase 1
  • Radoslav Vasilev 1
  • Rajat Dewan 1
  • Rajat Jain 1
  • Rich Martin 1
  • Richard Bustamante 1
  • Roshan Sembacuttiaratchy 1
  • Ruslan Khamitov 1
  • Sam Lee 1
  • Sean Jordan 1
  • Sebastian Dörner 1
  • Sharon Zhou 1
  • Shiva Garg 1
  • Siddartha Janga 1
  • Simran Basi 1
  • Stan Chan 1
  • Stephen Ng 1
  • Tejas Shah 1
  • Test Analytics 1
  • Test Engineer 1
  • Tim Lyakhovetskiy 1
  • Tom O'Neill 1
  • Vojta Jína 1
  • automation 1
  • dead code 1
  • iOS 1
  • mutation testing 1


Archive


  • ►  2025 (1)
    • ►  Jan (1)
  • ►  2024 (13)
    • ►  Dec (1)
    • ►  Oct (1)
    • ►  Sep (1)
    • ►  Aug (1)
    • ►  Jul (1)
    • ►  May (3)
    • ►  Apr (3)
    • ►  Mar (1)
    • ►  Feb (1)
  • ►  2023 (14)
    • ►  Dec (2)
    • ►  Nov (2)
    • ►  Oct (5)
    • ►  Sep (3)
    • ►  Aug (1)
    • ►  Apr (1)
  • ►  2022 (2)
    • ►  Feb (2)
  • ►  2021 (3)
    • ►  Jun (1)
    • ►  Apr (1)
    • ►  Mar (1)
  • ►  2020 (8)
    • ►  Dec (2)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Aug (2)
    • ►  Jul (1)
    • ►  May (1)
  • ►  2019 (4)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Jul (1)
    • ►  Jan (1)
  • ►  2018 (7)
    • ►  Nov (1)
    • ►  Sep (1)
    • ►  Jul (1)
    • ►  Jun (2)
    • ►  May (1)
    • ►  Feb (1)
  • ►  2017 (17)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Sep (1)
    • ►  Aug (1)
    • ►  Jul (2)
    • ►  Jun (2)
    • ►  May (3)
    • ►  Apr (2)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2016 (15)
    • ►  Dec (1)
    • ►  Nov (2)
    • ►  Oct (1)
    • ►  Sep (2)
    • ►  Aug (1)
    • ►  Jun (2)
    • ►  May (3)
    • ►  Apr (1)
    • ►  Mar (1)
    • ►  Feb (1)
  • ►  2015 (14)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (2)
    • ►  Aug (1)
    • ►  Jun (1)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (1)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2014 (24)
    • ►  Dec (2)
    • ►  Nov (1)
    • ►  Oct (2)
    • ►  Sep (2)
    • ►  Aug (2)
    • ►  Jul (3)
    • ►  Jun (3)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (2)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2013 (16)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Aug (2)
    • ►  Jul (1)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (2)
    • ►  Jan (2)
  • ►  2012 (11)
    • ►  Dec (1)
    • ►  Nov (2)
    • ►  Oct (3)
    • ►  Sep (1)
    • ►  Aug (4)
  • ►  2011 (39)
    • ►  Nov (2)
    • ►  Oct (5)
    • ►  Sep (2)
    • ►  Aug (4)
    • ►  Jul (2)
    • ►  Jun (5)
    • ►  May (4)
    • ►  Apr (3)
    • ►  Mar (4)
    • ►  Feb (5)
    • ►  Jan (3)
  • ►  2010 (37)
    • ►  Dec (3)
    • ►  Nov (3)
    • ►  Oct (4)
    • ►  Sep (8)
    • ►  Aug (3)
    • ►  Jul (3)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (3)
    • ►  Mar (3)
    • ►  Feb (2)
    • ►  Jan (1)
  • ▼  2009 (54)
    • ►  Dec (3)
    • ►  Nov (2)
    • ►  Oct (3)
    • ►  Sep (5)
    • ►  Aug (4)
    • ▼  Jul (15)
      • How to think about OO
      • Call for Attendance: Google Test Automation Confer...
      • The Plague of Blindness
      • Update! GTAC Keynote Speakers: Niklaus Wirth and A...
      • The Plague of Homelessness
      • Blog Stats
      • The Plague of Boredom
      • Blog Editor: Moderating Comments
      • The Advantages of Unit Testing Early
      • Software Testing Categorization
      • The Plague of Amnesia
      • Update! Google Test Automation Conference: Website...
      • Old habits die hard
      • Why are we embarrassed to admit that we don't know...
      • Separation anxiety?
    • ►  Jun (8)
    • ►  May (3)
    • ►  Apr (2)
    • ►  Feb (5)
    • ►  Jan (4)
  • ►  2008 (75)
    • ►  Dec (6)
    • ►  Nov (8)
    • ►  Oct (9)
    • ►  Sep (8)
    • ►  Aug (9)
    • ►  Jul (9)
    • ►  Jun (6)
    • ►  May (6)
    • ►  Apr (4)
    • ►  Mar (4)
    • ►  Feb (4)
    • ►  Jan (2)
  • ►  2007 (41)
    • ►  Oct (6)
    • ►  Sep (5)
    • ►  Aug (3)
    • ►  Jul (2)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (7)
    • ►  Mar (5)
    • ►  Feb (5)
    • ►  Jan (4)

Feed

  • Google
  • Privacy
  • Terms