Testing Blog

Minimizing Unreproducible Bugs

Monday, February 03, 2014
Share on Google+ Share on Twitter Share on Facebook
Google
Labels: Anthony Vallone

10 comments :

  1. Feyisayo AkinboboyeFebruary 4, 2014 at 2:04:00 AM PST

    Simply stated and enlightening. I am certainly going to incorporate these as soon much as possible. Thanks for sharing your thoughts.

    ReplyDelete
    Replies
      Reply
  2. Ken LeeFebruary 4, 2014 at 10:42:00 AM PST

    I wrote code that enabled me to read SQL 2000 deadlocks from the log by converting it into text that showed the tables and indexes involved and the process that was running. Certainly was a bug we couldn't reproduce but with the log we were able to figure out the odd locking order that was causing it with a read and an update.
    The update locked the table page and then tried to lock the index. The read had tried to use the index to read the table. The update couldn't lock the index, the read couldn't look at the data page.
    Altered the read to retrieve just the clustered PK field values. (A covering index read) Then read the table using the PK values so it is delayed by the update, not deadlocked.

    ReplyDelete
    Replies
      Reply
  3. Ken LeeFebruary 4, 2014 at 10:49:00 AM PST

    PS Your sign-in procedure exposed a bug. I wrote a comment, did not have a Google account, so signed up for it. Google sign-up procedure was successful, but then blew up when I tried to return to submit my comment. Had to rewrite the comment after going back in web history to your article.
    So, did I just find an unreproducible bug in your blog?

    ReplyDelete
    Replies
    1. Anthony ValloneFebruary 4, 2014 at 11:48:00 AM PST

      Hi Ken, Thanks for letting us know. Actually, this might belong in the reproducible category :) I reported this bug to the blogger team.

      Delete
      Replies
        Reply
    2. Reply
  4. Michael Kingsford GrayFebruary 7, 2014 at 10:26:00 PM PST

    Most "unreproducible bugs" are caused by using the execrable "C" language, with its inbuilt design flaws.
    Anyone programming in "C" (for a safety critical application) should be indicted for crimes against humanity.
    The total cost to the world for avoidable bugs directly as a result of employing "C" code or libraries possibly exceeds several billions of dollars.
    C++ is a willing accessory to these crimes against humanity.

    ReplyDelete
    Replies
      Reply
  5. Amandeep SinghFebruary 10, 2014 at 7:08:00 AM PST

    Good one Ken :D

    ReplyDelete
    Replies
      Reply
  6. Amandeep SinghFebruary 10, 2014 at 7:11:00 AM PST

    Well written Anthony - thought provoking... I have been in testing for more than 6 years now and it happens all the time when a reported bug doesn't reproduce especially when a developer updates you about it...

    This is a great list of guidelines to follow... thanks for sharing...

    Regards,
    Aman

    ReplyDelete
    Replies
      Reply
  7. kimkullingMarch 12, 2014 at 3:57:00 AM PDT

    Great article, thanks for sharing it!

    ReplyDelete
    Replies
      Reply
  8. AliMarch 27, 2014 at 4:24:00 PM PDT

    Interesting. We recently conducted an empirical study to characterize non-reproducible bugs. If you're interested, the paper is available here: http://salt.ece.ubc.ca/publications/docs/mona-msr14.pdf

    ReplyDelete
    Replies
      Reply
  9. Conrad BraamApril 22, 2014 at 6:21:00 AM PDT

    I'm not going to agree with Kingsford Gray on the C language bias standpoint at all. That's not the topic. But it is the paradigm! Some parts of a testers job do involve understanding the specific vulnerabilities the software under test has, based not only on historical knowledge of the kind of bugs your dev team inject, but also on the kind of bugs the software production process itself injects. As well as the compiler-toolchain and finally the language chosen to implement the software. So back to taking out locks and synchronization, when it comes to test-code I'm going to disagree with the author Mr Vallone.
    Test code and instrumentation code introduces synchronisation windows in the software being tested a lot of the time. Normally the "window" is quite wide like when turning on logging all threads end up having to access one log file. The tester who knows the exact affect this has on the system as well as about all the other synchronization points in the software will be able to exploit them to find timing bugs.
    And on this point I am your keen follower Anthony, being able to test timeouts properly as if they are simply another axis to your test driven approaches or simply another table-driven test test-case can uncover lots of bugs the devs might take years to find. Great guide, packed with nuggets :)

    ReplyDelete
    Replies
      Reply
Add comment
Load more...

The comments you read and contribute here belong only to the person who posted them. We reserve the right to remove off-topic comments.

  

Labels


  • TotT 70
  • GTAC 61
  • James Whittaker 42
  • Misko Hevery 32
  • Anthony Vallone 27
  • Patrick Copeland 23
  • Jobs 17
  • C++ 11
  • Code Health 10
  • Andrew Trenk 9
  • Patrik Höglund 8
  • JavaScript 7
  • Allen Hutchison 6
  • Zhanyong Wan 6
  • Harry Robinson 5
  • Java 5
  • Julian Harty 5
  • Alberto Savoia 4
  • Ben Yu 4
  • Philip Zembrod 4
  • Shyam Seshadri 4
  • Chrome 3
  • Erik Kuefler 3
  • John Thomas 3
  • Lesley Katzen 3
  • Marc Kaplan 3
  • Markus Clermont 3
  • Sonal Shah 3
  • APIs 2
  • Abhishek Arya 2
  • Alek Icev 2
  • Android 2
  • April Fools 2
  • Chaitali Narla 2
  • Chris Lewis 2
  • Chrome OS 2
  • Diego Salas 2
  • Dillon Bly 2
  • Dori Reuveni 2
  • George Pirocanac 2
  • Jason Arbon 2
  • Jochen Wuttke 2
  • Kostya Serebryany 2
  • Marc Eaddy 2
  • Max Kanat-Alexander 2
  • Mobile 2
  • Oliver Chang 2
  • Simon Stewart 2
  • Tony Voellm 2
  • WebRTC 2
  • Yvette Nameth 2
  • Zuri Kemp 2
  • Aaron Jacobs 1
  • Adam Bender 1
  • Adam Porter 1
  • Alan Faulkner 1
  • Alan Myrvold 1
  • Alex Eagle 1
  • Antoine Picard 1
  • App Engine 1
  • Ari Shamash 1
  • Arif Sukoco 1
  • Benjamin Pick 1
  • Bob Nystrom 1
  • Bruce Leban 1
  • Christopher Semturs 1
  • Clay Murphy 1
  • Dan Shi 1
  • Dan Willemsen 1
  • Dave Chen 1
  • Dave Gladfelter 1
  • Diego Cavalcanti 1
  • Dmitry Vyukov 1
  • Eduardo Bravo Ortiz 1
  • Ekaterina Kamenskaya 1
  • Elliott Karpilovsky 1
  • Espresso 1
  • Google+ 1
  • Goranka Bjedov 1
  • Hank Duan 1
  • Havard Rast Blok 1
  • Hongfei Ding 1
  • Jason Elbaum 1
  • Jason Huggins 1
  • Jay Han 1
  • Jeff Listfield 1
  • Jessica Tomechak 1
  • Jim Reardon 1
  • Joe Allan Muharsky 1
  • Joel Hynoski 1
  • John Micco 1
  • John Penix 1
  • Jonathan Rockway 1
  • Jonathan Velasquez 1
  • Josh Armour 1
  • Julie Ralph 1
  • Karin Lundberg 1
  • Kaue Silveira 1
  • Kevin Bourrillion 1
  • Kevin Graney 1
  • Kirkland 1
  • Kurt Alfred Kluever 1
  • Manjusha Parvathaneni 1
  • Marek Kiszkis 1
  • Mark Ivey 1
  • Mark Striebeck 1
  • Marko Ivanković 1
  • Matt Lowrie 1
  • Meredith Whittaker 1
  • Michael Bachman 1
  • Michael Klepikov 1
  • Mike Aizatsky 1
  • Mike Wacker 1
  • Mona El Mahdy 1
  • Noel Yap 1
  • Patricia Legaspi 1
  • Peter Arrenbrecht 1
  • Peter Spragins 1
  • Phil Rollet 1
  • Pooja Gupta 1
  • Project Showcase 1
  • Radoslav Vasilev 1
  • Rajat Dewan 1
  • Rajat Jain 1
  • Rich Martin 1
  • Richard Bustamante 1
  • Roshan Sembacuttiaratchy 1
  • Ruslan Khamitov 1
  • Sean Jordan 1
  • Sharon Zhou 1
  • Siddartha Janga 1
  • Simran Basi 1
  • Stephen Ng 1
  • Tejas Shah 1
  • Test Analytics 1
  • Tom O'Neill 1
  • Vojta Jína 1
  • iOS 1


Archive


  • ►  2019 (1)
    • ►  Jan (1)
  • ►  2018 (7)
    • ►  Nov (1)
    • ►  Sep (1)
    • ►  Jul (1)
    • ►  Jun (2)
    • ►  May (1)
    • ►  Feb (1)
  • ►  2017 (17)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Sep (1)
    • ►  Aug (1)
    • ►  Jul (2)
    • ►  Jun (2)
    • ►  May (3)
    • ►  Apr (2)
    • ►  Feb (1)
    • ►  Jan (2)
  • ►  2016 (15)
    • ►  Dec (1)
    • ►  Nov (2)
    • ►  Oct (1)
    • ►  Sep (2)
    • ►  Aug (1)
    • ►  Jun (2)
    • ►  May (3)
    • ►  Apr (1)
    • ►  Mar (1)
    • ►  Feb (1)
  • ►  2015 (14)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (2)
    • ►  Aug (1)
    • ►  Jun (1)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (1)
    • ►  Feb (1)
    • ►  Jan (2)
  • ▼  2014 (24)
    • ►  Dec (2)
    • ►  Nov (1)
    • ►  Oct (2)
    • ►  Sep (2)
    • ►  Aug (2)
    • ►  Jul (3)
    • ►  Jun (3)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (2)
    • ▼  Feb (1)
      • Minimizing Unreproducible Bugs
    • ►  Jan (2)
  • ►  2013 (16)
    • ►  Dec (1)
    • ►  Nov (1)
    • ►  Oct (1)
    • ►  Aug (2)
    • ►  Jul (1)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (2)
    • ►  Mar (2)
    • ►  Jan (2)
  • ►  2012 (11)
    • ►  Dec (1)
    • ►  Nov (2)
    • ►  Oct (3)
    • ►  Sep (1)
    • ►  Aug (4)
  • ►  2011 (39)
    • ►  Nov (2)
    • ►  Oct (5)
    • ►  Sep (2)
    • ►  Aug (4)
    • ►  Jul (2)
    • ►  Jun (5)
    • ►  May (4)
    • ►  Apr (3)
    • ►  Mar (4)
    • ►  Feb (5)
    • ►  Jan (3)
  • ►  2010 (37)
    • ►  Dec (3)
    • ►  Nov (3)
    • ►  Oct (4)
    • ►  Sep (8)
    • ►  Aug (3)
    • ►  Jul (3)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (3)
    • ►  Mar (3)
    • ►  Feb (2)
    • ►  Jan (1)
  • ►  2009 (54)
    • ►  Dec (3)
    • ►  Nov (2)
    • ►  Oct (3)
    • ►  Sep (5)
    • ►  Aug (4)
    • ►  Jul (15)
    • ►  Jun (8)
    • ►  May (3)
    • ►  Apr (2)
    • ►  Feb (5)
    • ►  Jan (4)
  • ►  2008 (75)
    • ►  Dec (6)
    • ►  Nov (8)
    • ►  Oct (9)
    • ►  Sep (8)
    • ►  Aug (9)
    • ►  Jul (9)
    • ►  Jun (6)
    • ►  May (6)
    • ►  Apr (4)
    • ►  Mar (4)
    • ►  Feb (4)
    • ►  Jan (2)
  • ►  2007 (41)
    • ►  Oct (6)
    • ►  Sep (5)
    • ►  Aug (3)
    • ►  Jul (2)
    • ►  Jun (2)
    • ►  May (2)
    • ►  Apr (7)
    • ►  Mar (5)
    • ►  Feb (5)
    • ►  Jan (4)

Feed

Subscribe by Email

follow us in feedly

Company-wide

  • Official Google Blog
  • Public Policy Blog
  • Student Blog

Products

  • Google for Work Blog
  • Chrome Blog
  • Official Android Blog

Developers

  • Ads Developer Blog
  • Android Developers Blog
  • Developers Blog
  • Google
  • Privacy
  • Terms