I got this question in email this morning from a reader:
"I am a test supervisor at --- and was promoted to a QA management position yesterday. I'm excited and terrified, so I have been thinking about how to organize the thought in my mind. After attending StarWest and following your blog for a while now, I am very interested in your opinion.
If you were a brand new QA Manager, and you knew what you know now, what are the top 5-10 things you would focus on?"
I am flattered by the confidence but in the event it is misplaced I wanted to answer this question publicly and invite readers to chime in with their own experiences. Besides, I am curious as to other opinions because I live with this same excitement and terror every day and could use a little advice myself. Here's my first couple and I'll add some more in future posts (unless of course you guys beat me to it).
Start living with your product, get passionate about it
Drink your product's kool-aid, memorize the sales pitch, understand it's competitive advantages but retain your skepticism. Test/QA managers should be as passionate about the product as dev managers but we need to temper our passion with proof. Make sure the test team never stops testing the functionality represented by the sales pitch.
Furthermore, part of living with your product is being a user yourself. I now live without a laptop and exclusively use my Chrome OS Netbook for my day to day work. As people see me with it in the hallways, I get to recite its sales pitch many times every day. Great practice. I also get to live with its inadequacies and take note of the things it has yet to master. This is great discussion fodder with devs and other stakeholders and also forces me to consider competitive products. When I can't do something important on my Chrome OS Netbook, I have to use a competing product and this spawns healthy discussions about how users will perceive our product's downside and how we can truthfully communicate the pros and cons of our product to customers. Every day becomes a deep dive into my product as an actual user.
This is a great way to start off on a new product.
Really focus on the test plan, make it an early priority
If you are taking over an existing role as test manager for an existing product chances are that a test plan already exists and chances are that test plan is inadequate. I'm not being unkind to your predecessor here, I am just being truthful. Most test plans are transitory docs.
Now let me explain what I mean by that. Testers are quick to complain about inadequate design docs: that devs throw together a quick design doc or diagram but once they start coding, that design stagnates as the code takes on a life of its own. Soon the code does not match the design and the documentation is unreliable. If this is not your experience, congratulations but I find it far more the norm than design docs that are continually updated.
Testers love to complain about this. "How can I test a product without a full description of what the product does?" But don't we often do the same thing with respect to our test plans? We throw together a quick test plan but as we start writing test cases (automated or manual) they take on a life of their own. Soon the test cases diverge from the test plan as we chase new development and our experience develops new testing insight. The test plan has just become like the design docs: a has-been document.
You're a new test manager now, make fixing these documents one of your first priorities. You'll get to know your product's functionality and you'll see holes in the current test infrastructure that will need plugging. Plus, you'll have a basis to communicate with dev managers and show them you are taking quality seriously. Dev managers at Google love a good test plan, it gives them confidence you know what you are doing.
Coming up next:
Understand your orgs release process and priorities
Question your testing process
Look for ways to innovate
Hmm. The live your product is right on. I am very dubious of the "really focus on your test plan" -- I would actually put "really focus on integrating the testers with the development team" -- go Agile! Having an integrated test group help to drive a fully functional team that will have ownership of the features they develop. Having a fosters an us versus them mentality.
ReplyDeleteI'd be interested in seeing what you consider a useful test plan. Most examples I've read on the web (aka the first few pages of search results) consist of what I would consider heavy, even bloated, process documents.
ReplyDeleteFor a small agile team (~10 devs) and 1-3 testers, a plan like this would be more hindrance than help.
What would you consider the most important aspects of a test plan that would be applicable for a small team?
What is being tested
DeleteWhat is not being tested
Risks
Gaps
What would you consider examples of Risks and Gaps, if you are not sure the developers disclosed all of the details of a new feature?
DeleteExcellent posting... as a Test Manager, I am always advocating understanding the purpose of the end deliverable. WHY you are building what you are building, and then testing whatever it may be. For many in conventional test roles, this approach seems a challenge to grasp, more are concerned with unit type testing versus a full understanding of what a piece of code is intended to accomplish.
ReplyDeleteProduct awareness is important no matter the role, or the product. In a previous life I was employed in a financial capacity for a Tier 1 auto supplier. Leadership mandated all employees attend a product awareness training event. Not only to give thanks for the product your company builds (said tongue in cheek), but also to understand how everyone's daily work activity contributes to the end product.
great post, looking forward to more...
Get your Communication methods and channels going immediately. As a manager you will interact with many different people from all parts of the company, including your own staff. Get to know your audience and how best to communicate with them. Also, work on being a better 'listener' and when and when not to voice your response/opinion.
ReplyDeleteA management position is a lot of "business" and "politics" so get used to being in that mindset. Don't be a "political player", but understand that you will be dealing with this everyday.
"understand it's competitive advantages" -> "understand its competitive advantages"
ReplyDelete"IT'S" ALWAYS MEANS "IT IS"
First, it's great that you are asking for help. That shows that you want to learn from others and that's something you want to model for the people who now report to you.
ReplyDeleteJames is right - definitely learn and live the product. But for my money, two other Ps are just as important: people and process.
Yours is a company of people. They make the enterprise tick. They come to the table with strengths and weaknesses; competencies and doubts. Work to help everyone you contact (your customers, contributors, C-types, contractors). We test mgrs are uniquely positioned to influence the nature and direction of our companys' culture. Have fun, expect the best, listen, and be supportive.
On the process front: Examine/Witness how things work - where value is created and where time seems to disappear. But don't try to fix everything a) at once or b) alone. Attempting to change too much results in process shock and regression. On the other hand, you do want to do something positive and visible in the first couple weeks. That will set the tone for your process improvement efforts. Engage others, forge alliances, and remain positive and calm.
Note: This is only one person's opinion but it comes from several years as a QA mgr and three decades in the software biz.
Good luck. Enjoy the ride.
Regards, Ian Savage
Two months back I wrote a blog post on Key practices for test managers here: http://blog.shino.de/2009/10/27/key-practices-for-software-test-managers/
ReplyDeleteI quite can't agree to some of the statements you made regarding test cases, test plans and design documents. The Swedish Army has a saying:
"If the map and the territory disagree, always trust the territory."
This means that the code is the final design of the software, and should really not be confused with the design document, which describe the state of thought at maybe a different time. The same goes with requirements, and all the other documentation we have.
What's about the testing team and their competence ? I think this is an important factor to consider too.
ReplyDeleteLeverage your people - look for opportunities and techniques to make one person-day worth five or more.
ReplyDeleteAutomated testing is the canonical example. It can be expensive to construct automated tests, and lots of QA testers don't have the skills to do so. But they are an excellent "force multiplier" if it becomes possible to run the test more often or if the test can be run for at least three QA cycles without change.
Unattended testing is usually harder than automated testing, but the payoff is even greater. Take the nightly build that your development team does and that passes their unit tests and subject it to your user- and functionality-focused automated tests. I guarantee you'll occasionally find important failures, and you'll find them while the code in question is still fresh in the developers' minds.
Bug clusters are your friend. Get more bug for your management's buck.
ReplyDeleteSee my Google tech talk "Building Software Smarter" for more
http://www.tinyurl.com/80-20-rules
cheers,
Erik Petersen
Hi,
ReplyDeleteMy name is Gil Zilberfld. We’ve discussed your post on our webcast “This week in testing”. We’ll be happy if you can comment, and if you like the discussion and content, let us know. And everyone else.
Thanks,
Gil Zilberfeld Typemock
Awesome, thanks Mr.Whittaker. I think we unconsciously realise that these are the bits to focus on, but never really focus in putting those thoughts to paper, so it's great to see this documented.
ReplyDeleteNetwork with other test professionals! I wrote an essay about this titled From Zero to Hero - Networking for QA Expertise: http://www.yvettefrancino.com/networking-for-qa-expertise.html
ReplyDeleteBesides living with your product and being passionate, how about "live" with your team. As a new manager I tried to understand my team, leverage what each person brings, grow, and evolve the team. Processes, products, tools, and techniques are necessary, but software is intellectual creation and testing software is even harder intellectually, so I had know the team and my customers/users as part of my "test team".
ReplyDeleteTest plan is only as good as the paper it's printed on.
ReplyDeleteI've worked with some managers who were unable to deviate beyond their test plan or spent their entire work day updating test plans that no one ever used.
Test plans are the final refuge of the incapable.
Lovely blog. I've just been nominate "QA Manager By Default," and my first step was to put my money where my mouth is: I started with communication and organization among the QA team, because those were the biggest complaints I had before landing here. A large part of that communication involves my "real" job - updating the existing technical and UA documents and creating the new ones made necessary by the enormous development effort over the last year.
ReplyDelete@Ian -
Our QA team wants to know very basic information: Who will perform the tests, What needs to be tested, How to report results, and Where can they find the necessary resources. That forms the framework for our test plan, and the details (the pass/fail criteria of the features tested) are filled in as we go. As the Business Analyst/Technical writer in our small, attempting-to-be agile shop, I work closely with the entire development group and borrow resources from technical support for the QA effort. Current test assignments are based on the work items and change requests in the build to be tested. Development is also doing their part by implementing Unit testing as part of the development process and updating their work item descriptions as they go (best thing ever! for making my documentation duties easier).
nice blog i have an advice too be innovative to prove your self worth full in organization and bring some thing called innovation in organization time to time . I mean to say present new ideas new practices that are not followed in your organization under previous manager like lets automation , Load performance test you will find so many things that you can introduce this would increase your team importance.
ReplyDelete