COOL to be a TE @ Google
Friday, May 08, 2020
By Anantha Keesara
Test Engineers are a part of Google’s Engineering Productivity (EngProd) Group. As mentioned in a previous post, we advocate for our users, provide comprehensive testing solutions, and play a key role creating successful and reliable products and platforms. At Google, Test Engineers are not manual testers; we are technical engineers whose focus is on advancing product excellence and engineering productivity.
In short, it’s COOL (Constant learner, Out-of-the-box thinker, Orchestrator, Leading-edge user) to be a Test Engineer at Google:
Constant learning is what keeps Google Test Engineers motivated. We understand holistically how all the pieces of the software stack are interconnected and what kind of coverage exists or is needed to test the connections between the stacks.This product knowledge makes us test experts. We work closely with Software Engineers from the very beginning of the development process to discuss the testability of the designs before the features are implemented. We develop test strategies, methodologies, and test plans; we write scripts, design systems, and build tools and test infrastructure. We review design docs, do deep dives into Google's massive codebase, evaluate stack traces, and determine the root causes of production outages. Through this constant learning, we not only build deep technical expertise and do risk management by identifying weak spots in the code base, we also find creative ways to break software and identify potential problems. Our job ladder also gives us the flexibility and independence to explore and learn new technologies like ML concepts and Cloud computing and to build new testing solutions or improve existing ones.
Test Engineers are a part of Google’s Engineering Productivity (EngProd) Group. As mentioned in a previous post, we advocate for our users, provide comprehensive testing solutions, and play a key role creating successful and reliable products and platforms. At Google, Test Engineers are not manual testers; we are technical engineers whose focus is on advancing product excellence and engineering productivity.
In short, it’s COOL (Constant learner, Out-of-the-box thinker, Orchestrator, Leading-edge user) to be a Test Engineer at Google:
Constant learning is what keeps Google Test Engineers motivated. We understand holistically how all the pieces of the software stack are interconnected and what kind of coverage exists or is needed to test the connections between the stacks.This product knowledge makes us test experts. We work closely with Software Engineers from the very beginning of the development process to discuss the testability of the designs before the features are implemented. We develop test strategies, methodologies, and test plans; we write scripts, design systems, and build tools and test infrastructure. We review design docs, do deep dives into Google's massive codebase, evaluate stack traces, and determine the root causes of production outages. Through this constant learning, we not only build deep technical expertise and do risk management by identifying weak spots in the code base, we also find creative ways to break software and identify potential problems. Our job ladder also gives us the flexibility and independence to explore and learn new technologies like ML concepts and Cloud computing and to build new testing solutions or improve existing ones.
Out-of-the-box thinking, a result of constant learning, is another thing that keeps us motivated. As Google Test Engineers, we champion Engineering excellence by providing optimized solutions to address engineering inefficiencies, testing gaps, and process gaps. We constantly think of ways to make machines do the work to increase testability and productivity. Hundreds and thousands of lines of code get checked-in every minute at Google. To maintain velocity, quality, and code health, we devise creative ways to test and debug the test failures -- like performing diff testing, building dynamic test cases from the logs, designing heuristic algorithms to identify culprits for test failures, building solutions to reduce the test run time, and implementing stubs, fakes, and mock objects and servers to help developers write stable unit and integration testing. Along with devising creative ways to test and debug the test failures, we also focus on improving engineering excellence and product excellence by defining and measuring productivity metrics and product health metrics like quality, stability, and performance. The testing of, for example, Search, Ads, Maps, YouTube, Cloud, self-driving cars, and Google Apps would not have scaled with traditional testing practices.
Orchestrating the testing efforts is a key responsibility of Google Test Engineers. As orchestrators we can collaborate with cross functional teams including Product Managers, Technical Program Managers, and Software engineers to define critical user journeys (CUJs), determine testing strategies, and ensure that the right tests are run on the right configurations/environments. With our strong communication and collaboration skills, we work with the cross-functional teams and play the role of evangelists in spreading the word on new tools, technologies, and best testing practices. We also have the opportunity to host Hackathons and Fixits, host interns, drive college recruiting events, engage with the open source community in testing the open source products, listen to feedback, and convert that feedback into product improvements.
Leading-edge user: the fun part of being a Test Engineer! We can engage with product development, participate in the review of product designs, documentation, and prototypes, play with features and products early on, and provide informed feedback. Best of all, as early adopters we get to wear wearables, ride in self driving cars, be in our own world with AR/VR, engage with Google Assistant to get our chores done, and have multiple laptops, phones, and smart display units!
Stay tuned to learn more COOL things about Test Engineering at Google!
Good to see this blog after a long time, looking for more frequent articles!
ReplyDeleteI am still curious about TE's relation with a product team. E.g, let's say for the search backend team, there is a cool new stateless service thunder2.0 which is going to replace the old one thunder1.0 . Does the search team have a few TE? Or is there a TE team work with this thunder2 development team? Who will decide what test(What's unittest coverage rate? correctness test? regression/perform test? Do we need loadtest or large scale cross system canary? ) do you need and where are you going to install those test(e.g for each commit? before commit? before release)? Who is going to responsible for set up new test? Let's say you are going to need a test platform to record and replay traffic for regression test. WHo is going to build and run this test platform? A infra SWE team? or a TE team? Once the test has been set up, who is going to take action when the test failed before release? Let's say the regression test shows 2% cpu regression and you can not find a commit with similar test result, who is going to responsible for the bisect?
ReplyDeleteTEs provide testing guidance and strategy to software engineers. For example in Search, TEs provide guidance to developers on what tests to build and where to run them. They design the needed test frameworks, implement tools, write test frameworks, decide (in partnership with dev) what tests to run and where to run, and engage with testing both at presubmit and post submit of the code commit.
DeleteDevelopers are responsible for writing tests and maintaining them. That said, it may not be true for all teams, especially if teams are just getting started or have unstable tests, but the goal is to make sure developers own the tests and maintain their tests as they make changes to the code. TEs are responsible for enabling the developers to write tests. TEs write test frameworks and may also write initial sets of tests to showcase how to use the framework and write tests. TEs may also add tests to cover any interesting or edge case scenarios, but they don’t write the majority of the tests themselves.
Developers work with TEs to make sure the right tools are available to them to help them take action in triaging and maintain their tests. TEs provide tools like bisect for developers to use when debugging a regression. Developers will be in the best position to bisect performance regressions as they know what changes they have checked in and what changes may have caused a regression.
Overall, TEs collaborate with the product team to provide test frameworks, tools, and processes at every phase of the development cycle and support SWEs to deliver high quality products.
Good insights on the TE role. I am curious to know which percentage of the time a TE at Google spends at planning, executing, and reporting on tests, versus building tools and test infrastructure. Is this something you could share based on your experience?
ReplyDeleteTEs are expected to split their time between coding, technical contributions, and testing consultation. TEs coding percentage time depends on the maturity of the project, where we are in the development cycle, and the testing needs of the project we are working on. If you are a TE on a project that just got started, then you will be spending more time on building the testing culture by setting testing strategies, defining test processes, proposing the needed test frameworks and defining the success metrics etc. If you are a TE on a matured project, then you will be spending more time on coding tasks like designing and implementing tools, writing test frameworks, and automating test processes. A TEs split may also shift depending on where in the product cycle their feature or product is with more time for tooling during development and more time for reporting near launch.
DeleteMost importantly, TEs at Google are expected to be proficient coders and to be able to leverage their skill set according to the project needs.
Hi, correct me if I am wrong, but TE in Google has nothing common with QA Engineers in other companies, because most of the time they are developing tools and not writing tests.
DeleteAt this point I think when Google is searching for TE, they are rather searching for Developer, that has some interest in testing, then for a QA Engineer, am I right?