I am still curious about TE's relation with a product team. E.g, let's say for the search backend team, there is a cool new stateless service thunder2.0 which is going to replace the old one thunder1.0 . Does the search team have a few TE? Or is there a TE team work with this thunder2 development team? Who will decide what test(What's unittest coverage rate? correctness test? regression/perform test? Do we need loadtest or large scale cross system canary? ) do you need and where are you going to install those test(e.g for each commit? before commit? before release)? Who is going to responsible for set up new test? Let's say you are going to need a test platform to record and replay traffic for regression test. WHo is going to build and run this test platform? A infra SWE team? or a TE team? Once the test has been set up, who is going to take action when the test failed before release? Let's say the regression test shows 2% cpu regression and you can not find a commit with similar test result, who is going to responsible for the bisect?
TEs provide testing guidance and strategy to software engineers. For example in Search, TEs provide guidance to developers on what tests to build and where to run them. They design the needed test frameworks, implement tools, write test frameworks, decide (in partnership with dev) what tests to run and where to run, and engage with testing both at presubmit and post submit of the code commit.
Developers are responsible for writing tests and maintaining them. That said, it may not be true for all teams, especially if teams are just getting started or have unstable tests, but the goal is to make sure developers own the tests and maintain their tests as they make changes to the code. TEs are responsible for enabling the developers to write tests. TEs write test frameworks and may also write initial sets of tests to showcase how to use the framework and write tests. TEs may also add tests to cover any interesting or edge case scenarios, but they don’t write the majority of the tests themselves.
Developers work with TEs to make sure the right tools are available to them to help them take action in triaging and maintain their tests. TEs provide tools like bisect for developers to use when debugging a regression. Developers will be in the best position to bisect performance regressions as they know what changes they have checked in and what changes may have caused a regression.
Overall, TEs collaborate with the product team to provide test frameworks, tools, and processes at every phase of the development cycle and support SWEs to deliver high quality products.
Good insights on the TE role. I am curious to know which percentage of the time a TE at Google spends at planning, executing, and reporting on tests, versus building tools and test infrastructure. Is this something you could share based on your experience?
TEs are expected to split their time between coding, technical contributions, and testing consultation. TEs coding percentage time depends on the maturity of the project, where we are in the development cycle, and the testing needs of the project we are working on. If you are a TE on a project that just got started, then you will be spending more time on building the testing culture by setting testing strategies, defining test processes, proposing the needed test frameworks and defining the success metrics etc. If you are a TE on a matured project, then you will be spending more time on coding tasks like designing and implementing tools, writing test frameworks, and automating test processes. A TEs split may also shift depending on where in the product cycle their feature or product is with more time for tooling during development and more time for reporting near launch.
Most importantly, TEs at Google are expected to be proficient coders and to be able to leverage their skill set according to the project needs.
Hi, correct me if I am wrong, but TE in Google has nothing common with QA Engineers in other companies, because most of the time they are developing tools and not writing tests. At this point I think when Google is searching for TE, they are rather searching for Developer, that has some interest in testing, then for a QA Engineer, am I right?
Good to see this blog after a long time, looking for more frequent articles!
ReplyDeleteI am still curious about TE's relation with a product team. E.g, let's say for the search backend team, there is a cool new stateless service thunder2.0 which is going to replace the old one thunder1.0 . Does the search team have a few TE? Or is there a TE team work with this thunder2 development team? Who will decide what test(What's unittest coverage rate? correctness test? regression/perform test? Do we need loadtest or large scale cross system canary? ) do you need and where are you going to install those test(e.g for each commit? before commit? before release)? Who is going to responsible for set up new test? Let's say you are going to need a test platform to record and replay traffic for regression test. WHo is going to build and run this test platform? A infra SWE team? or a TE team? Once the test has been set up, who is going to take action when the test failed before release? Let's say the regression test shows 2% cpu regression and you can not find a commit with similar test result, who is going to responsible for the bisect?
ReplyDeleteTEs provide testing guidance and strategy to software engineers. For example in Search, TEs provide guidance to developers on what tests to build and where to run them. They design the needed test frameworks, implement tools, write test frameworks, decide (in partnership with dev) what tests to run and where to run, and engage with testing both at presubmit and post submit of the code commit.
DeleteDevelopers are responsible for writing tests and maintaining them. That said, it may not be true for all teams, especially if teams are just getting started or have unstable tests, but the goal is to make sure developers own the tests and maintain their tests as they make changes to the code. TEs are responsible for enabling the developers to write tests. TEs write test frameworks and may also write initial sets of tests to showcase how to use the framework and write tests. TEs may also add tests to cover any interesting or edge case scenarios, but they don’t write the majority of the tests themselves.
Developers work with TEs to make sure the right tools are available to them to help them take action in triaging and maintain their tests. TEs provide tools like bisect for developers to use when debugging a regression. Developers will be in the best position to bisect performance regressions as they know what changes they have checked in and what changes may have caused a regression.
Overall, TEs collaborate with the product team to provide test frameworks, tools, and processes at every phase of the development cycle and support SWEs to deliver high quality products.
Good insights on the TE role. I am curious to know which percentage of the time a TE at Google spends at planning, executing, and reporting on tests, versus building tools and test infrastructure. Is this something you could share based on your experience?
ReplyDeleteTEs are expected to split their time between coding, technical contributions, and testing consultation. TEs coding percentage time depends on the maturity of the project, where we are in the development cycle, and the testing needs of the project we are working on. If you are a TE on a project that just got started, then you will be spending more time on building the testing culture by setting testing strategies, defining test processes, proposing the needed test frameworks and defining the success metrics etc. If you are a TE on a matured project, then you will be spending more time on coding tasks like designing and implementing tools, writing test frameworks, and automating test processes. A TEs split may also shift depending on where in the product cycle their feature or product is with more time for tooling during development and more time for reporting near launch.
DeleteMost importantly, TEs at Google are expected to be proficient coders and to be able to leverage their skill set according to the project needs.
Hi, correct me if I am wrong, but TE in Google has nothing common with QA Engineers in other companies, because most of the time they are developing tools and not writing tests.
DeleteAt this point I think when Google is searching for TE, they are rather searching for Developer, that has some interest in testing, then for a QA Engineer, am I right?