Interesting. I had never heard of Test Engineer being used as a job title before. Does the addition of the word engineer really represent what the job entails? Or do you think it is a reaction to the typically pejorative title 'tester' or 'QA'.
Note: I write test automation code for a living, and have always wondered what a suitable job title should be.
Interesting, sounds like all 3 roles require engineers that have coding skills. Is there any place for exploratory testers without scripting/coding skills?
sounds like the key function performed by Google SET is enabling testability. I think a lot of software companies have SET perform the functions of both SET and TE described here, minus the expectation of code refactoring for the system under test. in my opinion, refactoring code is probably best performed by one closest to code, developers who are more aware of all the implementation subtleties since they coded the subtleties in the first place.
also, testability can be a hard quality to define and measure in some situations. i wonder how Google assesses contributions from SET, thru deltas in code cyclomatic complexity before and after the refactoring?
I wonder how you feel a smaller development shop should approach testing, say a smaller engineering R&D group part of a larger organization. Since we have fewer engineering resources most of them are focused on development and QA is a smaller group staffed mostly with a few testers with less of a computer science background who do more functional and acceptance testing. Do you think this is the most effective way to approach testing or would we benefit from investing resources in order to make our testing department more like Google's?
James - the role of TE is new one I believe. Was this new? Late cycle testing, risk quality advising and lots of testing work (not testing by writing code).
My experience as a titled Test Engineer at an aerospace company aligns well with James' description with the exception of the statement that I spent most of my time writing automation. A majority of my time was spent working with the customers on how they use the system, researching test methodology & learning/testing the system I was working on in the lab. I typically performed those tasks in that order, over & over again, week by week until the project was complete. The automation I used as part of my testing was to simulate a random input generating live system that communicated with our system via an API. I needed to be able to control the input so that I could test different user scenarios & that's where automation became useful. However, I did not write the base test code, the code was written by an experienced developer who modified an existing script in a matter of minutes to do the basics of what I need. It was a fairly simple program so I was able to expend it to simulate exactly what I needed. The customers were impressed that I was able to not only create scenarios to test individual events & functions but that I was also able to create interesting real world scenarios to show how the functions integrated. I was able to design a large volume of user scenario tests because I was I had the benefit of a development expert modifying a piece of code for my own testing use that I could further expand as my test ideas expanded. A TE should be focused on how the customers use their system first & foremost. Automation is a tool to help them achieve the best possible scenarios in the smallest amount of time.
Такое разделение труда наталкивает меня на мысль, что Гугл практикует создание "индусских" dev фабрик.
Это же наверное медленно, когда ты пишешь код, потом ждешь пока кто-то напишет для него тесты, потом окажется что тесты надо подкорректировать и ты опять кого-то ждешь. Куда продуктивнее все делать самому, советуясь с более сведущими коллегами в случае необходимости.
BTW, can't share this post in the Google Reader. Go write better tests ;-)
Big 40 translated: This division of labor pushes me to think that Google is practicing the creation of "Hindu" dev factories. This is also probably slow, when you write code, then wait until someone writes tests to prove that the tests then have to correct you again and waiting for someone. Much more productive to do everything myself, consulting with more knowledgeable colleagues, if necessary.
Very interesting, SET and SWE must work very very closely and SET must understand SWE code very well. SET is like a white-box tester. So, what's the ratio of SWE, SET and TE in Google, like 3:2:1?
Thanks for sharing this great post. I'm just curious that if the SETs and SWEs are partnered together, then how do they deal w/ personality or other conflicts should they arise? The reason I ask is becuase in most companies especially the big ones, there's usually some individuals who are harder to get along and work with than others. So I'd just like to know how does Google handle this type of situation? Thanks.
Hi, I am very keen to know how does Google perform its UAT? Considering the shear size of the search engine, how does it actually perform its UAT? Or, does it perform some other tests instead of UAT? Can you please let me know?
The Google average is about 1:10 (SWE:SETI), but it varies based on testability, complexity, and criticality. Some teams have no SETIs (test automation is straight forward), and the highest ratio I've seen is 1:5 (very complex automation required and/or mission critical project).
And of course, there are teams that are just SETIs, because they are working on Google-wide development/test infrastructure/tooling.
Interesting. I had never heard of Test Engineer being used as a job title before. Does the addition of the word engineer really represent what the job entails? Or do you think it is a reaction to the typically pejorative title 'tester' or 'QA'.
ReplyDeleteNote: I write test automation code for a living, and have always wondered what a suitable job title should be.
good set of posts, thanks for sharing with the community
ReplyDeletelove you james
ReplyDeleteInteresting, sounds like all 3 roles require engineers that have coding skills.
ReplyDeleteIs there any place for exploratory testers without scripting/coding skills?
sounds like the key function performed by Google SET is enabling testability. I think a lot of software companies have SET perform the functions of both SET and TE described here, minus the expectation of code refactoring for the system under test. in my opinion, refactoring code is probably best performed by one closest to code, developers who are more aware of all the implementation subtleties since they coded the subtleties in the first place.
ReplyDeletealso, testability can be a hard quality to define and measure in some situations. i wonder how Google assesses contributions from SET, thru deltas in code cyclomatic complexity before and after the refactoring?
I wonder how you feel a smaller development shop should approach testing, say a smaller engineering R&D group part of a larger organization. Since we have fewer engineering resources most of them are focused on development and QA is a smaller group staffed mostly with a few testers with less of a computer science background who do more functional and acceptance testing. Do you think this is the most effective way to approach testing or would we benefit from investing resources in order to make our testing department more like Google's?
ReplyDeleteJames - the role of TE is new one I believe. Was this new? Late cycle testing, risk quality advising and lots of testing work (not testing by writing code).
ReplyDeleteShrini
How is the mix? How many TE per SWE per SET?
ReplyDeleteHow does this change in the differente Focus Areas?
My experience as a titled Test Engineer at an aerospace company aligns well with James' description with the exception of the statement that I spent most of my time writing automation. A majority of my time was spent working with the customers on how they use the system, researching test methodology & learning/testing the system I was working on in the lab. I typically performed those tasks in that order, over & over again, week by week until the project was complete. The automation I used as part of my testing was to simulate a random input generating live system that communicated with our system via an API. I needed to be able to control the input so that I could test different user scenarios & that's where automation became useful. However, I did not write the base test code, the code was written by an experienced developer who modified an existing script in a matter of minutes to do the basics of what I need. It was a fairly simple program so I was able to expend it to simulate exactly what I needed. The customers were impressed that I was able to not only create scenarios to test individual events & functions but that I was also able to create interesting real world scenarios to show how the functions integrated. I was able to design a large volume of user scenario tests because I was I had the benefit of a development expert modifying a piece of code for my own testing use that I could further expand as my test ideas expanded. A TE should be focused on how the customers use their system first & foremost. Automation is a tool to help them achieve the best possible scenarios in the smallest amount of time.
ReplyDeleteI have the position equivalent to a SET. We work with "SWEs" in our designs but target users are typically "TEs".
ReplyDeleteТакое разделение труда наталкивает меня на мысль, что Гугл практикует создание "индусских" dev фабрик.
ReplyDeleteЭто же наверное медленно, когда ты пишешь код, потом ждешь пока кто-то напишет для него тесты, потом окажется что тесты надо подкорректировать и ты опять кого-то ждешь. Куда продуктивнее все делать самому, советуясь с более сведущими коллегами в случае необходимости.
BTW, can't share this post in the Google Reader. Go write better tests ;-)
Big 40 translated: This division of labor pushes me to think that Google is practicing the creation of "Hindu" dev factories. This is also probably slow, when you write code, then wait until someone writes tests to prove that the tests then have to correct you again and waiting for someone. Much more productive to do everything myself, consulting with more knowledgeable colleagues, if necessary.
ReplyDeleteVery interesting, SET and SWE must work very very closely and SET must understand SWE code very well. SET is like a white-box tester. So, what's the ratio of SWE, SET and TE in Google, like 3:2:1?
ReplyDeleteHi, who is dog fooder?
ReplyDeleteWhat does it means?
Thanks.
@denisko: see Eating your own dog food
ReplyDeleteDeeply grateful for this! You are my Rockstar of the Month. Thanks!
ReplyDeleteJames,
ReplyDeleteThanks for sharing this great post. I'm just curious that if the SETs and SWEs are partnered together, then how do they deal w/ personality or other conflicts should they arise? The reason I ask is becuase in most companies especially the big ones, there's usually some individuals who are harder to get along and work with than others. So I'd just like to know how does Google handle this type of situation? Thanks.
Hi,
ReplyDeleteI am very keen to know how does Google perform its UAT?
Considering the shear size of the search engine, how does it actually perform its UAT?
Or, does it perform some other tests instead of UAT?
Can you please let me know?
Thank you,
Anand
what is the common ratio of SWE and SET in a delopement team?
ReplyDeleteRatio of SWE and SET depends on team to team and also depends on type of project.
DeleteGenerally it can be between 1:1 ratio to 1:3 ration (1 Test on 3 Dev).
The Google average is about 1:10 (SWE:SETI), but it varies based on testability, complexity, and criticality. Some teams have no SETIs (test automation is straight forward), and the highest ratio I've seen is 1:5 (very complex automation required and/or mission critical project).
DeleteAnd of course, there are teams that are just SETIs, because they are working on Google-wide development/test infrastructure/tooling.