Would be able to provide more detail with regards to expectations of the TE when it comes to their technical abilities, is it similar to SETI. Would TEs at google say that they are automation experts, or are they more focused and specialized on manual testing?
I believe the article is saying the TEs are good at making test plans, risk analysis, and manual testing while SEIT/SETI excel at automating tests, builds, releases, deployment, gathering metrics, and finding ways to reduce cycle time.
I find this to be a useful distinction for job descriptions and it's worth noting that both roles are very valuable to product teams.
Should TE and SETI be physically separate people? I believe they should be, as the skills and mind-set of a TE and SETI should be/are different, as they have a different focus?
As for should they be separate people, it depends on the team (and, thinking outside of Google, the company).
For a smaller team/company, it wouldn't make sense to have distinction between the roles because usually there'd only be one person responsible for everything. However, when products get larger and more complex, the team grows, and people tend to gravitate toward the responsibilities that interest them most.
It's not that SETIs can't do the work we primarily attribute to TEs, or vice versa. Many engineers find the primary responsibilities of one role or the other more interesting and more in line with what they want to do. Having separate roles makes things simpler versus having a combined role with a laundry list of responsibilities, many of which an individual engineer might not be doing.
To clarify, the purpose of this post was to describe the transition from Software Engineer in Test (SET) to Software Engineer, Tools & Infrastructure (SETI) and explain the new SETI role. Look for a future post on the evolution of the TE role at Google, but also refer to past post such as http://googletesting.blogspot.com/2013/01/test-engineers-google.html .
To answer some of the immediate questions: - The TE role does have a coding bar candidates need to meet when they interview for a TE position, and automated testing is part of the TE role. The technical bar is quite high for all individuals at Google: All engineers, and many others (e.g. people managers, program managers, etc.) also contribute code.
- As people's passions change over time, they can (and do) learn new skills, leading to a mobility within teams and roles. Via rotations, engineers can also try out other roles before committing to a change.
All engineers at Google are expected to contribute test cases for code that they write, not just TEs. We actively avoid situations where TEs end up writing all the test cases for code written by others.
From a team structure standpoint are SETIs embedded on the SWE teams that they support, or are they on their own smaller teams. What are the pros/cons with the current structure?
I think it's possible that you could have one of each. On an Agile Scrum team I think you would always want to have a TE. Having a SETI is the goal but it may not always be fiscally possible or needed for a mature team. More frugal organizations may have 'floating' SETI engineers which move from team to team as needed.
Then how about TE? Should they be embedded and report to the technical dev manager? Or should there by a separate group which the TE's belongs to, and are "loaned" to the feature/project teams?
Both models exist within Google, depending on team size, structure, etc. There are trade offs for both models. Perhaps this is something that we can address in a future blog post.
The key to long term success is to establish a community across Google, where SETI's and TE's can share ideas across organizational and product boundaries.
It is very interesting and excellent article showing the reality and evolution that has the area of testing. What I like most is that unlike the 2 roles (SETI and TE) and integrated work will allow a response time faster for the attention of the SDLC.
Many times you require 2 consulting roles to the same person
I agree with you, Alvaro, many times it takes disparate skills and team work to solve a problem. In an ideal world, the same person would have all the necessary skills, but that rarely happens, especially for complex issues.
I read this article recently: http://www.wired.com/2016/04/google-ensures-services-almost-never-go/
It talks about the SRE discipline at Google. What I find very interesting is the paragraph that talks about the two types of engineers that are hired, e.g. “set of technical skills that is useful to SRE but is rare for most software engineers"..
This post could not have come at a more appropriate time for my team. We are in the middle of the evolution from a more manual approach to an ever increasing set of automation and tooling. We have also recognized the necessity of our "Manual" testers, but it has been hard putting our finger on it in relation to the automation teams. This article helped to define that line a little better for us. I would love if you could provide any examples, allegorical or otherwise, about the two groups working together on a project? Maybe fodder for a future blog post?
A future blog post on this sounds like a good idea.
A suggestion: If you can attend GTAC (http://developers.google.com/gtac), feel free to reach out to any of the Google engineers there, I'm sure you'll get a lot of advice and ideas.
Agreed on bumpy and very rewarding! Persevere through it, it'll be worth it at the end. Please do share your milestones, we'd love to hear all about it.
Hi Ari, great article! I have a question on this excerpt:
"Their role evolved into a broad spectrum of responsibilities: writing scripts to automate testing, creating tools so developers could test their own code..."
So the TEs write scripts and create tools for the devs to use - not the SETs? Is this because the scripts and tools require domain / product knowledge which the TEs possess?
This distinction between the two roles is very interesting, as is the strong point made that engineers in general have coding skills. We are working towards this in my organisation but it is quite daunting, mainly due to the vastness of applications, products languages etc.
Great question, thank you for asking for clarification!
Regarding the sentence, "Their role evolved into a broad spectrum of responsibilities: writing scripts to automate testing, creating tools so developers could test their own code..."
We added that sentence to indicate that TEs, if they weren't already, started contributing tools and code in a big way as part of their evolution at Google. In fact, contributing technically is part of the job description for TEs. TEs at Google cannot be successful w/out technical contributions.
This task is by no means exclusive to TEs only. Every engineer within Google (TE or otherwise) ends up contributing tools and code during their career.
I hope this clarifies, feel free to follow up if you have any further questions.
Ari, Good article on your journey from SETs to SETI. Would like to know how the whole organization is now structured at Google specifically product development for various verticals and different/one productivity organization.
Do you think for a technically oriented Test Engineer inside of Google its possible to transfer into the role of Software Engineer in Test and Infrastructure? Thanks :-)
Thanks for sharing your eloquent story about Google evolution of quality assurance and the roles of the engineers , this is same experience we went through in our company. In order to close the test gap we improved our team cohesion and operational agility by redesign our organization away for a hierarchy organization to network organization. In addition be began merit badge system to encourage our QA to shift left by allowing them examine other tools and methods to determine if want to pursue them as a vocation. Looking foward to sharing experience and discussing idea about ways to increas engineering productivity.
I love this post, I need to read it a couple of time again before to get it totally. Very happy to hear about that kind of evolution in IT and Testing Engineering!
BTW: The g.con links at the end of the Post does not pass the Integration Test with careers.google.com.
Hi Ari, It's really great article! Just one quick question here, as you mentioned SETI are building tools to accelerate all other aspects of product development, now container technique are becoming mainstream. So is there any overlap between SETIs and REs? Will these two roles combine together in future.
Do the role SET's support cross platform organization? Meaning, SET's working for Software QA initiatives are expected to support Hardware QA initiatives as well? I am curious to understand how Google handles this after the new roles are coined, generally the Software QA engineers primarily focus on the productivity and efficiency measurements and they write and perform unit test using this as a basis whereas the Hardware engineers (in the testing organization) should have skills in the reliability level like Mean time between failure, Mean cycle between failure etc...
On GTAC blog (https://events.withgoogle.com/gtac-2017/), there was a mention about a plan to come up with an Engineering Productivity conference in 2018, but have not found any announcements or links yet on the same. Any idea when the new conference shall be held? Pointer to the details of such a conference would help.
I assume from the discussion that SETI's produce automation test harnesses. These would be used across multiple Agile/Scrum teams/squads. Are the SETI's embedded in the Agile entities? How do you resolve issues making use of this central investment? Are non-fuctional tests (chaos, stress, load, longevity, FMEA) performed by embedded individuals, and if so, how do you resolve cross-Agile-Entity impact?
Would be able to provide more detail with regards to expectations of the TE when it comes to their technical abilities, is it similar to SETI. Would TEs at google say that they are automation experts, or are they more focused and specialized on manual testing?
ReplyDeleteI believe the article is saying the TEs are good at making test plans, risk analysis, and manual testing while SEIT/SETI excel at automating tests, builds, releases, deployment, gathering metrics, and finding ways to reduce cycle time.
DeleteI find this to be a useful distinction for job descriptions and it's worth noting that both roles are very valuable to product teams.
Should TE and SETI be physically separate people? I believe they should be, as the skills and mind-set of a TE and SETI should be/are different, as they have a different focus?
DeleteSteve that's more or less the accepted narrative, but I think it's a bit insulting to both groups. Are we really saying that they can't learn a skill?
DeleteThere is an older, but still accurate, post about the expectations and responsibilities of test engineers at Google.
Deletehttp://googletesting.blogspot.com/2013/01/test-engineers-google.html
As for should they be separate people, it depends on the team (and, thinking outside of Google, the company).
For a smaller team/company, it wouldn't make sense to have distinction between the roles because usually there'd only be one person responsible for everything. However, when products get larger and more complex, the team grows, and people tend to gravitate toward the responsibilities that interest them most.
It's not that SETIs can't do the work we primarily attribute to TEs, or vice versa. Many engineers find the primary responsibilities of one role or the other more interesting and more in line with what they want to do. Having separate roles makes things simpler versus having a combined role with a laundry list of responsibilities, many of which an individual engineer might not be doing.
Thank you for your feedback.
DeleteTo clarify, the purpose of this post was to describe the transition from Software Engineer in Test (SET) to Software Engineer, Tools & Infrastructure (SETI) and explain the new SETI role. Look for a future post on the evolution of the TE role at Google, but also refer to past post such as http://googletesting.blogspot.com/2013/01/test-engineers-google.html .
To answer some of the immediate questions:
- The TE role does have a coding bar candidates need to meet when they interview for a TE position, and automated testing is part of the TE role. The technical bar is quite high for all individuals at Google: All engineers, and many others (e.g. people managers, program managers, etc.) also contribute code.
- As people's passions change over time, they can (and do) learn new skills, leading to a mobility within teams and roles. Via rotations, engineers can also try out other roles before committing to a change.
@ Ari, how good is to involve a TE in JUnit Scripting
DeleteHi Nithiya --
DeleteAll engineers at Google are expected to contribute test cases for code that they write, not just TEs. We actively avoid situations where TEs end up writing all the test cases for code written by others.
From a team structure standpoint are SETIs embedded on the SWE teams that they support, or are they on their own smaller teams. What are the pros/cons with the current structure?
ReplyDeleteI think it's possible that you could have one of each. On an Agile Scrum team I think you would always want to have a TE. Having a SETI is the goal but it may not always be fiscally possible or needed for a mature team. More frugal organizations may have 'floating' SETI engineers which move from team to team as needed.
DeleteThen how about TE? Should they be embedded and report to the technical dev manager? Or should there by a separate group which the TE's belongs to, and are "loaned" to the feature/project teams?
DeleteBoth models exist within Google, depending on team size, structure, etc. There are trade offs for both models. Perhaps this is something that we can address in a future blog post.
DeleteThe key to long term success is to establish a community across Google, where SETI's and TE's can share ideas across organizational and product boundaries.
It is very interesting and excellent article showing the reality and evolution that has the area of testing. What I like most is that unlike the 2 roles (SETI and TE) and integrated work will allow a response time faster for the attention of the SDLC.
DeleteMany times you require 2 consulting roles to the same person
I agree with you, Alvaro, many times it takes disparate skills and team work to solve a problem. In an ideal world, the same person would have all the necessary skills, but that rarely happens, especially for complex issues.
DeleteI read this article recently: http://www.wired.com/2016/04/google-ensures-services-almost-never-go/
It talks about the SRE discipline at Google. What I find very interesting is the paragraph that talks about the two types of engineers that are hired, e.g. “set of technical skills that is useful to SRE but is rare for most software engineers"..
I find this very analogous to the SETI/TE split.
This post could not have come at a more appropriate time for my team. We are in the middle of the evolution from a more manual approach to an ever increasing set of automation and tooling. We have also recognized the necessity of our "Manual" testers, but it has been hard putting our finger on it in relation to the automation teams. This article helped to define that line a little better for us. I would love if you could provide any examples, allegorical or otherwise, about the two groups working together on a project? Maybe fodder for a future blog post?
ReplyDeleteA future blog post on this sounds like a good idea.
DeleteA suggestion: If you can attend GTAC (http://developers.google.com/gtac), feel free to reach out to any of the Google engineers there, I'm sure you'll get a lot of advice and ideas.
I am at the same stage with my team. Transitioning from 100% manual to automation dominant approach. Bumpy road but very rewarding!
DeleteAgreed on bumpy and very rewarding! Persevere through it, it'll be worth it at the end. Please do share your milestones, we'd love to hear all about it.
DeleteThank you for replying Ari. I just submitted my application for GTAC!
DeleteHi Ari, great article! I have a question on this excerpt:
ReplyDelete"Their role evolved into a broad spectrum of responsibilities: writing scripts to automate testing, creating tools so developers could test their own code..."
So the TEs write scripts and create tools for the devs to use - not the SETs? Is this because the scripts and tools require domain / product knowledge which the TEs possess?
This distinction between the two roles is very interesting, as is the strong point made that engineers in general have coding skills. We are working towards this in my organisation but it is quite daunting, mainly due to the vastness of applications, products languages etc.
Simon Rigler
Hi Simon,
ReplyDeleteGreat question, thank you for asking for clarification!
Regarding the sentence, "Their role evolved into a broad spectrum of responsibilities: writing scripts to automate testing, creating tools so developers could test their own code..."
We added that sentence to indicate that TEs, if they weren't already, started contributing tools and code in a big way as part of their evolution at Google. In fact, contributing technically is part of the job description for TEs. TEs at Google cannot be successful w/out technical contributions.
This task is by no means exclusive to TEs only. Every engineer within Google (TE or otherwise) ends up contributing tools and code during their career.
I hope this clarifies, feel free to follow up if you have any further questions.
Ari, Good article on your journey from SETs to SETI. Would like to know how the whole organization is now structured at Google specifically product development for various verticals and different/one productivity organization.
ReplyDeleteThere is an older, but still accurate, post about the expectations
ReplyDeleteDo you think for a technically oriented Test Engineer inside of Google its possible to transfer into the role of Software Engineer in Test and Infrastructure? Thanks :-)
ReplyDeleteYes, ladder transfers happen between SETI and TE in both directions (and between other positions, too).
DeleteThanks for sharing your eloquent story about Google evolution of quality assurance and the roles of the engineers , this is same experience we went through in our company. In order to close the test
ReplyDeletegap we improved our team cohesion and operational agility by redesign our organization away for a hierarchy organization to network organization. In addition be began merit badge system to encourage
our QA to shift left by allowing them examine other tools and methods to determine if want to pursue them as a vocation. Looking foward to sharing experience and discussing idea about
ways to increas engineering productivity.
I love this post, I need to read it a couple of time again before to get it totally.
ReplyDeleteVery happy to hear about that kind of evolution in IT and Testing Engineering!
BTW:
The g.con links at the end of the Post does not pass the Integration Test with careers.google.com.
Thanks for the heads up! We've updated the links.
ReplyDeleteCurios to know the organizational structure, do the REs and SREs in the EP organization same as TE and SETI? or in other structure style?
ReplyDeleteHi Ari, It's really great article!
ReplyDeleteJust one quick question here, as you mentioned SETI are building tools to accelerate all other aspects of product development, now container technique are becoming mainstream. So is there any overlap between SETIs and REs? Will these two roles combine together in future.
Hi Ari, An informative article.
ReplyDeleteDo the role SET's support cross platform organization? Meaning, SET's working for Software QA initiatives are expected to support Hardware QA initiatives as well? I am curious to understand how Google handles this after the new roles are coined, generally the Software QA engineers primarily focus on the productivity and efficiency measurements and they write and perform unit test using this as a basis whereas the Hardware engineers (in the testing organization) should have skills in the reliability level like Mean time between failure, Mean cycle between failure etc...
On GTAC blog (https://events.withgoogle.com/gtac-2017/), there was a mention about a plan to come up with an Engineering Productivity conference in 2018, but have not found any announcements or links yet on the same. Any idea when the new conference shall be held? Pointer to the details of such a conference would help.
ReplyDeleteI assume from the discussion that SETI's produce automation test harnesses. These would be used across multiple Agile/Scrum teams/squads. Are the SETI's embedded in the Agile entities? How do you resolve issues making use of this central investment? Are non-fuctional tests (chaos, stress, load, longevity, FMEA) performed by embedded individuals, and if so, how do you resolve cross-Agile-Entity impact?
ReplyDelete