I have talked extensively about the DBA’s evolving role and how many DBA’s and operations professionals are now becoming SRE’s (site reliability engineers) or DBRE’s (database reliability engineers). Often, databases get blamed as the bottleneck for application slowdowns and issues, so DBAs have had to develop the skills needed to chase problems up and down the stack over the years. This full-stack approach to hunting out problems has resulted in many former DBAs and Sysadmins successfully taking on the role of an SRE/DBRE.
The question is, then, what are the most critical skills for this important role?
I personally have interviewed 1000’s of technical candidates over the last 10 years and have hired hundreds in various roles here at Percona. I often get asked the most critical skill for the next generation of technical engineers, SREs, or DBREs. The answer has been consistent for me over my career – I want engineers, SREs, etc., with good problem-solving skills and the ability to think outside the box. I am not looking for book knowledge or a detailed understanding of every concept; I want people who can see something new and…
- Be curious enough to ask “Why?” and want to know the answer.
- Will dig into the ambiguous and want to learn, and can learn the why.
- Can solve the issue, answer the question, and share that knowledge effectively.
From a technical perspective, while it is wonderful to have a great depth of knowledge, I generally am not looking for the experts’ expert. Rather, I look for people who are smart, passionate, and who learn quickly. I am not alone in this. I asked the question of those involved in hiring technical people here at Percona.
Peter Zaitsev (CEO) said the number one skill he is looking for is this: “Attitude and Ability to do independent research and find information to solve the problem at hand.” For many like Peter, having an encyclopedic knowledge of how things work or the right commands to use is secondary to solving problems never seen before. Many problems and issues that come up you cannot specifically train for. The unique nature of the workload, size, and way too many external factors often offer unique challenges to even the most experienced SRE. Peter added: “So many people now have this ‘I have not been trained on this’ attitude instead of doing some basic googling for the answer.” Indeed, there is a lot of information out there, and while searching for an answer quickly during an outage or performance event may seem like a no-brainer, more than half the people I have interviewed don’t even think about it. Thinking on your feet, reacting quickly, and restoring service can save companies millions of dollars of lost revenue and business in an outage.
Marco Tusa (MySQL Tech Lead) echoed Peter’s sentiment by saying that there are two important skills for him. One of these is the ability to learn what they don’t know. “This is because no matter what, often the best one on tech knowledge won’t know some important stuff. The will to learn is the key. “ Lenz Grimmer (Sr Director of Server Engineering) could not have agreed more, adding: “I’m seeking talent that is open-minded about acquiring new skills. So fast learners with the right sense of humility and the right attitude. “
Teamwork Makes the Dream Work…
Attitude and humility are critical in building an effective team (especially in a remote team). This was Marco’s second trait he is looking for. Marco went on to add he is also looking strongly into their fit with the team and if they will be a team player. The “no jerks” or “no soloist prima donna” mottos are very important. You have to be willing to share what you learned and look for help from your teammates.
This is the same thing Jay Janssen (Director of IT) said when asked about the number one thing he looks for: “Humility comes to mind — smart and humble is a good combination. While kind of cliche, it’s generally true.” We are all looking to hire smart people, but smart people who are Jerks or flaunt how smart they are generally don’t operate well in a team environment. You want someone who is smart but does not make other people feel small or insignificant.
Sanja Bonic (Head of Percona’s Open Source Program Office) also values teamwork and makes sure she tries to understand how people handle positive and negative interactions as a team. Sanja, who has previously led Engineering teams at OpenShift and now works with Percona’s community, asks people in interviews about “their best and worst experiences in teams they’ve previously worked with. This usually shows what people are paying attention to, and you’ll quickly get a hint of what people attribute value to.”
While you need people to work and learn independently, you equally also need them to function as a unit (or as a team). Remember to ensure the uptime, availability, and performance of the entire application spanning potentially hundreds or thousands of nodes, you need to use all the resources at your disposal when things go wrong, and having teammates who you trust, can help, and can augment your knowledge with is very important. You can’t do it all alone, so having the ability to “team-up” and work with others is a must.
“The strength of the team is each individual member. The strength of each member is the team.” ~ Phil Jackson
Sharing is Caring…
The ability for smart people to effectively share their knowledge and have good meaningful conversations is also critical in this role. Vadim Tkachenko (CTO) said he is looking for “People who have a brain and can have a meaningful conversation.” He went on to say he is looking for people who “Can speak well about previous relevant experiences.” This ability to share goes a long way internally to increase the collaborative spirit within the team. But this is not merely about speaking a single language; rather, it’s being able to talk about the technologies and match your audience’s expectations (or teammates).
Tate Mcdaniel (DBA Manager) says this is the number one thing he looks for when hiring people. His approach, in his words – “I ask questions about contentious and complicated things, and I look for answers that explain the complexity in ways a layperson can understand while giving pros/cons judiciously.” Taking the complex, explaining it, and educating others is of critical importance.
It is why Peter, Vadim, Jay, Marco, Tate, Lenz, and myself all said we look online at what people have written, where they have talked at conferences, what code they may have written, and other traces of their public persona before interviewing someone.
When I asked Lenz Grimmer if he looked at a candidate’s online persona, he said: “Absolutely, that’s one of the beauties of hiring in the open-source ecosystem. A public track record of contributions in various forms tells me much more than a CV. Forum and mailing list contributions, YouTube videos, all of which help get a better understanding of the candidate.”
One Person is an Island…
I personally highly value people’s willingness to share their insights, knowledge, and sometimes struggles. This is especially critical in the open-source space. I mentioned that no one person could manage a complex environment alone. Training and educating team members and others in the community is critical. The willingness to share and educate via online blogs, articles, and technical talks is, in my opinion, essential to the SRE/DBRE community as a whole.
So what do we see as the must-have skills?
- Problem-solving skills, the ability to troubleshoot unique and challenging problems.
- The passion and desire to learn, research, and acquire skills quickly.
- Humility and the ability to be a “team player” – No jerks allowed!
- The ability and passion for sharing their knowledge and educating others.
What do you think? Did we miss any?
