Why SuperOps.ai decided to not have a QA team and how we managed to pull it off

“We will not have a QA team.”

When I said these words at a meeting with my early engineering team in late 2020, I was met with pin drop silence. SuperOps.ai was just starting its journey as a fledgling startup and we were just beginning to build the platform that would be at the core of our company.

As the CTO and Co-founder of a tech startup in India, my announcement was an audacious one, but one in which I had full faith. 

But first let’s unpack why this was so audacious. At tech companies, at least in India, the Quality Assurance (QA) team is an integral and accepted part of engineering and development. The developers build the code and the QA team puts it through rigorous testing, identifies bugs, and alerts the developers on what’s breaking; the developer works on it again, the code goes back to the QA team who put the code through its paces once more and this repeats until a code is ready to be shipped. This is the accepted practice, and has been for years. 

So when I said we are doing away with a QA team I was asking my engineering team to get so out of the comfort zone that it wasn’t even visible anymore!

But, I was sure we needed to push the boundaries. Years ago I had lived through the transition from manual testing to automation for test cases that are at the heart of QA. For those from a non-software background—QA teams would create test cases for all the use cases against which they would test the code to uncover bugs. This was a time consuming process, with code taking six months, and even a year at times, to finally be released. With automation frameworks coming in about a decade ago, this timeline shrunk to about a month—still a long period considering how fast paced our world is. 

We are now at the next step in the evolution of the software development process.

Why we decided to do away with a separate QA team

Reducing the time from ideation to release of a feature is a significant requirement and I will come to it later, but that’s not the only reason.

For me, ownership is the key reason for deciding to follow this path.

For decades, the division of responsibilities between developer and the QA tech was clear, with the former writing code and the latter making sure that the code is of good quality. But, what this did, implicitly, was to transfer the responsibility of quality away from the developers. My belief is that the developer’s responsibility is not just to deliver a good code, it is to deliver a good feature. The end user, client or customer does not care about the code or if you wrote the code in Java or another programming language. The user cares only about whether the feature works for them and if it works well for them. The responsibility for creating a feature that meets the end user’s expectation should lie with the developer. 

Unfortunately, testing, which is a core requirement for ensuring the code and feature works without any breaks, is seen as someone else’s job and is even looked down upon in certain circles. 

We can link this mindset, I feel, to a lot of engineering problems that we have today and why a feature still takes a few weeks to move from idea to final release and launch. 
With ownership of quality of the end result on the developer and the shortening of the process, the cycle can come down to a few weeks, and even days. 

Then there is the aspect of talent utilization. QA engineers are technically very capable but they are stuck in a monotonous job of testing someone else’s work. Which is why you see many QA engineers trying to move to the developer role, since the latter are the ones truly building and creating something new. I feel QA engineers will be better utilized as developers.

We embrace testing and quality checks. But do it differently

But rigorous testing is an integral part of development, which is why the stunned silence when I told the team that we would not have a QA team. This did turn to shock when I informed them that we would not be doing away with quality checks and testing and that the developer would be driving it!

Now, the shock, I must say, was to do with the fact that a separate QA team was created to divide the load and to ensure efficiency. If developers have to follow the same process of testing without a QA team, then the very purpose of speeding up this process would be defeated.

So, I had to find a better way. 

I had to find a tool or system that would make quality testing a breeze, or close to one. 

In the age of no-code development there are multiple tools available to make different bits and pieces of the process easier. For instance, backend API testing is now quite easy.

But, it is a whole different story in the frontend, where even a slight shift in a single button changes everything around it. So, the test cases need to be redone and the whole process is time consuming, when done in the traditional manner. Especially in these times of A/B testing every aspect of the frontend multiple times a week.  

I was looking for a tool that could keep pace with these rapid changes and test for the changes simultaneously. I came across a tool called Testim.io, which heavily relies upon Artificial intelligence (AI) to run automated tests in the background so developers can know within a few minutes if there is a failure. Its AI learns and handles small UI changes intelligently without the need to re-record the test case. When the changes in the UI requires manual editing, it makes it absolutely easy by allowing developers to edit only a part of the test case. Think of it like editing a digital video. When you need to chop a scene, you cut just that frame without affecting the rest of the frames. That’s the freedom Testim.io gave us. 

We started using this tool over a year ago, well before SuperOps.ai went into private beta and it has worked out well for us. In fact, in the initial days I set up checks to ensure processes were being followed and tried to control as many aspects of this process as possible. This was in part to reassure the team and in part to reassure me that things would not break. But, in hindsight, none of that was required. 

We have reached a stage where the team is able to do one-day releases now. Developers build and test and release and can do so on their own, on their own systems. It is smooth, does not burden the developer and there is no compromise on quality. Our goal is to get to multiple releases a day.

I won’t say this was easy or the change happened overnight. Like I said in the beginning, the decision was met with worry and concern. All our developers came from companies where QA teams are the norm. What worked for us was educating them on the effectiveness of the new process, showing them that the change in process helps the developers become better at their job, and letting them experience firsthand that it would not add to their workload or make their work slower. 

We offered proactive training, we constantly reiterated the message on why we chose this path, we offered handholding and support where required, especially in the initial days when the team was getting used to the new system. Now, each time a new developer joins the team—and they are mostly used to having QA teams—we give them training. They also see how their peers work and see first hand the effectiveness of the new system. We also regularly do hackathon style sessions where all engineers get inside a room and just focus on the test cases. 

The impact is there for all of us to see. Developers can almost immediately see how their code is working and what the issues are. The feedback loop has shrunk dramatically. It has helped them improve the quality of their work. They also have a greater respect for testing. I have seen developers taking the initiative to create better test cases or volunteering to fix existing ones.

It is a joy to see the sense of ownership and pride. Developers own a feature and not just a piece of code. 

The best part of being a tech startup is the freedom to experiment, to follow your gut, and not worry (too much) about some of these trials not working out. We are changing the way Indian tech companies build products, and we are just getting started. 

(Want to join us? We are hiring)