QA is not UAT (and vice versa)
… and yet most startups act as if they are one in the same, go figure.
For those new to these terms…
QA is short for Quality Assurance, this is typically done by a dedicated resource (but sometimes an automated tool or service) that tests every variation of your app or website to ensure the experience matches what was intended to be built. Ie. Someone goes through your app and enters every possible combination in every flow to make sure there is code to handle those iterations well AND hasn’t introduced any new bugs in the existing experiences.
The last thing (well maybe one of the last things) you want is for a user to do something unexpected and that action produce crazy adverse effects on the system. Ex. someone entering a number in a name field and that crashes the app. Not good
So yeah, totallllyyy important. Particularly as a PM, you want to make sure that all the effort you and the team put in determining the right thing to build, also actually works.
Whereas UAT or User Acceptance Testing is where a PM or stakeholder tests the new experience from a functional perspective making sure that the experience still makes sense for the user now that its all built and that it looks as expected. Ex. A mobile design on a tablet still looks and functions well.
While they seem similar, they are not the same and more importantly one cannot replace the other. They each address different issues and together is where the magic happens.
So what are the options?
Well that depends on the company and where you want to put your money, because let’s be clear anything thats going to make the impact that you need will require an investment. Its either going to be a direct expense or its going to take time from someone already employed, which is also money at the end of the day.
But first, I recommend the following steps.
- Communicate about what activities need to occur
You need to get the entire team aligned that QA is important AND what explicit steps apply to your product. If they are not bought in, don’t pass go and don’t collect $200 dollars (bad monopoly pun), its not worth further conversation until everyone agrees in its importance and the investment.
Note at this point don’t bring up WHO might be doing the activities, because then people can start to get defensive by only thinking about themselves in that role vs the positive impact to the product. You’ll start to get the arguments like we don’t have time to do this, or its not my expertise. Aht Aht. We’re not there yet, we’re just focused on IF we should do this work or not.
2. What activities are important to your product
Once we’re agreed that QA is important, then be explicit about what do those activities look like. Does your product warrant a full regression test with each release? Do you need to do a slow roll out to ensure the new feature doesn’t impact current usage too much? Are there specific security concerns that you need to be aware of?
List out all the things that need to happen without the burden of who will do it, prioritize that list and once everyone is on board then continue to the next step.
3. Determine where in the product development work flow these activities should happen.
Some things are super clear, ie. we need to do a full regression before we get to staging because there are other teams in staging that are built on top of our stable code, so we can’t introduce any critical bugs in staging because it will take down 3+ teams in addition to our own.
Some things aren’t so clear, ie. we just need to do the security review before we get to production but it can happen at any time before that point.
Start with the activities with the hard requirements and work from there.
4. Now is when you figure out the who and the how
This is where things can get creative, you are only limited by your imagination and willingness.
The easiest thing to do is hire a person to do these activities, there are a ton of talented QA people out there that truly love this work and over time can get you set up with a heavily automated solution.
There are also outsourced QA teams that can do this really well. This is actually one of the best uses of an outsourced dev team in my opinion, they can seamlessly embed into the product development process and add value right away without a lot of onboarding (the better documentation you have the easier their onboarding is). And in my experience, this type of work is really great for teams that tend to stay really close to the letter of the contract, and doesn’t do much critical thinking. Either the thing works as its described in the card or it doesn’t. Easy Peasy.
You could invest in a tool, there are a bunch out there and even Browserstack has an automated testing option in their product. This of course would need some configuration and likely someone to manage it, but it would be less time than doing the manual testing themselves.
You could also make it part of the existing teams job, and this is where I see most startups land, which is fine. Just make sure that if this is the approach you want to take, be explicit and intentional about it. Be super clear about who is responsible for which activities and when. Put automations in place if you can, to ensure accountability. And then enforce that accountability.
If you don’t do all those things you might as well not even try, because you will end up just wasting time and money, introducing strife and frustration within the product development team and risking trust with your users by shipping buggy code.
If its everyone’s responsibility, its no ones responsibility.
And just let me say the need for QA is not a diss to any developer or PM for that matter. No ones perfect and we all become blind to the things that we work on extensively, and thats why we have a team of people with different skillsets and perspectives. So that together we produce a top quality experience for our customers because they deserve it, but we do it together. And QA is just simply a part of that equation.
Wrapping Up
This topic definitively started off as a rant because I’ve seen the same issue in startup after startup. But hopefully by the end of this you’ve seen my true respect for the QA function. Its extremely important and shouldn’t be left out of the product development process because of resource constraints.
With all the competition out there most startups don’t the luxury they think they do to continuously push buggy experiences. And its not a stakeholders or a PMs place to be the sole gatekeepers of quality, especially when thats not where their skillsets lay.
To hear me rant a bit more and add some addition context, check out the sister video on YouTube and also check out some of the other videos on topics like Working with Challenging Coworkers and The Power of Knowing your Strengths.
