Let me tell you a quick story about a common situation.
A Developer walks into the office and instantly notices that something isn’t right. His spider senses are going off. The room is electric, there’s a panicked look on a co-works’ face, as they types furiously. The two people in Customer Support look like they’ve been hit by a bus. He pulls out his phone, checking his email expecting the worse. It’s flooded with email. There was a software patch last night but the manual test team didn’t find any critical issues. “What is going on?”, they think.
Is Testing Top Of Mind?
If you’re like most startups you probably have so much going on that testing falls through the cracks. Things like getting more customers, creating new features, or trying to get a mention in ‘XYZ’ site stay on top of the list. Testing normally takes a back seat.
Finding bugs before your customers do should also be a priority. Why drive customers to buggy software? That’s a quick way to lose their confidence.
There are many different ways to go about testing software, here’s some reason why your startup should consider automated testing.
Regression
It’s important that the system works together as in integrated whole. Don’t sprint backwards! There’s times when recently checked in code breaks another area of the system. Its pretty much inevitable.
You hear that Mr. Anderson?… That is the sound of inevitability… –Agent Smith, The Matrix
Yes, you might have developers do code reviews, and write unit test. You might even have manual offshore teams regress the system.
Still It’s near impossible to beat the efficiency of automated robots running regression test mercilessly, on multiple instances of your application. To boot they’ll even try different user environments all without needing to eat, poop, or sleep.
Plus this “automation horde” can be triggered to run whenever you want. During every build, at the end of every sprint, or every release, nightly, hourly, or just every blue moon. Regression builds confidence in your system.
Exact Requirements
Unlike humans, computers don’t tend to lie. Well, not yet anyways!
A lie gets halfway around the world before the truth has a chance to get its pants on. –Winston Churchill
You write code that eventually gets turned into 1’s and 0’s. That’s it! There is little worry about ambiguity when it comes to automated test.
Different manual testers could run the same test case in different ways, and each time you could get a different result.
After it’s in code you have a pretty solid understanding of what the computer was trying to do.
Another benefit is that the test will be ran the same way every time. This is very helpful when debugging because you know the exact steps that reproduce the issue.
Data Combinations Done Programmatically
I was a pizza delivery guy for a few months during college and you would always see this kind of marketing.
We have over 10 Trillion Pizza Combinations, Come Try Them All –Local Pizza Joint, Failing At Marketing
They where about to make this claim because of topping combinations. Lets say you have 50 topping, 4 sizes, 3 crusts types, and 2 different sauces. That’s already 1,200+ pizza combinations.
Just think of the nightmare it would be to have to try and make all those different pizzas. You would go insane.
Most software application have a ton more data combinations. They still expect manual test teams to have great coverage, when faced with this challenge. No pun intended, but thats just a recipe for disaster.
Never send a human to do a machine’s job. –Agent Smith, The Matrix
Automated test could be the missing ingredient (see what I did their?), that turns things around.
With automation you can programmatically create different data sets to test. Combinations could even be stored in a file or data base and feed into multiple test. This greatly increase the efficiency of trying different data scenarios.
Back to our Startup a few months later…
A Developer commits code. This triggers an automated set of smoke test to be ran. Later that night a regression suite of test is kicked off. Two test fail during the run. The next morning the Developer walks into the office and checks their email. They see the two failed tests. The problem is identified quickly. A fix is committed, and the whole cycle starts again.
This might sound like a fairy tell for startups but it can be a reality for most. Handling regression, exact requirements, & programatic data combinations are just the tip of the automated testing benefits iceberg. Having it as a priority for your startup is the first step towards maintaining and improving the quality of your application.