I was working at a place where we had thousands of test cases in our database.
This wasn’t a problem until we realized that many of the test cases where dead.
By dead I mean they were completely absolute at the worst or just redundant at the best.
This initial build up happened slowly, and before we knew it we where waste deep in old scripts.
Of course we always said we would go through and clean things up. When we started working on it though something else more important would come up.
We where able to slow the progress but we never truly reversed it.
The chilling reality is most, old test cases never die. I’ve seen test cases turn with my own eyes. They can cluster, and disrupt test development.
I’m here to tell you that zombie test cases are real. Don’t be afraid, this post will hopefully help you survive a zombie test case outbreak.
To automate tests well, you have to be a pretty smart. Unfortunately, that big brain make highly attractive to zombies.
What’s A Zombie Test Case
Zombie test cases are test that are no longer needed but still sticks around. They’re redundant, brittle, or old test cases.
This normally happens for a few reasons: The system under test changes, test case was written poorly, test is brittle, testing tool changes, requirements are shifted or changes are made to the test framework.
The Problem With Zombies
You’re probably thinking that not having “clean” test cases isn’t a big deal.
Yet it isn’t safe, in fact, zombie test cases cause havoc.
One problem is the maintenance cost. If you add in the time of the full team it can soon runs into thousands of dollars but that’s not where the real problem lies.
Zombie test cases DON’T test functionality. Time and money are poured into them yet bugs still slip into production.
Too many test cases are redundant but will keep causing carnage until the clear up operation begins.
How To Spot A Test Case Zombie
It can be difficult to find and delete zombies. This is even more so when they’re numbers are still very low.
Finding test cases can be hard. It’s easier to spot the process that are likely to create zombie test cases and attacking that. Here are a few things that are very likely to produce zombie test cases.
- Rushing at the end of every sprint to add Automated test. Zombie.
- Adding hard delays in your scripts to handle slow loading web pages (example: Thread.sleep(8000); ). Zombie.
- Automating only by record and playback. Zombie.
- Test written by unskilled programmers. Zombie.
- Using the test count as your only metric for automation success. Zombie.
- Fallowing into the code coverage trap. Zombie.
- Just disabling test for known-broken features. Zombie.
- Ignoring flaky test, and not analyzing every failure. Zombie.
- Blankly automating test cases from another database. Normally a manual test cases. Zombie.
If you continue to do this then it’s likely that the Zombies are coming. Change course and spot them before they eat your brains.
How to Kill A Zombie Test Case
Creating a test case is easy, killing one is hard.
I recommend just deleting it.
If you’re still worried about documenting what’s happened then you can archive the case or change its status.
Delete the old test is attacking just the symptoms and not the causes. You’ll be better off changing the behavior that’s likely to create zombie test cases in the first place.
Top 4 Lessons form Surviving a Zombie Test Case Attack
1. Organize before they rise!
Prevention is the best course of action when it comes to zombie test cases. If you can prevent an attack before it even happens then you’ll be in much better shape.
You want to plan for the worst but hope for the best.
Having a plan in place will make everyone on the team feel at ease. It also helps
2. They feel no fear, why should you?
The main objection to deleting old test cases is the fear that you still need them. We normally have a strong attachment to the things that we create. It’s understandable. zombie test cases don’t show any fear and neither should you. They’ll just undead ones and zeros. Don’t be afraid to put these old test cases to rest.
3. Use your head: cut off theirs.
You’re the human in the equation. Use your head to remove test cases that are duplicates or cover the same functionality.
4. The zombie may be gone, but the threat lives on.
After you’ve cleared out the zombie test cases you might want to celebrate a job well done. Remember to stay alert because other zombies might popup when you least expect it, and if you don’t keep the numbers down you’ll be right back to where you started.
The Zombie Test Case Survival Guide offers complete protection through trusted, proven tips for safeguarding yourself and your loved ones against the living dead.
Don’t be carefree and foolish with your most precious asset—time. The hordes of undead test cases may be stalking you right now without your even knowing it.
It’s time to fight back.