A few years ago, I was walking through the office and someone asked if we had a test written for a bug that was just found. Was pretty proud to say that we did.

They then asked when was the last time it ran. I froze I had no idea. It was pretty embracing. Literally I had no idea. Couldn’t even give a ball park answer.

Didn’t want to lie so I said I would tell them later after looking.

Here are a few options if you have no idea when you should be running your test.

A] Nightly Runs The most common case. All test are ran nightly on the latest stable build. That 5+ hour test run time don’t matter that much if they run while everyones asleep.

B] Smoke Run After Commit Another common Cause. Unit & Smoke test are ran after every commit to a particular branch. These types of runs are normally done to save the test teams time. Just like it isn’t smart to test if the car can run, if it didn’t have a working engine. There’s no need to do in-depth testing on a broken build.

C] Release Regression Nothing new here. Before every release you’ll want to do some type of regression.

D] Feature Regression Sometime you’re making a change to the system that isolated to one area. You’ll want to hit that area pretty hard to make sure that no major bugs go through. That’s when a targeted automated regression it really helpful.

E] Unit Test Before Builds The point here is to get some quick feedback. Normally you want your build to max out at 10 minutes. It’s worth putting in concentrated effort to make it happen, because every minute you reduce off the build time is a minute saved for each developer every time they commit. Unit test normally run very quickly so they are a great candidate for this.