I was fortunate enough to be invited to the 2nd Midlands Exploratory Workshop on Testing (MEWT) which took place on the 13th September at the wonderful Attenborough Nature Centre.

(You can read more about the background to the MEWT concept and the day’s events on the MEWT blog.)

The theme of this year’s workshop was “Regression Testing”.

I’ve set out my presentation in long form below but if you want the original slides here they are: MEWT – Regression Testing

---

Initial thoughts

When I first read what the theme of this year’s workshop was I thought “Well, that’s a bit obvious. Surely everyone knows what Regression Testing is?”

It turns out there’s no such thing as a silly question, especially when it comes to testing.

So let me just set out what I understand regression testing to be, because I know already (thanks, Twitter) that there’s a difference of opinion in the room as to what that might be.

Again, I’m not going off what the “dictionary definition” says. I’ve deliberately not looked it up. I must present what I understand it to be, not what I’d like it to be, and certainly not what some other source says it is.

Regression Testing is…

Ensuring what has already been tested, and found to be good, remains so as the software continues to be developed.

Ideally you would re-run all your previous tests every time there is a change.

What’s the problem?

Q: Why is it the “bane of the tester”?

A: There is rarely (if ever) enough resource to cover a full regression test every time it is needed.

Reducing the load

There are some common tactics to reduce the workload…

1. Don’t retest everything.

• Wait until there are enough changes built up to form a release, then test that.
  • Reduces effort with minimal risk.

• Only test significant changes.
  • How do you determine “significant”?

• Only test those areas likely to have been affected.
  • Again, sensible, but how is this determined?

2. Don’t do it all yourself.

• Spread the testing effort out across the team.
  • Rerunning tests manually is dull. Rotate the tests. Share the pain.

• Outsource the testing.
  • Rerunning tests manually is dull. Let factory testers do it.

• Automate the testing.
  • Let the computer do the work. Now you have two problems.

Coverage

Regression Tests are…

• All the tests you have run to check against requirements.
• All the tests you have run to check the fixes are correct.

That’s a lot of tests.

Regression Testing vs Sapient Testing

Regression Testing is the antithesis of Sapient Testing.

• Use sapient testing to explore the way ahead.
• Use regression testing to shore up the road behind.

---

 

The one thing to take away from the workshop was that everyone came away with a different idea of what Regression Testing should be and could be than when they started that day. (As I’ve already said.)

I’d say that, on that basis alone, it was a worthwhile exercise.