cause and effect – more testing doesn’t necessarily improve software quality

Categories Development, Uncategorized, Working smart

A short story about cause and effect I had in my backlog.

Jim works for a software company or, why not, he may even own it.

Jim realizes that the quality of the software that gets built in his company decreases year-by-year, all over the board. He’s been seeing it for some time now but it seemed like the guys have it under control. You know, the guys know what they’re doing.

After awhile things seem to slip even further….

Some small widget doesn’t really work, that user guide doesn’t align to the standards, there’s a picture that is clearly stolen from the Internet, users say that the products don’t work, etc. etc. etc…. ad infinitum.

Jim is puzzled, perplexed, this quality thing is getting irritating. It’s costing the company a lot of money (in all the forms that money can take, including business lost due to a bad reputation). A spark ignites: “We have to fix it.”

Conversations unfold:

– “Didn’t we test this before shipping it?”

– “Oh, we did?”

– “Well, we weren’t thorough enough.”

– “We have to test better next time.”

– “Hey, how’s about revisiting our testing procedures, we could like… add more testing.”

– “Yup, that should fix it.”

…………… time passes and the testing procedure is updated.

New tools spring into action, probing and analyzing the innards of the tormented ware, while testing engineers, guided by never-ending test suites churn each feature and produce an endless stream of bugs in their shiny testing reports that they have to fill in weekly, to justify the minimum 40% testing LOE imposed on every development project.

Testing reports and indicators aggregated over the board show an increase in testing productivity, bug counts go up and developers jump at countless lines of code, beating them into submission. Fixing, isolating, replacing, identifying and applying workarounds… going steadily, fix by fix, toward something that works, or at least something that works for them… and the testers.

But the frustrated users keep bothering them with impossible bugs, heisenbugs, mandelbugs or even schrödinbugs…

But that can’t be! We’ve tested everything! By God, we even have automated testing and unit tests! Unit tests’ for God’s sake!

It’s pointless… everything is pointless, it never ends.


It seems that adding more testing to fix software quality is like adding sugar to make a dish less salty, every time you cook it. Or better yet, adding more testing to fix software quality is like hiring more police officers to lower the crime rate in a favela.

The morale of the story?

Go after causes not effects. Low software quality, just like a lot of things these days is an effect of industrial decadence, loss of purpose and meaning on behalf of the entire working collective.

Analysts analyze because it’s their job. Developers write code because being a software developer was an educated career choice and it pays way more than other desk jobs. Testers test because, you know, we have a project and a testing quota and by God we have all these test cases to check. An managers, oh my… managers manage. Because there’s no difference between managing a software project and a pet beauty saloon.

They don’t really know why they’re doing it, there is no time to think about it. They have so many things to do.

Anything worth doing is worth doing right and a software development team needs a goal, a purpose… a driving factor to deliver high quality software. Not more testing.

Coach the team, incentivise it. Make them feel that they can change the world. They need values, not rules.

Fixing effects pays off quick (even if it’s costly sometimes) but it pays in pennies. Maybe, just maybe, you’ll get a one digit improvement.

Going after causes will give you an order of magnitude improvement.

You just need to know the difference, and there’s just one way to find out. Look deeper, ask why.

Jim will say… c’mon, everybody knows that. Yeah sport, then why don’t you do it?

2 thoughts on “cause and effect – more testing doesn’t necessarily improve software quality

  1. “It seems that adding more testing to fix software quality is like adding sugar to make a dish less salty, every time you cook it.” – That phrase is epic.

    I am on a level 3 team now. Neither a dog, nor a cat. But I hear you well. Something is definitely degrading and slipping through fingers.

Leave a Reply

Your email address will not be published. Required fields are marked *