I got to thinking about the top reasons why people do NOT do automated testing after reading this article in Logic Magazine. The premise of the article is: “It’s impossible to test everything in an automated way, so why test anything in this way.” This is severely flawed logic! Not to mention highly ironic that it appears in a magazine with “logic” in its name.
Feeling inspired, I came up with some of the other, oh-so-common reasons, David Letterman-style, for why people don’t want to write tests:
10) I have no time to test
This is #10, but it is really #1 in my heart. At the end of the day, quality will demand its due in both time and money – it’s just a matter of when and how much. One can pay upfront when its still relatively cheap and the impact is small (i.e., fixing bugs while in the Dev and QA processes). Later, when bugs are out in the wild, the costs are much greater and manifold – it comes at the expense of:
- Lost and annoyed users
- The reputation of your company and product, and perhaps your client’s company and product
- The time of your personnel, across support staff, development, QA and everyone else that has to swing into action to understand some code they may not have looked at in months
But sure, it may be true that you don’t have time to test. And it is likely because you are spending so much more time dealing with production issues, grappling with super-slow development times due to crushing technical debt, and having a (demoralized) team that spends half its time just putting out fires. Why is this approach such a burden? Because it is 30X more expensive to fix a bug in production than in the design phase. If you don’t have good QA, you will spend all your time chasing your tail on these production issues, crowding out precious time for developing cool new features.
9) I don’t have the budget/tools needed for testing
Similar to an absence of time, the lack of money is illusory – payment is made for quality one way or another. Doing it on upfront QA is far cheaper than waiting for it down the road. Call it “enlightened self-interest”.
8) I don’t make mistakes in my code
There are few things more humbling than software engineering in terms of demonstrating one’s fallibility. Compilation and execution of code are demanding in a way that emails and power points are not. As a fundamental principle at Bespoken, we believe all code is guilty until proven innocent – that it is unproven until some form of test is written against it. This is based on personal experience – I have looked at a line of code over and over again, certain that it must work only to be proven wrong on first execution too many times to count. Don’t agree? Take a look at this study from the IEEE.
Still not convinced? Well, neither is Bill Gates 😆:
No! There are no significant bugs in our released software that any significant number of users want fixed.
– Bill Gates, 1995
Haha!
Thank God, Bill has moved onto bigger, more solvable problems like world hunger.
7) We can just do it manually
But this means you have to do it every time – even for simple apps, there are often hundreds if not thousands of scenarios. Testing those every time is phenomenally tedious and simply impractical. Even the most disciplined and well-resourced teams will fall short. It also encourages highly blindered thinking – people always naturally tend to focus on just testing the new/amended features. But bugs are just as likely to appear as side-effects, in existing, supposedly untouched features that are accidentally broken. Automation is the only way to really KNOW things are working consistently and comprehensively.
6) We have a QA team – it’s their job
This is one we hear from developers, who sometimes think unit-testing is a favor they do for the QA team. What nice guys – they tried it out before throwing it over the wall 🙂 At least they thought about it. Either way, the testing team should be grateful for anything, right? Of course, we’re just kidding. The fact is, unit-testing is an act of self-care. It makes debugging code faster and easier, and it means catching bugs before they get out into the wild, when they are much more expensive and time-consuming to fix. The best way to protect your time as an engineer is to test thoroughly and automatedly.
5) We are just getting started – we’ll test later
The best way to get feedback on a new product is NOT by doing something low-quality. Users will get stuck on the perception that it does not work well and won’t be able to see the idea beneath it. MVP means minimal functionality, not minimal quality. Some disagree, of course; the phrase “You waited too long to release if you are not embarrassed by your first version” is famous in startup circles. But we disagree – the MVP should be low scope, but high quality, as the diagram below illustrates:
4) That’s “Amazon”/”Google”/”WRITE_PLATFORM_HERE”’s Job
Sure, they can help, but they are rather busy themselves. Independent verification of these systems is the best way to protect your own reputation. Because if your app doesn’t work, people might say “Stupid Alexa” – but just as often, they will say “Stupid <YOUR ONCE HIGHLY RESPECTED BRAND HERE>”. The user doesn’t know where the problem lies, a strategy that protects your reputation is essential.
3) Automated tests are complicated and they are always breaking
I do have regard for this objection – they can be complicated, and they can break. And there is a point of diminishing returns – I’m looking at you 100% code coverage.
One needs to find a balance in the number of tests – they definitely can collapse under their own weight. Having a coherent philosophy on testing is important, such as this: “Write tests. Not too many. Mostly integration.”. And here’s another one:
Or my preferred one:
“Test until the fear goes away”
But regardless of your philosophy, I believe the honest reason people don’t test is inertia. Now, I can’t see inside your heart. There may be other reasons. But what have I found from experience? Unit-testing and testing, in general, is contagious. Once developers catch the habit, so to speak, it stays with them. Why? Because everyone likes doing high-quality work faster – once you realize that this is best achieved through testing, the right philosophy and balance that suits you and your organization will come naturally.
2) I don’t like testing
Hey, at least this is an honest response. Truth be told, I don’t really like testing either. But what do I like:
- Things that work well
- Debugging apps quickly
- Feeling proud of my work
- Spending time doing fun development
- Spending time with my family and friends
- Spending time on ANYTHING other than debugging and fixing complex, stress-inducing production issues!
1) Nothing changes with AI anyway….
Seriously, though, AI may not take over the world, but the nature of AI and machine learning is to constantly evolve. You need tools that help you to keep up and maintain peace of mind, ensuring releases are nearly flawless and keeping an eye out for anything that breaks once the app is live.
Now that you know all the reasons why you should NOT NOT test, just contact us via email or setup a FREE demo to learn how to get started. At Bespoken we specialize in making tools that ensure the quality of every aspect of your voice experiences. We also have monitoring tools to verify that they are up and running 24×7 and we provide a simple yet powerful way to ensure (in advance) that your users are being understood.
Yours in testing and debugging,
John – jpkbst