Testing – in nearly any form – does not typically get people excited. Except us. When it comes to voice apps, we are obsessed with tests.
We are committed to sharing different testing experiences and best practices, because we believe testing will drastically improve the quality and engagement of voice apps for millions of users worldwide. If enough voice-app players integrate a complete testing plan into their systems, we will see the legitimacy of the voice-first industry begin to take hold.
We are also obsessed with testing because it saves money and time. The benefits of testing produce a timely ROI. For example, an IBM study from its Systems Sciences Institute reported:
The cost to fix an error found after product release was four to five times as much as one uncovered during design, and up to 100 times more than one identified in the maintenance phase.
Relative Costs to Fix Software Defect
In addition to saving time and money, testing is a critical part of a successful system. We have an intimate understanding of how complex systems work. More about that in John Kelvie’s Diagnostics Manifesto – one of our most popular blog posts.
Thankfully, voice app testing has been gaining ground. Here are 7 ways we’ve seen people and companies embrace testing. Read their experiences, and find some helpful resources for testing your own Alexa skills.
Alexa Skill Testing Examples
1. A Treatise on Testability
By: 3PO-Labs
This blog post from March 2016 was one of the earliest pieces to identify some important voice-first dev shortcomings and see the relation with them through to ROI. A lot of relatable scenarios in here, and still worth a read almost 2 years later.
2. A Treatise on Testability, Redux
By: 3PO-Labs
Published February 2018, this thoughtful redux shows how far voice-app development and testing have come since 3PO’s original testing treatise. The piece also explains how rapid growth continues to cause problems along with offering a few solutions. It’s worth noting, we agree that the voice community today is much more interested in testing now compared to two years ago.
3. Testing Amazon Alexa Skills with Chai and Fixtures
By: Mike Hoolehan, Software Developer
See how Hoolehan rights his wrong after developing his first Alexa skill without writing any test cases. While the tools from Alexa testing support are good, Hoolehan seeks to find a faster way to test with his own custom solution using mocha/chai.
4. Building a Starship Computer with TDD and Bespoken’s Virtual Alexa
By: Mark Courquin
First, enable the Starship Captain skill and command your ship’s speed, shields, and phasers from your Echo. Then, read this 11-minute blog post to see how the skill was made and tested.
5. Automating Alexa Skill Dialog Testing
By: Macadamian
This straightforward post explains how one developer tested Alexa skill dialogs without speaking in a matter of seconds.
Get to know Pulse Labs – recently in the news for its latest round of funding from Bezos Expeditions, the Alexa Fund, Madrona Venture Group, and Techstars Ventures. This Seattle startup uses real people to get human feedback back to developers to help improve the voice-first industry.
7. Building Engaging Alexa Skills: Why Testing and Automation Matter
By: Amazon
After reading this Amazon Alexa blog post by guest contributor John Kelvie, you will have a holistic view of the 4 aspects that make a comprehensive plan.
Amazon has done a spectacular job creating a new industry and ecosystem around voice-first development. Though not perfect, it is clear Amazon understands developers, the impact and potential of voice-first technology, and the speed necessary to support a new platform. The number of skills available worldwide is now over 30,000, leading to the next step in Alexa’s evolution where we find ourselves today – quality and usability. And we are here to help; send us an email with your questions or comments to contact@bespoken.io.
Upcoming Skill Testing Webinar
Want to learn more about testing and see how to get started with Bespoken’s testing and monitoring tools?
Join us for our next webinar. Register here. Don’t know if you can make it? Register as usual and we will email you the recording.
[…] 7 Voice-First Experiences with Testing Alexa Skills […]