With the release of the new Bespoken Dashboard, you can now use the Validation feature to run automated testing for Alexa skills – and it can all be done without speaking aloud.
How does Automated Testing for Alexa Skills Work?
When you’re logged in to your Bespoken Dashboard, select the Validation tab and create a sequence of tests for your skill. We take your test sequence and run it through Alexa Voice Service (AVS) and your skill. Like this.
Let’s take a closer look at each interaction with Alexa in the script.
In the input column, the first line opens your skill. The expected column is what we expect the skill to say in reply. When we click Run, the expected is compared to what AVS reports back – in partial matches. For instance, in the example above the expected response “welcome” is accepted even though the whole response is “welcome to quick list, the quickest way to add to your list at any time”.
After the first line, we move to the next step in the conversation with your device. The session is maintained, so we don’t have to continue to invoke the skill every other line.
When you click Run, we see the results come back from Alexa in real time. In this example, all of our results are successful. However, an error will show up in red along with the response.
This automated testing for Alexa skills is made possible by using the Bespoken Virtual Device SDK. Here’s what it’s doing:
- Turns the script text into audio using text-to-speech (TTS)
- Sends the audio to AVS
- Receives the audio reply from AVS (and metadata, including card info for the Echo Show and Spot)
- Turns the audio back into text for you to see in your Dashboard
And there you have it. Now you have this convenient – and silent – way to perform automated tests on your Alexa skills.
How to Get Automated Testing for Alexa Skills
Follow these steps to get started. First, you will need a Bespoken account. If you don’t have one yet, sign up for a FREE 30 day trial.
- Log in to the Bespoken Dashboard
- Add a new voice app
- Click the “Virtual Device” icon on the left hand menu
- Create your automated test script by entering utterances in the Input column with the corresponding property. Also, add the expected result in the Expected column (Input example: Open SkillName; Output example: Hello, this is SkillName).
- Click “Save” and “Run”
We hope this automated testing feature supports your end-to-end testing process, and we hope you find it super helpful when you need to perform tests without speaking out loud.
Automated Monitoring for Alexa Skills
Now, for serious voice app developers and businesses, you will want to take your test sequence and run it continuously to ensure a satisfactory user experience. Lucky for you, we’ve built automatic monitoring into our software, and Bespoken will notify you when there is a problem with your voice app.
How?
We will take the script you saved in the Dashboard and check its status every 30 minutes. If we detect a problem, you will get an email right away. Best part is, it only takes one more click to start monitoring your voice app.
Just click the enable monitoring icon on your Alexa skill:
And that’s it. You can consider your continuous testing covered, and you can move on to completing the rest of your testing steps – manual testing, unit testing, and end-to-end testing.
More About Alexa Skill Testing
Voice app development requires a different type of maintenance.
Sure outages and issues happen to everyone, but only those who complete a full testing program will rise to the top. We can’t stop an outage or issue, but we can soften the impact, keep the majority of users happy, and keep you out of trouble with your stakeholders.
To get more help with testing Alexa skills, browse these blog posts and join us at our next webinar!
Best Practices for Testing Your Alexa Skills
Amazon Agrees – Building Engaging Alexa Skills Requires Testing and Automation
How to test Voice Apps: Alexa Skill and Google Action Testing Checklist with over 10 Resources
Testing Alexa Skills is Easy
[…] Note: Please see updated blog post. […]