Automated Testing for Alexa Skills

Bespoken Validation Automated Testing for Alexa Skills blog image

With the release of the new Bespoken Dashboard, you can now use the Validation feature to run automated testing for Alexa skills – and it can all be done without speaking aloud.

How does Automated Testing for Alexa Skills Work?

When you’re logged in to your Bespoken Dashboard, select the Validation tab and create a sequence of tests for your skill. We take your test sequence and run it through Alexa Voice Service (AVS) and your skill. Like this.

Bespoken Validation Automated Alexa Skill Testing Example Gif

 

Let’s take a closer look at each interaction with Alexa in the script.

 

Automated Testing for Alexa Skills with Bespoken Validation

 

In the input column, the first line opens your skill. The expected column is what we expect the skill to say in reply. When we click Run, the expected is compared to what AVS reports back – in partial matches. For instance, in the example above the expected response “how can i help you” is accepted even though the whole response is “welcome to bring how can I help you”.

After the first line, we move to the next step in the conversation with your device. The session is maintained, so we don’t have to continue to invoke the skill every other line.

When you click Run, we see the results come back from Alexa in real time. In this example, all of our results are successful. However, an error will show up in red along with the response.

This automated testing for Alexa skills is made possible by using the Bespoken Virtual Device SDK. Here’s what it’s doing:

  • Turns the script text into audio using text-to-speech (TTS)
  • Sends the audio to AVS
  • Receives the audio reply from AVS (and metadata, including card info for the Echo Show and Spot)
  • Turns the audio back into text for you to see in your Dashboard

And there you have it. Now you have this convenient – and silent – way to perform automated tests on your Alexa skills.

How to Get Automated Testing for Alexa Skills 

Follow these steps to get started. First, you will need a Bespoken account. If you don’t have one yet, sign up for free.

  1. Log in to the Bespoken Dashboard
  2. Add a new voice app
  3. Click the “Get validation token” link
  4. Create your automated test script by entering utterances in the Input column. Also add the expected result in the Expected column (Input example: Open SkillName; Output example: Hello, this is SkillName).
  5. Click “Save” and “Run”

We hope this automated testing feature supports your end-to-end testing process, and we hope you find it super helpful when you need to perform tests without speaking out loud.

Automated Monitoring for Alexa Skills

Now, for serious voice app developers and businesses, you will want to take your test sequence and run it continuously to ensure a satisfactory user experience. Lucky for you we’ve built automatic monitoring into our software, and Bespoken will notify you when there is a problem with your voice app.

How?

We will take the script you saved in the Dashboard and check its status every 30 minutes. If we detect a problem, you will get an email right away. Best part is, it only takes one more click to start monitoring your voice app.

Just click the enable monitoring icon on your Alexa skill.

 

 

And that’s it. You can consider your continuous testing covered, and you can move on to completing the rest of your testing steps – manual testing, unit testing, and end-to-end testing.

More About Alexa Skill Testing

Voice app development requires a different type of maintenance.

Sure outages and issues happen to everyone, but only those who complete a full testing program will rise to the top. We can’t stop an outage or issue, but we can soften the impact, keep the majority of users happy, and keep you out of trouble with your stakeholders.

To get more help with testing Alexa skills, browse these blog posts and join us at our next webinar: Quality Assurance for Alexa Skills on April 17 or 19.

Amazon Agrees – Building Engaging Alexa Skills Requires Testing and Automation

Alexa Skill Testing Checklist with over 10 Resources

Infographic: Alexa Skill Testing is a Piece of Cake

7 Voice-First Experiences with Testing Alexa Skills