Automated Testing for Alexa Skills and more

Welcome to our Bespoken Dashboard. From here, you have the power to do automated testing for your Alexa Skills and Google Actions – and you can do it all without talking!

Simply create a sequence of tests by just typing, pointing and clicking. These tests are then automatically run against the actualAlexa Voice Service (AVS) or Google Assistant and your voice app. It’s magical, it’s extremely useful for ensuring voice app performance, and it goes great with our bst speak command as well as our Virtual Device SDK. They all rely on the same underlying technology to make voice app testing easier than ever.

First, here is a quick demo of how to write your tests in our Bespoken Dashboard:

Pretty cool, right?

So what’s happening? Let’s take a closer look:

In the input column, the first line opens your voice app (in this case an Alexa skill called Quick List). The expected column is what we expect the skill to say in reply. When we click Run, the expected is compared to what AVS reports back – in partial matches. For instance, in the example above the expected response on the second line “add” is accepted even though the whole response is “added 1 items”.

You can move on to the next step in the conversation with your device. The session is maintained, so we don’t have to continue to invoke the skill every other line.

When you click Run, we see the results come back from the voice service in real-time. In this example, we have two successful results and one unsuccessful result. The error shows up in red along with the response as you can see on the first line. If you want to test multiple valid responses for the same input (“hi”, “hello”, “welcome back”, etc.), you can try our YAML editor.

How does the Bespoken Virtual Device SDK Work?

This automated testing is made possible by using the Bespoken Virtual Device SDK. Here’s what it’s doing:

• Turns the script text into audio using text-to-speech (TTS)
• Sends the audio to AVS or Google Assistant.
• Receives the audio reply (and metadata, including card info for the Echo Show and Spot)
• Turns the audio back into text for you to see in your Dashboard

And there you have it. Now you have this convenient – and silent – way to perform automated tests on your voice apps.

You can create your virtual device by accessing our Virtual Device Manager right on the Bespoken Dashboard.

How Do I Get Automated Testing for my voice apps?

Follow these steps to get started. First, you will need a Bespoken account. If you don’t have one yet, sign up for free.

1. Log in to the Bespoken Dashboard
2. Add a new Alexa Skill or Google Action
3. Create a Virtual Device
4. Write your automated test script by entering utterances in the Input column and define the property. Also, add the expected result in the Expected column (Input example: “Open VoiceAppName”; Output example: “Hello, this is VoiceAppName”).
5. Click “Save” and “Run”

We hope this automated testing feature supports your end-to-end testing process, and we hope you find it super helpful when you need to perform tests without the need to speak out loud.

How to Enable Automated Monitoring for voice apps?

Now, for serious developers and businesses, you will want to take your test sequence and run it continuously to ensure a satisfactory user experience. Lucky for you we’ve built automatic monitoring into our software, and Bespoken will notify you when there is a problem with your voice app. How?

1. We will take the script you saved in the Dashboard and validate it every 30 minutes.
2. If we detect a problem, you will get an email right away.
3. The best part is, it only takes one click to start monitoring your voice app.

Once you are there, we need to set up a virtual device with Alexa. This is done by clicking on the links at the top of the page:

Just click the enable monitoring icon at the top right of your test script on the Bespoken Dashboard.

And that’s it! You can consider your continuous testing covered, and you can move on to complete the rest of your testing steps – manual testing, unit testing, and end-to-end testing.

More on voice app Testing

Developing voice apps require a different type of maintenance.

Sure, outages and issues happen to everyone, but only those who complete a full testing program will rise to the top. We can’t stop an outage or issue, but we can soften the impact, keep the majority of users happy, and keep you out of trouble with your stakeholders.

To get more help with voice app testing, browse these blog posts and join us at our next webinar!