Bespoken LLM Benchmark: Does ChatGPT know more than Google and Amazon?
Learn more
October 3, 2017 in Blog

Unit Testing Alexa Skills Using Virtual Alexa

TL;DR Meet Virtual Alexa – an essential tool for unit testing Alexa skills. Unit test skills naturally and programmatically with a simpler interface. Try it now.

Meet Virtual Alexa – a magical emulator that is an essential tool for unit testing Alexa skills. Check it out on Github, or read the API docs here.

Virtual Alexa is based on our existing BSTAlexa emulator, but we’ve taken it to the next level. 🙂

What are our motivations for creating this? We wanted to provide:

  • An intuitive, natural way to unit test Alexa skills programmatically – as if you were speaking to the device
  • A simpler interface for startup and teardown of tests
  • A first-class project in its own right

We see the Virtual Alexa emulator as a standalone, highly useful project, and so we have moved it to its own repository. It really has taken on a life of its own, independent of our core CLI project, bst, as it is foundational to other new pieces we are working on.

To see how Virtual Alexa works in action, take a look at this project: Giftionary

This is a fun Alexa skill we’ve created that shows you an image from Giphy, then asks you to guess the search term behind it. For this new skill, we needed to create a thorough set of tests. Here’s a simple example:

test("Launches and plays", (done) => {
// Launch the skill
alexa.launch().then((payload) => {
// Test the correct text response comes back
expect(payload.response.outputSpeech.ssml).toContain("We show you images we got from Giphy");
// Once it launches, say Yes
return alexa.utter("yes");
}).then((payload) => {
expect(payload.response.outputSpeech.ssml).toContain("Take a look at this image");
// Cancel after the second interaction
return alexa.utter("cancel");
}).then((payload) => {
// Make sure the skill gracefully exists and ends the session
expect(payload.response.outputSpeech.ssml).toContain("Goodbye");
expect(payload.response.shouldEndSession).toBe(true);
done();
});
});
view raw SimpleTest.js hosted with ❤ by GitHub

Easy, right? We hope you agree that the syntax is intuitive – and wait until you see the Alexa unit tests perform with async/await. 😉 This sample uses Jest, but Mocha works just as well.

Backing up momentarily, here is how we configured Virtual Alexa for use:

let alexa = virtualAlexa.VirtualAlexa.Builder()
.handler("src/index.handler") // Lambda function file and name
.intentSchemaFile("./speechAssets/IntentSchema.json") // Path to IntentSchema.json
.sampleUtterancesFile("./speechAssets/SampleUtterances.txt") // Path to SampleUtterances
.create();

Virtual Alexa will also work with the new InteractionModel schema as well – your call. In either case, it parses the skill metadata so that you can call it with real utterances.

Here is a more in-depth example, that contains a complete interaction:

test("Plays once", (done) => {
alexa.utter("get started").then((payload) => {
expect(payload.response.outputSpeech.ssml).toContain("What is the search term for it");
return alexa.utter("incorrect guess");
}).then((payload) => {
expect(payload.response.outputSpeech.ssml).toContain("Nice try");
return alexa.utter("incorrect guess");
}).then((payload) => {
expect(payload.response.outputSpeech.ssml).toContain("That is not correct");
return alexa.utter("incorrect guess");
}).then((payload) => {
expect(payload.response.outputSpeech.ssml).toContain("No more guesses");
return alexa.utter("no");
}).then((payload) => {
expect(payload.response.outputSpeech.ssml).toContain("Goodbye");
done();
});
});

We even tied it into code coverage – here is a quick sample:

code coverage

Click here for the the full report.

We hope you find this useful! Here’s what we are planning next to make unit testing Alexa skills easier, faster, and even more quality-assuring:

  • AudioPlayer support
  • More sophisticated slot-handling
  • Integration with Slack

If you would like to get involved, please feel free to open a Pull Request, or talk to us about the project on Gitter.

Leave a Reply

Your email address will not be published. Required fields are marked *