Bespoken Release Roundup – January 2020

Hi! As a new year starts, we want your voice testing to be easier than ever. Here are our latest features and bug fixes that should help you accomplish that!

Dashboard

Improvements

• Blank "expected" fields will show the actual results: In some cases, you might want to add an utterance to your e2e tests but don’t need to run an assertion against it. In those cases, you can simply leave the "expected" field empty. We will return the transcript on the "actual" field and mark it as a success in the same way we do when asserting against "*".

• User errors are reported on monitoring: Related to a virtual device improvement described below, errors that are categorized as a user error will send an email explaining the issue. For example, setting a voice experience to be monitored with a virtual device whose permissions have been revoked.

Bug resolution

• Fixed an issue on the test page where, if a problem happened while testing, the user wouldn’t get notified about it. Eg.: Using a virtual device with revoked permissions.
• Fixed an issue that was preventing the option to subscribe to our newsletter from appearing in "My Account" page.
• Fixed an issue that prevented users from enabling monitoring on the YAML editor version of our test page.
• Fixed an issue when switching between the visual and YAML editors of the test page while having an empty assertion would cause the "expected" field to disappear.

Bespoken CLI

New Features:

• Configurable locations for testing.json and context: If you are like us, you might have both unit and end to end testing set up for your voice experiences. For those cases and more complex ones, we have added the possibility of specifying the location for your testing.json file and also set a custom context to resolve your file paths.

  bst test --config config/testing.json --context .

  Read more about it here.

• Using env variables as part of your settings: You can replace a setting inside your testing.json file by using the following format: ${ENV_VARIABLE}. For example, this:

  {
    "interactionModel": "${SKILL_LOCALE}.json"
  }
  

  … would result on the interactionModel being en-US.json if your SKILL_LOCALE env variable was set to "en-US".

Read more about it here.

Bug resolution:

• Fixed an issue where setting an utterance without an expected value would not return the actual value that came back from Alexa or Google Assistant. (skill-testing-ml #325)

Virtual Devices

Improvements

• A new property that returns the error category is returned over our API. Categories are "user" and "system". The first one is related to errors caused because of bad use of the API, while the second one refers to unexpected errors like issues with Google or Alexa voice services.
• A clearer message is shown if using a Google virtual device with revoked permissions.

Bug fixing

• Fixed an issue where any utterance sent from our virtual devices appeared twice on the Alexa history.
• Fixed an issue where utterances in german were not being recognized if using Polly voices.