Bespoken LLM Benchmark: Does ChatGPT know more than Google and Amazon?
Learn more
April 6, 2020 in Blog

The Bespoken Test Robot aka “The Robollama”

⚡ LIVE, INTERACTIVE 🏄‍♂️ Demo Via Twitter
TL;DR If you can talk to it, we can test it - Bespoken presents an easy and fun way to try out their new Test Robot offering. Read on to learn all about it.

What Is This?

We have set up two of our Bespoken Test Robots (which we affectionately refer to as RoboLlamas) for use on Twitter. They are configured to test an Echo Show and Google Nest Home Hub. This an easy, interactive way to try out Bespoken’s Test Robot product, as well as just have fun with Alexa and Google. Go here to try it out:

Alexa, how’s the weather?

How Do I Use It?

Any tweet that includes the hashtag #robollama and starts with “alexa” or “hey google” will be sent to our Test Robot. For example, you can say something like: “alexa what time is it” or “hey google who is Pete Townshend?”

We then take that tweet and do the following:

  • Turn it into audio via text-to-speech
  • Play the audio via the speaker on our Test Robot (housed in a remote, secure location 🙂)
  • Capture the reply from the device being tested using a microphone
  • Captures the display output via camera
  • Translates the audio response to text via speech-to-text
  • Translate the display output to text via OCR

Phew – it’s a lot of work! But it’s conceptually simple, and so is using it. You can ask us any of the questions you ask of Google Assistant or Alexa – have fun with it! And expect to hear more from us as we design some cool, fun contests in the coming months.

And to find out more about the Bespoken Test Robot, go here: https://bespoken.io/test-robot. Or to setup a custom demo, just email jpk@bespoken.io.

Common Questions

Does this work outside of Twitter?

It does! It works just like our “regular”, world-leading automated testing for voice (in-use everywhere from Spotify to Roku to the NFL) – you can write simple test scripts to interact with it, or use our powerful APIs. More info here: https://read.bespoken.io/end-to-end/getting-started

What Is This Good For?

The Bespoken Test Robot can test any device that uses voice as an interface. This includes:

  • Highly specific voice-first devices – such as the Echo Spot, Google Nest Home Hub Max
  • Mobile Applications – such as Erica from Bank of America, the Pandora voice search, etc.
  • In-car head units – Android Auto, Apple CarPlay,
  • Custom-made Devices – such as those made for healthcare and home, from pill dispensers to thermostats and everything in-between

The upshot is – if you can talk to it, we can test it!

What is the transcript?

The transcript is the result of applying speech-to-text to the actual audio response from the device being tested. It may be slightly imperfect – but don’t worry, we have simple ways to compensate for that in production testing implementations. If you are unsure what the device really said, just click on the audio link to listen to it.

What if more than one person talks to it at the same time?

We queue the tweets we receive, so if multiple people send tweets to the RoboLlama simultaneously, it might just mean it will take longer to get a response. Please bear with us!

What is this video?

The video includes the actual image we capture from the device when we are speaking to. It takes a picture of the screen five seconds after it begins its spoken response. This delay is easily configurable depending on the type of device and query being tested. We then take the image and put it together with the spoken response from the Test Robot.

How do I learn more and/or get my own test robot?

Just reach out to jpk@bespoken.io for an in-depth demo. We will get you set up fast.

Leave a Reply

Your email address will not be published. Required fields are marked *