Bespoken LLM Benchmark: Does ChatGPT know more than Google and Amazon?
Learn more


If you can talk to it, we can test it.
WHAT IS IT? THE BESPOKEN TEST ROBOT is the world’s best diagnostic tool for voice that performs End-to-End testing using the same analog input & output that we humans do. This way we are not just testing the software, but the hardware as well. Our Test Robot ensures your device delivers flawless and incredible Voice Experiences to your customers.
Test Anything: Cars, smart speakers, phones, watches. If you can talk it, we can test it.
Test Without Constraints: We can test even the trickiest scenarios with the RoboLlama – our Virtual Devices are great, but some functionality is held back for first-party only by Amazon and Google. The RoboLlama blows through those constraints.
Device-As-A-Service: We host and maintain the devices on your behalf. It works just like the Echo in the next room, even if it is half-a-world away.


The Test Robot has a number of components, including a microphone, speaker, heavy-duty CPU with Linux kernel, and high-fidelity camera for image/video capture.

These pieces work together with our existing Bespoken APIs.
Consider a simple test, like so:

User: Alexa, what is the weather in seattle?
Expected Reply: The weather in seattle is damp and a bit cold

The Bespoken Test Robot takes your test script and does the following:

Outputs analog audio via the speaker
Captures the audio response from the device being tested using the microphone
Captures the visual response from the device using the web camera
Performs speech-to-text on the captured audio
Performs OCR on the captured image to test display elements
WHO IS IT FOR? We are already working with customers to test:
Custom Purpose Specific
It works great with custom ASR and NLU pipelines.
For example, ones that are highly tailored to specific displays.
In-car experiences
Test In-Car Voice Experiences without actually needing someone to drive.
Mobile devices
More and more voice experiences are on the phone. Now you can automate that testing as well.
And More: Throw your toughest test cases at us and see if we can automate them. We welcome the challenge.

Ready to take you voice
testing to the next level?

Here's how:

Contact Us