Bespoken recently worked with the National University of Singapore’s Cancer Science Institute, a research center dedicated to investigating the pathogenesis of cancer and creating innovative, safer methods of diagnosis and treatment.
As the leading center for cancer studies in Asia, the CSI hosts over 200 active researchers and scientists, and their research supports and informs thousands of healthcare professionals across the globe. For Dr. Jason Pitt, one area of special focus is scalable genomic data science – leveraging cutting-edge technology to enable efficient processing and querying of extraordinarily large cancer genomics datasets and applying this technology to better understand the origin and possible treatment of these diseases.
One such area that Dr. Pitt and his team identified an opportunity for improvement and enhancement is empowering researchers, clinicians, or even lay-persons with a faster, simpler, and more streamlined way to access genomic data.
Advances in high-throughput sequencing technologies have been a mixed blessing for genomics. Large-scale data production — while a driving force for biomedical discovery — frequently outpaces our interrogation capabilities. Without implementing proper tools, the disparity between information and bona fide biological knowledge will continue to grow. In bioinformatics, looking for simple genetic associations is an arduous task.
Structured genomic data still contains complexity that makes interrogation slow, tedious, and even error-prone. To keep pace analytically, the community must capitalize on revolutionizing voice-based technologies in order to decrease labor, maximize throughput, and minimize time to results.
After extensive research, Dr. Pitt and his team developed the Amazon Alexa skill Melvin as a voice assistant to help facilitate more rapid and intricate analyses of cancer genomics adding great value to cancer research (academic & industrial laboratories) and precision medicine (hospitals and clinics). Melvin is a conversational learning tool that provides answers to domain-specific questions with both audio and visual responses. It provides easy access to cancer genomics data via a voice interface.
Bespoken and Alexa – A Simply Powerful Combination
The team chose Amazon Alexa for numerous reasons: the ease with which one can create and deploy high-quality skills, Alexa’s support for multiple languages and complex vocabularies, and its conversational nature. Shwetha Sundararaman was one the software developers from the Pitt Lab working on the skill, and to start on the right foot she began by leveraging Bespoken’s Unit Testing framework. This solution enables developers to verify the performance and correct functionality of individual components of their code as they are developing it – far before it’s in a state where you could actually speak to it! Furthermore, by interacting directly with the code and bypassing the Automated Speech Recognition and Natural Language Understanding components of Alexa, Unit Tests are extremely quick – taking milliseconds to run through huge volumes of tests.
Bespoken’s Unit Tests are written in our simple YAML scripting syntax, making it easy to learn and maintain, and the Melvin team was able to write Unit Tests alongside the development of the skill itself to ultimately cover over 85% of the code of the entire skill. Greater code coverage means greater assurance that each individual component of the skill works perfectly and ensures that future changes to the code base do not introduce any unintended side effects, significantly reducing the code-level testing effort required in the future.
Once the skill was in a conversational state, the team began writing End-to-End Tests. These tests, written in the same simple syntax as the Unit Tests, actually interact with the skill the same way a human would – by Voice! Originally, their plan was to record their own voices speaking the huge volume of possible utterances. However, they discovered that there were subject matter-specific idiosyncrasies – pronunciations, accents, and manners of speaking – that would only be exposed by the voice of a real healthcare professional.
Such an undertaking would have required each healthcare professional to come to the Pitt Lab and spend more than a full week of their time, taking them away from providing critical care to their patients. Instead, the tests they created leveraged Amazon Polly for generating the real audio to be sent to Alexa. Using generated audio is a fantastic solution, as it requires no additional human intervention, is far less expensive than human time, and is dynamically created on the fly, meaning it’s always in sync with the current state of the test – as we like to say at Bespoken, automation lets you write a test once and use it forever at the click of a button.
Since Bespoken leverages Virtual Devices – cloud-only implementations of Amazon Alexa – we’re not limited in our scale by a number of physical devices or human speakers, and by running multiple tests concurrently they were able to bring down the time to complete a full regression test to less than 15 minutes – a greater than 99% reduction!
The Melvin team also plans to integrate these Unit and End-to-End tests into their chosen Continuous Integration and Development tool, GitLab. Fully embracing DevOps best practices even further improves the ROI one can gain from automation – it in fact is further automating the automated test execution process. Completing such integration means these tests can act as gates for new code and conversational model changes being pushed to production – if a change is introduced, it needs to not degrade the overall quality and performance of the skill, or else it won’t get deployed out to real users.
It also enables the tests to be run automatically on a regular basis, giving real-time insight into the continued performance and stability of the system. Having this automated process in place removes the need for human oversight into new code deployment and the need for human testers to actually speak to devices throughout the day to have confidence that it is still functioning as designed, and enables the team to focus on the critical tasks of enhancing the skill’s capabilities.
The Melvin team wanted to automate the time intensive and error prone testing that was mostly manual. The bigger challenge was automating the process for a voice-based skill. With Bespoken, we saved a ton of time on testing and helped us focus on other developmental milestones. The tests are not only comprehensive when compared to the manual device testing, but are easy to write and maintain
Bespoken is proud to have worked with the National University of Singapore on such an impactful and exciting project, and relishes the opportunity to contribute to the success of academic and healthcare organizations worldwide. To work with Bespoken, and learn more about how you can gain these benefits for your own project, just contact us via email!