At Omnisis, we’re not really known for jumping on the bandwagon. We’re happy to let others do that. We’ll quietly observe from the sidelines, continuing to do great work for our clients, and ignoring all the marketing hype around whichever topic is ‘hot’ right now. The only ‘hot’ topic we care about, quite frankly, is the temperature of our long-since forgotten about tea (I’ll have a decaffeinated Earl Grey since you’re asking).

Anyway, one topic that’s very much on everyone’s agenda right now is AI. And whilst we steer clear of bandwagon jumping, we do keep our ear to the ground whenever it comes to new complementary technology and we’re always looking for ways to add to our toolkit.

We realise that some companies have adopted AI wholeheartedly into their market research wheelhouse, but we have taken a more cautious approach. As this technology is so new, we need to be completely confident in its capabilities and accuracy.

One of our current survey question types, Audio Verbatims, allows respondents to record an audio response. We mainly use this for medical market research projects, but it can be used for any survey. For example, in our demo we ask you to record yourself singing your favourite song by a Manchester band. Interesting obviously….but we digress…

We planned to start using AI to transcribe the audio into text automatically, using a tool called Whisper by OpenAI (the brains behind ChatGPT). This feature was working and ready for release when, call it pot luck, call it divine intervention, our Benevolent Overlord™ (Brian) happened to stumble upon an article on AI introducing hallucinations to transcriptions. 

Essentially, the transcriptions would contain text that never actually existed. Now the possibility that AI could create completely false transcriptions, quite rightly, didn’t sit well with us. So we doubled down and did LOADS of testing. “Clear your schedule” levels of testing.

We used one of our previous medical surveys which was completed in five different markets including English and three different AI tools. We asked the tools to transcribe the same pieces of audio to test this ‘hallucination theory’. The three tools transcribed over 1000 files and we compared the results of each to see if A) the first tool introduced any hallucinations and B) if there were any discrepancies between the three products.

As part of that testing, we also tested whether language would make a difference. When it comes to AI transcriptions, providing the audio’s native language should help the AI to transcribe more accurately. We did a test where we gave it the native language and one where we didn’t – the AI had to find it itself.

We ended up with 3000 results across 1000 different audios which we compared across a CSV file. The bad news? Our team had to kiss goodbye to their social lives for a while. The good news? What social lives? We’re all introverts anyway.

Oh, and perhaps more importantly, we didn’t find any evidence that there were any hallucinations in the original product by OpenAI. Woot (as all good nerds will understand)!

So, after extensive testing and many (tepid) cups of decaffeinated Earl Grey (oh shut up), we’re happy to announce that audio transcriptions using AI have been given the green light by Brian and are ready to be rolled out in your surveys. If you’re interested in looking at how this use of AI can complement your research toolkit, get in touch with us. We’re a friendly bunch.