The concept of voice as the next significant interface is gaining substantial traction, with substantial investments directed toward startups focused on voice AI. These companies are developing a range of applications, including customer service solutions, meeting transcription, and AI-driven dictation.
New voice models and tools that claim to replicate human conversation are launched weekly. However, some experts argue that the technology hasn’t yet reached its potential. Shawn Wen, CTO of the enterprise voice AI platform PolyAI, remarked that while full-duplex models—which can speak and listen simultaneously—have been created, voice AI has yet to experience its equivalent of a “ChatGPT moment.”
“We’ve made strides with full-duplex models. The next hurdle is to enhance reasoning speed, allowing the models to provide answers rapidly and create a more natural conversation flow,” he stated during a presentation at the recent HumanX conference.
Additionally, Wen emphasized that AI agents in customer service settings should avoid sounding robotic, should inspire confidence in users, and should be capable of resolving issues effectively.
“Once the voice technology reaches a certain level of quality, and customers feel comfortable interacting for a few exchanges, they may begin to trust that the AI can handle their inquiries without needing to escalate to a human,” he explained.
Alex Gay, CMO of the meeting transcription service Otter, suggested that advancements in speaker identification and intent understanding are essential for automation. The company is also developing digital twins that could represent individuals in meetings and emphasized that the AI’s voice must convey the emotional nuances of human speech.
“In today’s meetings, the most effective discussions arise from engaging debates and building relationships. If that connection cannot be formed with an avatar, it becomes merely a basic Q&A chatbot,” Gay commented.
Understanding and Transparency in Voice AI
Despite progress in voice AI, many assistants still struggle to accurately comprehend users, leading to incorrect transcriptions or summaries during meetings.
Wen pointed out that automatic speech recognition (ASR) often fails to capture vital keywords, which hinders the ability to grasp the full context.
Gay concurred, stating that Otter is continually enhancing its transcription capabilities. He noted that language remains a crucial area for voice model improvement.
“At Otter, we view transcription as merely the initial step toward achieving productivity enhancements. If the original transcription lacks precision, any follow-up actions taken based on that will be flawed. Mistakes lead to a loss of confidence in the platform. Thus, refining our ASR model is imperative, as it has far-reaching consequences,” he said.
As new voice technologies emerge, issues of transparency also arise. Tools should inform users if they are being recorded or speaking with an AI. Otter aims to foster trust during meetings, even implementing notification systems to alert participants when meetings are recorded. Wen from PolyAI echoed that it is vital for users to know they are interacting with an AI during enterprise discussions.
