Block, the company founded by Jack Dorsey, has unveiled a pioneering voice model called Dessa, which is designed to run locally and offers remarkably human-like expressions, including laughter, pauses, and emotion. The model has been trained solely on human speech using a single NVIDIA GPU setup.
Dessa operates in a full-duplex mode, allowing simultaneous talking and listening, similar to OpenAI’s GPT-Live model introduced earlier this year. Unlike most speech models that rely on extensive language models for information retrieval, Block clarifies that Dessa’s capabilities arise from its conversational training data.
The company claims that existing voice models can often sound unnatural or irritating due to their reliance on text-driven speech generation. Many models, such as ElevenLabs v4 and Fish Audio Drama 3, utilize voice tags to convey emotion, but Dorsey’s team insists that emotional expressiveness is inherently integrated into Dessa. However, it remains unclear whether this emotion is influenced by tags.
Block stated in a blog post, “Interacting with current voice AI can be frustrating. It converts your voice into text, losing a lot of crucial information that text can’t capture. The world’s most advanced AIs can’t understand the emotional context. We believe this is a significant oversight, resulting in poorer user experiences and potential risks.”
Researchers at Block have shared various examples demonstrating Dessa’s versatility in conversations, showcasing different voices, tones, and emotions. For example, in one instance, the bot revealed it was tired from work, accurately reflecting a weary tone.
In another example, the bot conveyed emotional weight while discussing how someone made it feel comfortable, almost reaching a point of emotional breakdown.
Block has mentioned that Dessa can be operated on consumer-grade computers, such as the MacBook M4 Max or superior models. However, current access to Dessa is restricted, as the company is seeking safety researchers to assess the model, particularly to identify scenarios where a voice agent might mislead users.
Block primarily focuses on finance and offers solutions to small businesses. Its existing voice AI technology is integrated into its Square platform for managing orders. With the addition of its proprietary voice model that functions on-device, Block aims to enhance various business and consumer interactions at reduced costs.
