Revolutionary French Startup ZML Launches Free Tool to Accelerate AI Chip Inference Speed

Nvidia continues to hold a substantial share of the market, but emerging competitors and options are surfacing rapidly.

ZML, a promising French AI startup supported by Turing Award laureate Yann LeCun, has introduced innovative inference-performance software. This allows multiple open-source large language models to operate across a range of hardware, including chips from Nvidia, AMD, Google’s TPU, Apple Metal, and Intel Arc.

With the launch of ZML/LLMD, the new inference server, the company aims to break down existing barriers and optimize diverse chips for AI applications at their highest speeds—and sometimes even faster. Founder Steeve Morin shared his insights on this ambitious endeavor.

As AI becomes woven into our professional and daily lives, optimizing inference—the process of handling requests—has outstripped model training in significance. However, it often appears inconsistent due to software and architectural limitations that contribute to vendor lock-in, according to Morin.

The potential for achieving top performance across different chips represents a significant technological accomplishment and could disrupt the current market amid rising concerns about AI expenses.

ZML aims to give companies the flexibility to utilize a combination of chips, including some that are potentially more affordable or energy-efficient. “Our goal is to empower users to build their systems, achieving real efficiency improvements that facilitate AI distribution,” Morin explained.

This software innovation may support emerging AI chip developers, particularly those based in Europe, like Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA. Morin emphasizes that the key aspect is ZML’s capacity to collaborate on innovative initiatives that have not yet been attempted globally.

Despite his enthusiasm for new technologies, Morin does not underestimate Nvidia’s position, partly due to its established production capabilities. He confirmed that ZML maintains a positive relationship with the AI chip leader, which is preparing for increased inference demands.

The surge in investment in inference has led to what is often referred to as the “inference gold rush.” Consequently, ZML faces competition from firms like Baseten, recently valued at $13 billion; Inferact, linked to the creators of the open-source project vLLM; and RadixArk, the company behind SGLang.

Both vLLM and SGLang offer partial competition to LLMD, but Morin envisions a broader scope for ZML. “We have reached the stage where we are co-designing chips,” he stated, praising the nimbleness of ZML’s 20-person team for its rapid development and future releases.

This agile team is also well-capitalized, thanks to Morin’s previous success as VP of engineering at Zenly, which was acquired by Snapchat in 2017 for a substantial sum. He secured $20 million from various venture capital firms, including notable investors like Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, and others.

Unlike ZML’s initial public effort—an ML framework focused on inference released in 2024 and updated in March—ZML/LLMD is not open source. However, it is launching as a free product to assess usage patterns. “I prefer to analyze usage before monetizing effectively, rather than jeopardizing growth through early greed,” Morin stated.

It remains uncertain when ZML/LLMD will transition to a paid model or how its adoption will unfold. Nevertheless, the startup’s ownership structure indicates that prominent founders, including Solomon Hykes of Dagger and Docker, Clément Delangue and Julien Chaumond from Hugging Face, and LeCun, who is now with AMI Labs, are closely monitoring its progress. This scenario supports the notion that European AI startups can thrive domestically. “I couldn’t establish ZML anywhere else but in Paris,” Morin noted.