General Intuition’s $2.3 Billion Gamble on Video Games as Real-World AI Training Grounds

Upon entering the R&D floor of General Intuition at their New York headquarters, Pim de Witte, the youthful co-founder and CEO, showcased a monitor where an AI was engaging in a game reminiscent of Fortnite. This wasn’t a human player.

“Our AI has played for 100 consecutive hours,” exclaimed Kent Rollins, the company’s chief product officer, clearly excited.

Before I could fully immerse myself in the AI’s gaming prowess, I heard the sounds of a large four-legged robot approaching.

“The very same AI that’s playing the game is also controlling the robot,” de Witte explained.

Data analyst Josh Duplantis, carrying a laptop that streamed a live feed from the robot’s camera, elaborated that the robot’s primary function was “exploration.”

Using its singular camera, the insect-like robot moved around the office, brushing against chairs and bumping into a trash can, mirroring the clumsiness of a young child still learning about spatial awareness. Duplantis mentioned it took just eight minutes of real-world data to refine the AI model for the robot, and intriguingly, that data was captured outdoors, not in the office environment.

Creating a model that can generalize across gameplay, simulations, and physical embodiment is the core mission of General Intuition. This innovative approach has attracted investments from prominent backers.

Recently, General Intuition announced it secured $320 million at a valuation of $2.3 billion. This funding brings the company’s total disclosed investment to $454 million following a $134 million round last October.

General Intuition originated from de Witte’s previous venture, Medal, which enabled gamers to share video game clips. The extensive hours of gameplay uploaded acted as the foundational dataset for training the model in spatial-temporal reasoning—essentially how to navigate space and time.

However, the essential data was not the gameplay footage itself, but the action labels attached to those clips, documenting the precise player actions. Most competitors attempt to infer actions from video data alone, which de Witte believes is insufficient.

“We see this as the future of pre-training,” he stated. “Our unified model can process Fortnite graphics and respond to real-world dynamics, surpassing the capabilities of typical large language models.”

De Witte set me up with a laptop running General Intuition’s world model, a simulated environment created frame-by-frame, not via conventional game design engines. In my tests, I walked directly into walls. Unlike other models I have used that ignore physical boundaries, this one understood that walls are solid, ladders are for climbing, and shadows cast by the sun change with its movement.

For General Intuition, this world model acts as a training lab or “gym.” The ultimate aim is to commercialize the agentic model itself, and de Witte emphasizes that the embedded action data allows the model to differentiate between self and environment, enriching its understanding of cause and effect.

While the company’s technology showcases significant potential, it faces competition in solving these challenges. Additionally, achieving reliable performance in real-world scenarios at scale remains untested. General Intuition believes that leveraging gameplay can provide an efficient alternative.

Investors support this approach. The recent funding round was led by Khosla Ventures, with notable contributions from General Catalyst, Jeff Bezos, Eric Schmidt, Nico Rosberg, and researchers from Google DeepMind and MIT.

A significant portion of this investment will enhance computational capabilities. General Intuition has partnered with CoreWeave and aims to focus on pre-training the next model version. They also plan to broaden access to their API by the end of summer.

Vinod Khosla, who spearheaded the latest funding round, is fascinated by de Witte’s vision and the company’s unique data resources.

“In the realm of large language models, the emergence of reasoning was revolutionary,” Khosla shared via phone. “In world models, I foresee a similar breakthrough with the onset of human-like intuition in AI. The human action data from games is crucial for this intuition.”

A Vision for the Future

General Intuition recognizes the significance of Medal’s human action data in constructing dynamic world models and versatile agents. According to Brianna Martin, the startup’s chief of staff, the company’s journey began partly after Medal declined an acquisition offer from a major laboratory, with more offers surfacing subsequently.

De Witte, alongside co-founders Eloi Alonso, Adam Jelley, and Vincent Micheli, is not keen on being acquired, and neither are their investors seeking a quick exit. The proprietary data amassed via Medal is one of the factors that encourage Khosla to view the startup as a multi-generational opportunity rather than a mere acquisition target; he sees the potential for creating a foundational framework for adaptive agents and world models.

“At this stage, an acquisition would simply mean obtaining data, which isn’t particularly interesting,” Khosla commented.

Part of this confidence also comes from de Witte’s guiding principles.

Having spent three years in humanitarian work, including with Doctors Without Borders, he has clearly outlined how General Intuition’s technologies will be used: they will not develop agents to inflict harm on humans.

“We refuse to be a part of any system that escalates violence,” de Witte stated. “Imagine if I announced we were venturing into lethal AI—what chaos would that stir globally?”

This ethical stance on military applications arises as the tech industry shows increasing interest in warfare, although de Witte expresses his readiness for his models to assist in humanitarian efforts like search and rescue operations.

Being of Dutch origin with a predominantly European team contributes to the company’s ethos. De Witte mentioned Martin’s hiring was influenced by her public resignation from Palantir over its collaboration with U.S. Immigration and Customs Enforcement.

“I’m perplexed by Silicon Valley’s choices,” he remarked. “It explains my absence from the area.”

De Witte’s ethical outlook extends beyond what the technology won’t enable. As an avid gamer who earned $1.5 million running a private RuneScape server as a teenager, he contemplates the impact on individuals who might be left behind due to advancements in AI.

General Intuition recently launched Nerve, a job marketplace allowing gamers to earn money by utilizing their existing setups. Initial tasks involve data labeling, eventually advancing to robotic teleoperation and other roles. As de Witte pointed out, Medal’s audience represents a generation highly vulnerable to AI-induced job displacement, and he aims for them to partake in what the future holds.

Cultivating a Data Ecosystem

De Witte envisions General Intuition as a facilitator of ecosystems like Anthropic or OpenAI—a provider of foundational models enabling others to innovate on its technology. Currently, the startup has several clients across gaming, simulations, and robotics.

“We don’t intend to create a self-driving vehicle company,” de Witte clarified. “Our goal is to simplify it for others to establish their self-driving endeavors.”

The company asserts that broadening their API access will allow for testing various applications, such as using their model to operate a robot in a digital factory setting, powering a humanoid bot in a gaming environment, or sending a quadruped into challenging terrains.

Though the quadruped is the first real-world manifestation General Intuition has trialed, they have also experimented with drones and other devices, including assessments in driving simulations.

“It can function with any system controlled by a game controller or a keyboard and mouse,” de Witte commented.

Ultimately, creating a robust data ecosystem remains a key objective.

“We’ll select clients that can enhance the diverse applications this generalized foundation model can service,” de Witte explained. “Our priority is to collaborate with those who offer valuable real-world data that can advance our research and possess agile internal teams with whom we can partner closely.”

Khosla emphasizes that General Intuition’s unique data has propelled its success thus far, and the continuous ability to gather data that others cannot will be crucial, especially as the reliability of transitioning from simulation to real-world scenarios at scale is still unanswered.

Note: The headline previously misstated the amount General Intuition raised in this funding round. The correction has been made.