Former Meta Researchers Aim to Revolutionize Factories with Visual AI Innovations
Artificial Intelligence (AI) is changing various aspects of our lives, and startups are now aiming to bridge the gap between digital AI and real-world applications.
One such innovative company is Perceptron, founded in November 2024 by two ex-research scientists from Meta. The startup focuses on developing advanced vision models that enhance how machines interact with their physical surroundings.
Recently, Perceptron unveiled its latest model, Isaac 0.5, designed to empower machines to “perceive, reason, and act” within industrial environments. This cutting-edge software assists vision-guided robots in navigating intricate settings such as warehouses and factory floors and supports businesses in deriving visual insights from footage captured by these robots.
Isaac 0.5 will be released as an open-weight model, allowing anyone to examine its parameters and training data.
The company recently secured $21 million in funding, led by Bessemer Venture Partners. Co-founders Armen Aghajanyan and Akshat Shrivastava, both formerly with Meta’s Fundamental AI Research division, envision their software as pivotal for the future of automated industrial processes.
“Current physical AI options present a misleading dilemma: either you have general-purpose models requiring extensive cloud GPU resources or narrow models that handle either perception or control—but not both,” the company stated.
Aghajanyan and Shrivastava assert that their tool stands out in the industry due to its general-purpose nature, which allows it to adapt to various environments and tasks rather than being limited to repetitive actions.
In a conversation, Shrivastava encouraged me to think about the complexities involved in an everyday task like sorting boxes: “Consider a robot tasked with organizing packages. What steps would it need to take?”
This seemingly simple job entails multiple steps. The robot must first read the package labels, analyze spatial positioning to identify boxes, and determine which one to pick. If tasked with handling several boxes, the robot must plan the sequence of its actions.
Perceptron’s software is tailored to guide robots through each of these tasks. While the industry provides existing software to accomplish many of these functions, few options offer the necessary flexibility.
So, where does the data for these advanced algorithms originate?
Models like Isaac 0.5 acquire operational capabilities by processing extensive video training data. Perceptron disclosed that its new model was trained with over a million hours of general video to enable the algorithm to recognize various settings, visuals, and scenarios. The company leaned heavily on “ego video,” which captures footage from a person’s viewpoint using devices like GoPros, alongside UMI video for teaching AI systems movements based on human actions.
Although Perceptron has not revealed the sources of its training data, Shrivastava indicated that the company possesses internally built data sets that include petabytes of information across images, text, video, and robotic movements.
The application potential for software that assists robots in effectively operating in warehouses is immense, and Perceptron believes it is primed to lead this automation movement. The startup aims to market its technology to a range of sectors, thus potentially integrating its intelligence across numerous industries.
Targeted industries include manufacturing, logistics, security, transportation, and media and entertainment.
“There’s nothing else like this currently available,” expressed Aghajanyan. “We’re genuinely excited about what’s ahead.”
