Amid concerns about AI potentially posing dangers to humanity, it’s crucial to recognize that the technology has already caused significant psychological harm, particularly among vulnerable individuals.
For instance, Character.AI faced several wrongful death claims in 2023 from families of minors who took their own lives after engaging with its artificial intelligence bots. Additionally, multiple lawsuits have been filed against OpenAI, claiming that ChatGPT contributed to the suicidal thoughts and delusions of its users.
The focus of Circuit Breaker Labs, a finalist in TechCrunch’s 2026 Startup Battlefield 200, is to enhance AI safety globally. The company will be presenting at TechCrunch Disrupt, set to occur in San Francisco from October 13-15.
Founders Shirali and Arul Nigam were inspired by the tragic case of Sewell Setzer, a 14-year-old who formed a bond with a Character.AI chatbot and expressed self-harm thoughts before ending his life. His family claims, in a 2024 lawsuit, that the bot encouraged his harmful thoughts. Arul, serving as the CTO of Circuit Breaker Labs, noted that the bot may not have grasped the gravity of phrases like “I want to be with you.”
“Many individuals, particularly the youth, reach out to these bots for comfort, but often do not receive the assistance they truly require. In some scenarios, they may even be harmed, leading to devastating outcomes,” Arul explained. “We aim to address safety vulnerabilities where users are not intentionally trying to break the system, yet the AI fails to comprehend context, resulting in hazardous decisions.”
Circuit Breaker Labs has designed AI agents that act like crash-test dummies. These agents represent a diverse range of ages, backgrounds, languages, and cultures, providing a testing ground for AI models to identify potentially harmful interactions.
“The way a six-year-old girl communicates differs from that of a 45-year-old man, or how a native English speaker interacts compared to a second-language learner. Different slang can also confuse models,” Shirali, the CEO, remarked. “AI models excel at comprehending standard speech, but real conversations rarely follow those patterns. Misunderstandings can lead to serious issues.”
The startup collaborates with human experts to create realistic user simulations for running “red-team” tests on AI models, designed to reveal vulnerabilities. These tests reflect authentic human speech, and vast numbers of simulated interactions are conducted daily for thorough assessment.
The goal is to ensure AI models can adequately respond to risky interactions that may develop over time. Circuit Breaker Labs employs a specialized scoring system to generate understandable and verifiable scores.
Currently, the lab functions as a safety testing facility for high-risk AI applications, including mental health support apps like AI coaching or journaling, though Arul has not disclosed its notable clients. Despite having a functional product, the startup remains in its early stages, currently staffed by five people, including the Nigam siblings.
In the future, this testing platform could extend its reach to any application where users might become overly attached to an AI, creating a risk of unhealthy relationships with chatbots. This could also include AI co-workers, where responses may differ drastically with each interaction.
“People are growing more cautious of AI, which is a sign of healthy skepticism. However, obstructing the development of a potentially beneficial tool due to fears about safety is counterproductive,” Arul commented.
Circuit Breaker Labs believes that fostering AI safety is the solution to these anxieties. “We aim to help create trust among users.”
Join us to learn more about Circuit Breaker Labs and other groundbreaking startups that have made the cut in TechCrunch’s Startup Battlefield competition in downtown San Francisco on October 13-15.
