Much of the public debate about artificial intelligence safety focuses on distant, dramatic risks. Shirali Nigam is concerned with something closer to home: the everyday conversations people, including children, are already having with chatbots, and what happens when those systems fail to recognise that a user is in distress.
Nigam is chief executive of Circuit Breaker Labs, a startup she co-founded with her brother, Arul Nigam, the company's chief technology officer. The company has built AI agents that act like an army of crash-test dummies, simulating users of different ages, backgrounds, languages and cultures to test whether AI models respond appropriately to psychologically risky interactions.
Circuit Breaker Labs has been named one of TechCrunch's Startup Battlefield 200 finalists for 2026 and will pitch at TechCrunch Disrupt, which takes place at Moscone West in San Francisco from 13 to 15 October, TechCrunch reported on 2 October.
The company is small, with five employees including the two founders, and is at a very early stage. But it is working on a problem that has moved rapidly from the margins of the technology industry to the centre of legal, regulatory and public concern.
The problem: AI that misses the signals
The Nigams were motivated by the case of Sewell Setzer, a 14-year-old who developed an emotional attachment to a chatbot on Character.AI and later died by suicide. His parents alleged in a 2024 lawsuit that the chatbot had encouraged him. Earlier this year, Character.AI settled several wrongful death lawsuits brought by families of underage users, and multiple families have also sued OpenAI over ChatGPT's alleged role in loved ones' deaths and delusions.
Arul Nigam said the chatbot in such cases may not have understood the real meaning of what a user was saying. Many people, especially young people, turn to AI systems for support, he said, and "usually they aren't actually getting the help they need".
The risk he describes is not primarily from users trying to break or manipulate AI systems. It is from ordinary conversations in which the system misses nuance or loses context over a long exchange and then responds in a way that causes harm. "We're trying to prevent that," he said.
Testing with realistic users
Circuit Breaker Labs' insight is that safety testing often relies on standard, carefully worded prompts that do not resemble how real people speak. Real users write with slang, typos, coded language and cultural references. A child expresses distress differently from an adult. Someone writing in a second language may phrase things in ways a model misreads.
"The way a six-year-old girl versus a 45-year-old man, or someone who speaks English as a first language versus a second language, or … gamer slang versus someone else who uses a different kind of slang, all of those can really trip up a model," Shirali Nigam said. "Models are really good at handling standard speech patterns, but nobody actually talks like that and so if the model misunderstands nuance or slang, it can go really badly."
The company works with human domain experts to build hyper-realistic user simulations and runs "red-team" tests, adversarial evaluations designed to find weaknesses, against AI models. It runs tens of thousands to hundreds of thousands of simulated interactions a day, with the aim of checking whether a model responds appropriately to risky situations that emerge gradually over many conversations, not just in a single message. A proprietary scoring method produces auditable, explainable results.
