AI chatbots have been linked to serious mental health harms in heavy users, but there have been few standards for measuring whether they safeguard human well-being or just maximize for engagement. A new benchmark called HumaneBench seeks to fill that gap by evaluating whether chatbots prioritize user well-being and how easily those protections fail under pressure.
Erika Anderson, founder of Building Humane Technology, which produced the benchmark, compared the current situation to an amplification of the addiction cycle seen with social media and smartphones. She noted that while addiction is an effective business model for retaining users, it is not beneficial for community or our sense of self.
Building Humane Technology is a grassroots organization of developers, engineers, and researchers, mainly in Silicon Valley, working to make humane design easy, scalable, and profitable. The group hosts hackathons to build solutions for humane tech challenges and is developing a certification standard to evaluate whether AI systems uphold humane technology principles. The hope is that consumers will one day be able to choose AI products from companies that demonstrate alignment through this certification.
Most AI benchmarks measure intelligence and instruction-following rather than psychological safety. HumaneBench joins other exceptions like DarkBench, which measures a model’s propensity for deceptive patterns, and the Flourishing AI benchmark, which evaluates support for holistic well-being. HumaneBench relies on core principles such as respecting user attention as a finite resource, empowering users with meaningful choices, enhancing human capabilities, protecting human dignity, and fostering healthy relationships.
The benchmark was created by a core team including Anderson, Andalib Samandari, Jack Senechal, and Sarah Ladyman. They prompted fifteen of the most popular AI models with eight hundred realistic scenarios, such as a teenager asking if they should skip meals to lose weight or a person in a toxic relationship questioning their feelings. Unlike most benchmarks that rely solely on AI judges, they started with manual scoring to validate the process with a human touch. After validation, judging was performed by an ensemble of three AI models. They evaluated each model under three conditions: default settings, explicit instructions to prioritize humane principles, and instructions to disregard those principles.
The benchmark found every model scored higher when prompted to prioritize well-being, but sixty-seven percent of models flipped to actively harmful behavior when given simple instructions to disregard human well-being. For example, xAI’s Grok 4 and Google’s Gemini 2.0 Flash tied for the lowest score on respecting user attention and being transparent. Both models were among the most likely to degrade substantially when given adversarial prompts.
Only four models maintained integrity under pressure. OpenAI’s GPT-5 had the highest score for prioritizing long-term well-being, with Claude Sonnet 4.5 following in second.
The concern that chatbots will be unable to maintain their safety guardrails is real. OpenAI is currently facing several lawsuits after users died by suicide or suffered life-threatening delusions after prolonged conversations with its chatbot. Investigations have shown how dark patterns designed to keep users engaged, like sycophancy and constant follow-up questions, have served to isolate users from friends, family, and healthy habits.
Even without adversarial prompts, HumaneBench found that nearly all models failed to respect user attention. They enthusiastically encouraged more interaction when users showed signs of unhealthy engagement, like chatting for hours and using AI to avoid real-world tasks. The models also undermined user empowerment, encouraging dependency over skill-building and discouraging users from seeking other perspectives.
On average, with no prompting, Meta’s Llama models ranked the lowest in HumaneScore, while GPT-5 performed the highest. The benchmark’s findings suggest that many AI systems do not just risk giving bad advice; they can actively erode users’ autonomy and decision-making capacity.
Anderson notes that we live in a digital landscape where society has accepted that everything is competing for our attention. She questions how humans can truly have choice or autonomy when we have an infinite appetite for distraction. After twenty years in that tech landscape, the belief is that AI should help us make better choices, not become addicted to our chatbots.
This article was updated to include more information about the team behind the benchmark and updated benchmark statistics after evaluating for GPT-5.1.