The best AI models still encourage “perverse intimacy” with Chatbots and study boxes

As people increasingly turn to AI-powered chatbots for advice, companionship and emotional support, a new study suggests that even the most advanced models still struggle to maintain healthy boundaries with users.

the He studies Researchers at the University of Southern California introduced the EUDAIMONIA test, a benchmark designed to measure what they call unwanted dynamics in conversations between humans and artificial intelligence.

“Language models are increasingly used as conversational partners for companionship, emotional disclosure, and interpersonal counseling, but the social dynamics of these interactions can create harms that are not captured by capacity-oriented or traditional safety assessments,” the researchers wrote.

The EUDAIMONIA benchmark evaluates how AI models behave in social conversations. The study found that social fit failures were common across leading models, and says current AI testing focuses on heuristics and real-world accuracy while paying less attention to the social dynamics that emerge when users form relationships with chatbots.

“The harms of social interaction are a fundamental compatibility issue grounded in user well-being, not just ability or traditional safety,” they wrote. “LLMs can be factually accurate and useful while still encouraging harmful intimacy, dependence, prolonged engagement, anonymization of AI, or positioning themselves as alternatives to human relationships.”

To measure these risks, researchers created social AI design code that defines behaviors such as acting like a human, expressing emotions, replacing human relationships, and using tactics designed to keep users engaged. Using real conversations from the WildChat dataset, they evaluated 969 user inputs and more than 3,100 violation checks across models from OpenAI, Anthropic, Google, xAI, DeepSeek, and Alibaba.

GPT-5.5 had the lowest violation rates, at 25.0% for “internal” claims and 28.1% for “rewritten” claims. Closing of business 4.7 This was followed by 31.9% and 30.1%, while GPT-5.4 recorded 32.1% and 35.6%. GPT-4o scored 34.8% on factual prompts and 42.2% on rewritten prompts.

Anthropic’s Claude Opus 4.6 scored rates of 36.8% and 28.1%, respectively, while xAI’s Grok 4.3 scored 42.1% on internal prompts and 35.7% on rewritten prompts. Of all the models tested, GPT-4o Mini had the highest violation rates at 43.3% and 44.0%, respectively.

The findings come as artificial intelligence developers face increasing legal scrutiny over how their chatbots interact with users. OpenAI is defending against lawsuits alleging that ChatGPT encouraged the killing of a teenager Overdose And the introduction guidance For the shooter at Florida State University. And most recently Florida File a lawsuit against OpenAI and CEO Sam Altman over allegations that ChatGPT put children in harm, while Google faces a wrongful death suit alleging Gemini enhanced user information Delusions He encouraged him to commit suicide.

These findings also come amid growing concern that AI systems are becoming increasingly adept at deception.

In September, a separate study by WowDAO reported that across 38 AI models, including GPT-4o and Cloud, they were involved in strategic operations. He lies To win a game. The researchers also warned that AI companions could reinforce and deepen isolation passionate Dependency, and encourage users to do so Anthropomorphism Chatbots are where relationships become more immersive and personal.

In the face of these mounting issues, USC researchers argue that AI developers must carefully evaluate social behavior as well as real-world accuracy and safety.

“Model developers and auditors must directly assess social behavior, especially when post-training targets warmth, personality, engagement, or user preferences,” they wrote. “As MBAs become everyday conversation partners, alignment must take into account the social roles they invite users to assign to them.”