“It’s not about whether Ryan was actually right or wrong,” said co-author Cinoo Lee, a Stanford social psychologist. “That’s not really ours to say. It’s more about the pattern that’s consistent across the data. Compared to an AI that didn’t overly affirm, people who interacted with this over-affirming AI came away more convinced that they were right and less willing to repair the relationship, whether that meant apologizing, taking steps to improve things or changing their own behavior.”

A self-reinforcing pattern

All these effects held across demographics, personality types, and individual attitudes toward AI. Everyone is susceptible (yes, even you). Even when the team altered the AI to be less warm and friendly and adopt a more neutral tone, it made no difference in the results. “This suggests that sycophancy can have a self-reinforcing effect,” said co-author Pranav Khadpe, a graduate student at Carnegie Mellon University who studies human/computer interactions. In fact, it’s built into the engagement-driven metrics. Any time a user gives positive feedback on a ChatGPT message, for instance, that feedback is used to train the model to replicate that “good behavior.” User preferences are aggregated into preference datasets, which are then used to further optimize the model.

“If sycophantic messages are preferred by users, this has likely already shifted model behavior towards appeasement and less critical advice,” said Khadpe, which translates into less social friction—not necessarily a good thing, because “some things are hard because they’re supposed to be hard.” In fact, Anat Perry—a psychologist at Harvard and the Hebrew University of Jerusalem, who was not involved with the study—argues in an accompanying perspective that social friction is both desirable and crucial for our social development.

“Human well-being depends on the ability to navigate the social world, a skill acquired primarily through interactions with others,” Perry wrote. “Such social learning depends on reliable feedback: recognizing when we are mistaken, when harm has been caused, and when others’ perspectives warrant consideration…. Social life is rarely frictionless because people are not perfectly attuned to one another. Yet it is precisely through such social friction that relationships deepen and moral understanding develops.”