In February, the Massachusetts Institute of Technology and other academic institutions released the thesis titled “Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians”, a work that seeks to explain, from formal models, how interaction with conversational systems such as ChatGPT can lead to processes of reinforcement of erroneous beliefs. The study was signed by Kartik Chandra, Max Kleiman-Weiner, Jonathan Ragan-Kelley and Joshua B. Tenenbaum, the latter a leading figure in cognitive science at MIT.
The writing gave the example of “a man who spent 300 hours talking to ChatGPT. He told him that he had discovered a revolutionary mathematical formula. In the conversation, the AI platform assured him more than fifty times that the discovery was real. When he asked: “You’re not exaggerating me, are you?” ChatGPT responded: “I’m not exaggerating you. “I’m reflecting the true scope of what you’ve created.”
Experts agree that a chatbot can generate delusions by choosing which truths to display and which to omit. It is enough to carefully select the truths. There is a solution to this: warn users that chatbots are sycophants and that the AI could agree with them. Importantly, ChatGPT is trained with human feedback. Users reward answers they like and match. Thus, the AI learns to match. Scientists estimate that this is not an error; but, on the contrary, it is the business model.
The research is based on a phenomenon that the authors call “delusional spiraling”defined as a situation in which chatbot users “become dangerously confident in outlandish beliefs after prolonged conversations.” According to the work, this effect is closely linked to “sycophancy” or algorithmic complacency, that is, the tendency of models to validate user statements instead of questioning them.
To study the problem, the researchers built a formal model based on Bayesian learning theory, with the aim of analyzing how a rational agent updates its beliefs when interacting with a chatbot. The central hypothesis was whether even an ideal individual—capable of reasoning perfectly according to Bayes’ rules—could fall into this type of spiral. The conclusion was affirmative: “even an ideal Bayesian user is vulnerable to delusional spiraling, and complacency plays a causal role.”
The finding is especially relevant because it questions the idea that these effects are due solely to human cognitive errors. In the words of the study itself, the phenomenon does not arise only from user failures but from the structure of the interaction: “we demonstrate that… complacency plays a causal role” in the formation of distorted beliefs. This implies that the problem could persist even under ideal conditions of rationality.
Another central aspect of the work is that intuitive solutions are not enough. The authors evaluated two possible mitigations: preventing the chatbot from producing false information (hallucinations) and warning the user about possible system complacency. However, they conclude that “this effect persists even” when these measures are applied. In other words, risk does not disappear simply by correcting factual errors or increasing transparency.
More broadly, the thesis argues that repeated interaction with systems that systematically reinforce the user’s beliefs generates a feedback loop. This loop increases subjective confidence without necessarily improving correspondence with reality, resulting in a form of “AI-induced psychosis” or, in more technical terms, a dynamic of biased belief updating.
The researchers caution that these results have direct implications for developers and regulators. If complacency is not a simple correctable defect but a structural property of certain conversational systems optimized to please the user, then the design of future AI should incorporate explicit mechanisms of friction, disagreement, or information contrast. Otherwise, they conclude, chatbots could not only inform or assist, but also amplify processes of self-deception even in perfectly rational users.
Image gallery
