In recent years, more and more people have been in the news who have taken their own lives after discussing their inner feelings with an online chatbot. That chatbot sometimes actively encouraged them to commit suicide, and in any case could not warn family or care providers. The English Wikipedia entry ‘deaths linked to chatbots‘ already contains more than ten such messages.
“The story that touched me most,” says computer scientist Erkan Basar, “was the Belgian man who committed suicide in early 2023 after conversations with a chatbot about climate change.” That man was in his thirties, married, with two young children. “A human conversation partner would have realized: he is now becoming very pessimistic, maybe I should tone it down a bit. An AI chatbot keeps bombarding someone with factual information.”
Basar received his PhD on November 25 from Radboud University in Nijmegen for research into building chatbots that can be used safely and effectively in healthcare. He focused on chatbots that help with smoking cessation and sexual health, but the method can be used in all kinds of areas.
The deceased Belgian man had chatted with a modern AI chatbot named after one of the very first chatbots: Eliza, built about sixty years ago by computer scientist Joseph Weizenbaum (1923-2008). That old, AI-free Eliza had a psychiatrist-like module that mainly reflected key concepts from what people themselves typed back to them in question form; that’s how the chatbot was programmed. Therapeutic chatbots after Eliza often respond to what someone says with texts written in advance by healthcare professionals. Although these texts are correct and clear, they make the healthcare bots sound rather stiff in a conversation.
Very different from today’s online AI chatbots, the large language models (LLMs), which calculate for themselves what is statistically the most likely next word in a conversation. They chat much better, but have often blindly adopted prejudices that prevail in society, and what they say is sometimes complete nonsense.
Pre-programmed chatbots provide more control. You know exactly what the chatbot will say and when
You wanted to create a healthcare chatbot without stiffness, without bias and without nonsense?
“Yes, but I prefer to formulate it the other way around. We wanted to combine the benefits of pre-programmed chatbots with the benefits of AI.”
What are those benefits?
“Pre-programmed chatbots provide more control. You know exactly what the chatbot is going to say and when. Human conversation is not so neatly structured, it flows more. But it is impossible to program all possible ways in which a conversation proceeds. And if a topic comes back, a pre-programmed chatbot always gives exactly the same answer. That becomes repetitive and boring and we know from research that this reduces involvement, and therefore the effect of the treatment.
“The advantage of LLMs is that they generate text during the conversation. They are not repetitive. And often more specific than the pre-programmed sentences of healthcare professionals. When a smoker says: ‘I don’t like the fact that my fingers are turning yellow’, a pre-programmed healthcare bot often says something general like: ‘of course you are concerned about your appearance’. We saw that LLMs become more specific: ‘yellow fingers, yes, that often happens when you smoke’, something like that. That specificity makes the conversation more fascinating. However, you cannot guarantee that there is no risk in talking to LLMs. They just put words together without understanding.”
How can you combine those two different models?
“You still have to pre-program the skeleton of the dialogue into the chatbot. But we used LLMs to generate a variety of possible statements: responses to what people who wanted to quit smoking or who wanted to learn more about sexual health had actually said. Then we had those responses, generated by different LLMs, rated by other people on how appropriate, specific, natural, and engaging they were. We saw, for example, that GPT-4 statements outperformed human statements on average. But some statements were less appropriate.
“So we recommend enriching the chatbot’s database with statements generated by LLMs, but first checking all statements manually. In our research, of the 380 generated statements, we removed 38 completely, modified 101 and used 241.”
If you ever feel the urge to smoke in the middle of the night, you can talk about it with a chatbot
So you shouldn’t let patients talk directly to LLMs, but use LLMs to increase your own database of possible answers?
“Yes. I would not recommend using LLMs directly as therapists. I am in favor of developing chatbots per domain, such as a ‘quit smoking’ chatbot. Such chatbots can serve as support. For example, you regularly talk to a human, but if you ever get the urge to smoke in the middle of the night, you can talk about it with a chatbot.”
What do you think about people pouring their hearts out at ChatGPT, for example?
“I’ve met people who said they talk to online chatbots about things they wouldn’t discuss with their best friends, and as a researcher in this field I immediately felt the urge to say: please don’t do that. Firstly, you won’t get the best advice and secondly, there are privacy issues.” He himself only uses such chatbots for non-sensitive matters, he says. For example, to practice his Dutch (his mother tongue is Turkish), and to improve his English and the code he programs.
Your research was about chatbots that helped with smoking cessation and sexual health. In what other areas would this method be suitable?
“In all kinds of areas. You can also create better customer service chatbots this way. Two years ago, a chatbot from Chevrolet suggested that the customer should buy a Tesla. That is of course not as sensitive as healthcare, but I can imagine that a company would not want something like that. I see it all as a matter of balance: how flexible can you be and how much loss of control can you afford?”
NEW: Give this item as a gift
As an NRC subscriber you can subscribe every month 10 items give as a gift to someone without an NRC subscription. The recipient can read the article directly, without a paywall.
The journalistic principles of NRC