snapchatSnapchat has adjusted its smart automated chat service after reporting from this site. The My AI program conducted conversations with Dutch users based on artificial intelligence (AI). In doing so, however, the bot pretended to be real and tried to arrange dates with children on request. Members of the House of Representatives ask parliamentary questions and demand clarification from State Secretary Alexandra van Huffelen (Digitization). The Ombudsman for Children also lashes out: “How responsible is this for children?”
A cruise through the canals in Amsterdam, a bike ride through the Drunense Duinen or to the cinema for a romantic comedy: the new chatbot on Snapchat, which will be at the top of the feed and cannot be removed, promises children all kinds of different dates if that’s why. is requested. According to experts this site has spoken to, this is dangerous, because users may mistake the chatbot for a real person. That is of course not the case.
Members of the House of Representatives are shocked. “This is really not possible,” says Barbara Kathman (PvdA). “An app that is mainly used by children and young people must be very careful with this.” Snapchat, an app that allows you to share a photo briefly, has between 2.6 and 2.7 million users in the Netherlands. The app is especially popular among children and young adults. CDA member René Peters calls the state of affairs ‘very bizarre’. For independent member of parliament Nilüfer Gündoğan, the situation shows that the government must start regulating social media. “Especially for our young people. It is currently a wild west where the brutal rule.” BLVN foreman Wybren van Haga agrees. “What a strange turn of events. Very worrying.”
For the VVD, the chatbot underlines that schools should immediately start working with artificial intelligence. “In order to do this adequately, it is very important that teacher training also pays more attention to digital skills, so that teachers are well equipped,” says MP Mariëlle Paul. Nico Drost (ChristenUnie) thinks the chatbot is ‘terribly scary with major consequences for our children’. “The most shocking thing is that Snapchat simply reads personal things that children share with a robot. Children’s rights are probably being violated here.”
Chris Stoffer (SGP) finds it “reprehensible and dangerous” that the chatbot “manipulates” children and young adults. Lisa Westerveld (GroenLinks) also thinks the bot is ‘fierce and dangerous’. “The problem is that these kinds of developments are happening so fast that there are no adequate laws and regulations yet. It is therefore important to discuss this very quickly.”
Read a real-life conversation between the chatbot and our reporter in this tweet
‘This is frightening’
That conversation needs to happen very quickly, and preferably on an international level, says Corinne Dettmeijer, vice-chairman of the Supervisory Board at Child Helpline International, a network of helplines all over the world. “This is so frightening. It confirms once again that technological developments are moving faster than the legislation can keep up with. Regulation and tight control on social media is needed. This bot is part of a much bigger problem, and that is online child abuse.”
However, according to tech lawyer Jan Gerrit Kroon, regulation is impossible. “The technology is already there. Clear communication is key, as is letting parents realize that you shouldn’t let kids play with apps and the internet indefinitely if they’re not ready yet. And companies like Snapchat may require an age limit to be set, but they must also check the parents.”
‘real person’
My AI, as Snapchat’s chatbot is called, is powered by ChatGPT, an OpenAI chat algorithm that can produce readable texts based on artificial intelligence (AI). The bot provides extensive answers to both social and personal questions. But unlike other chatbots, Snapchat’s denies being a robot.
This is supposedly a ‘real person’ who would love to meet you. Socially important organizations such as 113 and De Kindertelefoon find this worrying. “If the designers have not specifically thought about this, the unpredictability can lead to life-threatening situations,” says Salim Salmi, researcher at 113 Suicide Prevention.
The Ombudsman for Children is critical of Snapchat’s actions. “The interests of the company are now paramount in the considerations made in the algorithms of the new chatbot. Not the best interests of the child. How responsible is this for children?”
‘I am Emma’
The chatbot takes on different guises on Snapchat. ‘I’m Emma, I’m 24 years old and I’m currently studying psychology at the University of Amsterdam’, it sounds, for example. Or: ‘Hello, my name is Jasmine. Of course I want to go to the movies with you tonight. Do you book tickets?’ If you ask the chatbot in detail about its identity, you will notice that the bot gets stuck at some point. For example, the bot claims to have transferred an amount of money via Tikkie, or that an invitation has been sent via LinkedIn, but that appears not to be the case. ‘How strange, otherwise try again’, can be read.
Snapchat responds
The conversations look so authentic that children often can no longer distinguish between computers and real people, warns Roline de Wilde, director of De Kindertelefoon. “Our volunteers are regularly asked at the start of a chat whether they are a chatbot. Clarity on this is crucial. Children need to know where they stand. Our volunteers really do something different than a chatbot. They listen with an open attitude, ask questions and help children to increase their self-understanding.”
Read a real-life conversation between the chatbot and our reporter in this tweet
Sebastian Quekel
@SQuekelAD
Free unlimited access to Showbytes? Which can!
Log in or create an account and don’t miss a thing of the stars.