A teacher at Fontys educational institution in Eindhoven has encouraged colleagues to upload students’ personal data to the AI service ChatGPT, and may have done so themselves. This is contrary to privacy legislation. It happened via a tool that he developed himself. The teacher first denies it, but then reports that it only concerns his own data: “I am also a student myself.”
The teacher, who works in the ICT and Software Engineering course, developed a tool to quickly analyze student portfolios and make them searchable. In such portfolios, students describe their progress, areas for improvement and reflections on the learning process. This often concerns personal and sometimes sensitive information.
Testing with real portfolios
The teacher wrote enthusiastically about his tool on LinkedIn two weeks ago. He stated that it was a ‘local AI tool’ and called on colleagues to test it. They respond enthusiastically, with the exception of a few critical reactions. In comments he wrote that he is testing the tool ‘in practice with real portfolios’.
But the claim turns out to be incorrect. The AI did not run locally on his computer, but on the servers of OpenAI, the American company behind ChatGPT. On a website where he describes the development of the tool, the teacher later acknowledges this: ‘The tool now works with OpenAI. This is a privacy issue.” He says he wants to change this, but based on his previous messages it is likely that student information has been sent to OpenAI.
Not the first privacy issue
It is not the first time that a large institution has sent sensitive information to AI services such as ChatGPT. At the end of last year, it emerged that officials from the municipality of Eindhoven had uploaded data from vulnerable residents to an AI tool. Having the data removed by OpenAI turned out to be a hopeless mission.
Fontys has also previously been in the news due to a privacy problem. In April last year, hundreds of highly sensitive files, including medical data, were easily accessible to unauthorized parties.
‘Not to be trusted’
Professor Johan Jeuring from Utrecht University heads the National Education Lab AI, which develops and researches AI applications for education. “This doesn’t sound good,” is his first reaction to the teacher’s tool. “Using OpenAI for this application is simply not allowed by law, the GDPR prohibits this.”
“Sometimes I find the GDPR overly strict, but not in this case,” says Jeuring. Portfolios are by definition personal documents, and no one knows what OpenAI does with that information, or what it might do in the future. “You simply cannot trust OpenAI.”
He assumes good intentions on the part of the teacher, but emphasizes: “You are allowed to experiment, but always in accordance with the ethical conditions and the law. A tool like this should then run locally and not work with OpenAI.”
Teacher response
The teacher informs Omroep Brabant that his tool works ‘completely locally’. After further questioning, he says that OpenAI is no longer used as standard, although users can still check that option. “The issue has been resolved,” he says. An update to the tool will only appear a few hours later.
He also initially denied that student data had been uploaded, despite his earlier LinkedIn message. Later he says it was about his own portfolios. “I’m a student myself.”
Fontys response
Fontys says that experimenting with AI within the institution is normal, but that uploading student data to OpenAI is not permitted.
Fontys is convinced that no real student data was uploaded: only a ‘test portfolio, without personal data of real students’ was used. How do they know for sure? According to the school, the teacher stated this as such. Control is not possible. “We believe him,” the institution said.
It is striking that a technical employee of Fontys subsequently reports that the danger of OpenAI is often exaggerated. “That is a big misunderstanding in society. OpenAI itself says that they only store files for a short time and then delete them again.” He indicates that he has no reason to doubt this.
