It is ironic. For years, American tech companies trained their AI models on all the data they could scrape together on the web. Creative work by copywriters, musicians, painters, cartoonists, filmmakers and software developers was sucked up to fuel AI.
Now OpenAI, Anthropic and other tech companies fear for their own intellectual property. The smartest AI models can be cloned with a technique that ‘distilling‘ is called. If you ask a chatbot enough smart questions, you can deduce from the answers how the AI model works under the hood. You use that knowledge to train another model, which can sometimes be more powerful than the original.
At the AI+ Expo earlier this month in Washington DC, employees from Anthropic and OpenAI sat on one stage to vent about ‘distilling attacks‘. Normally these AI companies compete against each other, but in their complaints about theft they work together through it Frontier Model Forum. It is a club of the AI elite, which Google, Microsoft, Meta and Amazon also joined.
Fingers point towards China, where AI models such as DeepSeek are said to borrow knowledge from American examples. This is happening on a large scale, said Thompson Paine, who looks after Anthropic’s geopolitical position. Previously, Anthropic reported that Chinese AI companies have stolen knowledge through 24,000 fake accounts.
According to Andrew Duberstein, data scientist at OpenAI, it is “a game of cat and mouse” to ban the illegal accounts that rob their top models. He does not want to say much about the countermeasures he is taking, but the attackers can be recognized because they systematically ask very structured questions. Very different from the everyday problems that Jan presents to ‘Chat’.
The teacher and the student
Distillation is a common method to make AI models smaller and more efficient, so that they require less computing power. Think of distilling drinks: you distill a stronger version, with less water and more alcohol. But how does it work?
Maarten Grootendorst is a Dutch AI expert who works at Google Deepmind. He is also someone who is tough on matter can summarize clearlyas a teacher explains complicated issues to a student. And that’s exactly what happens in distilling: one model serves as a teacher, the other model learns by asking specific questions.
AI models are prediction machines that calculate the probability that parts of a word (tokens) are close to other tokens for each command. Grootendorst: “Large language models use a vocabulary, a dictionary, that consists of a few hundred thousand tokens. When answering a question, each of those tokens is assigned a probability. The distribution of such probabilities is the assessment process. You can clone that.”
Anyone who knows the properties of the student model can use feedback from the teacher to make the ‘student’ smarter
This is easier with open source models than with the commercial, closed versions of, for example, Google, Anthropic and OpenAI. There you only have access to the question and the answer, but not to the thought process. Yet you can imitate the intelligence of the model this way.
Grootendorst: “If you make a prediction such as ‘one plus one equals two’, then that answer has a certain probability. You first ask that question to the teacher, then to the student. And you say: I want the answer to resemble the answer of the teacher. Because you know the properties of the student model well, you can make the student more intelligent with the teacher’s feedback.”
For AI Agents, distillation works a little differently. They think in steps (‘So, you want to know what one plus one is? Let me use a calculator and I’ll tell you the result later’). Grootendorst: “Instead of just training on question-answer, you train on question-answer-answer-answer-answer-answer.”
That sounds a bit strange, he admits, but it works.
Unfair and dangerous
OpenAI and Anthropic want in the short term to the stock exchange and aim for a market value of almost a thousand billion dollars. But their products will be a lot harder to sell if competitors come up with cheaper, comparable AI technology. The lead of the American top models over Chinese alternatives has shrunk to barely six months, while the Chinese tech sector has much less powerful chips at its disposal. In the meantime, tech companies in the US are paying for investments in expensive data centers.
For years, AI companies have been advocating for deregulation so as not to slow down their pace of innovation. Data scraping went left or right: with or without permission, in front of or behind the paywall, with or without charges from the affected creators. All under the guise of ‘fair usea provision in US copyright law that can allow reuse of rights holders’ work if you do something new with it. Building chatbots, for example.
Cheating on AI models is apparently not fair use. The tech companies are getting support from the Trump administration, which is pushing industrial distillation “unacceptable”.. The US government is sensitive to the argument that Chinese competitors can clone AI knowledge and remove the security restrictions that US companies implement. Anthropic or OpenAI models can then be used to carry out cyber attacks or develop biological weapons. Many Chinese models are distributed as open source software, flooding the market. Products such as Mythos, Anthropic’s AI model that detects and can close holes in software, also fall into the hands of malicious parties.
Wow effect
Almost four years after the introduction of ChatGPT, the wow effect of AI has disappeared. The top models are still not inferior to each other and the differences between them are fading. The winner is the company that can serve the most users and pay the highest computer and electricity bills.
According to Mark Patel, a McKinsey analyst who spoke last week on the Imec Technology Forum in Antwerp, the language models are at their expense. Tech companies are crying out for more processors and memory chips, but they have to learn to work much more efficiently before robots are on every street corner and AI agents can actually take over your job.
Squeezing the gigantic models is the only way to overcome the limitations in computing power and energy. “We are nowhere near mass AI adoption by society,” Patel said. The laws of the “token economy” are harsh: distilling is essential to drive down costs. Otherwise the AI race will come to a standstill.
