Anthony Fleming chose a strategic spot to unfold his camping table. He stands along the sidewalk in the middle of the National Mall in Washington, the vast lawn between Congress and the White House. His mission: collect signatures to stop the unregulated rise of artificial intelligence. His weapons: brochures, buttons and a good conversation.
Fleming is the regional coordinator of Pause.AI. Together with two other young volunteers, he collected sixty expressions of support on a Saturday afternoon in May. Not bad, they think.
The Pause.AI movement started in the Netherlands but also has resonance in the US. In the country where ChatGPT was born, nothing seems to be slowing down the AI race anymore; tech companies are pouring hundreds of billions of dollars into new data centers and their AI models are eagerly embraced by the Pentagon.
“Ordinary Americans would rather see business slow down,” says Fleming. “There is a lot of local protest against the large data centers, and a growing fear that AI will takes over work or your children can drive suicide.”
The AI sector got a boost from the second Trump administration wide track and all the rules that the Biden administration drew up to keep AI safe, went into the trash. But Fleming feels change is in the air. He takes heart from the clash between the Pentagon and Anthropic. This leading AI developer refused to supply products that support mass surveillance or enable the deployment of lethal autonomous weapons. The Pentagon was planning to ban Anthropic: soldiers do not want to let a supplier dictate how they use their weapons.
But then Anthropic came up with Mythos, an AI model that quickly detects vulnerabilities in software, information that can be used to unleash cyber attacks. Mythos is said to be such a powerful weapon that for the time being a limited number of parties will have access to the software, including American security services.
“Mythos was a warning shot for the government: they now see that AI can also be a danger. That will set change in motion,” Fleming hopes. His hunch appears to be correct: last week, President Trump signed one decree to pre-check AI models that can serve as cyber weapons via a ‘clearinghouse’. This way, institutions that manage critical infrastructure can be the first to repair their software. The AI companies must cooperate ‘voluntarily’. Even if cautiously, it is the first time that Trump has tightened the reins.
AI aroma
For a long time, American tech giants stayed away from military applications, for fear of reputational damage or angry employees. That embarrassment is gone; tech companies are eager to sign defense contracts. Out of patriotism, but also to earn back their expensive AI data centers more quickly.
In May, the so-called AI+ Expo was held in Washington, where the stands of Amazon, Google, Meta, Microsoft and OpenAI stood side by side with those of arms manufacturers and US ministries. The leaders from the tech sector, politicians and military personnel meet each other in panels, in the corridors and in the long queue for a robot that serves coffee. Much slower than a human barista, but with AI flavor.
Katie Sutton, the deputy assistant secretary for cyber policy at the Department of War, cleverly sidesteps the Mythos discussion in an interview on the AI+ stage. “There is no point in focusing all the attention on one model. In six months there will be another product that is much better.”
Sutton was a cybersecurity specialist in the past. “The attacker always has the advantage,” she knows. This includes US government agencies and critical infrastructure, which proved extremely vulnerable to hacking campaigns such as Salt Typhoon (2023) and Volt Typhoon (2024). According to the US, the Chinese government was behind this.
The cyber war is escalating, Sutton sees: “Most cyber attacks so far have focused on espionage and data theft, but now hackers are embedding themselves in systems as a strategic preparation for a possible conflict.” America is also going with cyber weapons deploy offensivelyin combination with traditional weapons. Sutton: “We have now proven that this approach works with our actions in Venezuela and Iran, Operations Absolute Resolve and Epic Fury.”
Collapsed house
If you use a cyber attack to shut down power grids, water supplies, traffic lights or mobile networks, it will hit civilians just as hard as military opponents, Jonathan Horowitz warns. He is a legal expert at the International Committee of the Red Cross (ICRC). Their stand at the AI+ Expo stands out: a house that appears to have been hit by a rocket attack.
The ICRC tries to protect civilians in armed conflicts and therefore closely monitors new technologies that appear on the battlefield. That is why Horowitz talks a lot to defense specialists at the AI conference. “No soldier wants a weapon system that is unreliable, unpredictable and inaccurate. That is a waste of money and a tactical risk,” he says.
A cyber attack can cause death, injury or physical damage, which is why digital weapons must comply with the same rules as traditional weapons, according to Horowitz. “Just as you are not allowed to throw a bomb on a hospital, you are also not allowed to shut down that hospital with a cyber attack. That does not matter for international humanitarian law.”
Just as you are not allowed to throw a bomb on a hospital, you are also not allowed to shut down that hospital with a cyber attack
A hacker can work very accurately, but as soon as so-called ‘AI agents’ start blocking systems on their own, it quickly disrupts the entire society.
According to Horowitz, all hardware, software, networks and services from the commercial tech sector end up in the middle of the battle: “Payment systems, banking details and other data of citizens are also targeted.” That is one of the reasons that the Dutch cabinet blocked the sale of Digi-D manager Solvinity to an American party.
Maven’s misses
The Pentagon’s promise is that AI will allow you to strike enemy targets faster and artificial intelligence will help prevent human mistakes on the battlefield. But everything depends on using the right data. This became clear again at the end of February, when an American missile hit an Iranian girls’ school killed 156 people. That building was once part of a military complex, but that information apparently had not been updated.
Emil Michael does not want to comment on this dramatic mistake. Michael is Assistant Secretary at the US Department of War, responsible for introducing new technology to the Pentagon. He thinks that the use of AI reduces the chance of errors: “The more data you collect and combine with other real-time information, the less unintended collateral damage you cause.”
The American armed forces use the Maven Smart System to find and eliminate enemy targets as quickly as possible. The human assessment of such an attack is kept to a minimum to save time. The algorithms are trained with videos from drones and satellites; several commercial tech companies supply the software for image recognition and data company Palantir combines the information flows.
In her book ‘Project MavenBloomberg journalist Katrina Manson describes up close how a group of five US Marines set up this large-scale AI project in the Pentagon. Manson is at the AI+ Expo to explain her book: “The Project Maven team initially had great difficulty recruiting tech companies. They would have liked to collaborate with the ‘A-team’ of Google Deepmind, but they refused. They were forced to work with a company that specialized in image recognition for wedding photography.”
According to Manson, ‘Maven’ was intended from the start to identify enemy targets. One of the conclusions of her book is that algorithms invariably fail in new situations because there is too little data available. “The models trained for the desert do not work in the jungle or in the snow.”
Algorithms often cannot distinguish between men, women or children. Manson: “It is also difficult to indicate, based on three pixels, whether someone is carrying a weapon over their shoulder or an innocent backpack. The human assessors I spoke to believe they are better at this than AI. People understand more about the context.”
The Pentagon’s claims that AI would improve accuracy or precision are not supported, according to Manson. She cites the example of an action in Kabul in which seven children were killed. That was a human error of judgement, it turned out. “Maven was not used for that specific attack. But when the developers showed the same drone images to Maven, the system did not see the children either, even though the AI software was on its most sensitive setting.”