It seemed too good to be true: a breathalyzer that could determine within fifteen minutes that the blower did not have corona. In 70 percent of the participants, the SpiroNose breath test by the Leiden company Breathomix could immediately rule out a corona infection. From now on, only the remaining 30 percent would have to perform the expensive and time-consuming PCR test. Ideal for the test streets of the GGD, at events and in the office. No more hassle with nasal swabs, no more waiting a day for the results and suddenly a much larger test capacity.
The SpiroNose was “a wonderful Dutch innovation, which the whole world is jealous of,” said the then Minister of Health Hugo de Jonge (CDA) in the House of Representatives in February 2021. Shortly before that, he had performed the test himself, under great media interest, at the GGD test street in Amsterdam-Noord.
After the green light from the Outbreak Management Team (OMT), the ministry ordered 1,300 SpiroNoses from Breathomix in January, for an amount of 26 million euros. In October 2020, VWS had already ordered the first 500 devices. The aim was to deliver all those devices to GGDs across the country in the near future, as a promising addition to existing PCR testing capacity.
But the SpiroNose indeed turned out to be too good to be true. Pilots at the GGD Amsterdam and at the Eurovision Song Contest in Rotterdam showed that the reliability of the test left something to be desired. Shortly after the warm words of Minister De Jonge, several people in the test street in Amsterdam-North had wrongly received a false negative result; one person ended up in hospital.
An expert team assembled by the ministry after this incident concluded in May that the test was not reliable enough, after which all pilots were stopped. Last September, VWS decided to terminate the contract with Breathomix and no longer pay the outstanding bills. At that time, a forensic investigation into Breathomix was already underway, so reported the FD earlier this week. This would have shown that Breathomix adjusted research results in its favor, so that it appeared that the test worked better.
According to the Leiden company, VWS broke off the collaboration on improper grounds. “The test certainly works, although it has a user manual just like any other test,” says Breathomix founder Rianne de Vries. “It was included in the contract, but the GGD has not adhered to that.”
Before the judge
Breathomix now demands that the ministry still pays a bill of 24.7 million euros. The matter will be brought to court for the first time on Friday in The Hague.
A crucial question in the conflict is: did the ministry know there was a risk that the breath test would not work properly? And did the VWS study sufficiently in the small company that received an order worth tens of millions of euros?
At the start of the corona crisis, Breathomix, founded in 2018, was a start-up with four employees and equity of 320,000 euros, according to data from the Chamber of Commerce. The founders had been working for almost two years on a breath test that should be able to detect lung cancer and COPD, but the company did not have a working product.
This is not surprising, by the way: science has been working for decades on a breath test to detect diseases at an early stage. “In thirty years and after hundreds of studies, no test has yet been developed worldwide that has received approval from the health authorities,” says lung specialist Lieuwe Bos of the Amsterdam UMC, who obtained her doctorate on the subject.
pilots
Breathomix was approached by RIVM in March 2020, says founder De Vries: “The RIVM questioned whether a corona infection could be determined with the help of a breath test. After careful consideration, since we were a small start-up after all, we decided to start a first scientific study into this.” The Franciscus Gasthuis in Rotterdam and the LUMC are also involved in this.
A second company, eNose Company from Zutphen, is also participating in the trial. “I thought it was a great idea,” says Maastricht surgeon and professor Nicole Bouvy, who has been working at eNose for some time on a test for the detection of thyroid cancer. “Certainly for the detection of Covid, there were few alternatives. Given the limited testing capacity and the desire to unlock the country again, we started to look into it.”
After an initial pilot, the eNose team concludes that it is not realistic to develop a breath test for detecting corona in the short term. In a very stable environment, the test might work, but in practice the sensors in the device turn out to be far too sensitive. Temperature fluctuations, exhaust fumes, what someone has eaten and the presence of alcohol in the breath influence the result in such a way that it is not reliable.
Bouvy: “We came to the conclusion that it would probably go wrong in the validation phase and we stopped the study.”
In June 2020 Bouvy will also let you know in meetings with VWS. “We then said: our test is not stable enough,” says Bouvy. “After that, I looked with frowns at the speed and drive with which Breathomix managed to get it done.”
In the same period, RIVM received an anonymous report about the reliability of Breathomix, after the company posted a message about the pilot on LinkedIn. An employee of the RIVM is advised to be careful, because the technology used by Breathomix would be uncontrollable for outsiders – so there would be a risk of manipulation of research data. A spokesperson for the RIVM says that he is not aware of this report. According to him, the initiative for the pilot did not lie with RIVM, but with VWS.
Results not public
Once de Volkskrant in October 2020 reports that the Ministry of Health, Welfare and Sport wants to buy 500 breath test devices from Breathomix, three pulmonologists in training from the Amsterdam UMC write a critical opinion piece in the same newspaper. They also point out that the test results in practice may differ from those in a controlled setting. But their main objection is: the test results from the tests that Breathomix has carried out are not public. No one has been able to verify the design except the researchers themselves, and the results of the study have not yet been published anywhere.
“The accuracy they claimed, excluding 70 percent non-infected people, had never been practiced in any other test,” says Lieuwe Bos, one of the letter writers, now. “Yet it went straight from a relatively small study to clinical application. That was not justified. It seems that they were hounded by the ministry: everything had to be faster, faster, faster.” He emphasizes that Breathomix could also have stepped on the brakes itself.
The company MolGen, from Veenendaal, is unknown, but one of the success stories of the corona crisis is due to its test technology. Also read: From three men in an office active worldwide in no time, due to corona
On January 11, 2021, the OMT gave the green light to use the SpiroNose. A negative test result with the breath test would correspond in 98 percent of the cases with the result of the PCR test: a ‘good sensitivity’ to use the test as ‘first screening method’, according to the OMT advice† Shortly afterwards, the ministry ordered the next 1,300 devices. The first preprint study was only published on February 16, the results of which have still not been validated by other scientists.
Rianne de Vries denies that the development went too fast: “Breathomix has always carefully thought out every step and implemented it in consultation with GGD Amsterdam and VWS. Careful milestones have been defined for both the research and the development process. After each study and development phase, there was a joint analysis.” According to her, both internal and external experts were involved.
She attributes the fact that things went wrong with the SpiroNose to a “break of trust” between her company and the GGD Amsterdam. After a conflict in February 2021, the GGD would have refused to roll out the test further and also “deliberately spread incorrect information” about Breathomix. Mariken van der Lubben of GGD Amsterdam does not want to respond substantively: “The case is now before the court. I am very sorry that it has come to this. We have done our utmost to make this a success.”
A version of this article also appeared in NRC on the morning of February 11, 2022