There are many smartphone apps that can identify plants from photos. But tests now show that some are only accurate to 4 percent. As a result, foragers foraging for food may be at risk and endangered plants may be falsely labeled as weeds and eradicated.
Ecologist Julie Pecock from the University of Leeds in the UK and her colleagues evaluated six of the most popular apps: google lenses, iNaturalist, Leaf snap, Pl@ntNet, Plant snap and Seek. With each app, they tried to identify 38 plant species in their natural habitat, at four locations in Ireland. The team found that some apps scored extremely poorly, with even the best failing to reach 90 percent accuracy. They published their results in the journal PLOS ONE.
“It’s important for a lot of reasons that either the apps are accurate, or that people are aware that these apps are a guide, but by no means perfect,” says Peacock. For example, people may misidentify important native species as invasive and remove them from their garden. Conversely, people can consume potentially dangerous wild plants, thinking it is a harmless species.
READ ALSO
These seven chemistry ideas are going to change the world
Increasingly clever chemical ways of manipulating molecules bring us all kinds of progress. We highlight seven of the most promising …
But Peacock doesn’t think people shouldn’t use these apps, as long as they understand the limitations. ‘An advantage is that the apps could lead people to become more involved with plants,’ she says.
Flowers and leaves
The apps use artificial intelligence (AI) algorithms trained on large numbers of photos of plants with captions. During the training, the AI program not only learns to recognize the training photos, but also to see similarities between these photos and new photos, enabling it to identify plants.
Overall, the apps were all better at recognizing flowers than leaves, which the researchers say is because the greater variety of shapes and colors gives the AI more clues. But this was not always the case. The app iNaturalist could correctly identify only 3.6 percent of the flowers and 6.8 percent of the leaves. Plant snap correctly identified 35.7 percent of the flowers and 17.1 percent of the leaves. The highest accuracy was achieved by Pl@ntNet: This app correctly identified 88.2 percent of the flowers and 80.4 percent of the leaves.
Extensive data sets
Computer scientist Alexis Jolly from the Inria research institute in Montpellier in France, one of the researchers behind the non-profit project Pl@ntNet, says the app’s success is due to its datasets, which come from botanists, scientists and informed amateurs. In addition, algorithms try to de-bias common types and instead rank several possible candidates for each query.
“This is sometimes a thankless task because people would rather see a single result with 100 percent certainty, even if it’s not the right one, than three possible types with 33 percent each, which do represent reality with respect to the photo taken,” says he. “But it looks like our strategy is paying off.”
Mislabeled
Biologist Stephen Harris from the University of Oxford says Peacock’s concerns are valid, and that he’s also encountered problems with such apps. Instead, he relies on a good science book. The problem is that the apps rely on images uploaded to the internet, which are often mislabeled, he says.
“People tend to take pictures of the same things. So you get certain plants that are obvious that everyone wants to take a picture of. On the other hand, there are also really interesting plants that happen to be very small and don’t have attractive flowers, for example. You’re not going to get many pictures of that,” says Harris. “It’s very unlikely that people walk around in ponds, pull weeds out and take pictures of them.”
Google declined a request for an interview, and the makers of the other apps didn’t respond.