To an algorithm, every slum looks different

Africa is urbanizing rapidly. In 2018, 43 percent of the population lived in cities, in 2050 this is according to expectations of the UN almost 60 percent. More than half of the urban population in sub-Saharan Africa lives in slums. Often in wooden buildings, close together and in risky locations: vulnerable to natural disasters such as floods.

“Urban planning is needed to improve the way of life in slums. For that you need information about what is in the area. Suppose you work for a municipality, want to improve an area and have a lot of money for that. What should you do with that pot of money? Sometimes you don’t even know how to walk through a slum,” says Caroline Gevaert. She is a geo-informatician and assistant professor at the University of Twente. In December 2021 is she chosen as a new member of The Young Academy of the KNAW, her membership will start from 22 March. “To make a good development plan for an area, you need to know exactly what is there: how many houses are there, where are the roads, and where are the facilities.” To answer these questions, Gevaert maps slums by taking drone images and processing them automatically with AI algorithms.

Are these kinds of maps of slums not yet available?

“It varies from area to area. In 2015 I started fieldwork in Rwanda. They had an aerial photo of that area from a few years earlier, but it was now outdated. And because the level of detail is not as high there as with a drone, you can see certain things less well. You get a bit of an idea from those maps, but often not detailed enough. Drone images are more detailed and flexible, you can fly wherever you want. Also under the clouds, which sometimes complicate satellite images.

Open Buildings of Google has mapped 516 million buildings in Africa. But that has limitations. “Open Buildings are better for getting a first look at an area, while drones can provide much more detailed information.”

How do you process drone images into a useful map?

“When you fly a drone, it always takes separate photos, which you can combine into one orthomosaic” [2D plattegrond in een bovenaanzicht]. But you can also use the images to create a point cloud, which is a 3D model. Such a 3D image can be very useful for flood models, for example, because you can also see how the height varies in the neighbourhood.

“To go from the orthomosaic or the 3D image to a map, you can manually draw objects that you recognize. For example, houses, vegetation or roads. But I focus on automating this with algorithms. The outcome is similar.

“We first show those algorithms a few examples: this is what I consider a house, this is also a house, and this is also a house. You can do the same with roads or with vegetation. From these examples, the model learns to recognize patterns and can find them throughout the image. Then you get a map of the entire area.”

You also see that the roofs can look very different per house

Gevaert shows the different steps on her computer. The drone image shows a slum in Rwanda in great detail, a pixel is 4 by 4 centimeters. Gevaert points out stairs that run through the neighbourhood, laundry hanging to dry in the garden, and sandbags placed against flooding problems. “You also see that the roofs can look very different per house, newer roofs have different colours. Sometimes people don’t have the money to replace their entire roof at once. Then it is partly one color and partly another. That makes it difficult for an algorithm to recognize.”

The moving 3D model she shows is almost like walking through a video game, albeit at a lower resolution. After the algorithm has interpreted the images, a clear map is displayed. Houses in red, vegetation in green and the ground in beige.

Is that algorithm universally applicable?

“We would like to, but what a slum looks like differs from city to city. For example, the architectural style differs. I also tested the algorithms I made for Rwanda on slums in Tanzania and Uruguay. It works partly, but you have to make adjustments in the model. In Uruguay, for example, the trees are a bit higher, so buildings are partly covered by trees. You then have to take into account that a drone cannot see through a tree.

“If you apply the algorithms that we use to recognize slums in Europe, they sometimes indicate the city center as a slum. This is because some of the characteristics of slums – very dense buildings and streets that intertwine instead of forming a rectangular network – also exist in a city center. That shows that the algorithm is extremely context sensitive.”

It also helps determine the flood hazard

What happens next to the cards you produce?

“They are used by the government and planners for urban planning, but also for disaster management. For example, the impact of a flood is related to which buildings are exposed, how vulnerable those buildings are, social vulnerability – a wealthy family with two working parents has more resistance than a single mother – and the risk of flooding itself. A drone map helps to estimate exposure: what is in the area that is going to flood? It also helps, together with flood models, in determining the flood hazard: which areas are more likely to flood than others? You can then plan improvements in the most risky places.

Gevaert is currently researching ethics and responsibility surrounding the use of artificial intelligence for geographic applications. She also works part-time as a consultant for the World Bank. “From my work there I often think: just leaving it at a map is not enough. You also have to be able to explain to the end user where your algorithm is certain, and where less. The user must be able to trust the information. But I still look at ethical questions from a technical point of view: what do I need to technically adjust to include those questions?”

When using algorithms, it is often pointed out that prejudices can sneak into an algorithm, allowing an artificial intelligence to make racist choices, for example. What about your field of research?

“You see that certain areas are mapped much better than others,” says Gevaert, showing an example of a Microsoft project to digitize all buildings in Tanzania and Uganda. Next to it the same floor plan, but with manual recognition of the buildings. “In this case, AI is missing a large part of the buildings in a slum. That’s an example of a bias [vooroordeel]. Such a bias is especially a problem if it is linked to socio-economic groups. Then, for example, you can better protect wealthier people in a larger house against a flood, because their house was recognized on maps. That is a hypothetical example, but something to take into account in our research.”

Residents of slums are often very vulnerable. Can the extra information thanks to your cards also be misused against that group? “In a hypothetical case, yes. Slums often arise in areas that are deliberately not built on, for example because of the risk of flooding. If those neighborhoods are then mapped and the maps come online, the government can use that argument to expropriate those buildings. That is also their right, but you have to be careful with that. As researchers, we discuss such matters with the ethics committee. The images I made in Rwanda are available, but you are not allowed to use them just like that. You must first contact us and tell us what you want to do with it. That is to prevent abuse.”