In recent years, the podcast has become an important medium for background information on the news. But for the approximately 1.5 million Dutch people who are deaf or hard of hearing, this genre is unattainable. That has to change, thinks concept developer Geert-Jan Strengholt, who heads VPRO Medialab together with editor-in-chief Annelies Termeer. At 61, he has been the engine of digital innovation at the VPRO for 17 years.
Medialab, together with innovative companies and start-ups, explores what new technologies can mean for public broadcasting. Initially we worked from an old Philips factory in Eindhoven, now we meet in the colorful maze of Villa VPRO in Hilversum, where the team has been based since 2019. The latest project is called Doofpod, a web app that unlocks the existing podcasts of the NPO in a completely new form for the deaf and hard of hearing. A guide to podcast makers will then be published so that they know how to make their productions suitable for playback in the Doofpod app. But we are not there yet.
:strip_icc()/s3/static.nrc.nl/bvhw/files/2022/10/doofpod.gif)
The podcast Bob from VPRO Medialab in a prototype of the Doofpod app.
“We started this project with the question of what the podcast of the future should look like,” says app designer Delyan Pragov in English. The hip, bespectacled, late thirties is originally Bulgarian and for a few years now, in addition to his work for Medialab, has also been responsible for the design of vpro.nl.
Geert-Jan Strengholt adds: “Our designer Marie van Driessen is deaf herself and was disappointed that media is not always accessible to the deaf and hard of hearing. Then we started interviewing the deaf and hard of hearing. Some people turned out not to know what a podcast is at all, and others said, “I’ve never been able to listen to a podcast, so I don’t know what I’m missing.”
“We soon discovered that this community is not uniform and that there are different wishes. People who became deaf suddenly and late still have Dutch as their mother tongue, so they can follow subtitles well. But people who are deaf from birth have sign language as their mother tongue. There are also huge gradations between people who are hard of hearing. For example, people who have a certain amount of residual hearing can listen to a regular podcast if they get some help with that, for example by subtitling. While we have to come up with a completely new product for people who are deaf that does not involve any sound at all.”
In recent months, Medialab has been working on a prototype of Doofpod. An episode from the popular podcast series Bobabout the past of the demented Elisa, can be played in a podcast player via a laptop or smartphone in this prototype. “In the beginning, we opted for the simplest solution by adding a visual transcription to the audio of the podcast, so that people could read or read the spoken word,” says Strengholt. „No thickened subtitles, in which the richness of the language is often lost, but a beautiful ‘translation’. But as soon as we looked more closely at that, we realized that reading and listening require two different speeds. That’s why at a certain point we left out the sound, the podcast itself. But in testing, it turned out that the hearing impaired still missed the audio. Now we are going to look for a solution where we can connect those tracks in a new way so that it becomes usable.”
The current prototype of Doofpod consists of white and yellow text on a black background. Why the choice for those colors?
Pragov: “Accessibility also means readability, so you have to have enough contrast between the foreground and the background. We decided to use different colors and fonts for the voice-over than for the dialogue. But it’s complicated, because how you experience color is culturally determined. The color black is associated with funerals in Western Europe, but in China the color white is used for that. It might be a good idea for creators or users to decide for themselves which background color to choose.”
In addition to transcription of the spoken text, background noise or music is described in brackets in the app. In the prototype I read sentences such as: ‘Music continues to summer sultry’ and ‘Obvious tones push dark cloud into view.’
You can also translate sound coming in through a window as a warm breeze
Geert-Jan Strengholt VPRO Medialab
Strengholt: “Descriptions of sound for the deaf and hard of hearing are often meaningless or kill the atmosphere. Then it says, for example: ‘people are talking in the cafe.’ We have worked with a writing interpreter. We noticed that audio terms are often used that mean nothing to people who have been deaf for a lifetime. Because do you know what ‘sad music’ is, if you’ve never heard music? We tried to choose terms that refer to color, taste or touch. Because if you’ve never heard anything creak, you don’t know what that sounds like, but you do know what a rough surface is, because you can feel it. If you want to tell that sound comes in through a window on a warm summer evening, you can also translate that as a warm breeze. Instead of referring to sounds that we as hearing people know, we believe that poetry can be much more powerful. Because that stimulates the imagination.”
What is the most important insight so far?
Strengholt: “That almost everyone thought the episode was too long.”
Pragov: “With a podcast like this, of forty minutes, the transcription becomes longer than the spoken text anyway. It is too tiring to read focused on a screen for so long.”
Strengholt: “With a traditional podcast you can sit down and muse, but you are actively involved. In addition, user feedback on a prototype showed that most simply turn on the app at home on their computer, and not casually on their mobile phone. We had blindly assumed the latter.
“Fifteen minutes seems like the maximum length. If you know that, you can already take this into account with a new podcast production. We like to be inspired by someone like Niels ‘t Hooft, who developed the Immer app, in which he reviews books. He comes from the corner of literary games and has researched how you can make the reading experience on a mobile phone pleasant and less tiring.”
VPRO Medialab is not the only organization experimenting with new forms of podcasts, Strengholt shows on his laptop. How the BBC develops adaptive podcasts, where people listening to a football game can choose in an app to make the commentator’s voice louder than the stadium sound. Vox Media made the podcast More Than This a kind of longread that the user can scroll through himself. “They make graphic jokes with deletions and corrections and you can customize the backgrounds as a user.”
Strengholt clicks the program Hands Up! that KRO-NCRV recently developed for deaf children. “They call it Vodcast – a podcast with visual support. They have the spoken text converted into images by artificial intelligence. On the right is a sign language interpreter in the picture to translate everything.”
Why not choose this?
„With this you are handing it over to artificial intelligence, and I don’t think the makers of Bob are waiting for an uncontrolled visual representation of their story. KRO-NCRV discovered that it was incredibly laborious. With AI you quickly think: it does everything for me, but in practice you have to prepare and adjust a lot.”
You have now tested Doofpod with a panel of three hundred people. What assumptions did you have to say goodbye to?
Pragov: “That colors can enhance the story. We thought: this passage is very sensitive, maybe we should also add soft shapes or colors in a soft voice, so that the viewer can follow the emotion. We experimented with all kinds of lines, shapes and fonts, but it didn’t work. We thought it would be interesting to let the sound waves move along behind the text. But that also caused confusion. We just had to make things more subtle to get people to follow the story.”
Strengholt: “Not all senses need to be continuously stimulated.”
The main character of Bob is an elderly lady. Have you been tempted to show her photo when she is speaking?
Pragov: “No, because with a photo we give the viewer a very concrete image. We prefer that viewers form an image of that woman in their own heads. If you use photos it just becomes a video again. We are trying to develop a new design language.”
VPRO Medialab hopes to be able to launch the Doofpod app plus the manual for podcast makers in the spring of 2023. The plan is to design the app in such a way that it can be used internationally.