There has been a large influx of AI tools in recent months. Whether for image editing, online searches or even as a language program – there is now a program based on artificial intelligence (AI) for almost everything. One of them is called “HeyGen” and specializes in translation. TECHBOOK tested the tool
The “HeyGen” AI has been around for quite some time. But users only recently became aware of the tool. “HeyGen” is a program that lets people speak in another language. To be more precise, you record any video in your native language, for example, talk for around 30 seconds and then let the AI convert what is spoken into the target language. TECHBOOK editor Isa Kabakci took a closer look at the tool and was surprised by the results.
What is “HeyGen” anyway?
“HeyGen” is a start-up from the USA that was founded in 2020. The main business is explained relatively simply: You can create videos including text with AI-generated avatars and voices. This is particularly useful for content creators, influencers, explainer videos and also for people in the marketing sector. Basically you don’t need a camera, no people or actors and no recording studio.
It is also possible to create your own videos in another language. Not only is the spoken language converted, but the lips are also synchronized. What is also astonishing is that the intonation of your own voice is also adapted to the target language.
The currently available languages are Spanish, French, Hindi, Italian, English, German, Polish, Portuguese, Turkish, Mandarin, Korean, Japanese and Dutch (some languages were later integrated). However, it is important to mention that the AI can only translate the video if the source language is also supported by the tool. More about this “problem” below.
Also interesting: Tech founder: “AI offers the greatest opportunity humanity has ever had”
Suddenly we speak Japanese
The “HeyGen” AI went to X some time ago (formerly Twitter) viral. The well-known tech influencer Jon Finger uploaded a video in which he suddenly speaks German, even though he doesn’t actually speak the language. That made us curious, so we tested it too.
brother
75€ cashback
Get €75 cashback for your 4-in-1 All In Box ink multifunction device
TECHBOOK editor Isa Kabakci grew up bilingual (German and Turkish) and tested the AI in both languages. The first test about “HeyGen Labs” was initially in Turkish and was to be translated into Portuguese. But that went wrong because the Turkish language was not available at “HeyGen” at the time – Turkish was later integrated. Therefore, another video was recorded in German instead, which the tool then translated into Japanese. First the original video in German, then the AI-based version in Japanese:
Advantages and disadvantages of translation AI
There are many advantages, but also some disadvantages, with “HeyGen”. Let’s start with the positive aspects first. As already mentioned, this tool is quite suitable for content creators and media professionals. You don’t necessarily have to buy expensive equipment (camera, microphone, etc.), but you can create videos with text relatively easily using AI by letting avatars speak the content. “HeyGen” is also very practical if you want to record a video in another language, as long as it is supported by the tool. This means that foreign audiences can be reached quickly.
You don’t want to miss any more important (or strange) news on technology topics? DThen follow us on WhatsApp!
The disadvantages, however, are manifold. Among other things, AI can be used to easily create deepfakes and thus specifically spread false news. Especially when you put false statements into the mouth of a well-known person. The issue of pricing could also potentially deter users. Although you can have such an AI-based video translated for free, if in doubt you have to wait several days until it is finished. If you get a subscription, which starts at around 22 euros per month, you can edit the videos directly – provided you have enough credits.
Scary good
“I’ve known “HeyGen” for a long time, but I didn’t know whether the translations were actually good. However, after recording the video and having it translated into Japanese, I was very surprised at how frighteningly good the lip syncing and voice matching were. However, a person with very good knowledge of Japanese has to judge whether the content has been translated well.” – Isa Kabakci, editor