Feijoo, Belen Esteban and El Fary speak in perfect English: automatic video dubbing is already devastating and confusing

This article is a part of the weekly Technology newsletter, which is sent every Friday. If you want to sign up to receive it in its entirety, with similar but more varied and brief topics, You can do it at this link.

This week It is the work of the growing popularization of apps that make the most of artificial intelligence. This week’s explosion is due to the release last Friday of the new video translation application from a startup called HeyGen.

The dubbing of Feijoo, the leader of the PP known for not being fluent in English, has caused some, like the journalist Isabel San Sebastian, to fall into deception. After applauding “the excellent diction and very well chosen words” by the candidate for the presidency of the Spanish Government in a tweet this Thursday, has acknowledged his frustration an hour later.

“They tell me that this video is the work of artificial intelligence. The voice is the same, the verisimilitude, absolute. Depending on what it is used for, this tool contains an enormous danger, because it is impossible to distinguish it from reality,” the opinion member has lucidly warned.

The video of Leo Messi in English by Argentine content creator Javi Fernandez went viral on Monday. Shortly after, American user Jon Finger recorded himself in English and within 10 minutes he had his own tone of voice dubbed into French and German with his lips slightly adapted to the new language. It went viral and in a couple of days in Spain the North Korean account saw that video, posted it and started posting other examples with another app (HeyGen was overloaded with work), which seems to fit the lips worse.

Belen Esteban and El Fary began to appear in videos speaking excellent English, with a tone that seems to recall Donald Trump’s accent. Wait times in the app went from a few minutes to a day. Wednesday there were more than 120,000 users waiting for the HeyGen servers to do their job. Although there are also applications that manage to do it in real time.

Here is the youtuber technology Marques Brownlee speaking in Latin Spanish. It’s not perfect, but it’s very fine. “Since we launched the video translation app last week, we have received a lot of interest from the community,” Joshua Xu, founder of HeyGen, wrote on X. Xu sees the best application of his tool on YouTube and in online education courses. Today the main ones are in English. If they can be translated into all languages ​​effortlessly, their impact will be greater. For now, what works and is impressive is monetizing translated implausible content in X:

EL PAIS already explained more than a year ago the risk that AI implied for dubbing, with lost jobs and union manifestos between organizations in several countries to defend themselves from the avalanche. Now, as happens in technology, apps arrive that popularize a technology that until recently was only in the hands of companies.

It is likely that most of the impact of an app like HeyGen will be on videos that would never have hired a voice actor. Or at least not all. Mr Beast, the most popular YouTuber in the world, has channels in other languages ​​that do just that: dub his videos into other languages. Now it will be easier and cheaper, as long as the machine is capable of catching the speech turns.

In addition to the economic or educational impact of dubbing, there are other possible consequences in these videos. One of the easiest to foresee is the confusion and misinformation that seeing celebrities speaking in other languages ​​and saying anything can cause on social networks. The case of Feijoo cheating on Isabel San Sebastian is very clear, perhaps only the first step. As with the deepfakes, the real problem remains to be discovered: perhaps also audio messages with threats or lies? Or fake calls? Or videos where a politician says something in one language is translated into saying another with fake subtitles? It’s all to be seen.

There is, however, more and more prevention. This week a tiktok by Argentine comedian Jero Freixas was released. He was commenting on an interview with Messi where he said that he likes paddle tennis. Freixas joked that Messi can only like football: “This is crazy,” he said, and added: “They created it with artificial intelligence.” There will necessarily be more people with the easy or artificial resource of saying that “this is a lie.”

The increase in fake videos raises people’s caution. This week, YouTuber Carles Tamayo published an experiment on his channel. He created a hoax about him where an alleged employee accused him of labor exploitation. He created a setup with messages, fake X accounts, paid bots and even bought a misleading article on a news website with the headline “The fallen YouTuber.”

The initial reaction on the internet was variants of phrases like “no wonder” and “he deserves it for being a liberal liar.” But within a few hours, more and more people discovered the weaknesses of the story and quickly began sharing videos explaining that everything was a bait orchestrated by Tamayo. The YouTuber was right about people’s prejudices and his speed when judging him. But beyond the paid article, there was hardly any media coverage. And between the reaction of users who were suspicious and the media that did not increase the false trend in X, there is some hope with the skepticism of society when it sees something that seems too good to be true.

You can follow EL PAIS Technology in Facebook and Twitter or sign up here to receive our weekly newsletter.