Imagine watching a movie in Korean sitting right here. Now, that happens or has been happening for quite a few years, right, thanks to subtitles. But were there instances when you were busy following the subtitles, and you missed out on a crucial scene in the film? To solve this problem and make the watching more seamless, two Ph.D. scholars from the International Institute of Information Technology – IIIT-H’s Centre for Visual Information have come up with a deep tech Artificial Intelligence web app.
The software is already patented by a start-up called NeuralSync AI, mentored by IIIT-H. It can morph authentic-looking speeches with a single video clip of a human or digital avatar. And this, the app can do within minutes in the language of your choice. For example, if it is integrated into Netflix, then you can have the luxury of watching the movie of your choice in the language of your choice.NeuralSync’s Way2Lip is a compact video editor with an AI algorithm that is trained to generate accurate lip movement based on the target’s speech. Apparently, the project is already a hit as per NeuralSync’s software developers Rudrabha Mukhopadhyay and Prajwal KR and CEO Pavan Reddy.
They reveal that they have already been getting offers worth several thousands of dollars. Incidentally, the US-patented technology owned by IIIT-H was invented by Rudrabha, Prajwal, Prof C Jawahar, and Prof Vinay Namboodiri.
As of now, the app takes 45 seconds to make one minute of video, and they say they are working on bringing down the time to 15-20 seconds. This also means that the operating costs will be cut down drastically. Just about all European and Asian languages can be dubbed using the app, though for now, Mandarin is a challenge to it.NeuralSync is pretty clear with regards to its future direction and says they are now keen on targeting e-learning entities to create e-learning videos and marketing video films. Then, they are also in talks with a few metaverse companies in China. However, their ultimate goal is films, for which they will create their product in a small Google Chrome extension. They have already made a pact with a Bollywood producer to execute a few scenes.