Deepfakes: How it works, apps & some examples

Deepfake photos and videos are catching the attention of many around the world and causing researchers and lawmakers to ask: What’s next?

The deepfake term stands for “deep learning” and fake, referring to the use of deep learning methods from artificial intelligence to create a fake copy of any media.

Many are alarmed at how easy it is to fake pictures and videos using deepfake apps, while others find it highly entertaining. However, deepfakes have become a thing.

So, we are taking a closer look at this technology to discover how it works and what it means for the world.

Deepfakes vs Artificial Intelligence

First and foremost, it is important to note the difference between standard AI-generated output and deepfakes. AI algorithms have produced different types of media outputs for ages, but you can hardly call them deepfakes.

The term deepfake refers to a faked media, such as videos, pictures, or anything else, that is an AI-edited version of a pre-existing copy.

Artificial intelligence developed immensely over the last decade, thanks to cheaper, more powerful computers, as well as to new developments in the field.

Where AI-generated imagery was easily recognizable a decade ago, the algorithms have gotten better in the last few years, making it easier to produce highly realistic results. It is this high level of authenticity that makes AI-generated images in general, and deepfakes specifically, very impressive.

AI and Deep Learning

To understand how computer algorithms evolved to create the highly realistic human faces you can find on generated.photos and thispersondoesnotexist.com, you will need a primer in artificial intelligence.

There are different areas in AI, depending on the method you are using and what you plan to achieve. You will find everything from probability methods like the Bayesian filter used for spam detection to fuzzy logic, evolutionary algorithms that develop on their own, and artificial neural networks, which aim to simulate the human brain.

Neural Networks

Just as there are real neurons in the human brain, with their dendrites and axons connecting in highly complex networks that span millions to billions of neurons, artificial neurons also connect. But their numbers are limited by computing power.

The goal of a neural network is to provide an output or answer, to every input. This is achieved by first teaching the network as you would teach a child. After that, it can now make predictions based on your teachings.

You should also note that more neurons in a network often mean better results and more training data improves results as well. This is exactly how the human brain works, at least, theoretically.

How deepfaking works

There are many things and areas where you can apply artificial intelligence. Deepfaking is just one of them, which evolved from using neural networks to edit images. The earlier results were promising, but they were very basic until Ian Goodfellow and friends came up with GAN in 2014.

GAN or Generative Adversarial Network is a framework for teaching neural networks. So, instead of you having to prepare and teach the network on your own, you let another network compete with the first network by criticizing its outputs. This leads to the generation of great results based on any training set.

The results of the GAN approach were as phenomenal back then as they are today. It has also become the bedrock of many AI solutions and applications, including deepfakes. Plus, the improvements in computing power have made it easier to do amazing stuff on even smartphones.

Possible deepfake applications

Asides from the highly popular images and videos circulating on the net, there are wide applications for using the deepfake technology for other more useful purposes.

Deepfakes can help speech-impaired patients to find a voice, using audio synthesis. It can find uses in the classroom, where historical figures are brought back to life, as it happened in the Dalí Museum in St. Petersburg, Florida.

The movie industry can also find uses for deepfakes, as it can significantly slash the cost of CGI (Computer Generated Imagery) in movies. Dead entertainers and actors can be resurrected and featured in new movies.

Deepfakes can enable anyone to produce specialized videos in multiple languages, as happened with David Beckham’s “malaria no more” campaign and Manoj Tiwari’s political campaign in India, where he spoke fluently in multiple languages.

Gaming can also find good uses for deepfakes, as players can immerse themselves in the game for a deeper virtual reality experience.

Even social media marketers are finding uses for computer-generated social media personalities created and managed entirely on the computer. Though not a real deepfake, Calvin Klein’s Lil Miquela and her 3 million Instagram followers show what’s possible.

Some deepfake examples

To better understand this technology and the promises it holds, here are some of the most impressive deepfakes out there.

• Obama announcement – This is one of the best known, oldest, and most shocking deepfakes. Released in 2018, this video was created as a warning to enlighten people about the possibilities of the technology.
  
  It featured Obama making a Public Service Announcement and includes him calling Trump a “dipshit”. The creator is Jordan Peele and the tools he used include Adobe After Effects and FaceApp.
  
  
• Mona Lisa – Most people know Leonardo da Vinci’s masterpiece, the Mona Lisa. But in 2019, people marveled the world over, as they saw her smile and move for the first time, thanks to Samsung’s Russian AI research labs.
  
  Labeled “realistic neural talking heads”, the researchers used 7,000 images gathered from YouTube to train this neural net. Then all you need is a single photo to match the facial features and animate the face. They also did it with Albert Einstein, Marilyn Monroe, Salvador Dalí, and others.
  
  
• Zoom Calls – In 2020, two Russian researchers demonstrated how you could use the customizable background feature of Zoom video calls to create real-time animated deepfakes of anyone you wanted. They could appear as a live-and-speaking Albert Einstein, the Mona Lisa, Donald Trump, or Boris Johnson on the web calls.
  
  
• Salvador Dalí – In 2019, the Dalí Museum in St. Petersburg, Florida, hosted the “Dalí Lives” exhibition. It featured a deepfaked version of the dead artist, and they needed over 1,000 hours of machine learning and 6,000 frames to pull it off.
  
  
• DeepNude – Also in 2019, a team of developers released an impressive app called DeepNude. All it needed was a picture of a woman in bikinis and it would strip her completely naked. Then it added a “FAKE” watermark on the picture, which you can remove for $50.
  
  The app amazed and outraged so many people. And the pressure was so much, that the developers had to pull it off the web. Its open-sourced code on GitHub was also removed, but websites like http://deepnude.to and a telegram bot show that DeepNude lives on.

Notable deepfake apps

There are also lots of deepfake generator apps out there, with some being more impressive than others. They are mostly for the smartphone and make it easy for anyone to quickly manipulate photos and videos.

These apps include:

• DeepFaceLab – Available on GitHub, DeepFaceLab is a leading software solution for creating deepfakes. It lets you swap and de-age faces, replace the head, and manipulate the lips of politicians and other people of interest. Many YouTube channels also use it.
  
  
• MyHeritage – The Deep Nostalgia offer from MyHeritage.com makes it easy to bring antique family photos to life. MyHeritage is a platform for discovering your family tree. So, bringing your ancestors back to life can be a creepy but impressive experience.
  
  
• FakeApp – Developed and released by a Reddit user, FakeApp makes it easy to swap or play with faces on a video for free. It was used in creating the famous re-mastering of young Princess Leia in the Star Wars: Rogue One prequel. It only took minutes to make, apparently, but it looked better than the movie original, which took weeks and cost much more.
  
  
• Reface – Another impressive and fun-to-use app available for Android and iOS. It comes with ads though, which you can remove with a monthly subscription.
  
  
• Zao – Chinese deepfake app that lets you create new videos in just seconds, but with impressive results. It is only available in China.
  
  
• Deepfakes Web – A cloud-based deepfake app that works on the web. Just get to the website, upload a video and click a button. Then the system will learn the video and create a new one for you. You can also train it for better results.

Conclusion

Coming to the end of this deepfake post, it should be obvious how far this technology has come by now. And this includes both the moral and immoral approaches, as the porn industry has always been a source of innovation on the internet.

Also given the relatively cheaper cost of AI-deepfakes over traditional CGIs, the future is surely bright for deepfake use in the movie industry, as well as in other applications beyond entertainment.