Whether you are a Facebook user or not, you must have heard about the accusations made against Facebook in the news in recent weeks. The accusations forced Facebook to rebrand in a smart move. The new name of the company will be Meta (for now and in this article we will continue to use the name Facebook). Surely, this rebranding is one of the things that will make history and can be talked about a lot.
But what accusations did Facebook have? One of them was the use of Facial Recognition System. A system based on artificial intelligence. Of course, Facebook announced that it will no longer use this system. A lot can be said about this. But our discussion in this article is none of these.
In this article, we want to introduce one of the exciting and safe features of Facebook. A feature that allows its users to convert any two-dimensional (2D) photo into a 3D or three-dimensional photo and share it. A feature that is 3 years old has been added to it. A feature that still uses artificial intelligence, but not to recognize the user’s face. Facebook has long invested in the use of artificial intelligence in its social network.
Before answering these questions, what are Facebook 3D photos and how do they work?, it is better to answer another related question: What is a 3D photo and how to make them? 2D photos are the ones you and I take every day with our mobile phones of trees, flowers, books, etc. But how long have you been able to turn these photos into 3D photos with the help of applications, some smartphones and of course Facebook.
Table of Contents
How do Facebook 3D photos work?
In a 3D photo, a scene is photographed twice, in one of which the distance from the center of the photo is slightly deviated and shifted. 3D photos are more realistic and give 2D images depth and even the illusion of movement. So, they are more attractive. It is for this reason that Facebook wanted its users to be able to easily take more interesting and attractive photos, without the need to work with a special software or application. Why don’t the memories and photos of Facebook users become deep and animated when everything is moving towards 3D and 3D designs are used in all fields (from 3D cinema to advertisements)?
Facebook wanted its regular users to easily convert any 2D photo into 3D and share it in the News Feed. For this reason, he defined a project with the same name: Facebook’s 3D photo project. The result of that project was made available to users in two stages. The first phase introduced in 2018 and the more advanced phase in 2020.
But why two steps? What needed to be changed and what was the most important thing that changed in these 2 years? Facebook used machine learning to fix the deficiency that was in the first stage and prevented a number of users from 3D rendering of photos. First, I will explain the first step to see how the 3D photos of Facebook first worked.
3D photos of Facebook at the beginning (2018)
Let’s hear from Brian Thivierge, design director at Facebook, how these 3D photos were made in the first stage:
In essence, 3D photo technology takes the subject in the foreground and measures it against whatever is in the background. This data is then used to create an accurate depth map and, when combined with Facebook’s custom software, gives photos movement and depth when you scroll past them in News Feed. The effect is a little like peeking into a magic window on your computer or smartphone screen.
But there was a point. Facebook could only 3D photos taken with a dual-camera or dual-lens smartphone (Portrait Mode), not every 2D person. Because only those photos had a depth map. Facebook created a 3D model of the photo using the Interpolation Algorithm, and then using the three.js framework, that 3D model was shown moving when the user scrolled the page.
3D Facebook Photos Now
Facebook explains the second stage and the reason for using artificial intelligence and machine learning as follows:
To bring this new visual format to more people, we have used state-of-the-art machine learning techniques to produce 3D photos from virtually any standard 2D picture. This system infers the 3D structure of any image, whether it is a new shot just taken on an Android or iOS device with a standard single camera, or a decades-old image recently uploaded to a phone or laptop.
So now you can 3D any photo you have taken with any device and share it on Facebook. Facebook has not reached this technology easily. As he said, many technical challenges had to be overcome. Part of the challenges was related to this system running well on any mobile device so that the user who works with the application does not have a problem in creating and viewing 3D photos.
How does the Facebook 3D photo system work?
To overcome these challenges, we trained a convolutional neural network (CNN) on millions of pairs of public 3D images and their accompanying depth maps, and leveraged a variety of mobile-optimization techniques previously developed by Facebook AI, such as FBNet and ChamNet.
So Facebook built a neural network (CNN) that was trained with a huge amount of 3D photos to be able to estimate the depth map of each photo (depth estimation neural network). Then this network is equipped with artificial intelligence so that the quality of the photo is not reduced in the 3D process. As a result, both the process and the result in mobile are high quality and fast.
How to make a 3D Facebook photo?
Is it really easy to make 3D photos with Facebook? Yes! I will show you this. I had never done this before and Lee was very relieved. Believe me, it was a lot of fun and I want to make all the photos I took in 3D! pay attention to these points:
- Making a 3D photo with a computer and mobile phone has different steps.
- If the photo you want to make 3D with Facebook is on your laptop/computer, the png file should be. Or jpg. Save the image along with its depth map file (with the extension depth_). (If you have problems, refer to the Facebook guide.)
- Of course, as mentioned, you shouldn’t worry about what mobile phone you took the 2D photo with. But check the Facebook guide to make sure your device is compatible. Because the list of compatible mobile phones (Android and IOS) is written in that guide.
Artificial intelligence, machine learning and 3D design
Facebook has announced that it plans to evolve this system so that users can create 3D videos. Facebook is looking to create more 3D experiences for its users. That is, it wants to make the virtual world more real, which it has done now with photos. If you are a regular user, you may not care much about the complex steps that have been taken so that you can make your desired photo in 3D in less than 1 minute.
But if you are a graphic designer or a visual and product designer or a photographer or somehow deal with 3D designs, what does all this mean for your career? And what message does it give you about the future of 3D design? Artificial intelligence and machine learning have shown that it can revolutionize the world of 3D designs and provide designers with many possibilities to create more realistic and complex 3D designs. Of course, it will make their work easier.