How Holograms Could Improve Your Smartphone

Chatting with friends via a hologram on your smartphone soon could be possible.

Using artificial intelligence, scientists can now rapidly generate photorealistic color 3D holograms even on a smartphone, according to a new study. One company is planning to launch a system later this year for displaying holograms on smartphones.

"The emotional impact is palpable, and it can add a new and unique dimension to the utility of smartphones," Joe Ward, CEO of IKIN, a company that is working on holograms for phones, said in an email interview.

"The human eye is much more captivated by a three-dimensional image, and the physiology that occurs when viewing a hologram translates into deeper and more satisfying interactions. This impact can be felt in a video conversation between two people, playing a game as a hologram, or even viewing old photographs in 3D."

Solving the Data Problem

Generating holograms takes a massive amount of data and computation, so they haven’t been practical for personal electronics. Researchers have developed a new way to produce holograms using a deep learning-based method so efficient that it can make holograms in the blink of an eye, they revealed in the recent paper. 

"People previously thought that with existing consumer-grade hardware, it was impossible to do real-time 3D holography computations," Liang Shi, the study’s lead author and a PhD student in MIT’s Department of Electrical Engineering and Computer Science, said in a news release.

"It’s often been said that commercially available holographic displays will be around in 10 years, yet this statement has been around for decades."

The researchers used deep learning to allow for real-time hologram generation. The team designed a convolutional neural network—a processing technique that uses a chain of trainable tensors to mimic how humans process visual information.

Training a neural network typically requires a large, high-quality dataset, which didn’t previously exist for 3D holograms.

The team built a custom database of 4,000 pairs of computer-generated images.

To create the holograms in the new database, the researchers used scenes with complex and variable shapes and colors, with the depth of pixels distributed evenly from the background to the foreground, and with a new set of physics-based calculations to handle occlusion.

That approach resulted in photorealistic training data. Next, the algorithm got to work.

The research "shows that true 3D holographic displays are practical with only moderate computational requirements," Joel Kollin, a principal optical architect at Microsoft who was not involved with the research, said in the news release.

He added that "this paper shows marked improvement in image quality over previous work," which will "add realism and comfort for the viewer." 

Holograms Are Already on the Market

Holograms are heading to a smartphone near you. IKIN already is selling holographic solutions to some businesses, and the company plans to release consumer options later this year, first on Android devices and then on Apple iOS.  

"The challenges are significant," Ward said. "Our technology works in ambient light, which means that users can view the holograms without having to wear any goggles, headgear, or other equipment. This can require significant processing power and battery life, which are problems that we solved."

Another gadget that produces holograms is also on the market for corporate customers, although it’s a permanent installation and not for smartphones. ARHT Media produces the HoloPod, which the company touts as a way to avoid business travel.

For example, the firm Sun Life Financial wanted one of their executives to appear at an event in Vancouver, despite a scheduling conflict that had obliged him to stay in Toronto, according to the website. 

ARHT Media mounted a holographic display at the Vancouver event and had Sun Life’s executive captured and transmitted live as a hologram to the event from its Toronto studio.

"He was able to see the audience and interact with them in real-time as if he was attending the event and present in the room," the website boasts.