It’s likely going to be difficult to find a phone with a single-lens camera in 2018 as Qualcomm is introducing three new camera modules, with two specifically designed to boost the depth sensing prowess of our devices.
Just as Qualcomm is one of the biggest names in phone processors, these imaging modules, which are a part of the company’s Spectra Module Program, could find their way to dozens of Android smartphones next year, including some of the 2018’s flagships.
By developing and licensing the camera parts to hardware partners, Qualcomm aims to speed up the time it takes to deliver depth sensing to consumers in phones as well as head-mounted displays.
The three new modules are as follows: an iris authentication camera module, a passive depth sensing module and an active depth sensing module.
The iris authentication module will allow for the biometric security feature to be embedded into phones. Qualcomm says it works through sunglasses and also has liveness detection, meaning you won’t be able to fool it with a photo or a (super creepy) 3D model of someone’s face.
The passive depth sensing module is for entry- and mid-tier devices. Two cameras create two images of the same object, and in sensing the disparity between the images, can infer how close the object is to you.
While this module brings basic depth sensing to lower-end devices, it also requires ample light to cut down on noise as well as a background that’s differentiated enough to pick up the object, i.e. the background can’t be a starch white wall.
To that end, Qualcomm created the active depth sensing module, also known as its structured light image sensor. It has two essential components: an infrared illuminator and an infrared camera. This module doesn’t require a dual camera, though it’s more likely than not phones that use it will feature both an infrared camera and a standard one.
With this module, the infrared illuminator is constantly projecting a laser in a known light pattern. The infrared camera reads the light pattern and figures out any discrepancy caused by an object that’s in its path.
According to Qualcomm, this sensor can work in complete darkness and at a range between three and five meters. By casting a pattern of light with over 10,000 points of depth collected, it can monitor changes in depth as small as .1mm apart, making it the most accurate on the market. It also doesn’t struggle with round edges, like passive depth sensing might.
Depth sensing has a number of useful applications, such as improving a phone camera’s bokeh effect (distinguishing an object in the foreground from the background).
But these also extend, at least in the active depth sensing module’s case, to more accurate facial detection, recognition and authentication as well as real-time depth mapping and hand tracking and object avoidance in VR and AR.
So, we could soon be jumping into Google Daydream and using our hands to explore virtual worlds.
We’ll have to see how accurate the module’s recognition is when phones that house it hit the market, and how it compares to the iPhone 8, but Qualcomm is setting up the Android ecosystem to compete, if not dominate, depth sensing in 2018.
More stable images on the way
Also announced today is the second generation of the Spectra image signal processor (ISP), which has been rearchitectured and will be included in Qualcomm’s next-gen system-on-a-chip (SoC).
The new chip is rumored to be unveiled later this year, and is expected to be included in flagship Android devices launching in 2018.
What exactly does the revamped Spectra ISP bring to the table? It, too, has depth sensing capabilities, however, not active depth sensing as found in the module. With two cameras, devices with the Spectra ISP can do passive depth sensing.
The second-gen Spectra ISP also features multi-frame noise reduction, meaning it filters and blends frames as your camera takes bursts of photos for the highest image quality. It also allows for crisper video capture with motion compensated temporal filtering (MCTF), and accelerated electronic image stabilization cuts down on vibrations during recording.
Qualcomm demonstrated the impact of MCTF with an image shot from a drone, and the result was a cleaner photo than the muddier image of the same scene taken without MCTF.
Buildings and lights in the MCTF image were distinct from the background, and it’s no stretch to see the Spectra ISP making itself welcome in drone photography in addition to phones.
The Spectra ISP also supports machine learning-backed computer vision to assist in tasks like face detection and computation photography. In practical terms, your phone’s camera will be better at picking up faces in a photo, which will help when adding filters in Snapchat, for example.
Finally, the second-gen Spectra ISP features improved head/body motion tracking and simultaneous location and mapping, capabilities that will allow for improved experiences in AR and VR.