How Does a Depth Sensor Camera work? (The SIMPLE Answer)

If you’ve heard about the latest smartphones with depth sensor cameras, you might have wondered what that actually means and what it does for you.

Depth sensor cameras, also known as Time of Flight or ToF cameras, use pulses of light to render a detailed, three-dimensional map of an image. The smartphone can use that map to apply filters or effects to images and videos in real time.

The depth map can also be used to give measurements of an object.

What Is a Depth Sensor Camera?

A depth sensor camera, or ToF camera, sends out individual pulses of laser light each time a picture is taken. Most of that light is reflected back to the sensor, and because the speed of light is constant, the time that takes is used to create a three-dimensional map of the image.

ToF cameras have been around for decades, but it’s only in the last few years that the technology has trickled down to consumer electronics.

Especially when it comes to higher-end smartphones, ToF cameras have started to replace separate depth sensors to improve Augmented Reality apps and camera apps.

ToF cameras have a lot of advantages for smartphone makers. The technology is proven to work and doesn’t require a lot of processing power, meaning they stay cooler and use less battery.

What Does a Depth Sensor Camera Do?

One of the very first consumer electronics to use a ToF camera was the second generation Kinect with the Microsoft XBOX One. The camera was used for motion tracking, and some smartphones use ToF cameras for the same purpose today. The LG G8 ThinQ uses a ToF camera to track gestures, for example.

They can be used to improve Augmented Reality, as well, by placing things in the scene in a more realistic way.

Some of the latest offerings from Samsung, including the Galaxy Note10, Galaxy S20+ and Galaxy S20 Ultra also use the ToF camera as a 3D camera, offering instant measurements of objects on screen.

Depth Sensor Cameras and Photography

The most common use for a ToF camera is to offer better background blur in images. This feature, often referred to as portrait mode because that was the name Apple used, has been common for a few years, but ToF cameras take it to another level.

Originally, portrait mode used a separate depth sensor to create the blur, with the effect applied after the image was taken. With ToF cameras, though, blur can be applied to an image in real time.

Another major advantage to a ToF camera is the ability to have a detailed 3D map of a scene to help with autofocus.

Depth Sensor Cameras and Video

One reason ToF cameras have started to come to smartphones is because they offer better autofocus and depth detection, making it easier to offer blurred backgrounds and highlight the subject.

Those same attributes are just as desirable for video, as well, and cameras like the Galaxy Note10 and Galaxy S20+ have a new function called Live Video that offers them. You can switch between subjects on the fly and shoot video with a softly blurred background in real time.

The effect is available on both the front and rear facing cameras, making it possible to shoot selfie video with a blurred background.

Limitations of Depth Sensor Cameras

There is a drawback to the ToF camera on the Samsung offerings when it comes to video, however. While 4K video has become standard on smartphones and some high end phones can even shoot 8K video, the ToF camera shoots video in 480p at best.

On the Galaxy Note10, the camera shoots VGA resolution video, or 640 pixels x 480 pixels, and the S20+ and others have lower resolution.

The resolution of the cameras is likely to keep improving, however, making this a temporary drawback.