Light Field Features

We’ve taken a brief look at the definition and principle of light field photography in What is the Light Field. On this page, we’d like to show the possibilities that come with light field imaging.

Software Refocus

One of the most prominently marketed features of light field photography is the possibility to change the focus after the picture is taken. Knowledge of the direction of light rays in a recording allows us to reconstruct different versions of a photo with infinitely adjustable focus:
Post-capture software refocus animation

 
 

Synthetic Aperture and Variable Depth of Field

The information contained in a 4D light field volume includes a 3D model of the captured scene, as well as the direction of all light rays arriving at the sensor. Using this rich information, we can use complex mathematics to change optical parameters which, in conventional photography, would be fixed when the image is recorded. One of these parameters is aperture, which determines the depth of field. This makes it possible to produce images with very shallow depth of field (e.g. a very blurry background), or infinite depth of field (where everything is in focus), from the same recording.
“Extended depth of field”, “all in focus” or “total focus” are commonly marketed terms describing the “sharper image” side of the synthetic aperture feature, but shallow depth of field and more bokeh are also possible, and may especially be desired for artistic effects.

Light field feature: variable depth of field

To illustrate the potential of synthetic aperture, below are five versions of the same light field picture with identical focal plane, processed at synthetic apertures of f/1, f/2, f/4, f/8 and f/16. The raw light field volume was recorded at f/2.

Synthetic aperture of f/1 - very shallow depth of field Synthetic aperture of f/2 - shallow depth of field Synthetic aperture of f/4 - moderate depth of field Synthetic aperture of f/8 - wide depth of field Synthetic aperture of f/16 - very wide depth of field


 

Parallax and Perspective Shift

By selecting specific light rays that travel through different parts of the main lens system, we can produce different views or perspectives of the recorded scene. The resulting parallax effect – showing differences between two different lines of sight – is similar to moving your head around slightly to peek behind an object.
The strength of this parallax effect is dependent largely on the diameter of the main lens system. In case of the Lytro LightField Camera, you can shift about 5 cm from the very left to the very right.
Other terms used for parallax are “multiview” (e.g. with Raytrix products) or “perspective shift” (e.g. with Lytro products).

Parallax / Perspective Shift animation based on a single light field exposure Parallax / Perspective Shift animation with extended field of view, based on a single light field exposure

Since light field technology is based on tracing light rays, it correctly reproduces the effects of optical elements (e.g. lenses or reflecting surfaces) in an image:
Parallax / Perspective Shift animation of a technical laser setup correctly reproduces changing views through a lens (animation based on a single light field exposure)
 

3D – Single lens, single exposure

Utilizing different views from parallax (see above), we can reconstruct 3D images with a single exposure from a single lens. In contrast to conventional two-lens 3D recording, the baseline is not fixed at recording. Instead it’s possible to rotate a picture and still produce a correct 3D effect, without having to tilt your head in the same direction.
3D stereo is included in the standard software packages by Raytrix, and in Lytro Desktop 3.1 and higher.

Single-lens, single-exposure 3D output from a light field recording, shown as anaglyph stereo output


 

Distance Measurement

If the optical properties of the light field system are known precisely, it is possible to translate the relative depth data into absolute distance from the lens. This can be useful in many use cases, from inspecting manufactured parts for quality assurance, to measuring the dimensions of microscopic samples.
“3D depth estimation” is available for Raytrix cameras as a software add-on.

Distance measurement based on depth data of a light field image, shown as a depth map overlay (illlustration) Distance measurement based on depth data of a light field image, shown as a picture overlay (illlustration)

 

Depth Scaling

LightField pictures can be stretched or compressed in depth. Similar to the “dolly zoom” video effect (or “vertigo zoom”) that is created when zooming in on a subject while moving in the opposite direction, objects in the background seem to move closer to the subject. When zooming out while moving closer, objects in the background seem to move farther away.
Depth scaling has already been demonstrated by both Lytro and Raytrix. Raytrix’ Android App (sadly, no longer available) allowed users to experience the feature first-hand.
Depth scaling animation (screen capture of the Raytrix Light Field Viewer app for Android)

 

Change lighting conditions in post-processing

Using the depth data that can be derived from a light field volume, it is also possible to change the lighting of a scene after the picture was taken. Digitally moving the light source in post-processing produces realistic, new shadows in a captured, real scene.
 

LightField Video

All of the features above apply to both, still light field imaging as well as light field video. Processing large lightfield datasets requires lots of computing power and very high-end hardware, and processing in real time remains a technological challenge.
However, Raytrix has already demonstrated live 3D light field video at up to 25 frames per second: