We’ve taken a brief look at the definition and principle of LightField Photography in What is the LightField. In this article, we’d like to elaborate on what becomes possible when you record not just a flat projection, but the entire Light Field.
The important thing to note here, is that plenoptic computational imaging is only taking its first steps. We’re excited to see what other uses scientists will come up with in the future.
One of the most prominent and popular features of the Light Field is the possibility to change or set the focus after the picture is taken. Software Refocus is available with cameras by Lytro and Raytrix, the latter even offers pixel-precise focus as a software add-on.
All in Focus
Since virtually every pixel in the LightField picture can be focussed upon, we also have all the information necessary to create a picture with infinite depth of field, i.e. a picture where everything is in focus.
Lytro has earlier announced this feature for the Desktop Application “by the end of 2012“, but we’re still waiting for it. Open-source solutions already exist, but they are merely workarounds that stack the processed focus layers from the web-optimized, compressed -stk.lfp files.
Variable Depth of Field
The intermediate of the two above features is variable depth of field: You can not only choose the focal plane, but also and how much of what’s before and behind the plane is still in focus.
Raytrix software offers this feature under the name “Extended Depth of Field”.
Perspective Shift / Parallax
By selecting Lightrays that travel through different parts of the main lens system, you can move the perspective around a bit. In other words, you can “tilt your head” a little bit in every direction – virtually, and after the picture was taken. The strength of this parallax effect is dependent largely on the diameter of the main lens system. In case of the Lytro LightField Camera, you can shift about 5 cm from the very left to the very right.
Raytrix offers the Parallax feature under the name “Multiview”, Lytro’s Software does “Perspective Shift” since version 2.0 (December 2012).
3D – Single-lens, single-shot
With the extremes of perspective shift, 3D images can be reconstructed with a single picture from a single lens. What sets LightField data apart from conventional two-lens 3D recording, is that the baseline can be freely chosen, i.e. you can rotate the picture and still experience a correct 3D effect without having to tilt your head.
3D stereo is included in the standard software package by Raytrix, for Lytro, it’s the third feature that was originally said to be available “by the end of 2012“.
Doing a fair bit of math, it is possible to translate the relative depth data into actual, absolute distance from the lens. This can be useful for anything from inspecting manufactured parts (quality assurance) to measuring the dimensions of microscopic samples.
“3D depth estimation” is available for Raytrix cameras as an add-on.
LightField pictures can be stretched or compressed in depth. Similar to the video effect that is created when zooming in on something, objects in the background seem to move closer or farther away from the foreground.
Depth scaling has already been demoed by both Lytro and Raytrix. Raytrix’ Android App lets you experience the feature yourself.
Change lighting after the Fact
Using the depth data that can be derived from a LightField picture, it is also possible to change the lighting of a scene after the picture was taken. This results in new shadows that can be computed from the 3D model of a scene.
All of the above features apply both to still LightField Photography as well as LightField Video. However, processing LightField data is still very resource-intensive, and doing it at 30 frames per second is still a technological challenge.
Raytrix has already demonstrated live 3D LightField video, and is planning to bring Full-HD 3D video at 30 fps to the market by the end of 2012.