MIT Camera Culture: Simple, Cheap Method for Light Field Photography at Full Sensor Resolution

At its current development stage, light field photography (based on microlens arrays) poses a compromise between spatial information and resolution: The more refocus or prespective a camera is required to provide, for example, the more of its sensor resolution is sacrificed. In the Lytro Light Field Camera, an 11 Megapixel sensor takes pictures that result in pictures of 1.1 Megapixels, so only about 10 % of the sensor resolution make it into the final image.

As reported previously, the MIT‘s Camera Culture group has come up with a new method to capture light fields, which is both cheaper and more effective. In a new article published by MIT News, the researchers explain what their system, named “Focii”, is capable of:

At this summer’s Siggraph — the major computer graphics conference — they’ll present a paper demonstrating that Focii can produce a full, 20-megapixel multiperspective 3-D image from a single exposure of a 20-megapixel sensor.

Instead of an expensive microlens array, Focii uses a small piece of plastic film with a unique printed checkerboard pattern (a so-called coded mask), which can be inserted between the sensor and lens of any ordinary digital SLR camera.
The rest of the work is done by some clever algorithms:

Focii represents a light field as a grid of square patches; each patch, in turn, consists of a five-by-five grid of blocks. Each block represents a different perspective on a 121-pixel patch of the light field, so Focii captures 25 perspectives in all. (A conventional 3-D system, such as those used to produce 3-D movies, captures only two perspectives; with multiperspective systems, a change in viewing angle reveals new features of an object, as it does in real life.)

The key to the system is a novel way to represent the grid of patches corresponding to any given light field. In particular, Focii describes each patch as the weighted sum of a number of reference patches — or “atoms” — stored in a dictionary of roughly 5,000 patches. So instead of describing the upper left corner of a light field by specifying the individual values of all 121 pixels in each of 25 blocks, Focii simply describes it as some weighted combination of, say, atoms 796, 23 and 4,231.

More information is available here:

• Multiview 3-D photography made simple – MIT Media Relations
• Compressive LightField Photography enables Higher Resolution LightFields in a Single Image
• Marwah K., Wetzstein G., Bando Y. & Raskar R. (2013): Compressive Light Field Photography Using Overcomplete Dictionaries And Optimized Projections, Technical Paper, ACM Siggraph 2013