LightField Toolbox: Lytro Image Processing in Matlab

Processing Lytro LightField Pictures is generally done within the free, official Lytro Desktop Software. If you want more control over your LightField results, though, there are a few tools available to do just that. (Note that the Desktop Software is still needed to transfer the pictures from the camera.)

The newest addition to the list of openly available LightField tools is LightField Toolbox v0.1 for Matlab, by Donald G. Dansereau.
It its initial release, it is limited to basic features such as loading, decoding, colour correction and visualization (including a basic perspective shift effect) of Lytro light field images. But there’s more to come!

This version of the toolbox and documentation are focused on Lytro imagery. Adapting this to other cameras should be straightforward. Future releases will support additional input formats, camera calibration, image rectification, and depth and volumetric filtering for improved rendering quality.

A sample light field pack is provided as well:

  LightField Sample Pack 1 for LightField Toolbox v0.1 (72.8 MiB, 2,257 hits)

LightField Toolbox v0.1 includes the following functions:

Decoding / Input

• LFDecodeLenseletSimple
  Decodes a 2D lenselet image to a 4D light field using a white image and grid model. See example usage in LFExampleDecodeLenseletImage.
• LFExampleDecodeLenseletImage
  Demonstrates decoding of lenselet-based light fields using LFDecodeLenseletSimple and LFSelectWhiteImage.
• LFSelectWhiteImage
  Selects the white image appropriate for a given zoom and focus setting. See example usage in LFExampleDecodeLenseletImage.
• LFUtilProcessLytroCalData
  Processes a folder populated with Lytro calibration data. Only white files and their associated metadata (.TXT) files are used. This generates a grid model (.grid.mat) for each white image, and a white image database (WhiteFileDatabase.mat) used by LFSelectWhiteImage. Darker white images are automatically detected and skipped.
  This function will generally only be used once for each Lytro camera.

Image Adjustment

• LFColourCorrect
  Applies a colour balance vector, an RGB colour correction matrix, and
  gamma correction. Usage is demonstrated in LFExampleColourCorrect.
• LFExampleColourCorrect
  Demonstrates use of LFColourCorrect to apply colour and gamma correction based on light field metadata. Also demonstrates use of LFHistEqualize to adjust the brightness of a light field based on histogram equalization.
• LFHistEqualize
  Adjusts the brightness of a light field based on histogram equalization,
  usage is demonstrated in LFExampleColourCorrect.

Visualization

• LFDispMousePan
  Display 2D slices of the light field with a rudimentary parallax effect. Click and drag in the image to change the point of view. Note that darkening at the edges of the image means that the effect is best near the center of the spatial range. For an automatically animated display,
  see LFDispVidCirc.
• LFDispVidCirc
  Animated display showing 2D slices of the light field, similar to LFDisp-
  MousePan except the motion is preset to a circular path. An optional argument allows the radius of the circular path to be set.

File I/O

• LFReadMetadata
  Reads the Lytro metadata files, which are in json format.
• LFReadRaw
  There is no standard raw file format, and different tools generate differently formatted files. This function reads the 12-bit packed raw files generated by the LFP Reader tool when extracting white images and lenselet images.

Utility / Convenience

• LFGatherWhiteImageInfo
  Builds a list of white images and associated camera settings by crawling through a prescribed folder. Called by LFUtilProcessLytroCalData to build the database of white images.

Support Functions

• LFBuildLenseletGridModel
  Function for building a grid model from a white image. Called by LFUtilProcessLytroCalData.
• LFRotz
  Builds a 3D rotation matrix: rotation around the z axis.

The image decoding process will be published later this year:
D. G. Dansereau, O. Pizarro, and S. B. Williams: Decoding, calibration and rectification for lenselet-based plenoptic cameras. In: Computer Vision and Pattern Recognition (CVPR), IEEE Conference on. IEEE, Jun 2013.