LightField Toolbox: Lytro Image Processing in Matlab
Processing Lytro LightField Pictures is generally done within the free, official Lytro Desktop Software. If you want more control over your LightField results, though, there are a few tools available to do just that. (Note that the Desktop Software is still needed to transfer the pictures from the camera.)
The newest addition to the list of openly available LightField tools is LightField Toolbox v0.1 for Matlab, by Donald G. Dansereau.
It its initial release, it is limited to basic features such as loading, decoding, colour correction and visualization (including a basic perspective shift effect) of Lytro light field images. But there’s more to come!
This version of the toolbox and documentation are focused on Lytro imagery. Adapting this to other cameras should be straightforward. Future releases will support additional input formats, camera calibration, image rectification, and depth and volumetric filtering for improved rendering quality.
A sample light field pack is provided as well:
LightField Sample Pack 1 for LightField Toolbox v0.1 (72.8 MiB, 1,389 hits)
LightField Toolbox v0.1 includes the following functions:
Decoding / Input
Decodes a 2D lenselet image to a 4D light field using a white image and grid model. See example usage in LFExampleDecodeLenseletImage.
Demonstrates decoding of lenselet-based light fields using LFDecodeLenseletSimple and LFSelectWhiteImage.
Selects the white image appropriate for a given zoom and focus setting. See example usage in LFExampleDecodeLenseletImage.
Processes a folder populated with Lytro calibration data. Only white files and their associated metadata (.TXT) files are used. This generates a grid model (.grid.mat) for each white image, and a white image database (WhiteFileDatabase.mat) used by LFSelectWhiteImage. Darker white images are automatically detected and skipped.
This function will generally only be used once for each Lytro camera.
Applies a colour balance vector, an RGB colour correction matrix, and
gamma correction. Usage is demonstrated in LFExampleColourCorrect.
Demonstrates use of LFColourCorrect to apply colour and gamma correction based on light field metadata. Also demonstrates use of LFHistEqualize to adjust the brightness of a light field based on histogram equalization.
Adjusts the brightness of a light field based on histogram equalization,
usage is demonstrated in LFExampleColourCorrect.
Display 2D slices of the light field with a rudimentary parallax effect. Click and drag in the image to change the point of view. Note that darkening at the edges of the image means that the effect is best near the center of the spatial range. For an automatically animated display,
Animated display showing 2D slices of the light field, similar to LFDisp-
MousePan except the motion is preset to a circular path. An optional argument allows the radius of the circular path to be set.
Reads the Lytro metadata files, which are in json format.
There is no standard raw file format, and different tools generate differently formatted files. This function reads the 12-bit packed raw files generated by the LFP Reader tool when extracting white images and lenselet images.
Utility / Convenience
Builds a list of white images and associated camera settings by crawling through a prescribed folder. Called by LFUtilProcessLytroCalData to build the database of white images.
Function for building a grid model from a white image. Called by LFUtilProcessLytroCalData.
Builds a 3D rotation matrix: rotation around the z axis.
The image decoding process will be published later this year:
D. G. Dansereau, O. Pizarro, and S. B. Williams: Decoding, calibration and rectification for lenselet-based plenoptic cameras. In: Computer Vision and Pattern Recognition (CVPR), IEEE Conference on. IEEE, Jun 2013.