LightField Toolbox: Lytro Image Processing in Matlab

LightField Toolbox: Lytro Image Processing in Matlab Processing Lytro LightField Pictures is generally done within the free, official Lytro Desktop Software. If you want more control over your LightField results, though, there are a few tools available to do just that. (Note that the Desktop Software is still needed to transfer the pictures from the camera.)

The newest addition to the list of openly available LightField tools is LightField Toolbox v0.1 for Matlab, by Donald G. Dansereau.
It its initial release, it is limited to basic features such as loading, decoding, colour correction and visualization (including a basic perspective shift effect) of Lytro light field images. But there’s more to come!

This version of the toolbox and documentation are focused on Lytro imagery. Adapting this to other cameras should be straightforward. Future releases will support additional input formats, camera calibration, image rectification, and depth and volumetric filtering for improved rendering quality.

A sample light field pack is provided as well:

  LightField Sample Pack 1 for LightField Toolbox v0.1 (72.8 MiB, 2,107 hits)

LightField Toolbox v0.1 includes the following functions:

Decoding / Input

  • LFDecodeLenseletSimple
    Decodes a 2D lenselet image to a 4D light field using a white image and grid model. See example usage in LFExampleDecodeLenseletImage.
  • LFExampleDecodeLenseletImage
    Demonstrates decoding of lenselet-based light fields using LFDecodeLenseletSimple and LFSelectWhiteImage.
  • LFSelectWhiteImage
    Selects the white image appropriate for a given zoom and focus setting. See example usage in LFExampleDecodeLenseletImage.
  • LFUtilProcessLytroCalData
    Processes a folder populated with Lytro calibration data. Only white files and their associated metadata (.TXT) files are used. This generates a grid model (.grid.mat) for each white image, and a white image database (WhiteFileDatabase.mat) used by LFSelectWhiteImage. Darker white images are automatically detected and skipped.
    This function will generally only be used once for each Lytro camera.

Image Adjustment

  • LFColourCorrect
    Applies a colour balance vector, an RGB colour correction matrix, and
    gamma correction. Usage is demonstrated in LFExampleColourCorrect.
  • LFExampleColourCorrect
    Demonstrates use of LFColourCorrect to apply colour and gamma correction based on light field metadata. Also demonstrates use of LFHistEqualize to adjust the brightness of a light field based on histogram equalization.
  • LFHistEqualize
    Adjusts the brightness of a light field based on histogram equalization,
    usage is demonstrated in LFExampleColourCorrect.

Visualization

  • LFDispMousePan
    Display 2D slices of the light field with a rudimentary parallax effect. Click and drag in the image to change the point of view. Note that darkening at the edges of the image means that the effect is best near the center of the spatial range. For an automatically animated display,
    see LFDispVidCirc.
  • LFDispVidCirc
    Animated display showing 2D slices of the light field, similar to LFDisp-
    MousePan except the motion is preset to a circular path. An optional argument allows the radius of the circular path to be set.

File I/O

  • LFReadMetadata
    Reads the Lytro metadata files, which are in json format.
  • LFReadRaw
    There is no standard raw file format, and different tools generate differently formatted files. This function reads the 12-bit packed raw files generated by the LFP Reader tool when extracting white images and lenselet images.

Utility / Convenience

  • LFGatherWhiteImageInfo
    Builds a list of white images and associated camera settings by crawling through a prescribed folder. Called by LFUtilProcessLytroCalData to build the database of white images.

Support Functions

  • LFBuildLenseletGridModel
    Function for building a grid model from a white image. Called by LFUtilProcessLytroCalData.
  • LFRotz
    Builds a 3D rotation matrix: rotation around the z axis.

The image decoding process will be published later this year:
D. G. Dansereau, O. Pizarro, and S. B. Williams: Decoding, calibration and rectification for lenselet-based plenoptic cameras. In: Computer Vision and Pattern Recognition (CVPR), IEEE Conference on. IEEE, Jun 2013.

You may also like...

1 Response

  1. Donald says:

    The paper this work is based on, “Decoding, calibration and rectification for lenselet-based plenoptic cameras”, is available here: http://www-personal.acfr.usyd.edu.au/ddan1654/PlenCal.pdf

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.