Light Field Microscope: New Imaging System Allows Real-Time 3D Microscopy

One of the most exciting fields in science where light field imaging (or plenoptic imaging) has a great potential is microscopy: Not only is the depth of field very limited in microscopy, it also enables us to observe things much smaller than what the eye can see. Extended depth of field and 3D reconstruction would offer many opportunities, such as to better understand the three-dimensional internal structure of plant- or animal cells and tissues, to name just one example.
3D imaging using multi-camera approaches is very difficult in microscopy due to space limitations and the strong effects of parallax, but light field imaging can solve these problems. Until now, however, light field microscopy is still largely defined by resource-intensive post-processing, which limits real-time applications and observations.

In a recent publication in the journal Optics Express, researchers from Seoul National University and Harvard Medical School in Boston present a novel light field microscopy system that enables light field microscopy with real-time 3D display.

Abstract: We propose a real-time integral imaging system for light field microscopy systems. To implement a 3D live in-vivo experimental environment for multiple experimentalists, we generate elemental images for an integral imaging system from the captured light field with a light field microscope in real-time.
We apply the f-number matching method to generate an elemental image to reconstruct an undistorted 3D image. Our implemented system produces real and orthoscopic 3D images of micro objects in 16 frames per second. We verify the proposed system via experiments using Caenorhabditis elegans.

In the presented setup, light is emitted from an incoherent light source and passes through the specimen, after which it is transmitted through the objective lens and onto the micro lens array (125 µm lens pitch, 2.5 mm focal length). The coded picture is picked up by a relay lens (in this case, a Canon EF 100 mm f/2.8 Macro USM), and finally imaged at the CCD sensor at the top of the setup. Captured images are first analyzed using a real-time pixel mapping algorithm by Jung et al. (2013), followed by additional image processing (introduced in the paper), to create “elemental images” that are optimized for the display lens array.
On the image output side of the setup, the created elemental images are displayed on a high resolution LCD (IBM 22 inch, 3840 x 2400 pixels, 0.1245 mm pixel pitch), and pass through a 1 mm lens array (3.3 mm focal length) before reaching the eye of the observer(s) as an orthoscopic 3D live view.
The CCD collects image data at a frame rate of 32 fps, but due to current limitations of the pixel mapping algorithm, elemental images can only be computed in real-time at 16 fps.

The new system not only allows real-time tracing of microscopic samples, it also allows multiple viewers to observe the same sample on the display. Be sure to also check out the video links below.

Supplemental videos:
Media 1: Synchronized view of a sample at multiple perspectives
Media 2: Perspective views of reconstructed 3D images with generated elemental image
Media 3: Conceptual video of real-time 3D experiment

More information: Ki J, Jung J-H, Jeong Y, Hong K, Lee B. 2014. Real-time integral imaging system for light field microscopy. Optics Express 22: 10210-10220. http://dx.doi.org/10.1364/OE.22.010210