New Reciprocal Light Field System Captures and Displays 360-Degree Light Fields

Lightfield recordings contain data much richer than a traditional photo, but even so, they usually only contain a rectangular section of a recorded scene. Further down the light field path, the display of light field volumes is a very separate, ressource-intensive process that requires additional hardware limited in size and viewing angle.
An upcoming research article in the Journal of the Optical Society of America A aims to combat these limitations and introduces a “reciprocal 360-degree 3D light-field image acquisition and display system”.
Ali Özgür Yöntem and colleagues at the University of Cambridge have designed a novel system which uses a combination of lenses and a curved mirror (also known as a catadioptric system) to record and display 360 degree light fields, all in one setup.

New Reciprocal Light Field System Captures and Displays 360-Degree Light Fields. Fig. 2 in Yöntem & Chu (2018) gives an impression of the prototype design in display mode (a) and shows how the 360-degree light fields are recorded (b) and restored (c). Image adapted with permission from Yöntem & Chu 2018, The Optical Society (OSA).

Here’s how it works:
The proposed design for a 360-degree light field camera and display system uses a beam splitter (BS) to switch between recording (a) and displaying (b). Image: Fig. 3 adapted with permission from Yöntem et al. 2019, Optical Society of America. For the recording part, incoming light from a 360 degree cylindrical volume is first reflected off a parabolic mirror and into a field lens, which produces a spherical intermediate imaging volume. This intermediate volume is captured using a plenoptic camera, containing a microlens array.
To display a recorded scene, light follows the reverse path: A high-resolution display, with properties matching the camera sensor, generates light which is modulated by an (identical) microlens array and passes through the field lens. From there, the light rays reach the parabolic mirror and are directed back to their original location.
A beam-splitter, situated between the field lens and microlens array, allows switching between recording and displaying light fields.

The proposed optical configuration means that light fields can be captured and reproduced in one system, allowing viewers around the parabolic mirror to see objects in 3D and with parallax, as if they were physically there (like when the original light field was recorded).

To test a real-life version of this reciprocal light field system (see below for photos), the authors had to make some modifications to the proposed design:
Instead of a microlens array, the plenoptic camera was built using a micro photon sieve (mPSA) – an array of pinholes which focus light by means of diffraction and interference – with a Sony A7R II (42 Megapixel full-frame sensor). mPSAs have lower diffraction efficiency but are much more cost-effective.
The display part of the setup had to be scaled up due to the pixel size mismatch between the full frame image sensor and available displays. The adjusted setup utilised a 4K mobile phone display (Sony Xperia XZ Premium), and reproduced only about 25% of the entire captured image. However, the proof-of-concept system successfully demonstrated reconstruction of real images reflected from the parabolic mirror.

Schematic and real-life prototype for the recording part - 360-degree 3D light field camera setup. Image: Fig. 10 adapted with permission from Yöntem et al. 2019, The Optical Society (OSA).

Schematic and real-life prototype for the displaying part - 360-degree 3D light field display setup. Image: Fig. 9 adapted with permission from Yöntem et al. 2019, The Optical Society (OSA).

The upcoming research article builds upon a recent conference paper describing the general design of the system (Yöntem & Chu, 2018), and adds modelling data, simulations, as well as experimental results.
The full paper (Yöntem et al., 2019) is currently available through “Early Posting” access via institutional subscriptions, and will be published in an upcoming issue of JOSA A.
Update: The full paper (Yöntem et al., 2019) is now available in JOSA A (see below).

References:

Figures were adapted with permission from Yöntem & Chu, 2018, and Yöntem et al., 2019 and the The Optical Society (OSA).

You may also like...

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.