New Reciprocal Light Field System Captures and Displays 360-Degree Light Fields

Lightfield recordings contain data much richer than a traditional photo, but even so, they usually only contain a rectangular section of a recorded scene. Further down the light field path, the display of light field volumes is a very separate, ressource-intensive process that requires additional hardware limited in size and viewing angle.
An upcoming research article in the Journal of the Optical Society of America A aims to combat these limitations and introduces a “reciprocal 360-degree 3D light-field image acquisition and display system”.
Ali Özgür Yöntem and colleagues at the University of Cambridge have designed a novel system which uses a combination of lenses and a curved mirror (also known as a catadioptric system) to record and display 360 degree light fields, all in one setup.

Here’s how it works:
For the recording part, incoming light from a 360 degree cylindrical volume is first reflected off a parabolic mirror and into a field lens, which produces a spherical intermediate imaging volume. This intermediate volume is captured using a plenoptic camera, containing a microlens array.
To display a recorded scene, light follows the reverse path: A high-resolution display, with properties matching the camera sensor, generates light which is modulated by an (identical) microlens array and passes through the field lens. From there, the light rays reach the parabolic mirror and are directed back to their original location.
A beam-splitter, situated between the field lens and microlens array, allows switching between recording and displaying light fields.

The proposed optical configuration means that light fields can be captured and reproduced in one system, allowing viewers around the parabolic mirror to see objects in 3D and with parallax, as if they were physically there (like when the original light field was recorded).

To test a real-life version of this reciprocal light field system (see below for photos), the authors had to make some modifications to the proposed design:
Instead of a microlens array, the plenoptic camera was built using a micro photon sieve (mPSA) – an array of pinholes which focus light by means of diffraction and interference – with a Sony A7R II (42 Megapixel full-frame sensor). mPSAs have lower diffraction efficiency but are much more cost-effective.
The display part of the setup had to be scaled up due to the pixel size mismatch between the full frame image sensor and available displays. The adjusted setup utilised a 4K mobile phone display (Sony Xperia XZ Premium), and reproduced only about 25% of the entire captured image. However, the proof-of-concept system successfully demonstrated reconstruction of real images reflected from the parabolic mirror.

The upcoming research article builds upon a recent conference paper describing the general design of the system (Yöntem & Chu, 2018), and adds modelling data, simulations, as well as experimental results.
The full paper (Yöntem et al., 2019) is currently available through “Early Posting” access via institutional subscriptions, and will be published in an upcoming issue of JOSA A.
Update: The full paper (Yöntem et al., 2019) is now available in JOSA A (see below).

References:

• Yöntem A, Chu D. 2018. Design for 360-degree 3D Light-field Camera and Display. Imaging and Applied Optics 2018. OSA Technical Digest: paper 3Tu5G.6.
• Yöntem A, Li K, Chu D. In Press (2019). A reciprocal 360-degree 3D light-field image acquisition and display system. Journal of the Optical Society of America A: 36, A77-A87.

Figures were adapted with permission from Yöntem & Chu, 2018, and Yöntem et al., 2019 and the The Optical Society (OSA).