Sony Patents Light Field Sensor with Full-Resolution 3D Stereo Output

by markus · 2014-07-01

With today’s light field sensors, extracting 3D stereo images from light field recordings typically results in a lowered effective image resolution – but that limitation may soon be history: Sony has developed a novel sensor design with overlapping pixels in two layers, that will allow 3D output without the typical decrease in image resolution. In Sony’s recently granted US Patent, Nr. US20140071244, author Isao Hirota introduces a dual level microlens array setup in combination with a sensor that consists of two layers of light sensitive pixel grids – front-facing and back-facing grids that are rotated at, for example, 45 degrees.
The described configuration allows different neighbouring pixels to share the same information from a single microlens while being allocated to either the left or right stereo views, resulting in higher-resolution 3D stereo output from a single-lens, single-sensor device (i.e. a “monocular 3D stereo camera”).

Fig. 8 from the patent application shows an example setup which uses both a multilens array (34) as well as an additional on-chip lens array (OCL, 35) and color filter array (33) to create a stereo light field image.

Fig. 18 from the patent application is a diagram illustrating a CMOS image sensor in which pixels are arranged in a matrix of 2x2 and, for the second layer of pixels, rotated by 45 degrees for multiple perspectives output in the Bayer arrangement.

Fig. 19 from the patent application illustrates how the setup in Fig. 18 allows for nine perspective images (incl. 6 actual parallax images and 3 interpolated ones), making the setup suitable for a monocular 3D stereo camera.

Fig. 27 provides examples of electric potential distribution of the light receiving face (back face) in a typical square (A) and triangular (B) pixel, showing that the microlenses can be circular, ellipsoidal or polygonal shapes to improve the lenses' extinction ratios.

Patent abstract:

There are provided a solid-state image pickup device and a camera system that include no useless pixel arrangement and are capable of suppressing decrease in resolution caused by adopting stereo function. A pixel array section including a plurality of pixels arranged in an array is included. Each of the plurality of pixels has a photoelectric conversion function. Each of the plurality of pixels in the pixel array section includes a first pixel section and a second pixel section. The first pixel section includes at least a light receiving function. The second pixel section includes at least a function to detect electric charge that has been subjected to photoelectric conversion. The first and second pixel sections are formed in a laminated state. Further, the first pixel section is formed to have an arrangement in a state shifted in a direction different from first and second directions that are used as references. The second direction is orthogonal to the first direction. The second pixel section is formed in a square arrangement along the first direction and the second direction orthogonal to the first direction.

For more information, check out the full patent details here: Patent US20140071244 – Solid-state image pickup device and camera system