Photography is just one possible application of LightField Technology. No wonder, that movie corporations are also very interested in its potential. Computer animation studio Pixar is one of these companies, and they’ve applied for several LightField related patents starting in 2010:
In the three most notable patent applications with the US Patent and Trademark Office, entitled “Super Light-Field Lens”, “Super Light-Field Lens and Image Processing Methods”, and “Super Light-Field Lens with Doublet Lenslet Array Element”, Pixar presents plenoptic lens systems that can be combined with conventional cameras to create light field imaging systems, as well as methods necessary to process the recorded rich LightField data.
Pixar isn’t just interested in the resulting 2D images, right eye/left eye 3D images, or refocus, depth of field and perspective shift capabilities (“moving the camera angle”). They also talk about rendering and manipulating images, which is probably where their animation expertise comes in.
In the patent application “Super Light-Field Lens” (March 4 2010, application no. 20110169994), the authors describe a method to create LightField lenses that can be “mated with a variety of conventional cameras (e.g., digital or photographic/film, image and video/movie cameras, and DSLR cameras) to create light field imaging systems”.
Such a Super LightField Lens may contain a field lens, microlense array (or lenslet array) which is located proximal to the focal plane of the prime lens, and a “relay optical element” which then gathers and projects the LightField data onto the ordinary image sensor of, say, a DSLR camera.
Submitted a year later, “Super Light-Field Lens and Image Processing Methods” (April 4 2011, application no. 20110273609) presents ways to compute the acquired LightField data created with a microlense array.
Using depth information, the resulting “adjusted image” allows 2D and 3D representation, adjusting the focus, changing the camera angle or virtually moving the camera position forward or backwards, tilting and panning, or zooming the camera. All of these possibilities may be executed for single images as well as videos.
Finally, in “Super Light-Field Lens with Doublet Lenslet Array Element” (April 4 2011, application no. 20110249341), Pixar researchers discuss coupling two microlens arrays between the field lens and relay element that, again, collects and projects the resulting data onto an ordinary sensor.
The two lenslet arrays, close in proximity to each other, may differ in lens type (positive/negative), materials used (fused silica, borosilicate, or fused quartz) and/or glass type (crown glass, flint glass).
Pixar’s approaches differ from other solutions, in that the lens is actually what collects the LightField data, independently of the sensor used.
Disney’s Research Website shows that these patent applications were not just short-lived projects: David DiFrancesco, principal author of the patent applications, is stated to be “researching instrumentation to capture lightfields for use in 3D cinematography and videography” at the Pixar Research Group in Emeryville.
But what is Pixar’s interest in recording actual video, rather than just animating their content? The answer to that question is two-fold. For one, much of Pixar’s portfolio is based on motion capture, e.g. the recording of actual human (and other physical) movement to create a 3D model that can later be overlayed with animated content.
On the other hand, award-winning director and animator Brad Bird recently stated he was “interested in moving into the live action realm with some projects” while “staying at Pixar [because] it’s a very comfortable environment to work in.1”
Recent Comments