Video Presentation: Raytrix demoes live 3D LightField Video and more

Graphics processor- and chipset manufacturer Nvidia regularly hosts an in-house conference titled “GPU Technology Conference” (GTC). The event is primarily targeted at engineers, scientists and journalists. At GTC 2012 Earlier this year, Raytrix co-founder Christian Perwass introduced the company’s LightField cameras, their fields of application and the technology behind the LightField:

Video description (from gputechconf.com):

In this session you will learn what a lightfield camera is, how it works and what you can do with it. Next to the theoretical presentation we give a live demo of the camera system developed by our company Raytrix that gives you 3D live video from a single camera through a single lens currently at up to 10fps with a maximum effective resolution of 3 megapixels synthesized from an 11 megapixel sensor using CUDA algorithms on a GTX580. Post-production features include pixel-wise focusing, depth zoom, variable stereo base-line and base-line rotation.

In his talk, Perwass also demonstrates some features of the Raytrix LightField Software, including mask-based refocus (produced using the reconstructed all in focus image and subsequent blurring according to the depth data), actual LightField refocus (based on the RAW LightField data), re-lighting the picture (adding a virtual light source), and the application of these features to an entire LightField video. Furthermore, the parallax effect (changing the perspective after the fact) 3D stereo viewing, and baseline rotation (rotating the picture data while keeping a 3D effect in “normal” upright orientation of the beholder’s head) are shown.
The best part of the presentation is the demonstration of these features in a live 3D video, which also shows aspects of Raytrix’ software user interface. Depth estimation for each frame (4016 x 2688 pixels on the sensor, 2008 x 1344 effective pixels) takes just 94 milliseconds in this demonstration (using a single core GTX580 graphics card). The depth estimation quality can be adjusted through the software (higher quality takes longer to compute).

In the chapter “Technology”, Perwass presents the advantage of Raytrix’ heterogeneous microlense array compared to standard arrays (as used e.g. in the Lytro camera): It includes microlenses of three different focal lengths (“plenoptics 2.0” or multifocus plenoptics) and increases the effective image resolution to 25 % of the sensor resolution.

Slides from the presentation are available here: Live 3D-Video with a Lightfield Camera – Christian Perwass (Raytrix GmbH)

via, via