Apple’s latest light field patent describes the use of a camera array for immersive augmented reality (AR), live display walls, head mounted displays, video conferencing, and similar applications based on a user’s point of view. The patent application, simply titled “Light field capture”, talks about AR video conferencing where the user’s background can be replaced with other information (e.g. their own view of a scene, or live sports).
The invention also includes concepts including pixel culling (i.e. following the user’s movements and cropping to the interesting parts of the entire camera view), conversion of 3D data to 2D views for the left and right eyes of the second party,
Interestingly, the authors also mention the possibility for a hybrid display/camera-array that would integrate both devices into a single, light-field sensing screen.
Virtual Reality and Augmented Reality (or Mixed Reality) headsets have evolved quite a bit over the last few years. Improvements in resolution, lag, and other factors, have led to new, extremely immersive systems such as the HTC Vive. Hovewer, one missing feature is still holding back the technology:
Generally speaking, most of today’s displays consist of a two-dimensional display that’s placed at a fixed distance from the user’s eyes. This creates a conflict for our eyes and brain, which in the real world are used to a linked adjustment of the angle between the eyes (“vergence”) and the focus plain (“accommodation”). Recent proof-of-concept systems use up to three display planes, allowing us to experience discrete near, mid-range and far layer to focus on, but for a better, more immersive 3D experience we’ll need the ability to experience at almost continuous focal range.
The most promising solution to this problem is light field technology: For instance, Nvidia’s light field display prototype has shown successfully (though at low resolution) that it is possible to construct a light field image that allows placement of multiple objects at different focal planes or virtual distances. The Nvidia prototype uses a microlens array, much like in light field cameras from Lytro or Raytrix. Magic Leap is another company working on light field technology. While the company has teased a head-mounted light field display on several occasions, they have yet to explain how exactly their system works, let alone present a working prototype to the public.
Now, another company has entered the light field space. Head-mounted display maker Avegant has announced a new display that uses “a new method to create light fields” to simultaneously display multiple objects at different focal planes. While all digital light fields have discrete focal planes, according to Avegant CTO Edward Tang, the new technology can interpolate between these to create a “continuous, dynamic focal plane”. “This is a new optic that we’ve developed that results in a new method to create light fields,” says Tang. Continue reading →
Rony Abovitz, CEO of the secretive startup Magic Leap, was expected to reveal the company’s rumoured Augmented Reality headset recently at a TED talk in Vancouver, but canceled a few days before. Instead, the company released a 90 second promo video teasing an AR game that it says is “currently being played at the office”, and it looks pretty awesome.
Video description: Unfortunately, we couldn’t make it to TED, but we wanted to share one of the things that we’d planned to share at the talk. This is a game we’re playing around the office right now (no robots were harmed in the making of this video).
A rather secretive startup from Hollywood, Florida, recently made headlines for raising a spectacular investment for their vision of the next generation of Virtual Reality. Big names like Google, Qualcomm Ventures, Andreessen Horowitz, and others have put together the sum of 540 million US-Dollars for a company called Magic Leap, but the public isn’t even sure what the company is working on.
The official press release reads: “Magic Leap is going beyond the current perception of mobile computing, augmented reality, and virtual reality. We are transcending all three, and will revolutionize the way people communicate, purchase, learn, share and play.”
…and Magic Leap’s website doesn’t provide many details either.
The company is reportedly working on Dynamic Digitized Lightfield Signals” (Digital Lightfield, in short), a “biomimetic” technology that “respects how we function naturally as humans”. What that means precisely, the company doesn’t explain. However, Technology Review has dug up some interesting patent applications by Magic Leap which may give us a glimpse into what convinced their investors: Continue reading →
Earlier this year at Augmented World Expo, Nvidia researcher Douglas Lanman gave a talk about Near-Eye Light Field displays, i.e. electronic glasses which allow users to experience both 3D and depth. When asked about Augmented Reality (AR) applications during the discussion, Lanman noted that creating a set of transparent glasses that would also include microlenses (or something equivalent) but still allow “normal” see-through vision, was a real challenge. He very briefly teased “pinlight displays”, which were to be presented at the same conference, but no further information could be found online.
In the Emerging Technologies section of the Siggraph 2014 conference (10-14 August 2014), Adam Maimone and colleagues from the University of North Carolina at Chapel Hill and Nvidia will be presenting their new invention in a talk entitled “Pinlight Displays: Wide-Field-of-View Augmented-Reality Eyeglasses Using Defocused Point-Light Sources”. Continue reading →