Light field photography comes with features so revolutionary, it is considered by some to be the next major imaging revolution after the pinhole camera.
With the timeline below, we’re following the history and development of light field imaging, from its earliest stages to today’s technology.
Please note that this page is a work in progress, so it will continue to grow (slowly, but steadily). If you think we’ve missed an important milestone, please let us know in the comments.
For a quick overview of milestone headlines, click on “Light Field Photography” below.
Light Field Photography
- Leonardo Da Vinci: Pyramids of Light
ca. 1530
Leonardo Da Vinci writes about vision and perspective, and describes the distribution of light rays in space, calling them “pyramids of light”:
The body of the atmosphere is full of infinite radiating pyramids produced by the objects existing in it. These intersect and cross each other with independent convergence without interfering with each other and pass through all the surrounding atmosphere; and are of equal force and value—all being equal to each, each to all. And by means of these, images of the body are transmitted everywhere and on all sides, and each receives in itself every minutest portion of the object that produces it.
While these “light pyramids” fill space, intersecting and crossing each other at every point, Da Vinci notes that the image information that is contained in any particular location can be revealed using a pinhole camera (camera obscura):
I say that if the front of a building—or any open piazza or field—which is illuminated by the sun has a dwelling opposite to it, and if, in the front which does not face the sun, you make a small round hole, all the illuminated objects will project their images through that hole and be visible inside the dwelling on the opposite wall which may be made white; and there, in fact, they will be upside down, and if you make similar openings in several places in the same wall you will have the same result from each. Hence the images of the illuminated objects are all everywhere on this wall and all in each minutest part of it.
Sources:
- Codex Forster II, Reale Commissione Vinciana, Rome, 1930-6 (in the Library of the Victoria and Albert Museum at South Kensington)
- J.P. Richter. 1888. The notebooks of Leonardo da Vinci: Vol. 1. New York: Dover (republished 1970).
- E.H. Adelson, J.R. Bergen. 1991. The plenoptic function and the elements of early vision. In Computational Models of Visual Processing, edited by Michael S. Landy and J. Anthony Movshon. Cambridge, Mass.: MIT Press.
- Faraday: Light as Lines of Force
1846
In his 1846 lecture entitled “Thoughts on Ray Vibrations”, Michael Faraday suggests that light and other radiation occurs in “lines of force”, similar to the “lines of magnetic force”. In today’s terms, he suggests that light manifests a field, similar to magnetic fields.
The point intended to be set forth for consideration of the hearers was, whether it was not possible that vibrations which in a certain theory are assumed to account for radiation and radiant phaenomena may not occur in the lines of force which connect particles, and consequently masses of matter together.
(…)
All I can say is, that I do not perceive in any part of space, whether (to use the common phrase) vacant or filled with matter, anything but forces and the lines in which they are exerted. The lines of weight or gravitating force are, certainly, extensive enough to answer in this respect any demand made upon them by radiant phaenomena; and so, probably, are the lines of magnetic force: and then who can forget that Mossotti has shown that gravitation, aggregation, electric force, and electro-chemical action may all have one common connection or origin; and so, in their actions at a distance, may have in common that infinite scope which some of these actions are known to possess?Sources:
M. Faraday. May 1846. “Thoughts on Ray Vibrations”, Philosophical Magazine, S.3, Vol XXVIII, N188, May 1846. via
- Ives: Parallax Stereogram Patent
1903
In 1903, Frederic E. Ives is granted a US patent for a recording and viewing parallax stereograms using a single-lens apparatus that features a line-screen at the image plane, in front of the photographic film, thus recording two separate views for the beholder’s eyes in one exposure.
When viewing these stereograms through a compatible viewing device, another line-screen produces a virtual, parallax view of the recorded scene.
The patented invention is a close predecessor to Gabriel Lippmann’s integral camera (1908), which instead of a lens plus line-screen uses only an x/y-array of lenses, and is also very similar to Adelson & Wang’s single-lens plenoptic camera (1992), which uses a main lens and x/y-array of small lenses.
More details in the original patent: US725567A – Parallax stereogram and process of making same. Google Patents
- Lippmann: Integral Photography
1908
Franco-luxembourgish physicist Gabriel Lippmann introduces “integral photography”. He proposes an analogue multi-lens photo plate that is “reminiscent of the insect’s compound eye, where a large number of crystalline lenses are arranged hexagonally, like the cells of a bee hive”. His setup allows mapping the direction of light rays to their position in the recording. This makes him the inventor of the world’s first analogue light field device, though the term “light field” wouldn’t exist for another 30 years.
By placing a print from such an exposure behind the same lens array, and illuminating it from behind (think “light field display”), he is able to reproduce a virtual, 3D image of the recorded scene, complete with parallax and perspective shift when the observer moves:
If we looked at the photographic print from behind, we could only see a system of small juxtaposed images.
The observer must be on the anterior side, his eyes at an arbitrary distance from the plate, which is being illuminated by diffuse light (applied for example against a frosted glass). In this case, he sees a unique, full-size image that is projected in space.Indeed, during the exposure, the light rays emitted from point “A” converge at the bottom of any cell at an image point of “A” (Fig. 1). This image is revealed and fixed by the photographic process. When we then light up “a”, the rays leaving from “a” will converge to “A” under the principle of reversibility of light.
This applies to all cells that have received rays emitted by “A”: all bearing images “a”, “a”, “a” which have the conjugate image point “A”, all emit light beams which converge towards “A”.Later in 1908, Lippmann wins the Nobel prize for colour photography.
While Lippmann describes the theoretical basics for light field photography, his technique remains experimental due to limitations in optical quality. When he dies in 1921, he reportedly has a working system with only 12 lenses.
Sources:
- G. Lippmann. 1908. Épreuves réversibles donnant la sensation du relief. J. Phys. Theor. Appl., 7 (1), pp.821-825
- K. Timby. 2015. 3D and Animated Lenticular Photography.
- Gershun: The Light Field
1936
Andrey Gershun defines the term “Light Field” in a scientific publication, in which he discusses the radiometric properties of light in three-dimensional space.
We may define the physical field as a part of space, studied from the standpoint of a definite physical process happening within that space. Analogously, we shall introduce the concept of the light field, as a part of space studied from the standpoint of transmission of radiant energy within that space.
Until recent times, photometry limited itself to concepts concerning the emission and absorption of light by bodies, while the transmitting medium was ignored. The older photometric science was a peculiar manifestation of the concept of actio in dislans. The modern study of the light field consists of an investigation of the space-distribution of luminous flux.
Sources:
A. A. Gershun. 1936. The Light Field, Unify Technical Press, 1936. Translated by P. Moon and G. Timoshenko in Journal of Mathematics and Physics, Vol. XVIII, MIT, 1939, pp. 51–151.
via, via
- First Single-lens plenoptic camera
1992
Edward Adelson and John Wang publish work on the first single-lens plenoptic camera, featuring a lens array placed at the sensor plane. In their laboratory setup, the authors combine a 50 mm (main) lens from an SLR camera, and a plano-convex field lens followed by a 2D lens array made by crossing two 64 line-per-inch lenticular sheets (producing 100 x 100 lenses). The image formed at the back of the lens array is reimaged via a 60 mm relay lens (macro lens) to a CCD video camera. The unprocessed light field frames measure 512 x 480 pixels, with an effective resolution of approx. 100 x 100 pixels.
Side notes:
- The term “plenoptic” is coined one year earlier by Adelson & Bergen (1991), from the word roots for “complete” and “view”.
- Note that Frederic Ives developed a similar optical setup, but used a line screen instead of lenses, and thus captured only horizontal parrallax, in 1903 (see above).
Original publication: E.H. Adelson, J.Y.A. Wang. 1992. Single Lens Stereo with a Plenoptic Camera. IEEE Transactions on Pattern Analysis and Machine Intelligence – Special issue on interpretation of 3-D scenes—part II 14: 99-16
- 4-D Light Field Notation, Camera Gantry Setups
1996
In their technical report “Light Field Rendering”, Marc Levoy and Pat Hanrahan devise a simplified mathematical representation of light field data, from Adelson and Bergen’s 7-dimensional plenoptic function (or its shortened 5-dimensional version) down to a 4-dimensional notation.
The 4-D light field gains popularity among scientists, outperforming its predecessors due to the reduced number of dimensions.The report’s experimental part also describes methods for recording light fields, including computer-assisted camera gantries where a single, moving camera takes pictures at known positions, and/or objects are rotated for recording.
More details: M. Levoy, P. Hanrahan. 1996. Light field rendering, Technical report, Stanford University.
- The Largest Known Light Field is recorded
1999
Marc Levoy and Jonathan Shade record the largest light field known: A light field volume of Michelangelo’s statue of Night in the Basilica di San Lorenzo in Florence, Italy is recorded through a grid of 62 x 56 viewpoints in 7 light field slabs, resulting in 24,304 individual pictures of 1.3 Megapixels each. The entire dataset amounts to 16 GB of 1300 x 1030 RGB JPEG files.
More information: A light field of Michelangelo’s statue of Night
- Camera Arrays for Light Field Capture
2002
Jason Yang and colleagues at the MIT Laboratory for Computer Science develop a “Real-Time Distributed Light Field Camera”, an array of 64 commodity webcams (8 x 8 OrangeMicro iBot firewire video cameras) and a 1.5 GHz compositor computer that is used to render live dynamic light fields. The system generates approx. 18 frames (320 x 240 each) per second. Light-field capture time is reduced greatly when compared to camera gantry systems, but the current setup is lacking the ability to synchronise image recording sufficiently to capture fast-moving dynamic scenes.
Nevertheless, the researchers successfully demonstrate software refocus (despite rather severe, uncorrected rendering artefacts) and stereo 3D output.
Original publication: J. Yang, M. Everett, C. Buehler, L. McMillan. 2002. A realtime distributed light field camera. In Proceedings of Eurographics Workshop on Rendering 2002, 77–86.
Another notable light field setup is Bennet Wilburn et al.’s “Multiple Camera Array” at Stanford University. Development of this camera array started in 1999, and a comprehensive study was published in 2005.
The setup consists of 100 Omnivision 860 image sensors with either Sunex DSL841B lenses (6.1mm focal length, f/2.6, 57° diagonal field of view) or Marshall Electronics V-4350-2.5 lenses (50 mm focal length, f/2.5, 6° diagonal field of view), respectively, depending on the application. Wilburn’s array records VGA videos (640 x 480 pixels) from 100 cameras at 30 frames per second.
The study presents several imaging applications, from high-dynamic range and high resolution rendering, to high-speed video, matted synthetic aperture (the researchers demonstrate filming people through a dense cover of foliage). While the publication itself doesn’t talk about processing and generating light fields per se, the setup is effectively recording light fields, and is further used for light field technology.
More details: B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Levoy, M. Horowitz. 2005. High performance imaging using large camera arrays. 765–776. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2005).
- Hand-held Light Field Camera
2005
During his PhD research, Ren Ng combines the idea of software refocus with the physical concept of a light field camera in 2005. He further miniaturizes light field technology (most notably, Adelson’s single-lens plenoptic setup) and develops the first hand-held light field camera. Ng’s plenoptic camera prototype is a modified Contax 645 medium format camera with Kodak KAF-16802CE 16.8 Megapixel sensor, which has a 296 x 296 microlens array (part 015-0.5-s made by Adaptive Optics Associates) placed 500 µm (= microlens focal length) in front of the image sensor.
Recorded light field images have an effective resolution of 296 x 296 pixels and directional resolution of 12 x 12 at each pixel.
In the following years, Ng goes on to found Refocus Imaging, which would later turn into Lytro, Inc. Read more about Lytro’s history and developments in our Lytro Timeline.
More information:
- R. Ng. 2005. Fourier slice photography. ACM Trans. Graph. 24: 735–744.
- R. Ng. 2006. Digital Light Field Photography. PhD thesis, Stanford University.
- Light Field Photography with a Hand-Held Plenoptic Camera (project page)
- Raytrix R11: First Commercial Plenoptic Camera
2010
German LightField specialist Raytrix announces the world’s first commercial plenoptic camera, the Raytrix R11. The first version of their light field camera uses a 35mm CCD-sensor (10.7 Megapixels, 9 µm square pixel size) to record up to 3.5 monochrome pictures per second, with 40,000 microlenses and an effective resolution of 3 Megapixels. The R11 measures 8.9 x 7.8 x 10.4 cm, weights in at 900 g, and features a USB 2.0 interface and F-mount for a variety of lenses.
The product is targeted at industry and research sectors, and pricing is reportedly in the range of 20,000 to 30,000 Euros.
The cheaper R5 model, announced later that year, costs just over 10,000 Euro for the camera, various accessories, and refocusing software.
Related articles from 2010/2011:
- Lytro: First Consumer Light Field Camera
2012
After making headlines worldwide, announcing the world’s first light field camera for the consumer market in 2011, Lytro ships their first cameras in February of 2012.
The Lytro Light Field Camera is a cuboid form-follows-function camera with 1 Megapixel effective resolution and a constant f/2 lens with 8x optical zoom (35 – 300 mm focal length).
The camera sells for 399 (8 GB internal storage) or 499 US-Dollars (16 GB), respectively, and comes with a free Mac application for processing and online sharing.More details are available on our Lytro Light Field Camera product page the Lytro Timeline, and in Lytro’s 2011 product announcement: Lytro, Inc. Unveils the World’s First Consumer Light Field Camera
Recent Comments