CrowdCam: Watch a Scene from any Angle, using Everybody’s Cameraphones

Events such as concerts, public performances or weddings have two things in common: Virtually everybody’s taking pictures, and it can be quite unsatisfying to be in the wrong spot. What if you could just switch your perspective to someone’s in the first row in order to get the perfect view, or even move around a scene as you like?

CrowdCam is a smartphone app concept by Aydin Arpa (MIT) and colleagues, designed to do that and more, using everybody’s smartphone cameras: The app, which is currently in development, compares photos taken at the event and estimates the different angles between camera views. It then arranges these pictures according to their relative location in the scene, giving users the ability to swipe between different points of view, while stabilizing the image and transition and keeping the image centered on the main object of interest.

In other words, the app creates a collaborative network of cameras and views, allowing you to find the best view and virtually move around in any scene.

Publication abstract:

We present a near real-time algorithm for interactively exploring a collectively captured moment without explicit 3D reconstruction.
Our system favors immediacy and local coherency to global consistency. It is common to represent photos as vertices of a weighted graph, where edge weights measure similarity or distance between pairs of photos. We introduce Angled Graphs as a new data structure to organize collections of photos in a way that enables the construction of visually smooth paths. Weighted angled graphs extend weighted graphs with angles and angle weights which penalize turning along paths. As a result, locally straight paths can be computed by specifying a photo and a direction. The weighted angled graphs of photos used in this paper can be regarded as the result of discretizing the Riemannian geometry of the high dimensional manifold of all possible photos.
Ultimately, our system enables everyday people to take advantage of each others’ perspectives in order to create on-the-spot spatiotemporal visual experiences similar to the popular bullet-time sequence. We believe that this type of application will greatly enhance shared human experiences spanning from events as personal as parents watching their children’s football game to highly publicized red carpet galas.

Two main challenges may decide whether this concept can be turned into a successful product: All pictures to be included in one scene need to be recorded at the same time (in the case of specific events rather than a sightseeing tour), and there must be a certain density of participating users in order to create an interesting multi-perspective view.

More information: Arpa A, Ballan L, Sukthankar R, Taubin G, Pollefeys M, Raskar R (2013) CrowdCam: Instantaneous Navigation of Crowd Images using Angled Graph. International Conference on 3D Vision, Seattle WA

What do you think about this concept? Would you join the crowd, once the app is ready?

via