Research Group Projects and Descriptions

The Camera Culture group is building new tools to better capture and share visual information. What will a camera look like in ten years? How should we change the camera to improve mobile photography? How will a billion networked and portable cameras change the social culture? We exploit unusual optics, novel illumination, and emerging sensors to build new capture devices and develop associate algorithms.

With over a billion people carrying camera-phones worldwide, we have a new opportunity to upgrade the classic bar code to encourage a flexible interface between the machine world and the human world. Current bar codes must be read within a short range and the codes occupy valuable space on products. We present a new, low-cost, passive optical design so that bar codes can be shrunk to fewer than 3mm and can be read by unmodified ordinary cameras several meters away.

Computational photography is an emerging multi-disciplinary field that is at the intersection of optics, signal processing, computer graphics and vision, electronics, art, and online sharing in social networks. The first phase of computational photography was about building a super-camera that has enhanced performance in terms of the traditional parameters, such as dynamic range, field of view, or depth of field. We call this 'Epsilon Photography.' The next phase of computational photography is building tools that go beyond the capabilities of this super-camera. We call this 'Coded Photography.' We can code exposure, aperture, motion, wavelength, and illumination. By blocking light over time or space, we can preserve more details about the scene in the recorded single photograph.

Our goal is to exploit the finite speed of light to improve image capture and scene understanding. New theoretical analysis, coupled with emerging ultra-high-speed imaging techniques, can lead to a new source of computational visual perception. We are developing the theoretical foundation for sensing and reasoning using transient light transport, and experimenting with scenarios in which transient reasoning exposes scene properties that are beyond the reach of traditional machine vision.

We are creating a portable and high-speed tomography machine. We record all the data in a single snapshot without mechanical movement or time-division multiplexing of emitters.

The goal is to build a wearable fabric that supports millimeter-accurate location and bio-parameter tracking at thousands of points on the body. Such a fabric can compute and predict 3-D representations of human activity and use them with closed-loop tactile feedback to augment human performance. This will be able to provide a detailed analysis and control of higher-level human activity. The basic technology uses a new, optical motion-capture method we have recently developed.

We present a new method for scanning 3-D objects in a single shot, shadow-based method. We decouple 3-D occluders from 4-D illumination using shield fields: the 4-D attenuation function which acts on any light field incident on an occluder. We then analyze occluder reconstruction from cast shadows, leading to a single-shot light field camera for visual hull reconstruction.

The ray-based representation cannot be directly used to analyze diffractive or phase-sensitive optical elements. We exploit tools from wave optics and extend the light field representation via a novel "light field transform." We introduce a key modification to the ray-based model to support the transform. We insert a "virtual light source," with potentially negative-valued radiance for certain emitted rays. We create a look-up table of light field transformers of canonical optical elements. The two key conclusions are (i) in free space, the 4-D light field completely represents wavefront propagation via rays with real (positive as well as negative) valued radiance, and (ii) at occluders, a light field composed of light field transformers plus insertion of (ray-based) virtual light sources represents resultant phase and amplitude of wavefronts. For free-space propagation, we analyze different wavefronts and coherence possibilities. For occluders, we show that the light field transform is simply based on a convolution followed by a multiplication operation. This formulation brings powerful concepts from wave optics to computer vision and graphics. We show applications in cubic-phase plate imaging and holographic displays.