Research Group Projects and Descriptions

Our group explores how the distribution of computational intelligence throughout video and audio communication systems can make a richer connection between the people at each end. In particular, we seek to build systems which represent content as a collection of meaningful objects accompanied by procedural metadata. To support this vision, we develop not only applications and tools, but also novel content-understanding methods and hardware/software systems.

Connectibles are small "keepsake" objects that are exchanged by individuals, establishing a communication portal between them. As an ecology of objects, the Connectibles system physically embodies a social network; as an ecology of "smart" objects, it can automatically build rich representations of social networks. Any gift from one person to another contains information about the relationship between the giver and the receiver, but this information remains locked away in each object. If we could capture this information, we might expose social networks that have both more detail and greater consonance with a user's "reality-based" social network than current virtual, Internet-based social network applications. We might also open up a new, tangible way to interact with social networks, using as phicons the very same sentimental objects that represent people in the user's social network.

Alumni Contributor(s): Jeevan James Kalanithi

The goal of this project, building upon work begun by Stephen Benton and the Spatial Imaging group, is to create an inexpensive desktop monitor for a PC or game console that displays holographic video images in real time, suitable for entertainment, engineering, or medical imaging. To date, we have demonstrated the fast rendering of holo-video images from OpenGL databases on off-the-shelf PC graphics cards; current research addresses new optoelectronic architectures to reduce the size and manufacturing cost of the display system.

Alumni Contributor(s): Wendy Plesniak and Tyeler Quentmeyer

We are using consumer-grade inkjet printers with shrinkable print media in order to enable the rapid printing of extremely inexpensive holograms and diffractive optics.

We are developing inexpensive, efficient, high-bandwidth light modulators based on lithium niobate guided-wave technology. These modulators are suitable for demanding, specialized applications such as holographic video displays, as well as other light modulation uses such as compact video projectors.

Linear arrays of surface-emitting semiconductor lasers provide the basis for very small, low-power video projectors requiring minimal optics. Such projectors are suitable for handheld devices like phones.

Alumni Contributor(s): Wilfrido Sierra and Ben Dalton

This project examines the creation of modular computational elements which can be used to build smart rooms, linked meeting rooms, and other sensor- and display-equipped intelligent spaces. These units are intended to tile the walls of a room, and act as a scalable, self-organizing system. Each tile incorporates networked communications, sensing, intelligence, and actuation/display. Cells coordinate their operations in order to provide complex sensing and display applications. Most recently this platform has been the basis for a gesture-based distributed telecollaboration and simulation system. This is a joint project between the Media Laboratory and the Information and Communications University (ICU) in Korea.

Alumni Contributor(s): Ben Dalton, Diane E. Hirsh, Jacky Mallett and Arnaud Pilpre

We have built several handheld devices that combine grasp and orientation sensing with pattern recognition in order to provide highly intelligent user interfaces. The Bar of Soap is a handheld device that senses the pattern of touch and orientation when it is held, and reconfigures to become one of a variety of devices, such as phone, camera, remote control, PDA, or game machine. Pattern-recognition techniques allow the device to infer the user's intention based on grasp. Another example is a baseball that determines a user's pitching style as an input to a video game.

uCom (the "u" stands for "ubiquitous") is a follow-on to our iCom (Media Lab/Media Lab Europe, 1999) system for connecting architectural spaces to enable collaboration by distributed groups. uCom takes advantage of input/output resources (e.g., displays, cameras, speakers, sensors) already in place; it is not restricted to one "window" in one location, but rather creates multiple video and audio portals between the spaces; it scales in richness as the number of input and output devices increase; and it has both a real-world and a virtual-world presence. The virtual-world model can be accessed by users in other places, or for replay of past events. uCom also enables the aggregating of sensor data to allow processes to draw higher-level inferences (e.g., for understanding the actions of the community of users), monitoring individual users (e.g., for health or disability reasons), or augmenting the space with additional virtual information.

Using the Eye Society network of robotic cameras, this project seeks to capture three-dimensional representations of static interior spaces for use with holographic displays. The project directs robotic cameras to gather images needed to generate novel 2-D renderings of the scene, directing cameras to move so that they can look behind objects that might be obstructing parts of their view.

Alumni Contributor(s): Jacky Mallett

Mobile devices with cameras have enough processing power to do simple machine-vision tasks, and we are exploring how this capability can enable new user interfaces to applications. Examples include dialing someone by pointing the camera at the person's photograph, or using the camera as an input to allow navigating virtual spaces larger than the device's screen.