Research Group Projects and Descriptions

Robots are an intriguing technology that can straddle both the physical and social world of people. Inspired by animal and human behavior, our goal is to build capable robotic creatures with a "living" presence, and to gain a better understanding of how humans will interact with this new kind of technology. People will physically interact with them, communicate with them, understand them, and teach them, all in familiar human terms. Ultimately, such robots will possess the social savvy, physical adeptness, and everyday common sense to partake in people's daily lives in useful and rewarding ways.

We are developing a novel interface for controlling the behavior of physical (e.g., personal robots) or virtual (e.g., animated agents such as in Second Life). As the morphologies of these avatars become more sophisticated, it becomes more difficult to convey, compellingly and effectively, the remote human's communicative intent while mitigating cognitive load. Puppeteering devices such as motion-capture systems can control all joints of a robot, but are too expensive for personal use; gamepads are affordable, but are often unintuitive and difficult to learn and master. We are developing an intuitively understandable and affordable device to control personal robots such as the Huggable and Leonardo, as well as sophisticated avatars in virtual worlds. A new puppeteering device can control an avatar by capturing a human operator's motion directly through an IR vision-based technology as well as other wearable sensors such as low-cost, 6-axis intertial measurement units. This multi-modal, real-time data can be used to recognize the intentions of an operator's movements to evoke compelling animations or sound effects.

AUR is a robotic desk lamp—a collaborative lighting assistant that sheds light on the right thing at the right time. It serves as a platform to investigate notions of fluency in joint action, helpfulness, and timing. Through its movement and change of color and light intensity, it is also aimed to evoke a personal relationship with its human partner without resorting to human-like features, encouraging us to rethink the inanimate.

The robot AUR, a robotic desk lamp, played a character part in Rony Kubat's short play "Talking to Vegetables" alongside two human actors, using a novel hybrid control interface for robotic theater acting.

The Huggable is a new type of robotic companion for health care, education, and social communication applications. The Huggable is designed to be much more than a fun, interactive robotic companion; it is designed to function as an essential team member of a triadic interaction. Therefore, the Huggable is not designed to replace any particular person in a social network, but rather to enhance that human social network. The Huggable is being designed with a full-body sensitive skin with over 1500 sensors, quiet back-drivable actuators, video cameras in the eyes, microphones in the ears, an inertial measurement unit, a speaker, and an embedded PC with 802.11g wireless networking. An important design goal for the Huggable is to make the technology invisible to the user. You should not think of the Huggable as a robot but rather as a richly interactive teddy bear.

Alumni Contributor(s): Daniel Bernhardt (Cambridge University) and Kuk-Hyun Han (Samsung)

The goal of this project is to research and test existing compact methods of actuation that are viable for robot applications, and to develop new actuators that will augment the performance of robots intended to interact with people. Metrics for performance include power and force density, controlability, smoothness of motion, ease of implementation, cost, and shape. Current explorations include the use of long-travel voice coils as drop-in replacements for DC motors or pneumatic cylinders, and the development of miniature hydraulic actuators to combine the high force-density of hydraulics with the high power-density of electromagnetic actuators.

Alumni Contributor(s): John McBean

We are progressing in the development of a synthetic skin capable of detecting pressure and location with acceptable resolution over the entire body, while still retaining the look and feel of soft skin. We are particularly interested in having a robot recognize the affective content of touch. We are creating a tactile sensing system where a distributed grid of quantum tunneling composites are placed over the robot's core and under its silicone skin or fur. Using the homunculus distribution of sensing resolution as a guide, we are varying the density of sensors so that the robot will have greater resolution in areas that are frequently in contact with objects or people. We are developing a distributed network of tiny processing elements to lie underneath the skin to acquire and process the sensory signals. These sensing elements will cover the entire body of our Huggable platform, a robotic teddy bear intended for therapeutic applications for the elderly in assisted living situations and for children in hospitals.

Communication technologies today fail to produce remote physical presence. By controlling a robotic avatar in a remote location, we can produce this presence. However, Internet latency, synchronization, and the cognitive load of operating a high-tech robot can complicate this interaction process. The Huggable project solves these problems through its unique Web interface that allows for the puppeteering of a multiple degree-of-freedom robot. This interface empowers the avatar's operator with low-level control (direct manipulation of the robot's limbs) and high-level control (initiating long sequences of actions at the click of a button). The interface communicates how the robot is being interacted with and allows the operator to look through robot's eyes, speak through its mouth, and hear through its ears. Our interface targets a wide variety of users, ranging from grandparents to grade school teachers to experts.

Alumni Contributor(s): Robert Toscano

A storytelling system designed to enhance the experience of reading a children's story. An animation engine visualizes the story, allowing the storyteller and child novel ways to experience each retelling of the story. The story visually express the lexical changes by the storyteller.

The Sociable Robots project aims to build capable and appealing robots that can work collaboratively with, communicate with, and learn socially from people. In a unique collaboration with Stan Winston Studio (the creators of "Teddy" in the Kubrick/Spielberg movie A.I.), this project seamlessly merges artistry of character, engineering of robotic technology, and the science of artificial intelligence and psychology to develop robots with social intelligence.

Alumni Contributor(s): Andrew Brooks, Matt Hancher and Andrea L. Thomaz

Many new applications for robots require them to work alongside people as capable members of human-robot teams. These include—in the long term—robots for homes, hospitals, and offices, but already exist in more advanced settings, such as space exploration. A robotic member of such a team must be able to work towards a shared goal and be in agreement with the human as to the sequence of actions that will be required to reach that goal, and to adjust dynamically its plan according to the human's actions. We are researching the social and psychological workings of teamwork, and working towards equipping humanoid robots with the social skills needed to perform as useful team members in human-robot teams.

Alumni Contributor(s): Andrea L. Thomaz

With the aim of allowing professional actors to naturally control robotic film characters, we are developing a novel puppeteering interface that uses—among others—the actor's neck and eye movement to control a robotic character's gaze behavior. We are developing a head-mounted hardware interface, as well as software combining computer vision, pattern recognition, robotic control, and synthetic character animation to create a transparent interface that will—for the first time—allow a single actor to control the behavior of a whole animatronic character.

Robotic systems that aim to collaborate effectively with humans in social environments must be able to respond flexibly to the intentions of their human partners. Dynamic environments may further require robots to respond intelligently to the actions of humans with false or incomplete situational beliefs. We are developing an integrated architecture which incorporates simulation-theoretic mechanisms to allow a robot to infer the task-related beliefs and intentions of its interaction partners based on their observable motor behavior and visual perspective. We demonstrate the performance of this architecture on a set of novel benchmark tasks requiring our robot to exhibit appropriate collaborative behaviors in the presence of potentially false beliefs. We compare our results against human performance on similar collaborative tasks.

Alumni Contributor(s): Andrea L. Thomaz

Robots can currently perform animations, as well as brute imitations of user input (typically given through telemetry devices). More recently, humanoid robots have begun to learn motor tasks through imitation. But, as of yet, no robot has the ability to learn new manipulation skills by watching a user perform those tasks, with any understanding of what the task is accomplishing—the intent of the task. We intend to teach our robot Leonardo how to manipulate objects in goal-directed ways through human demonstration. With a higher-level control system, Leo will be able to watch a user perform a task several times, and slowly take over control of his own body as he gains confidence in the task at hand.

The ability to interpret demonstrations from the teacher's perspective plays a critical role in human learning. Robotic systems that aim to learn effectively from human teachers must similarly be able to engage in perspective taking. We are devloping an integrated architecture wherein the robot's cognitive functionality is organized around the ability to understand the environment from the perspectives of both a social partner and itself. To better understand perspective taking in humans, we are examining its importance in human learning, and have found that it focuses the agent's attention on the subset of the problem space important to the teacher. This constrained attention allows the agent to overcome the ambiguity and incompleteness often present in human demonstrations, thus learning what the teacher intends to teach. We are developing our architecture to use perspective in similar ways, to allow the robot to learn correctly in ambiguous teaching situations.

Alumni Contributor(s): Andrea L. Thomaz

Social referencing is the tendency to use the emotional reaction of another to help form one's own affective appraisal of a novel situation, which is then used to guide subsequent behavior. It is an important form of emotional communication and is a developmental milestone for human infants in their ability to learn about their environment through social means. We have implemented a biologically inspired computational model of social referencing for our expressive, anthropomorphic robot. Our model consists of three interacting systems: emotional empathy through imitation, a shared attention mechanism, and an affective memory system. These systems interact to enable the robot to demonstrate social referencing behavior similar to that of human infants. This work is an important milestone towards social learning in robots. Additionally, our model presents opportunities for understanding how these mechanisms might interact to enable social referencing behavior in humans.

Alumni Contributor(s): Andrea L. Thomaz

Learning by human tutelage leverages structure provided through interpersonal interaction. Teachers direct learners' attention, structure experiences, support learning attempts, and regulate the complexity and difficulty of information. Our approach to machine learning takes tutelage as its model. We are studying how social guidance interplays with traditional inference algorithms (such as Bayesian hypothesis testing) in an interactive-learning scenario. In our demonstration, the robot Leonardo pays attention to verbal guidance as well as nonverbal spatial cues that human teachers naturally provide. The robot communicates its current understanding through demonstration and expressive social cues. The human can quickly and effectively help the robot to learn action sequences and secret constraints associated with an interactive construction task. The robot integrates these learned components via hierarchical planning, taking advantage of the human's social guidance to successfully complete various puzzles.

Alumni Contributor(s): Guy Hoffman and Andrea L. Thomaz

The fusion of "intelligence" and "gentleness" is the foundation of Toyota's partner robot project. Currently, Toyota is focused on implementing a partner robot to enhance the interactive partner robot experience, for example, for patients in hospitals. "Social graces"—the ability of a partner robot to interact with a person in a socially skillful and pleasant manner that is likeable and engaging—is fundamental to the realization of human and partner robot coexistence. To this end, Toyota proposes a collaborative research initiative with the Media Lab to examine how expressive face and neck movements of a partner robot contribute to a human's perception of the "social graces" of a robot using the Media Lab's MDS platform.

The MeBot is a semi-autonomous, mobile phone robotic avatar that allows the caller to better control its presence in an interactive way in front of a receiving audience. A lot of emphasis is put on the robot conveying the non-verbal channels of social communication. It will take advantage of the current advanced technology in wireless communications and the ever-expanding capabilities of mobile devices. MeBot is a push toward a future where remote presence can be achieved easily in a way that saves traveling time but still achieves the same experience as "being there." We propose to do this by means of robot-mediated presence.

People on the autism spectrum face a number of challenges, including motor movement issues that can cause limbs to cease activity. Circumstantial evidence suggests that autonomic nervous system influences related to stress and overload may arise from and contribute to these problems. We propose to allow individuals to monitor several physiological parameters to see if there are patterns that recognize or predict the onset of their individual motor problems. We plan to develop new, wearable technology to treat these problems via the use of tiny, vibrotactile devices carefully placed at the joints. We hypothesize that some methods of touch-feedback and vibration at the joints may enable individuals to recover motor functioning during episodes of intermittent loss. We are also exploring the development of personally controlled devices that facilitate finer motor movement for augmenting communication as needed for assisting in typing or pointing.

Persuasion is a fundamental part of human-human interaction, though very little is known about how this social concept applies to human-robot interaction (HRI). The goal of truly sociable robots requires a deep understanding of how people perceive and respond to robots across the spectrum of social interaction. A recently completed study at the Museum of Science in Boston explored the way in which people perceive, and are influenced by, the MDS robot.

The vision in Human Robot Interaction is for autonomous robots to have the ability to understand common-sense behavioral patterns. We have developed a multi-modal online game as a method of learning these common-sense behavior patterns. In this two-player game, the players collaborate to create a salad through selection and discussion of available items.

We are demonstrating how a heterogeneous group of robots (MDS and helicopters) can be used as a first disaster response before human teams enter the perimeter. The goal is to have robots engage in the search for victims, build a map of the environment, and report back to human operators at a safe distance through a wireless link. Information collected by the robots is displayed on a remote operator interface in real time so that human personnel can easily create new tasks that reflect the state of the environment. The autonomous helicopters scan the area for potential interest points, while the MDS robots can engage in a dialogue with victims to get direct advice about how to structure their search for more victims. The demonstrated system is semi-autonomous and allows the human operator to be in the loop during the entire search. The robots possess enough autonomy to reduce cognitive load on the operator as much as possible. Results are presented with simulated versions of MDS and helicopter units in the USARSim simulator.

A robotic computer that moves its monitor "head" and "neck," but that has no explicit face, is being designed to interact with users in a natural way for applications such as learning, rapport-building, interactive teaching, posture improvement, and to explore how embodiment and affect interact with cognitive performance. In all these applications, the robot will need to move in subtle ways that express its state and promote appropriate movements in the user, but that don't distract or annoy. Toward this goal, we are giving the system the ability to recognize and have subtle expressions.

Alumni Contributor(s): Guy Hoffman and Alea Teeters

Humans are fundamentally social animals. Why not design cars to leverage this natural propensity for social interaction and understanding? We are working with Audi and the SENSEable City Lab to redefine the relationship between car, driver, and passengers. We are currently developing a new type of in-car system that acts as a partner or friend, providing important information, and intelligently responding to the mood and behavior of the driver.

Squash-Stretch is an innovative platform aimed at new types of interaction with children through the use of cartoon-animation-style movement. A mix of active and passive compliant components enables this new platform to generate more organic movement while standing up to the physical toll induced by children. Potential applications are numerous, but include language learning and early-stage autism diagnosis.

Two people repeatedly performing an activity together naturally converge to a high level of coordination, resulting in a fluent meshing of their actions; we seek a more fluent meshing of human and machine activity. Toward this goal, we have developed an adaptive, anticipatory action-selection mechanism for a robotic teammate. We have analyzed our model in a cost-based framework of coordinated shared-location action, and have compared it to a purely reactive agent, demonstrating a theoretical improvement in efficiency. Using an online game, we have tested the performance of the algorithm in a group of untrained human subjects working with a simulated version of a robot (named Symon) using our anticipatory system. We found significant improvements in task efficiency when compared to a group working with a reactive agent, and a significant difference in several measures of the perceived commitment of the robot to the team and its contribution to the team's fluency and success. Grounding these perceptions in behavioral measures of the human-robot team, we found that the groups differ significantly in a number of proposed fluency metrics including amounts of concurrent motion of human and robot and of human idle time, and the time between the human's action and the robot's uptake on it.

TIKL is a wearable robot system that uses real-time vibrotactile feedback to accelerate the learning of movement skills. Expert and novice movements for a specific motor skill are recorded using a VICON optical motion-capture system to millimeter accuracy. In real time, TIKL compares the novice attempt to the expert model to vibrate small actuators embedded in a Lycra suit worn by the novice. The sequence of vibrations cues the novice how to adapt their joint rotation or flexion to reduce the error for that degree of freedom. We have demonstrated that the addition of vibrotactile feedback results in a significant learning improvement (improved steady state performance in learning rate) over visual feedback alone.