News » Cover Story

A system that combines a humanoid robot with a hacked XBox Kinect helps autistic children learn

How Soon Is NAO



Vanderbilt professor Nilanjan Sarkar didn't set out to design a therapeutic robot for autistic children. He had been working in the area of man-machine interaction — trying to improve how robots are able to detect and respond to human emotions and implicit nonverbal cues. But he was focused on physiological responses like heart rate increase and designing machines that could correctly interpret and respond to them.

Then Sarkar spent some time with a young relative in India who happens to have autism. And as he learned more about autism spectrum disorder, Sarkar recalls, one aspect stood out: "One of the salient deficits is [that autistic children] cannot understand emotion," he says, "and they cannot express their own emotion." In Sarkar's mind, that connected with some research he had read about, showing that many children with autism are fascinated by robots. "Since we are working with robots and human computer interaction with emotional cues," he remembers thinking, "can this system be designed specifically for children with autism so they can learn some skills?"

So Sarkar and his team bought and reprogrammed a NAO (pronounced "now") — a French-made humanoid robot. In the first round of experiments, NAO helped autistic children improve an ability known as joint attention (appropriately coordinating attention with surrounding people and objects) by pointing, saying the child's name and adjusting the intensity of its cues depending on how quickly the child responded. Though limited in scope, the initial study showed that over time, the children retained their interest in the robot therapist, and their performance in joint attention skills improved. A paper detailing their findings has been tentatively accepted for publication by the Journal of Autism and Developmental Disorders.

Now in its third iteration, the system also utilizes a reprogrammed Kinect — an accessory for Microsoft's Xbox gaming system that can detect motion and translate it digitally — and will focus on imitation learning, another area of deficit for autistic kids. Sarkar hopes to eventually get to a point where the system is affordable for families. (The NAO robot used in the experiments costs $12,000.)

"Our goal is not to have them hooked up with the robot all their life," Sarkar explains. "Ultimately, the robot serves as a mediator. Initially, get [the child's] attention, make them learn some skills, and slowly another caregiver or peer comes into interaction. That's the holy grail in the end. We are not there yet."

But they've come pretty far already.

More Innovations ...

ESL to Go Truck
The Bridge Program
Mini-Maker Faire
Institute for Digital Learning
KidSense Car Seat
Institute for Software Integrated Systems
Ingram Barge Pilot program
Innovation Update

Add a comment