Coming to grips with robot learning

A ‘living’ artificial hand that learns through imitation has been developed, enhancing human-machine communication and paving the way for novel prosthetic aides.

“The last decade has seen enormous advances in the design and implementation of all kinds of robot platforms,” says Alois Knoll of Technische Universität München and coordinator of the ArteSImit project. “However, their abilities to ‘learn’ how to solve even simple tasks are still very restricted.”

According to Knoll imitation is a powerful shortcut to successful task learning. “If a robot learner is shown a certain behaviour necessary for solving a task by an instructor, the motion alternatives for producing the goal-reaching behaviour with the learner’s own devices and effectors can be dramatically reduced,” says Knoll. “The learner would then try to mimic the coarse motions of the instructor and adapt them to achieve the same goal.”

Exploring the functional and neuropsychological mechanisms of imitation learning was the aim of the ArteSImit project funded by the Future and Emerging Technologies initiative of the IST programme. The overarching goal was to reveal the neurophysiological structures for finger and hand movements in humans and monkeys, and design a computer-operational dynamic model of imitation learning. The consortium constructed a fully functional visual-motor system for a limited domain to control a bio-analogue hand, which is thus capable of mimicking imitation behaviour.

The system implements the full visual serving loop in that it observes the environment with a camera, recognises the instructor’s gestures from a set of predefined gestures, makes the necessary decisions to achieve the goal, i.e. the appropriate sequence of actions of the arm, hand and finger movements. This is optimised in such a way that it is achieved with the minimum amount of motion. The visual system is highly innovative and unique: it recognises finger positions in a precise way through a novel algorithm with just one camera.

“Our final objective was to implement imitational learning on this artificial hand and to suggest applications of the new methods and models to other artefacts with many degrees of freedom that cooperate with humans,” says Knoll.

“We believe that the methods we developed in the ArteSImit project could be directly applied, for example, to new generations of prostheses. These could react for example to finger movements, controlled by a camera applying the concept of imitation, and additionally, they could be linked directly to the nerves.”

Research will be continued within the follow-up project JAST, which took up some of the ArteSImit findings. JAST aims to develop jointly-acting autonomous systems that communicate and work intelligently on mutual tasks in dynamic unstructured environments expanding the concept of group to human plus artificial agents.

Further potential applications of ArteSImit findings are numerous. “It could be applied to control instruments used for medical operations or even to capture and detect a person’s movement within a crowd of people. This method could be interesting for example for security systems,“ Knoll says. Although some of the results were mostly interesting for further basic research, there are some results, e.g. in the area of computer vision and object tracking, that could be phased out and be developed into useful products in their own right.