Helping human and robot firefighters work as a team

Imagine a firefighter scrambling through a burning building, searching for survivors of a devastating explosion. Injured people on the far side of a brick wall, but out of reach. However, the partner on the other side promptly smashes through the wall, clears a path so both can help the survivors. Science fiction perhaps? No, this is exactly the scenario that partners in the PELOTE project have been working on.

Libor Preucil, from the Czech Technical University in Prague and coordinator of this IST project, admits that the participants had the events of 9/11 in New York very much in mind when writing the project. “We wanted to design principles for systems where firefighters could be aided by semi-automated robots, especially in conditions of high hazard or limited visibility.”

Robots as part of the team

To date, says Preucil, robotic assistance in such scenarios has been limited to remotely-operated machines with a camera. “We wanted to see how robots with a certain amount of intelligence could share tasks and communicate with their human partners, despite problematic environments and severe limitations on their capabilities.”

“Robots can operate in conditions that are difficult for human beings,” he says. “They can lift heavier objects, see and navigate better in poor visibility and can withstand higher temperatures. Human beings, however, are better at adapting fast to changing circumstances. Our aim was to enable robot and human to work together in partnership and share their complementary skills.”

Personal navigation system for each team member

PELOTE focused on how human firefighters and their robot counterparts would make use of a personal navigation and localisation system which could guide their movements, at the same time informing the external command centre of the exact location of each team member.

Researchers developed a backpack for the firefighter which uses inertial guidance systems, rather than GPS, to provide the location as shown on a personal display screen, as well as that of the command centre. “The idea is that team members would download a map of the interior before entering the building, and with the start point being calibrated at, say, the entrance, this personal map would enable them to know where they are – no matter how bad the visibility.”

Thanks to this personal backpack, both the mission controller and the firefighters know the exact location of each team member at all times. The team also shares sensor data (e.g. temperature) and video images via wireless communication. If an area is highly polluted with toxic fumes, the mission controller can send in a robot which explores and takes measurements. Another robot can navigate through a part of the building which has been extensively destroyed and where human progress is difficult or dangerous, search for victims and act as a communication platform between the controller and the victim.

A key focus throughout the project was to how to present human firefighters as a telematic entity on the system. This is why the firefighters’ backpack plays a key role in acting as a localisation device, as well as supporting personal navigation. Both mission controller and individual firefighters all see identical information showing the position of each team member on their personal screens. Wireless communication was based on standard Wi-Fi technology, and the technology between the screen representations was based on C++ and Java.

User reactions positive

PELOTE completed at the end of March 2005 with an experimental prototype that was demonstrated to students and volunteer firefighters at the Juliux Maxmilian University of Wurzburg in Germany. “The human demonstrator wore dark-shaded glasses – in effect he was blindfolded,” says Preucil. “As a result he had to rely totally on the information he could see on his personal screen, plus the guidance given by the mission controller, to navigate around the test area.”

Overall user reactions to demonstrations of the system were positive, he says. Some saw the weight of the backpack, around 15 Kg, as a potential issue. However, Preucil emphasises, this was a prototype, and further engineering development would be needed. “What was noticeable was the speeding-up of any rescue mission. This was quite remarkable, especially in environments where visibility is limited by smoke or the shape and structure of the environment is not well known.”

The PELOTE partners now hope to take the concept further within a further research project. “We have done a lot of development work in the basic project, and the next steps for us would be in engineering development, for example making the backpack lighter and the system more robust.”

Preucil notes that the firefighting scenario is just one potential upshot of the technology. Other likely applications include service robots for home environments, remote control of robots in online education, tele-maintenance of industrial transport robots and even planetary rovers.