Transmitting the sense of touch via the Internet

Compression standard for haptics published.

• With the help of the new HCTI standard, sensations can be transmitted over the Internet.
• The tactile counterpart to JPEG, MP3 and MPEG was developed by a consortium led by the Technical University of Munich (TUM).
• HCTI can be used in tele-surgery, tele-driving or in the gaming industry.

What JPEG, MP3 and MPEG are for images, audio and video, haptic codecs are for transmitting the sense of touch via the Internet. After eight years of standardization work under the consortium leadership of the Technical University of Munich (TUM), a standard for the compression and transmission of the sense of touch has been published for the first time under the name “Haptic Codecs for the Tactile Internet” (HCTI). It lays the foundation for tele-surgery, tele-driving and new online gaming experiences.

When audio or video files are sent over the internet, the process is quite simple from today’s perspective: every 20 milliseconds, a data packet is put together from which information irrelevant to human vision and hearing is already filtered out. This reduces the amount of data. Information is only sent in one direction, to the recipient. When transmitting haptic information, both the sender and receiver play an equal role. To change the position of a robot arm remotely, for example, the user indicates this intention by moving. If the hand on the robot arm grips a tennis ball, the user senses this from a distance. Information must flow in both directions. A global control loop is created, with the commands to the robot in the remote environment and the haptic feedback transmitted back to the user influencing each other. Ideally, the haptic information must now be transmitted in a millisecond, a speed usually used in physical interactions with robots.

To reduce the amount of data to be sent, programs known as codecs encode and decode data for transmission. This enables data to be transmitted efficiently. “The IEEE standard 1918.1.1, recently published for the first time, defines a codec as the standard for tactile data transfer,” explains Prof. Eckehard Steinbach, Head of the Chair of Media Technology at TUM. It records the sensations for movements, namely the positions of the limbs and forces acting on them, and for the sensitivity of the skin, to make it possible to feel paper or metal, for example. These two haptic codecs are supplemented by a standardized protocol for exchanging device properties, known as handshaking, when establishing a connection.

How HCTI works: Up to 4,000 data packets per second

In contrast to image, audio and video compression standards, it was previously common practice to send data packets in both directions up to 4,000 times per second for the transmission of tactile information. “This places very high demands on the communication network that transports the data packets,” explains Steinbach. The advantage of the high clock rate is that the teleoperation is realistic and the transmission is highly robust, even if individual data packets are lost. Nevertheless, the researchers want to reduce the clock rate to around 100 times per second. “That’s close to the human perception threshold,” says Steinbach.

In 2014, a working group within the IEEE Standardization Association was launched with researchers from Imperial College London, New York University of Abu Dhabi (NYU Abu Dhabi), Dalian University in China and the German Aerospace Center (DLR), among others, under the consortium leadership of TUM. The aim was to develop a standard for haptic communication. “The new codec is something like JPEG or MPEG, only for haptics,” explains Prof. Steinbach, who has led the standardization group for the last eight years.

For the new compression standard, called HCTI, the researchers have optimized both the control loop between the sender and receiver and the compression of the data. The crucial feature: Even if data packets are sent over long distances, this should not be noticeable at the other end of the line. “The integrated control has a stabilizing effect. The forces exerted by a distant robot, for example, are slightly dampened. Hard surfaces feel softer,” says Steinbach, describing the now standardized solution, in which Prof. Sandra Hirche and Prof. Martin Buss from TUM were also involved.

Possible applications for the tactile standard

The standard will be of interest for various future applications:

– Telesurgery: The new standard avoids oscillations over arbitrary distances. This means that an operating theater robot can be controlled just as well remotely as onsite. Experts from leading heart centers in Munich or New York, for example, can be called in for certain operations and perform the surgery themselves.

– Teledriving: Companies working on autonomous driving are already offering teleoperation services. Drivers do not sit at the wheel of the vehicle, but in “driving centers” from which they control vehicles remotely.

– Ultrasound in the ambulance: Paramedics can provide first aid but are not authorized to take ultrasound images. In critical situations, this can be done by a doctor while en route to hospital.

– Gaming and film industry: HCTI can bring computer games and films closer to reality and deliver convincing sensations. An exoskeleton can be used to transmit vibrations in a car or centrifugal forces in tight curves, for example.

– Shopping: Online clothing shoppers will not need to have items shipped to them to find out how they feel.

“In the case of JPEG, MP3 and MPEG, many applications emerged after the standards were made public,” explains Prof Eckehard Steinbach from TUM. “I expect the same from our new haptic codecs.”

Further information:
– At the Munich Institute of Robotics and Machine Intelligence (MIRMI), Prof Eckehard Steinbach is one of the board members responsible for start-ups and infrastructure. With MIRMI, TUM has created an integrative research institute with leading expertise in the fields of robotics, perception and data science. The Executive Director is Prof Sami Haddadin. https://www.mirmi.tum.de/

– The CeTi Cluster of Excellence plays an important role in the development of the tactile internet. At the Centre for Tactile Internet with Human-in-the-loop (CeTi), researchers from the Technical University of Munich (TUM) and the consortium leader Dresden University of Technology have been working together since 2019 with the aim of taking collaboration between humans and machines to a new level. https://ceti.one/

– Preparatory work in the SFR453: Collaborative Research Centre: Information sent at the speed of light through fiber optic networks travels a maximum of 300 kilometers in one millisecond. From Germany to Japan, the data would then take 30 milliseconds to travel. And that’s just the pure transmission time. Further delays are caused by the display in the end device and the sensors. Researchers from the Technical University of Munich (TUM) and the Institute of Robotics and Mechatronics at the German Aerospace Centre (DLR) in Oberpfaffenhofen have been working together to shrink the data transmission delay to the point where it is not noticeable to humans, even over long distances. In the Collaborative Research Centre SFB453, funded by the German Research Foundation (DFG), they developed a new generation of haptic codecs back in 2008 that brought two things together – the control for stabilizing “teleoperation in the presence of delays” and “perception model-based compression of haptic data”. These developments were also made possible through the research of the then DLR Professor Gerhard Hirzinger, whose team made data transmissions between the ISS space station and Earth manageable using a stabilization algorithm.

Additional editorial information
Photos for download:
https://mediatum.ub.tum.de/652209?show_id=1747402
https://mediatum.ub.tum.de/652209?show_id=1747401
https://mediatum.ub.tum.de/652209?show_id=1747400

Wissenschaftliche Ansprechpartner:

Prof. Dr. Eckehard Steinbach
Technical University of Munich (TUM)
Chair of Media Technology
Eckehard.steinbach@tum.de

Originalpublikation:

IEEE Standard for Haptic Codecs for the Tactile Internet
https://standards.ieee.org/ieee/1918.1.1/6835/

https://www.tum.de/en/news-and-events/all-news/press-releases/details/transmitting-the-sense-of-touch-via-the-internet