Remote-controlled robotics for surgery applications

What potential do 6G and AI unfold?

We are already experiencing the shortage of physicians and its consequences for patient care. Can mobile surgical robots provide a solution? Combining approaches from robotics with AI methods using the future mobile communications standard 6G, researchers at RPTU and the German Research Center for Artificial Intelligence (DFKI) are investigating this question. Their objective: An analysis that shows potentials of remote-controlled robotics for surgical operations and defines requirements for AI and communication networks. The researchers will be presenting their project at the trade fair “Medica” held from 13 to 16 November at the Rhineland-Palatinate research stand (hall 3, stand E80).

Using robotics during operations is not a fundamentally new idea. What is innovative is the approach of making such systems operable remotely or mobile. Project leader Marc Ruffing, who is doing research at the Institute for Wireless Communication and Navigation at RPTU and in the research area “Intelligent Networks” at DFKI, outlines the challenges involved: “Until now, the person performing the operation has to be present near the operating room, because mobile use of remotely controlled robotics is still impractical due to the size and infrastructure of such systems. Moreover, the control of the systems is not intuitive.” The person who operates uses joysticks and uses a shielded screen. Both natural motion and haptic feedback are missing. Unlike humans, robots are not sensitive enough.”

High latency requirement

Overcoming these hurdles is what Ruffing and his team have made their research task. To do so, they use a demonstrator that consists of two collaborative robot arms and thus represents a classic setup. One robot arm can be controlled by human hands through the guidance of the other. The system enables virtual haptic feedback in the form of force feedback, which is transmitted from the controlled robot arm to the controlling robot arm. This allows the controlling person to sense what he or she is doing. Communication between the two robots takes place via a network. During this process, no large data packets are transmitted – rather, the tricky part is a different one: “Particularly with such highly sensitive activities as remote-controlled operations, there are especially high requirements in terms of latency times. Control commands from the person performing the operation must arrive at the treatment site without any delay,” explains team leader Christoph Lipps. “That’s why we’re using the test scenario to define the requirements that the future 6G mobile communications standard must meet in terms of real-time control.”

More natural control

At the same time, the team is researching how intelligent technologies can be used to improve the system’s operation. Among other things, they are testing a near-infrared-based motion capture system. This allows objects such as a hand and their movements to be captured with millimetre precision in space. This would eliminate the need for unnatural joystick control.

It is also possible to integrate a human-machine interface (brain-computer interface, BCI) into the system. “By measuring an individual’s brain waves using electroencephalography or near-infrared spectroscopy, we can obtain data that provide information about their state of mind,” says Matthias Rüb, a research associate on the team. “An artificial neural network, which is an application from the field of machine learning, is used for the evaluation. It scans the data measured by BCI and assigns states of mind to them. If the doctor’s attention decreases or their stress level increases, for example, a warning message could be sent.”

Contribution to “Open6GHub”

The team will use the findings from the project to contribute to the specifications for real-time remote control of surgical robots. “We are not developing medical devices,” Ruffing summarizes. “We are more concerned with defining requirements for 6G and for AI in order to bring the technology into use. For example, in the form of a mobile operating room or built into an ambulance.”

The researchers will use the demonstrator to convey the status of their research to interested visitors at the Medica trade fair.

The project is part of the “Open6GHub” which is coordinated by Professor Schotten, Head of the Institute for Wireless Communication and Navigation at the RPTU and Head of the Intelligent Networks Research Department at the DFKI. In addition to RPTU and DFKI, other universities and research institutes are also involved. The partners in the research alliance want to contribute to the development of an overall 6G architecture and also to launch end-to-end solutions in the following and other areas: Advanced network topologies with highly agile so-called organic networking, security and resilience, Thz and photonic transmission methods, sensor functionalities in the networks and their intelligent use and further processing, and application-specific radio protocols.

As they move forward, the researchers are open to dialogue and collaboration: “We are looking for an early and interactive dialogue with the public and are equally ready for collaborations with industry and users,” explains Schotten. “To this end, we will install OpenLabs and open experimental fields. Last but not least, we want to promote an open innovation system by involving SMEs and startups and their results.”

Klaus Dosch, Department of Technology and Innovation, is organizing the presentation of the researchers of the RPTU at the Medica. He is the contact partner for companies and, among other things, establishes contacts to science.
Contact: Klaus Dosch, E-mail: klaus.dosch@rptu.de, Phone: +49 631 205-3001

Questions can be directed to:
Marc Ruffing
Institute for Wireless Communication and Navigation
Phone: +49 631 205 75 1826
Email: marc.ruffing@rptu.de

About the DFKI
The German Research Center for Artificial Intelligence GmbH (DFKI) was founded in 1988 as a non-profit public-private partnership (PPP). It combines scientific excellence and business-oriented value creation with social appreciation. DFKI has been researching AI for humans for more than 30 years and is oriented towards social relevance and scientific excellence in the crucial future-oriented research and application areas of artificial intelligence. It is among the most important “Centers of Excellence” in the international scientific community. DFKI runs facilities in Kaiserslautern, Saarbrücken, Bremen and Lower Saxony, laboratories in Berlin and Darmstadt, and branch offices in Lübeck and Trier. Approximately 1,560 employees from more than 76 nations are currently conducting research in the field of innovative software solutions. The financial budget in 2022 was EUR 82.6 million.

https://rptu.de/en/newsroom/detail/news/ferngesteuerte-robotik-fuer-operationseinsaetze-welches-potenzial-eroeffnen-6g-und-ki