3D technologies for human-machine interaction

Making human-machine interaction more efficient and safer with the help of innovative 3D technologies – that was the goal of the “3Dsensation” research alliance. After eight years, the joint project funded by the German Federal Ministry of Research and Education (BMBF) with 45 million euros, is now coming to an end. Together, the partners from research, industry and business can look back on groundbreaking developments within the alliance – such as health monitoring for newborns, forgery-proof personal identification or merchandise management in retail.

Machines have long since become irreplaceable partners in our everyday lives. It is not only in large production facilities or complex industrial manufacturing processes that human and machine-based processes are closely intertwined. In our private lives, too, assistance systems are playing an increasingly important role in ensuring our safety and comfort. 3D technologies are an essential basis for enabling interaction between machines and their environment.

Eight years, 81 partners and 77 projects with over 200 subprojects

Against this background, the “3Dsensation” research alliance has been dedicated since 2013 to the challenge of developing innovations for even safer and more efficient interaction between humans, machines and robots in the future. The alliance was coordinated at the Fraunhofer Institute for Applied Optics and Precision Engineering IOF in Jena, Germany. In close cooperation with at times up to 81 cooperation partners from science, business and industry, 77 research projects with over 200 subprojects were implemented in the alliance during its eight-year term.

“The goal of our research alliance was to combine strengths of man and machine,” sums up Prof. Dr. Andreas Tünnermann, head of Fraunhofer IOF as well as chairman of the steering committee. “This means flexibility and creativity on one hand and innovation and power on the other. Starting from a concrete technological challenge, we as an alliance together with our partners have developed into a consortium that has worked in an extremely applied and interdisciplinary manner and thus ultimately addressed socially relevant issues,” Tünnermann continues. “Especially due to the Covid-19 pandemic, the technologies we have been researching since 2013 have experienced a particular boom – for example, the contactless measurement of biometric data. But even beyond that, 3Dsensation has produced innovations that I am convinced will successfully establish themselves on the market. In this way, the know-how developed in the network can create value for society, industry and the economy.”

Three collaborative projects that emerged from “3Dsensation” have a special lighthouse character for the future of human-machine interaction:

“NeoVital” monitors vital parameters of premature and newborn babies

The start in life is not always easy. Even if everything seems fine when a child is born, their state of health can change rapidly shortly after. Premature babies in particular require intensive monitoring in order to deal with sudden complications. Contact-based measuring devices for recording vital parameters are not only inconvenient for the young patients – they also do not allow seamless monitoring, for example when it comes to transporting them in the hospital or during certain diagnostic methods.

“NeoVital” aims to close this gap: In an interdisciplinary joint project, a contactless and multispectral 3D sensor was developed. It allows the vital signs of newborns and premature babies to be recorded in real time. “For this purpose, we have adapted already existing optical components for multispectral 3D detection to the requirements of a clinical environment,” explains Jan Sperrhake, leading researcher in the project. The sensor is installed in a spherical device that is placed in close proximity to the infant, for example, on the baby’s bed or on a treatment table. “This allows continuous monitoring of the child in the existing clinical infrastructure,” Sperrhake added. A demonstrator has already been permanently installed in the pediatric clinic of Jena University Hospital during a pilot study and was used there during ongoing operations. Current study results underline the precision of the measurements.

Moreover, the “NeoVital” project shows that “3Dsensation” not only promoted innovative research ideas, but also concrete entrepreneurial action: The start-up “Xsight Optics” based in Jena emerged from the project. “The system for contactless health parameter measurement developed in our company is intended to optimize the process of complex and time-consuming routine care documentation and to increase the quality of care,” explains Maria Nisser, a former employee at Jena University Hospital and now COO of the start-up company.

Contactless personal identification with “3D4F”

“Show me your finger and I’ll tell you who you are”: fingerprints are an increasingly popular method of uniquely identifying people. However, sensory capture via the touch of a scanner is associated with the risk of transmitting viruses and bacteria – especially when the scanner is used at facilities with large amounts of traffic, such as airports or residents’ registration offices. Therefore, not only since the Covid-19 pandemic have alternatives been in demand that allow the hygienically safe capture of fingerprints for large crowds.

This is where the “3D4F” sensor comes in. It is a four-finger scanner based on non-contact 3D capture technologies. Completely contactless, it meets the FBI’s globally recognized security standards for the first time. This is achieved by pairing a high-resolution camera with a lens-projector combination developed specifically for the sensor. This is how the necessary resolution and image quality of the scan can be guaranteed, which is required to comply with the FBI standard. Another special feature is that specially developed algorithms are used to convert the image of the hand captured by the 3D scan into a 2D image. In this way, an interface can be created for comparing this information with already existing databases.

In order to make this solution applicable to a wide range of scenarios, JENETRIC GmbH from Jena and its partners have developed a sophisticated and patented interaction concept. This allows any person, regardless of individual experience with biometrics, to work intuitively with the system.

Service robot “TORY” assists staff in retail stores

Inventory in retail is a necessary but resource-intensive task. Not only do employees have to invest valuable working time in this task – in some cases, stores even have to be closed completely in order to record the inventory. In the future, the service robot “TORY” will offer an alternative: The robot moves independently through the sales areas, digitizes the article inventory and enters the collected data into the merchandise management system. The system was developed by the company MetraLabs, based in Ilmenau, Germany as part of the 3Dsensation joint project “ROTATOR”.

“The robot can record inventories ten times faster and with a significantly lower error rate than a human,” explains Dr. Andreas Bley, co-founder and managing director at MetraLabs and coordinator of the project. “There is a significant reduction in the workload of employees while at the same time increasing efficiency in merchandise management.” The system is already being used by the first customers in Germany, Europe and Australia.

“3Dsensation” research alliance part of BMBF’s “Zwanzig20” funding program

The “3Dsensation” research alliance emerged in 2013 from a competition for the BMBF’s “Zwanzig20 – Partnership for Innovation” funding program. The aim of the funding program was to support eastern German companies in particular with innovations and cooperations.

Wissenschaftliche Ansprechpartner:

“NeoVital” project
Jan Sperrhake
Friedrich Schiller University Jena
+49 3641 807-806
jan.sperrhake@uni-jena.de

“3D4F” project
Dirk Morgeneier
JENETRIC GmbH
+49 3641 3219950
d.morgeneier@jenetric.de

“ROTATOR” / “TORY” project
Dr. Andreas Bley
MetraLabs GmbH | Neue Technologien und Systeme, Ilmenau
+49 3677 667 431 11
andreas.bley@MetraLabs.com

Weitere Informationen:

https://www.iof.fraunhofer.de/en/pressrelease/2022/research-alliance-3Dsensation…