Replacing hand gestures with holography

Car2Human communication at LASER 2023…

How do you decide whether a pedestrian needs to wait or it’s safe for them to cross the road in front of a car? In today’s world, car drivers and pedestrians simply exchange a brief eye contact or small hand gestures to express their intentions to one another. But how will future autonomous cars communicate? Researchers involved in the MaMeK project are seeking to answer that question. They will be presenting their findings at the LASER World of PHOTONICS trade fair in Munich from June 27 to 30 (Booth 415, Hall A2).

Imagine a situation in which a cyclist isn’t sure whether an approaching car is giving him way or not — but then a bright projection appears in front of the vehicle, indicating that it has detected the bike and is waiting for it. This is one example of how cars and humans might communicate with one another on the streets of the future. Together with his team at the Fraunhofer Institute for Applied Optics and Precision Engineering IOF in Jena, Norbert Danz is looking at scenarios of this kind within the MaMeK joint project, which is focusing on projection systems for human-machine communication and involves partners including Audi AG. Two technological approaches are being pursued as part of this: displays shown directly on the car itself and holographic projections on the ground surrounding the vehicle. Fraunhofer IOF is responsible for the technology on which the latter of these cases is based.

“Many cars already feature LED-based symbol projection systems that project images onto the ground when the driver gets out. But they’re not bright enough yet for daylight conditions and the images they produce are static, so they’re not suitable for enabling autonomous cars to communicate with their environment,” explains Norbert Danz. With this in mind, the challenge was to create an ultra-bright projection that would be visible even in sunlight and able to display information dynamically. It also had to be capable of illuminating a sufficiently large area to cover all directions around the car with just a few projectors.

Crystal-clear projections in sunlight and over wide angles

The researchers at IOF are using state-of-the-art laser technology to put their dynamic micro-projector into practice. Four laser diodes illuminate an image generator known as a spatial light modulator (SLM), which distributes the light in a way that forms a particular image on the road. To achieve the right level of brightness and ensure that the projection area is as large as possible, the system generates four sub-images that are then merged into one. “As the projector is very near to the ground and the projection angle is very wide, it’s difficult to create one large image, so instead we distribute the image across multiple laser diodes and widen each individual image using micro-optic telescopes. Additionally, the direction of projection is determined by an array of microprisms, allowing us to achieve a projection area of 100 x 30 centimeters at a distance of less than 50 centimeters,” explains Danz.

With dimensions of just 7 x 7 x 5 centimeters, the system developed by Fraunhofer IOF can be integrated into any car sill. Installing multiple projectors makes it possible to display pictograms on the road all the way round the vehicle — and the modular architecture also opens up the possibility of depicting dynamic elements. What really makes the system stand out, however, is the brightness level of around 10,000 lux (depending on the image), which allows cyclists and pedestrians to see road projections even on a sunny day. In addition to the use of lasers, this is achieved by elements including the SLM image generator from Berlin company Holoeye Photonics, one of the project partners. Docter Optics SE, a medium-sized enterprise based in the German state of Thuringia, is also involved in the construction of the projectors.

The overall system developed as part of the MaMeK project is now being integrated into a demonstration vehicle and networked with appropriate sensors from a (highly) semi-autonomous vehicle. The Fraunhofer researchers will be exhibiting a laboratory demonstration model of the micro-projector for the first time at LASER World of PHOTONICS in Munich from June 27 to 30, 2023, at stand 415 in hall A2.

Development in 3Dsensation joint project

MaMeK emerged from the 3Dsensation joint project, which represented one of ten consortia from the Twenty20 — Partnership for Innovation funding program run by the German Federal Ministry of Education and Research between 2014 and 2022. Together, the alliance partners pursued the aim of using innovative 3D technologies to help machines capture complex scenarios visually and interpret them. The consortium focused on the principles underpinning safe and efficient interaction between humans, machines and the environment in production, health and mobility — key areas of life and work.

Weitere Informationen:

https://www.fraunhofer.de/en/press/research-news/2023/june-2023/replacing-hand-g…