New technology gives smart cars ‘x-ray’ vision, detecting hidden pedestrians, cyclists

Australian researchers have developed disruptive technology allowing autonomous vehicles to track running pedestrians hidden behind buildings, and cyclists obscured by larger cars, trucks, and buses.

The autonomous vehicle uses game changing technology that allows it to “see” the world around it, including using x-ray style vision that penetrates through to pedestrians in blind spots and to detect cyclists obscured by fast-moving vehicles.

The iMOVE Cooperative Research Centre-funded project collaborating with the University of Sydney’s Australian Centre for Field Robotics and Australian connected vehicle solutions company Cohda Wireless has just released its new findings in a final report following three years of research and development.

The technology’s applications, which are being commercialised by Cohda, involve an emerging and promising technology for intelligent transportation systems (ITS) called cooperative or collective perception (CP).

Using roadside ITS information-sharing units equipped with additional sensors such as cameras and lidar (‘ITS stations’), vehicles can share what they ‘see’ with others using vehicle-to-X (V2X) communication.

This allows autonomous vehicles to tap into various viewpoints. By being hooked up to the one system significantly increases range of perception, allowing connected vehicles to see things they wouldn’t normally.

The engineers and scientists developing the technology said it could benefit all vehicles, not just those connected to the system.

“This is a game changer for both human-operated and autonomous vehicles which we hope will substantially improve the efficiency and safety of road transportation,” Professor Eduardo Nebot from the Australian Centre for Field Robotics said.

“The connected vehicle was able to track a pedestrian visually obstructed by a building with CP information. This was achieved seconds before its local perception sensors or the driver could possibly see the same pedestrian around the corner, providing extra time for the driver or the navigation stack to react to this safety hazard,” he said.

Another experiment demonstrated the CP technology’s ability to safely interact with walking pedestrians, responding based on the perception information provided by the roadside ITS station.

The three-year project also demonstrated the expected behaviour of a connected vehicle when interacting with a pedestrian rushing towards a designated crossing area.

“Using the ITS system, the connected autonomous vehicle managed to take preemptive action: braking and stopping before the pedestrian crossing area based on the predicted movement of the pedestrian,” Professor Nebot said.

“The pedestrian tracking, prediction, path planning and decision making were based on the perception information received from the ITS roadside stations.

“CP enables the smart vehicles to break the physical and practical limitations of onboard perception sensors,” he said.

Lead project researcher Dr Mao Shan said the research confirmed using CP could improve awareness of vulnerable road users and safety in many traffic scenarios.

“Our research has demonstrated that a connected vehicle can ‘see’ a pedestrian around corners. More importantly, we demonstrate how connected autonomous vehicles can autonomously and safely interact with walking and running pedestrians, relying only on information from the ITS roadside station,” he said.

Cohda Wireless Chief Technical Officer Professor Paul Alexander said the new technology “has the potential to increase safety in scenarios with both human operated and autonomous vehicles”.

“CP enables the smart vehicles to break the physical and practical limitations of onboard perception sensors, and embrace improved perception quality and robustness,” Professor Alexander said.

“This could lower per vehicle cost to facilitate the massive deployment of CAV technology.”

Professor Alexander said using CP for manually driven connected vehicles “also brings an attractive advantage of enabling perception capability without retrofitting the vehicle with perception sensors and the associated processing unit”.

iMOVE Managing Director Ian Christensen said the project was a great example of industry collaborating with scientists to unleash new innovations and commercial and public good not just for the benefit of Australians, but road users all over the world,  including pedestrians and cyclists.

“When we bring industry and scientists together, we can achieve many great things as a nation. iMOVE CRC is proud to have initiated this exciting project and many others like it – which are all about getting our best and brightest minds working together to develop new technologies and innovations for real-world problems and needs,” Christensen said.

The report (https://pdfhost.io/v/h1Xd1YCBM_Cooperative_Perception_Final_Report66) details technology testing, which included trials on public roads in Sydney, Australia.

Video and high-resolution images are attached, showing how the technology works:

• How a connected autonomous vehicle using the technology sees the world around it.
• How it uses futuristic x-ray style vision to penetrate pedestrian blind spots.
• How it detects a cyclist obscured by a fast-moving bus.

RECORDED WEBINAR https://imoveaustralia.com/news-articles/intelligent-transport-systems/video-c-its-projects-leading-the-way-in-australia/

Media contacts

Jacqueline King, Communications Manager, jking@imoveaustralia.com, 0404 045 293

Rajiv Maharaj, Story Inception Media Relations, newsroom@storyinception.com, 0416 148 541

Luisa Low, University of Sydney, luisa.low@sydney.edu.au, +61 438 021 390

ABOUT iMOVE

iMOVE is the national centre for collaborative R&D in transport and mobility. It facilitates, supports and co-funds research projects that improve the way people and goods move in Australia. It has 44 industry, government and academic partners and has over 50 projects completed or currently underway in a broad range of transport areas. Find out more about iMOVE: https://imoveaustralia.com/

ABOUT THE AUSTRALIAN CENTRE FOR FIELD ROBOTICS, UNIVERSITY OF SYDNEY

The Australian Centre for Field Robotics (ACFR), part of the Sydney Institute for Robotics and Intelligent Systems, has been instrumental in developing breakthrough technologies, conducting world-renowned research, and developing field robotics principles and systems. The centre’s mission is to undertake research to develop new field robotics and intelligent systems theories and methods, and apply them in industrial, social, and environmental settings. For more information about ACFR: https://www.sydney.edu.au/engineering/our-research/robotics-and-intelligent-systems/australian-centre-for-field-robotics.html

ABOUT COHDA WIRELESS

Cohda Wireless is a global leader in the development of Connected Vehicles and Connected Autonomous Vehicle software with proven applications for Smart City, Mining, and other environments. Cohda’s technology connects vehicles with infrastructure and pedestrians to make streets, cities and working environments safer, smarter, and greener. Cohda is headquartered in Australia and has offices in Europe, China, and the USA. For more information about Cohda Wireless: https://cohdawireless.com/

Method of Research: Experimental study
Subject of Research: Not applicable

Media Contacts

Rajiv Maharaj
Story Inception Media Relations
newsroom@storyinception.com
Office: 61-041-614-8541

www.storyinception.com

Luisa Low
University of Sydney
luisa.low@sydney.edu.au
Cell: +61 438 021 390

www.sydney.edu.au

Jacqueline King
iMOVE CRC
jking@imoveaustralia.com
Cell: +61 404 045 293

www.imoveaustralia.com