Intelligent surveillance increases public transport security

Sophisticated tools used to survey and monitor passenger flows through busy metro stations may result in unmanageable data loads. ADVISOR’s decision support tools reduce the workload of operators and increase the utility of the data output.

ADVISOR, which stands for Annotated Digital Video for Intelligent Surveillance and Optimised Retrieval, “is a significant aid to the operators in charge of metro security,” says project coordinator Michael Naylor. “The principal of the [ADVISOR] system is to automate recognition, reporting and archiving of suspect or dangerous human behaviour captured from video surveillance cameras around metro stations to provide a better and safer service for passengers.”

Using cameras to detect human behaviours

“The system takes a live video feed provided from the metro network as input,” says Charles Attwood, technical consultant for Thales Research & Technology (UK). “First, a time-base corrector system assigns and aligns the timing on the video feeds. It then digitises the analogue signals, compresses them and sends them to the ‘crowd monitor’ system provided by Kingston University in the UK. This analyses the video provided and returns alarms based on movements related to crowd behaviour – overcrowding on station platforms, for example.

“Next, a motion detector, developed by INRIA of France, recognises and tracks individuals as they move about the metro system,” continues Attwood. “Data is then analysed by a behaviour recognition unit, which makes decisions based on the type of behaviour being perceived and raises alarms based on that data which it will send to the human-computer interface (HCI) designed by Vigitec in the control and command centre. The final unit is an archive server, provided by Bull, which stores the digitised and annotated video images and responds to requests from the HCI to either search for or retrieve particular sequences of video based on specified behaviours.”

The image processing is totally transparent to the operator. In detecting a scene of violence, for example, ADVISOR detects moving zones by computing the difference between each current image and the background image. It then computes the type of each moving region to identify individuals and groups of individuals. Once detected it follows them through the video sequence, drawing trajectories for each individual. Finally the system uses 3-dimensional spatio-temporal reasoning to recognise the behaviours pre-defined by security experts e.g. trajectories and size of the group and sudden accelerations of people within the group.

ADVISOR in tests on metro networks

The ADVISOR system was tested in real-life situations by the operators of the Barcelona and Brussels metro services. According to Daniel Huts, a Safety and Security Control Officer at STIB’s control centre in Belgium, ADVISOR offers a supplementary aid to the operators: “We currently have 800 video cameras in Brussels filming throughout the metro network, and it is hard for one human being to manage so many of them. When an alarm occurs the relevant camera view takes over the screen, but the decision to intervene in the situation is left to the controller, giving him control of the situation.”

At the beginning of the project, the metro operators were asked to describe their needs in terms of the desired alarms. The following situations were used in design: overcrowding, access blocking, the detection of ticket barrier jumping, vandalism, and scenes of violence such as fights and muggings.

Archiving and searching video footage

Apart from just recognising behaviours and provided users with alarms, ADVISOR is also a complete archiving tool for managing video sequences. The operator can search the video database by alarm type, place, and time. And, the operator can save an image frozen on the screen in an open format to be used by other programmes or to be transmitted to authorities. This process could be used, for example, to report an act of vandalism to the police.

After three years of research the ADVISOR project has demonstrated the feasibility of automating human behaviour recognition. After several days of real-life tests on the metro networks the operators were very positive about the ADVISOR system. The evolution of the prototype has reached a stage where it can start to be included in operational systems.

“At this stage the technology works best with fixed cameras so it’s easy to envisage the system in a range of other applications from car parks to shopping centres. Imagine a situation where a security guard could be visible to the public as they patrol a location and via a wireless device could be alerted to, view and respond to incidents ‘on the ground’ rather then sitting in a distant control room.” As Naylor concludes: “The system has huge potential for commercialisation.”