Innovative autonomous system for identifying schools of fish

The University of Haifa (Israel) and two teams from the IMDEA Networks Institute have developed an innovative autonomous system, SYMBIOSIS, to monitor real-time schools of fish. This system, which combines optical and acoustic technologies, will be environmentally friendly and will provide reliable information about the condition of marine fish stocks, something that at the moment is practically impossible to achieve without investing enormous resources.

The SYMBIOSIS system integrates acoustic and optical technologies without human intervention. The system is the product of an international scientific initiative under the EU's Horizon 2020 program. Real-time monitoring of schools of fish will inform the development of fishing policy and lead to enhanced protection of the marine environment.

An international scientific initiative under the auspices of the European Union's Horizon 2020, project SYMBIOSIS is developing an autonomous system to identify schools of fish, including information about their size and movements in deep waters.

The University of Haifa is leading the project, while two teams from IMDEA Networks Institute in Madrid are contributing to the development effort. The SYMBIOSIS system integrates acoustic and optical technologies that require no human intervention, and will be able to transmit real-time warnings to coastal stations. These data will contribute to the formulation of ocean fishing policies and to enhanced protection of the marine environment.

“The system will be environmentally friendly, not only in its operation which will be non-invasive and won't impact the marine ecosystem, but more importantly because it will provide reliable information about the condition of marine fish stocks. At present, it's virtually impossible to collect such information without investing enormous resources. Using the latest optical and acoustic technology, we hope to change attitudes towards marine resources”, explained Dr. Roee Diamant of the School of Marine Sciences at the University of Haifa, who is coordinating the initiative.

The development of fishing technology since the twentieth century has resulted in the growing realization that fishing is one of the most serious problems facing marine ecosystems. According to some estimates, if over-fishing is not brought under control, the world's fish stocks may collapse by 2048. Global fishing authorities are hoping to curb over-fishing with new regulations and enforcement based on fish stocks.

But, there are very few methods presently available for real-time monitoring of those stocks. Most involve surface boats attempting to locate schools of fish using sonars. According to Dr. Diamant, these methods require considerable resources and personnel to monitor and interpret the sonar findings. Consequently, they have limited viability in cost-benefit terms.

Also, the use of sonar usually limits the search for fish stocks to narrow areas (those beneath the ship doing the sampling), impairing any subsequent decision-making. The limited statistics provided by this random and short-term sampling of the marine environment mean the process is prone to numerous sampling errors.

Combining optical and acoustic technologies, the SYMBIOSIS system will monitor the marine environment, and in particular the size of the fish stock, within a radius of one kilometer. It works on an entirely autonomous basis, collecting underwater data over long periods and transmitting this information to a coastal center.

The research is focusing on the identification of six large fish species that are in especially high demand from the fishing industry: two species of tuna; scad (a species of mackerel; Trachurus mediterraneus); Atlantic mackerel (Scomber scombrus); mahi-mahi (Coryphaena hippurus); and swordfish (Xiphias gladius). This will provide authorities with concrete and actionable information.

The solution has a processing chain that begins with the acoustic discovery and classification of fish, based on their typical speed and movement characteristics. Acoustic sensors also measure the size of the fish and the total biomass of the fish in the area. Once the acoustic system identifies one of the six selected species, it activates the optical system, which features several cameras and sophisticated data processing with various image identification algorithms using deep learning. When the optical system confirms the identification of one of the six selected species, it transmits the information via underwater acoustic communications, and then by radio communications to a coastal station.

IMDEA Networks' researchers are focused on the design of an efficient fish localization system, and on the visual recognition of the selected fish species. The two teams from the Madrid-based research institute are the Ubiquitous Wireless Networks laboratory led by Dr. Paolo Casari, IMDEA PI and Scientific Manager for the project; and the Global Computing Group led by Dr. Antonio Fernandez Anta.

“Using acoustics to localize specific fish species is very challenging”; said Dr. Casari. “Firstly, the acoustic processing chain has to incorporate cost-effective components, and it needs to be highly energy-efficient. The signal processing algorithms deployed in the acoustic fish identification system have to strike a good tradeoff between complexity and accuracy. On top of this, the underwater environment contains many background acoustic noise sources and reflectors, and the signals from fish around the SYMBIOSIS system will be much weaker than acoustic interference coming from the environment. The algorithms need to be robust enough to cope with these shortcomings.”

“For optics, the marine environment is characterized by low visibility and elements in the water volume that distorts the image. The big challenge is to secure good detection performances and to minimize false alarms. This needs to happen autonomously in a deep-sea environment, where there's practically no possibility of human intervention,” continues Dr. Diamant.

“The optical classification of fish species has its own particular challenges, too. There are very few pre-classified images available with which to train the deep-learning classifier. And many of the images that are available were taken under very different visibility conditions to those the system will encounter.

In SYMBIOSIS, we are dealing with this uncertainty by leveraging public databases of fish pictures, many of them provided by scuba divers and underwater photographers. To address the lack of a large image dataset, we are starting with pre-trained neural networks for object recognition, and we'll add more images from SYMBIOSIS test environments once we enter the experimental phase of the project,” concludes Dr. Fernández Anta.

SYMBIOSIS was selected for funding by the European Commission's research and innovation programme, Horizon2020. Four institutions are participating in the SYMBIOSIS project: the University of Haifa, Israel (coordinator); IMDEA Networks Institute in Madrid, Spain; Wireless & More company from Italy and EvoLogics Gmbh from Germany. The project includes the development of innovative discovery and classification algorithms, the application of dedicated hardware, and the completion of a large number of marine trials.

As part of the project, a prototype is being developed including a system of acoustic sensors, a network of cameras, sophisticated processing units, and an energy unit permitting autonomous activity. The goal of the project is to sample the performances of the prototype system in three different marine environments: shallow Mediterranean, deep Mediterranean, and a tropical environment in the Canary Islands. The project will run until November 2020, and will offer novel solutions to the distributed and large-scale monitoring of the underwater environment, with a positive impact on marine biology research, conservation, and policy making for fisheries in Europe and worldwide.