What kind of sea ice is that? Ask Knut!

A new app under development is using deep learning and artificial intelligence to classify different kinds of sea ice.

If you’ve watched Netflix, shopped online, or run your robot vacuum cleaner, you’ve interacted with artificial intelligence, AI. AI is what allows computers to comb through an enormous amount of data to detect patterns or solve problems. The European Union says AI is set to be a “defining future technology.”

And yet, as much as AI is already interwoven into our everyday lives, there’s one area of the globe where AI and its applications are in their infancy, says Ekaterina Kim, an associate professor at the Norwegian University of Science and Technology’s (NTNU) Department of Marine Technology. That area is the Arctic, an area where she has specialized in studying sea ice, among other topics.

“It’s used a lot in marketing, in medicine, but not so much in Arctic (research) communities,” she said. “Although they have a lot of data, there is not enough AI attention in the field. There’s a lot of data out there, waiting for people to do something with them.”

So Kim and her colleagues Ole-Magnus Pedersen, a PhD candidate from the Department of Marine Technology and Nabil Panchi, from the Indian Institute of Technology Kharagpur, decided to see if they could develop an app that used artificial intelligence to identify sea ice in the Arctic.

You may think there’s not much difference between one chunk of sea ice and another, but that’s just not so.

In addition to icebergs, there’s deformed ice, level ice, broken ice, ice floes, floe bergs, floe bits, pancake ice and brash ice.

The researchers wanted the app to be able to distinguish between the different kinds of ice and other white and blue objects out there, like sky, open water and underwater ice.

Different kinds of ice really matter to ship captains, for example, who might be navigating in icy waters.  Actual icebergs are nothing like brash ice, the floating bits of ice that are 2 metres in diameter or less. Think of it –the Titanic wouldn’t have sunk if it had just blundered into a patch of brash ice instead of a big iceberg.

Another factor that adds urgency to the situation is climate change, which is dramatically altering sea ice as oceans warm. Even with the help of satellite images and onboard ship technologies, knowing what’s in icy waters ahead can be a difficult challenge, especially in fogs or storms.

“Ice can be very difficult for navigation,” Kim said. “From the water (at the ship level) It can be hard to detect where there is strong ice, multiyear ice, and different ice.  Some ice is much more dangerous than other types.

The team began teaching their app’s AI system using a comprehensive collection of photographs taken by another NTNU ice researcher, Sveinung Løset.

But an AI system is like a growing child — if it is to learn, it needs to be exposed to lots of information. That’s where turning the AI into an app made sense. Although the COVID-19 pandemic has shut down most cruise operations, as the pandemic wains, people will begin to take cruises again — including to the Arctic and Antarctic.

Kim envisions tourists using the app to take pictures of different kinds of ice to see who finds the most different kinds of ice. And every one of those pictures helps the app learn.

“If the app is used for ‘infotainment,’  accuracy isn’t that important,” Kim said.  “It can even be fun when the model makes mistakes.”

As the AI learns, Kim says, the increasingly complex dataset could be taken into the classroom, where navigators could learn about ice in a much more sophisticated way.

Currently, students just look at pictures or listen to a PowerPoint presentation, where lecturers describe the different kinds of ice.

“So this could revolutionize how you learn about ice,” she said. “You could have it in 3-D, you could emerge yourself and explore this digital image all around you, with links to different kinds of ice types.”

The researchers are planning an AI in the Arctic workshop in September to explore AI applications in these remote areas.

“There are extreme challenges unique to the Arctic, from human activities and impacts in remote Arctic locations to Arctic data acquisition, sharing, and quality,” Kim said. “We need to direct AI applications towards solving Arctic challenges that are important for the world as well as to highlight the ‘black holes’ or knowledge gaps and raise awareness on what does not work, needs improvements.”

###

For more information about the workshop, see https://www.ntnu.edu/imt/aidingarctic

Reference: N. Panchi, E. Kim and A. Bhattacharyya, “Supplementing remote sensing of ice: Deep learning-based image segmentation system for automatic detection and localization of sea ice formations from close-range optical images,” in IEEE Sensors Journal, doi: 10.1109/JSEN.2021.3084556.