When Machines Learn How to Act on Their Own

Carlo D’Eramo is new at the University of Würzburg. The computer science professor works in the field of artificial intelligence. He is an expert in a special form of machine learning called reinforcement learning.

An intelligent camera surveillance system is supposed to autonomously detect stray pieces of luggage or other suspicious objects at a railway station. To do this, it has to know what suitcases and bags look like. To achieve this, humans have to feed the system with training data – in this case with as many images as possible of different pieces of luggage.

A form of machine learning called reinforcement learning works in a completely different way. “Data input from humans is not necessary here,” says computer science professor Carlo D’Eramo. Instead, the algorithms gather their own experience by solving tasks in many simulation runs according to the principle of “trial and error”. Correct decisions are rewarded, for example with points credited to the algorithms. This positive reinforcement causes desired actions to be carried out preferentially – the system learns.

“You can imagine this kind of machine learning like a baby that gradually explores its environment. It makes good and bad experiences in the process, and it remembers them,” the professor explains. The baby will avoid “bad” actions in the future, and repeat “good” ones.

Multitasking and transfer learning as goals

Carlo D’Eramo’s group is working on further improving the methods of reinforcement learning. Among other things, the goal is to teach the self-learning systems multitasking and transfer learning.

On the subject of transfer learning, the researcher again draws a comparison to humans: Football players know that they belong to a team that plays against another team in a precisely defined space. They can assess the running routes of the others and have a feel for the ball. These and other experiences are useful when footballers learn other ball sports: they transfer their knowledge from football to the other field. It is precisely this kind of knowledge transfer that Carlo D’Eramo wants to make usable for machine learning.

BMBF project in robotics

The computer scientist has given his working group the name LiteRL. RL is the abbreviation for Reinforcement Learning, Lite stands for “light” – with this, D’Eramo wants to express that he relies on learning models that are as simple as possible, but which can handle complex tasks.

“We do research in theory and practice. We test whether our theoretically developed methods are effective in simulations and real applications, for example in the fields of finance, games and robotics,” explains the professor.

In the field of robotics, for example, Carlo D’Eramo receives 750,000 Euros from the German Federal Ministry of Education and Research (BMBF) for a project that has just started. The aim is to use RL to teach a four-legged robot to walk accident-free on various surfaces.

“It’s easy to make a robot with four legs walk on flat, solid ground,” says the researcher. But this task becomes more difficult when the ground is slippery or uneven or when there are obstacles in the way. It is precisely this problem that the BMBF project aims to solve. The grant money will also be used to fund three Ph.D. students.

Career of the new professor

Carlo D’Eramo was born in 1989 in Pescara (Italy). He studied computer science at the Polytechnic University of Milan and at the University of Illinois in Chicago (USA). He received a Double Degree Master from Milan and Chicago.

Even as a student, the computer scientist was oriented towards artificial intelligence – because it combines exactly the subjects he is very passionate about: Computer Science, Mathematics and Logic.

After completing his doctorate in Milan in 2019, he went to the Technical University of Darmstadt as a postdoc in the Intelligent Autonomous Systems Group. There, from 2022, he led his own research group, which is also located at the Hessian Research Centre for Artificial Intelligence (hessian.AI).

In November 2022, Carlo D’Eramo accepted the call to the professorship for Reinforcement Learning and Computational Decision-Making at the University of Würzburg. This position was newly created within the framework of the Würzburg Center for Artificial Intelligence and Data Science (CAIDAS). The professor will continue to lead his group at hessian.AI until his two doctoral students graduate there.

Wissenschaftliche Ansprechpartner:

Prof. Dr. Carlo D’Eramo, professorship for Reinforcement Learning and Computational Decision-Making, University of Würzburg, T +49 931 31-83457, carlo.deramo@uni-wuerzburg.de

Weitere Informationen:

https://www.informatik.uni-wuerzburg.de/rlcdm/home/ Carlo D`Eramo’s Website

http://www.uni-wuerzburg.de