In tune with learning music electronically

When music teachers and students first heard about the IMUTUS interactive tuition many were sceptical about its ability to improve upon traditional learning. Today, evaluation evidence suggests that it has a really strong great potential of making music education more fun, entertaining and effective than ever before.

The result of more than two years of work by music experts and software developers in Greece, Italy, France and Sweden, the IST programme-funded project has created what project coordinator Spyros Raptis describes as “a truly efficient and effective music learning environment which holding much promise for the future”.

Using real-time audio processing and recognition technologies, 3D virtual reality modelling, and evaluation and feedback tools, IMUTUS opens a new interactive world to music students whether they are at home, in the classroom or even taking classes remotely over the Internet.

Complementing traditional teaching

“The aim was not to replace music teachers, as some of them may have feared when we proposed the system, but to create a complementary tool to improve the way students learn music,” Raptis, a researcher at the Institute for Language and Speech Processing in Athens, explains. “The initial scepticism of teachers and the uncertainties of students were soon overcome when they started to use the system, with tutors seeing their students improving faster and the students themselves finding it fun and engaging to use.”

For the purposes of the project the system was targeted at children between the ages of 9 and 14 learning to play the recorder, a wind instrument that is commonly used in schools and conservatories across Europe in the initial stages of music education. The main role of IMUTUS is to help students learn more effectively when they are practicing at home after class, giving them much needed feedback about how they are performing musical scores.

“Whereas students traditionally went home and practiced alone, with IMUTUS their performances are recorded and evaluated in real time, letting them know if they made any errors, where they went wrong and how to correct it,” the coordinator notes.

By loading the software into a normal PC with a microphone, IMUTUS records and recognises what the students are playing, comparing it to uploaded scores presented in XML format. An evaluation module analyses how their performance sounds and offers constructive feedback when they are done playing to improve their interpretation of the score. A three-star overall grading system supplements the detailed feedback and gives appreciated rewards as the performances improves. Using 3D modelling the system incorporates a virtual reality avatar that can play any uploaded score, allowing the student to see where their fingers should be placed.

“We used advanced algorithms as well as score matching and score following to ensure accurate audio recognition and its conversion into MIDI output, which in the case of monophonic instruments such as the recorder is relatively simple,” Raptis says, noting that only minor changes would be needed to use the system with other wind or brass instruments. “Polyphonic instruments such as the piano or the guitar, where many notes are played at the same time, are harder to address.”

Trials underline the benefits

The real benefits of the system come to light when the student returns to the classroom, as two trials at music schools in Stockholm and Florence showed.

In the Swedish evaluation 12 students were divided into six pairs of equal musical ability. One student in each pair used the IMUTUS system for their homework in a controlled environment, while the other did not. “The results were beyond all expectations,” Raptis says. “The students who used IMUTUS returned to class and played with far fewer mistakes than those in the control group. Over the course of a couple of weeks the error rate of those using the system declined much faster than the rest.”

“The improvement was stable across the three weeks and there is a persistent effect on students’ musical skill, even now a month later,” one teacher commented after the tests.

“I am positively overwhelmed that students found IMUTUS such fun. It exceeded my expectations,” said another.

A more subjective approach was taken in the Italian trial, with the teachers themselves asked to evaluate their students’ performance.

Indeed, for students the IMUTUS system provides a more “engaging and entertaining” environment in which to complete their homework, with many students playing scores repeatedly until the evaluation system gave them a three-star mark. “Fun is something that is essential if students are to maintain an interest in what they are learning,” Raptis notes.

With the project having ended in February, the partners are now planning to develop the system further and are looking to carry out more evaluation trials. “We would like to test it with all the students of a music school over a prolonged period of time,” the coordinator says. A mid-term goal is to produce a commercially available software package incorporating the in-home assistant, teaching aids for tutors and Internet connectivity that would allow “a student to learn from a teacher no matter where they are, something that could lead to the creation of a pan-European online music school.”