XR and AI enable innovative rehabilitation methods

TeMoRett project develops computer-assisted therapy.

The Fraunhofer Heinrich-Hertz-Institut (HHI) is coordinating the newly launched project “Technology-supported Motor Rehabilitation for People with Rett Syndrome” (TeMoRett). Intending to devise an interactive computer-assisted motor rehabilitation therapy, TeMoRett is designed as a collaborative project for the duration of three years and will run until January 2026. Generously funded by the German Federal Ministry of Education and Research, the total budget is 1.62 million euros. The Fraunhofer HHI receives about 650 thousand euros from the overall amount.

Rett syndrome depicts a rare genetic disorder that predominantly affects girls. A central complaint of female subjects with Rett syndrome consists of hand movement disorders – aligned with challenges in communication. This constraint leads to significant limitations in simple everyday activities, such as being able to eat independently, and thus significantly reduces social participation. The severity of these impairments can vary widely, especially regarding intended hand movements induced by, for instance, stereotypies such as repetitive ‘hand washing’ movements.

Correspondingly, personalizable and adaptive exercises can support the development of voluntary hand movements while reducing stereotypies. The core purpose is to provide a fun therapy approach that can be applied seamlessly at home, improving directed arm use by Rett patients. In context, the TeMoRett system will consist of low-cost components that can be easily operated by those caring for the individual such as relatives, therapists, or physicians. Furthermore, a telemedical feature will allow patients to profit from care delivered by remote experts.

In analogy, the project idea ties back to the results of game research by Dr. Pamela Diener of Georgetown University Medical Center (Washington, D.C., USA): “In previous work, we were able to demonstrate the basic feasibility of the game idea and the telemedical approaches for training arm use, associated postural reactions and attention to task in Rett patients. TeMoRett conveys an essential next step to further develop and explore these training procedures.”

Accordingly, a high-caliber, interdisciplinary consortium has gathered to implement the set benchmarks. In addition to Georgetown University Medical Center (Washington, D.C., USA) and the Interactive & Cognitive Systems Group of Fraunhofer HHI (Berlin), the Max Planck Institute for Human Cognitive and Brain Sciences (Leipzig), the Clinic for Pediatrics with a focus on neurology and the Social Pediatric Center (Berlin) of the Charité – Universitätsmedizin Berlin, as well as Rett Syndrom Deutschland e.V. (Rösrath) are contributing their extensive expertise to the project.

Paul Chojecki, coordinator of the TeMoRett consortium and Fraunhofer HHI project manager, states an explicit prognosis for the future: “TeMoRett fits perfectly into our sustained strategy of linking perceptual human-technology interaction technologies with medical applications. I am looking forward to leading this high-caliber alliance and jointly developing new therapeutic approaches to motor rehabilitation for Rett patients.”

Fraunhofer HHI is taking a leading role in technology development. In particular, the scientists from the “Vision & Imaging Technologies” department focus on precise, real-time 3D hand detection and user-friendly augmented reality (AR) interaction modules. Constituting a crucial part of the TeMoRett project, technical methods for entertaining exercise programs will be developed, in which intended hand movements are automatically recognized and positively reinforced by rewards. Important to consider, the range of possible movement and control varies considerably from person to person, posing a technical challenge. Moreover, input technologies must not be intrusive to distract patients as little as possible.

Dr. Vera Raile, Charité – Universitätsmedizin Berlin, summarizes the TeMoRett project succinctly: “Our goal is to develop a playful and motivating therapy for girls with Rett syndrome that enables motor learning to accompany everyday life and thus leads to improved hand function.” Augmenting this prospect, Dr. Michael Gaebler, Max Planck Institute for Human Cognitive and Brain Sciences, adds the significance of “scientifically based optimization of learning progress, i.e., through personalized content and adaptive difficulty levels.”

Subsequently, the reward systems already indicated (e.g., multimedia output) compose another vital element of the project, as they are individually adaptable to provide comprehensive support for the test subjects. More precisely, the difficulty degrees can increase adaptively (“shaping”), depending on the performance in previous sessions. The game-based content, presented on the table in front of the participants using projective extended reality (XR) methods, will form an essential part of the project development and evaluation. Combined with training evaluations, data from video-based recordings of behavior (e.g., head movements) and psychophysiology (e.g., heart rate) will channel into the adaptation. The outcome of this research practice facilitates an improved matching of subjects, difficulty levels, and reward mechanisms.

Finally, analyzing the training data and visualizing the results allows a quantifiable control of training effects for physicians, researchers, and therapists. This aspect of scalability is not the only reason why “it is particularly important that we already take into account the needs of the patients within the project, but also those of the other user groups in terms of the future TeMoRett system,” proclaims Claudia Petzold, Chairwoman of Rett Syndrom Deutschland e.V. The association is a central point of reference for Rett patients in German-speaking countries. Raising awareness, the association’s networking in the international Rett community will support the consortium in establishing fruitful exchanges with potential user groups, conducting evaluations in families, and communicating TeMoRett results.

In summary, TeMoRett specializes in the robust real-time capture of voluntary hand movements. Computer vision techniques merge with machine learning for rapid evaluation. For successful project implementation, it is thus critical to provide a user-friendly training application that motivates frequent use. The TeMoRett approach offers numerous advantages for ensuring so, such as contactless hand movement capture, personalizable XR training content, and telemedical support.

Wissenschaftliche Ansprechpartner:

Martina Müller
Press Officer
Phone +49 30 31002 242
Email martina.mueller@hhi.fraunhofer.de

Dipl.-Psych. Paul Chojecki
Project Manager
Phone +49 30 31002-281
Email paul.chojecki@hhi.fraunhofer.de

Originalpublikation:

https://newsletter.fraunhofer.de/-viewonline2/17386/771/5/6RFhct0v/5hYZPUpSH7/1

Weitere Informationen:

https://www.hhi.fraunhofer.de/abteilungen/vit/temorett.html