Back to search

IKTPLUSS-IKT og digital innovasjon

Multimodal Elderly Care Systems (MECS)

Alternative title: Multimodale systemer for eldreomsorg (MECS)

Awarded: NOK 12.2 mill.

The ratio of elderly in the world's population is increasing. Thus, the number of older people living alone at home for some time of their life is increasing, and this trend is expected to continue. The challenge then would be how to provide technology that can operate in the complex and different environments found in homes. Further, technology can easily be seen as a threat to privacy and lack of interpersonal contact. This project has addressed these issues through user-centred design of robot systems and the development of adaptive technology. A part of this has been to demonstrate the benefits regarding both performance and privacy being improved by applying sensors like cameras on a robot companion rather than having them permanently mounted in a home. These can be used for detecting falls and other non-normal situations. Using new sensor technology, we have targeted to explore if it is possible to remotely monitor what activities a person performs and whether it is a normal or abnormal one. Rather than requiring elderly to activate their personal security alarm themselves in an emergency situation, a goal of this project has been to develop methods that can contribute to automatic activation. The research has been documented through 56 peer-reviewed scientific papers in journals and conferences, and more than 100 other public and user-oriented disseminations. Important contributors to the work have been two PhD candidates and 11 MSc students who have graduated with project-related work and reports. Many systems for elderly have been designed, but few have been adopted on a large scale. We think a key reason for this is limited user involvement and few iterations of user testing. Therefore, we have focused specifically on involving older people throughout the project. In the first phase of the project, it included collaboration with residents and management at Kampen Omsorg+ housing facility in the form of workshops, interviews and testing out robot technology. One study investigated the encounters between residents and a robot-prototype. We also investigated elderly and their reported experiences with off the shelf vacuum cleaning robots in the elderly´s home. In parallel, we have been researching technology about recognition of human activity using different sensors like depth camera, thermal, and ultra-wideband sensors. Facial expression recognition using RGB-depth camera images has also been studied. The models used for recognition are trained with state-of-the-art deep learning algorithms and also include a set of different feature extraction methods being tested and compared. To increase privacy for a user, we have had a special interest in testing the performance of a new ultra-wideband sensor. There are no datasets available, so we have collected our own data for a number of different human activities and heart rate levels, respectively. This was with a goal of training a system to distinguish between normal and abnormal condition of a person in a bedroom. The results obtained are promising. Regarding robot control, we have worked on motion planning for mobile robots where the focus has been on providing efficient navigation strategies to handle the high complexity of a home environment. To have a more robust robot control, the idea of ``navigation without a map´´ was investigated and resulted in the ``virtual experience model´´, which empowers the robot to move in a completely unknown terrain. To have a more privacy-friendly robot companion, we have also been working on using laser range finder sensors for mapping, navigation, and human detection. In addition to user testing at Kampen Omsorg+, user testing has also been investigated through a collaboration between the MECS project team, Eindhoven Univ. of Technology, and Vitalis home for elderly in Eindhoven in the Netherlands. We have through this collaboration undertaken several user studies focusing on different verbal and nonverbal behaviours of a robot in the interaction between humans and robots. We have also been investigating how we can move robots using techniques from film and computer animation, and how this affects people´s opinions about the robot. We have examined this in cooperation with the University of Hertfordshire. This included a MECS researcher on exchange running experiments with over 30 participants using one of the university´s robots and their Robot House facility. This work was an important part of a PhD thesis resulting from the project and which was defended in March 2020. Finally, we have been involved in committee work for defining an IEEE standard for ethically driven robotics and automation systems. Two papers have also been published related to ethics.

The impact of the project has been in progressing research on user-centred design of a mobile robot system that can act as an automated personal safety alarm. This has been addressed through research on robot sensing and control for a robot that can monitor the behaviour and condition of older people. We regard the project to have resulted in social, academic and industrial benefits through its 56 peer-reviewed scientific papers in journals and conferences more than 100 other public and user-oriented disseminations. Three of the paper have more than 60 citations at the end of the project. Another project outcome is that two PhD candidates and 11 MSc students have graduated with project-related work and reports. They benefit from getting competence in undertaking technical development and research in close collaboration with users. In summary, we think the project can contribute to future technology product development that can both increase life quality and reduce societal costs.

The number of elderly people living at home is increasing and this trend is expected to continue. The challenge then would be how to provide technology that can handle the complex and different environments found in homes. Further, technology can easily be seen as a threat to privacy and lack of interpersonal contact. This proposal addresses these issues by user centered design of systems and the development of adaptive technology. We will further in this project demonstrate the benefits regarding both performance and privacy being improved by applying sensors like cameras on a robot companion rather than having them permanently mounted in a home. These would be used for detecting falls and other non-normal situations. Using new sensor technology, we would also like to explore if it is possible to remotely monitor medical states like pulse, breathing etc. Rather than having elderly themselves activating their personal security alarm in the case of an emergency situation, a target of this project is to make this automatic and not dependent on the person carrying an alarm device. Many systems for elderly have been designed but few have been applied. We think a key reason for this is limited user involvement and few iterations of user testing. Therefore, we will focus specifically on developing our systems with a large degree of user participation. In conclusion, we will in this project like to implement and evaluate robotic elderly care systems with multimodal sensors that are able to sense, learn and predict future events. That is, by using complementary sensor technology and machine learning analysis and modeling techniques, we will target the development of novel monitoring systems to be applied in home care applications. Further, we will improve their usability through participatory design, involving users in actual use contexts.

Publications from Cristin

Funding scheme:

IKTPLUSS-IKT og digital innovasjon