Integration of Multimodal Sensor Data for Targeted Assessment and Intervention

Abstract

Physical and Occupational Therapy have been used for many years to help people who have suffered an injury of some kind. This injury could be caused by a physical injury, such as falling or breaking a bone, or a brain injury, such as a stroke. Traditional interventions involve having a therapist watch a patient perform any prescribed interventions to see if they are done correctly and to assess progress, or to have a patient perform exercises at home unsupervised. Patients, once discharged, do not always adhere to the prescribed intervention. They begin to not keep scheduled appointments and not complete the at home exercises. This limits the patient’s chances for a full recovery. Tele-rehabilitation is a potential answer to improve adherence and rate of recovery. By using gamification and Virtual Reality (VR), rehabilitation exercises can be turned into more engaging and entertaining activities called exergames. Patients show a preference towards these exergames over traditional interventions. The sensors required to gather input for these games often include motion tracking technologies. These input modalities can be used to gather more information about a patient than just the score achieved in a game. Detailed information about the patient’s motions can be collected. With new low-cost alternatives available, such as the Microsoft Kinect, Leap Motion, and HTC Vive, these types of tele-rehabilitation systems are becoming more affordable. This research aims to develop a virtual recreation of an Occupation Therapy assessment using newer low-cost equipment. Data gathered from gameplay and from sensors are used to assess a user’s performance. Evaluation of different types of gameplay will be done to find the best way to administer computerized or virtual versions of this Occupational Therapy task.