Neuro RecoVR is a stroke rehabilitation platform that utilizes simulated modules to aid in post-stroke recovery. This AR/VR treatment employs activities and mini-games that are carefully designed to stimulate damaged parts of the brain. I am the lead designer on the project and had a hand in the conceptualization and design of each of the rehab modules. Through focused meetings with leading doctors and therapists in the neurological field, we were able to create a handful of modules designed specifically to address key areas of stroke rehabilitation. The system is being built in 2 stages, a treadmill-based solution focusing on lower body motor function through in-hospital care and a follow-up system that utilizes consumer VR headsets to continue a patient's cognition and upper-body motor function treatment at home.
NeuroRecoVR is a 2 phase treatment solution that uses new visualization technology to rehabilitate stroke patients and track their recovery journey.
February 2021 – Present
Current stroke rehabilitation is mainly done one-on-one with in person therapists. Often these treatments are under administered and are expensive due to a lack of therapists working in the field.
To develop a new digital tool allows therapists to work with multiple patients at once by utilizing a platform that instructs the patient to play targeted minigames and tracks their progress remotely.
Lead UX designer, lead UX researcher, lead UI designer.
User research, wireframing, UI design prototyping
The identity for Neuro RecoVR had to be instantly iconic while also conveying themes of health and regeneration. The brain was chosen to be the central icon as it is the primary organ that the procedures look to repair. The use of simplified line work was used to relay the curvatures of the brain and the gradient is meant to invoke a sense of regeneration.
The vast majority of current stroke rehabilitation procedures do not use any kind of modern technology. Therapists usually work one-on-one with patients guiding them through activities and exercises that if repeated often enough will recondition the brain and allow for recovery in motor function and cognition. While the work done with therapists is important and leads to successful degrees of rehabilitation, the approach is not without it's flaws.
First and foremost the demand for stroke rehabilitation therapists is currently extremely high which not only makes them very expensive but also means they are often overbooked and may not be able to see patients as frequently as needed. Therapists are also usually much more available to people near cities, patients in rural locations are usually under supported due to travel time and limited access to nearby medical centers.
Last but not least the Covid-19 pandemic showed the world that we need solutions that could be implemented without in-person contact. During the pandemic many stroke rehabilitation patients went without care for extended periods of time.
The in-hospital Neuro RecoVR platform utilizes a split belt treadmill system that doctors and therapists can use for focused rehabilitation of lower body motor function. The treadmill also has the functionality of pressure tracking through plates below the belt. We utilized these features in the treadmill to create custom software modules that focus on specific areas of a patient's rehabilitation. Three modules were designed to facilitate growth in lower body motor function, each one increasing in challenge and movement complexity.
Patient rehabilitation data tracking is at the forefront of the Neuro RecoVR platform. Every session on the system tracks the challenges the patient is tasked with and response to the treatment. Overtime with continued use of the platform the system will compile logs of patient data and through data visualizations doctors and therapists are able to see where the rehabilitation process is working or falling short. They can then use this collected data to make informed decisions about the patient's next steps and focused treatments.
The goal of the pre-gait module is to get patients that no longer have the ability to stand, up and applying weight to their injured extremity. Utilizing the harness the full weight of the patient can be reduced on their lower body. The therapist then has the ability to allocate a target on the belt for foot placement, the amount of weight they want the patient to apply and the length of time the patient should hold their pose. Through this process and repeated sessions the patient should regain ability in their lower body to the point where walking (or gait) treatments can begin.
The colors used in the patient UI are all high contrast and adhere to colorblind hue recommendations.
Once a patient is able to stand it is time to train their lower body to walk again. The gait module does this by utilizing the split belt treadmill. On screen the patients will be viewing a 360 degree video of walking along the bow river. While again strapped into a harness the patient is tasked with taking a summer walk along the river. The therapists have the ability to independently change the speeds on each side of the belt. The goal is for the patient to exercise and rehab their affected leg so that they can gradually speed up that belt to the point where the two legs are functioning at the same walking speed.
After a patient has regained their ability to walk at a moderate pace it's time to cognitive exercises into the mix. Most healthy people have no problem dealing with distractions while walking, however a stroke can impair a patients ability to focus on their walking balance leading to dangerous falls. In the final module the harnessed patient is tasked with walking down a digital path while looking for specific animals. The therapists have fully custom digital sandbox they can use to spawn animals or distractions. the patient can then point to the animals and using a hand tracking system the software can tell if they are pointing at the right location awarding the patient points.
Most medical software is not known for well designed UI. I really wanted to create a front end that was intuitive and colorful. Taking the colors from the brand I wanted to design clean and clear log-in screens that would be a fresh departure of classic Windows XP interfaces that therapists are so often stuck working within.
Once the therapist has logged-in with their credentials they are greeted with a patient login screen. Due to laws and regulations regarding the collection and storage medical data, the patient's identity needs to be separated from their tracked information. For this we choose to issue a patient ID number (or PID) for each user of the platform. If lost the therapist does have the ability to recover a patient ID as that information is stored on a separate local server.
The dashboard is the first thing the therapist sees once they are logged into the platform. The chart at the top shows a visual representation of the patient's journey with the Neuro RecoVR system. Each of the 3 modules are graphed with by the patient's session score over time. The session overview panel shows the order in which the modules were completed, the date in which they were performed and the patients final score by percentage. As more specific data tracking features are integrated into the platform additional panels will be added.
Through our research we found that music has incredible secondary healing properties when combined with physical stimulation. Therapists need to be constantly focused on the treatment they are administering and thus was decided it was easier and more practical to build the music selection straight into the platform rather than fumble with a CD or MP3 player. Beats per minute can also be very useful when working with a patient's walking ability as the beat can help to build a consistent stride. We utilized the Spotify API to relay the musical information to the therapists.
The UI for the pre-gait controls had to be carefully designed. The therapists wanted a wide range of options so that they can cater to varying degrees of disability while also using condensed and simple interface. Foot placement and limb weight allocation are the primary functions of this treatment and as such have the most visual dominance. It was also important to include sliders for an acceptable range of weight so that patients can gradually improve their balance without getting frustrated.
%20%E2%80%93%201.png)
The gait interface lets the therapist directly increase or decrease the speed of each side of the split belt treadmill. At the start of the session the therapist can choose a location for the treatment, a summer walk along the Bow River or a fall hike through a paved mountain path in Canmore. These locations were chosen to provide a beautiful setting for the treatment but also because they are familiar areas to local Calgarians. Each of the 360 degree videos were recorded at multiple speeds which allows the therapists to choose the speed and difficulty of the session.
%20%E2%80%93%201.png)
Last but certainly not least is the user interface for the gait and awareness module. While getting a severally injured patient walking again is an amazing feat, the lower body motor function is only half of the challenge. Because a stroke can also affect cognition it's important to also make sure that a patient can recognize the always shifting environment around them.
Like the gait UI the therapists have the ability to directly adjust the belt speeds but also have the added functionality of adding target animals, fruit and distractions into the environment. The patient is tasked with walking through a digital park setting while also identifying specific objects that are spawned based on the parameters the therapist creates. Each of these targets once added to the scene each have their own specifications that can be adjusted on the fly. Some of these parameters include object size, spawn location, spawn frequency and whether the object is interactable or just there as a distraction.
The UI is designed so that each target can be engaged or disengaged quickly while the therapist is physically tending to the patient.
.png)
After a patient has been discharged from the hospital additional rehabilitation is often needed. The out patient phase of the Neuro RecoVR platform utilizes consumer VR headsets to engage patients with minigames specifically designed to improve upper-body motor function and cognitive abilities. The data from these sessions are sent to a secure server where medical professionals can monitor a patients progress remotely and adjust the treatment as they see fit.
The goal of this out patient rehabilitation platform is to allow therapists to work with many patients at once through the use of new digital tool. It is therefore important for the user experience to have a digital guide, as therapists are not regularly on site to administer treatment. The largest demographic that interact with the platform are the elderly who have very little familiarity with digital engagements. Because of this, it was critical to create an avatar that would be a relatable character and guide for the patient. All of the instructions are relayed through Sarah so it was crucial that she clearly communicates the expectations and objectives of each module. Stroke recovery is a long and frustrating experience so she is also designed to be an encouraging voice that continually motivates the patient to continue with their treatment.
The eye-spy module is specifically designed to rehabilitate visuospatial awareness and upper-body motor function. Patients are asked to identify specific animals walking around a wooded environment. This exercise was built to get the patient scanning a large environment and after repeated uses the system can identify impairments in visual ability. While unfortunately the visual dark areas will never return, the system can instead help train the patient understand their visual limitations and use head and body movement to compensate for the loss of visuospatial awareness. Upper-body motor function is also targeted in this simulation as the patient is required to point at the animal with their impaired limb. This movement exercises the shoulder, arm and even fine motor function in the wrist and fingers.
In the barrel breaker module the patient is tasked with bouncing a ball towards target barrels stacked along the back wall. Using their disabled arm the patient has to control a transparent sliding blocker used to keep the ball in play. This module is designed to rehabilitate visual object tracking and upper-body motor function. The game pushes the patient to focus on a moving ball that slowly increases in speed, while also navigating the sliding blocker. The combination of these functions help retrain the brain's ability to multitask cognitive ability with movement motor function. Score is kept so that over the course of repeated sessions the patient can see the progress of their rehabilitation.
The patient's comfort and engagement must always be the highest priority and for many users this platform will be their first experience trying virtual reality. Post stroke rehabilitation can be a long and frustrating journey, because of this it's important to make sure the technology never interferes with the recovery process. Even the smallest detail can have profound effects on the patient's experience, so every effort must be made to ensure the user is comfortable while in their VR treatment session.
The color pigment of the patient's hands are set before the beginning of any treatment sessions so there is no disconnect between their actual skin tone and their virtual representation.
Strokes also affect the way a person absorbs information. The platform allows for all instructions to be verbally communicated and written while a tutorial of the activities are being demonstrated.
Both the in-hospital and out-patient Neuro RecoVR systems are currently in patient trials and early results look very promising. Additions, adjustments and features are constantly being added to the platform all guided by the incoming data from user testing. Additional modules that focus on other areas of stroke rehabilitation such as speech and memory recovery are in development and will be integrated into the system when tested and approved by neurological therapists and doctors.
© Mike Lohaus - All rights reserved