Mike Lohaus logo

NeuroRecoVR

Neuro RecoVR is a stroke rehabilitation platform that utilizes simulated modules to aid in post-stroke recovery. This AR/VR treatment employs activities and mini-games that are carefully designed to stimulate damaged parts of the brain. As the lead designer on the project I had a hand in the conceptualization and design of each of the rehab modules. Through focused meetings with leading doctors and therapists in the neurological field, we were able to create a handful of modules designed specifically to address key areas of stroke rehabilitation.

The system is being built in 2 stages: a treadmill-based solution focusing on lower-body motor function through in-hospital care, and a follow-up system that utilizes consumer VR headsets to continue a patient's cognition and upper-body motor function treatment at home.

Current problems to overcome

The vast majority of current stroke rehabilitation procedures do not use any kind of modern technology. Therapists usually work one-on-one with patients guiding them through activities and exercises that if repeated often enough will recondition the brain and allow for recovery in motor function and cognition. While the work done with therapists is important and leads to successful degrees of rehabilitation, the approach is not without its flaws.

First and foremost the demand for stroke rehabilitation therapists is currently extremely high, which not only makes them very expensive but also means they are often overbooked and may not be able to see patients as frequently as needed. Therapists are also usually much more available to people near cities; patients in rural locations are usually under-supported due to travel time and limited access to nearby medical centers.

Last but not least the Covid-19 pandemic showed the world that we need solutions that could be implemented without in-person contact. During the pandemic many stroke rehabilitation patients went without care for extended periods of time.

Our solution is to provide doctors and therapists with a digital tool that will empower them with data to quantify a patient's rehabilitation journey.

Split-belt treadmill

Neuro RecoVR in-hospital patient care

The in-hospital Neuro RecoVR platform utilizes a split belt treadmill system that doctors and therapists can use for focused rehabilitation of lower body motor function. The treadmill also has the functionality of pressure tracking through plates below the belt. We utilized these features in the treadmill to create custom software modules that focus on specific areas of a patient's rehabilitation. Three modules were designed to facilitate growth in lower body motor function, each one increasing in challenge and movement complexity. Patient rehabilitation data tracking is at the forefront of the Neuro RecoVR platform. Every session on the system tracks the challenges the patient is tasked with and response to the treatment. Over time with continued use of the platform the system will compile logs of patient data, and through data visualizations doctors and therapists are able to see where the rehabilitation process is working or falling short. They can then use this collected data to make informed decisions about the patient's next steps and focused treatments.

Module conceptualization

When building medical technology software, it is crucial to understand the needs of the people using it. We achieved this by engaging with stroke rehabilitation doctors and therapists at every stage of the software development process, from requirements gathering to testing and validation. It was a very collaborative process and of the utmost importance to listen to their feedback and suggestions, and to incorporate their perspectives into the design and functionality of Neuro RecoVR. Additionally, it is essential to have a thorough understanding of the clinical workflow and how the software will be used in real-world medical settings. This can help ensure that the software is intuitive, user-friendly, and tailored to medical professionals' specific needs and requirements. Ultimately, by focusing on the needs of medical professionals, software developers can build technology that truly meets the needs of the people who will be using it, improving patient outcomes and streamlining healthcare processes.

Patient in a harness on the treadmill during a pre-gait session

The pre-gait module

The goal of the pre-gait module is to get patients that no longer have the ability to stand, up and applying weight to their injured extremity. By utilizing the scaffold mounted harness the full weight of the patient can be reduced on their lower body. The therapist then has the ability to allocate a target on the belt for foot placement, the amount of weight they want the patient to apply and the length of time the patient should hold their pose. Through this process and repeated sessions the patient should regain ability in their lower body to the point where walking (or gait) treatments can begin. The colors used in the patient UI are all high contrast and adhere to colorblind hue recommendations.

Gait module running on the treadmill screen

The gait module

Once a patient is able to stand it is time to train their lower body to walk again. The gait module does this by utilizing the split belt treadmill. On screen the patients will be viewing a 360 degree video of walking along the Bow River. While again strapped into a harness the patient is tasked with taking a summer walk along the river. The therapists have the ability to independently change the speeds on each side of the belt. The goal is for the patient to exercise and rehab their affected leg so that they can gradually speed up that belt to the point where the two legs are functioning at the same walking speed.

Gait and awareness module on the treadmill screen

The gait and awareness module

After a patient has regained their ability to walk at a moderate pace it's time to add cognitive exercises into the mix. Most healthy people have no problem dealing with distractions while walking, however a stroke can impair a patient's ability to focus on their walking balance, leading to dangerous falls. In the final module the harnessed patient is tasked with walking down a digital path while looking for specific animals. The therapists have a fully custom digital sandbox they can use to spawn animals or distractions. The patient can then point to the animals, and using a hand tracking system the software can tell if they are pointing at the right location, awarding the patient points.

NeuroRecoVR therapist dashboard on a monitor

User interface

Therapist login screenPatient dashboardPatient dashboard with music playlistPre-gait session launch dashboardGait session launch dashboardGait and awareness session launch dashboard

Designing clean and intuitive user interfaces is crucial when building medical technology software. These interfaces should be designed with simplicity and ease of use in mind, to minimize the risk of user errors and improve efficiency in medical settings. Clear and concise navigation, using simple and recognizable icons, can help medical professionals quickly and easily find the information and tools they need. Many of the therapists commented that current medical software has not aesthetically matured since the days of Windows XP. Because of this I created an interface that used high-contrast colors, gradients, and bold text to improve readability, reducing the risk of human error in fast-paced clinical environments. By designing clean and intuitive user interfaces, medical software developers can improve the experience of the professionals using it and help ensure that patient care is delivered more efficiently and effectively.

NeuroRecoVR VR headset

Neuro RecoVR out-patient care

After a patient has been discharged from the hospital additional rehabilitation is often needed. The out-patient phase of the Neuro RecoVR platform utilizes consumer VR headsets to engage patients with minigames specifically designed to improve upper-body motor function and cognitive abilities. The data from these sessions are sent to a secure server where medical professionals can monitor a patient's progress remotely and adjust the treatment as they see fit.

Eye-spy module: animals in a wooded VR environment

The eye-spy module

The eye-spy module is specifically designed to rehabilitate visuospatial awareness and upper-body motor function. Patients are asked to identify specific animals walking around a wooded environment. This exercise was built to get the patient scanning a large environment and after repeated uses the system can identify impairments in visual ability. While unfortunately the visual dark areas will never return, the system can instead help train the patient to understand their visual limitations and use head and body movement to compensate for the loss of visuospatial awareness. Upper-body motor function is also targeted in this simulation as the patient is required to point at the animal with their impaired limb. This movement exercises the shoulder, arm and even fine motor function in the wrist and fingers.

Barrel breaker module: a ball and barrels in VR

The barrel breaker module

In the barrel breaker module the patient is tasked with bouncing a ball towards target barrels stacked along the back wall. Using their disabled arm the patient has to control a transparent sliding blocker used to keep the ball in play. This module is designed to rehabilitate visual object tracking and upper-body motor function. The game pushes the patient to focus on a moving ball that slowly increases in speed, while also navigating the sliding blocker. The combination of these functions helps retrain the brain's ability to multitask cognitive ability with movement motor function. Score is kept so that over the course of repeated sessions the patient can see the progress of their rehabilitation.

Sarah, the digital guide avatar

Meet Sarah, the patient's digital guide

The goal of this out-patient rehabilitation platform is to allow therapists to work with many patients at once. It is therefore important that the user experience has a digital guide, as therapists are not regularly on site to administer treatment. The largest demographic that interacts with the platform are the elderly, who have very little familiarity with digital engagements. Because of this, it was critical to create an avatar that would be a relatable character and guide for the patient. All of the instructions are relayed through Sarah so it was crucial that she clearly communicates the expectations and objectives of each module. Stroke recovery is a long and frustrating experience so she is also designed to be an encouraging voice that continually motivates the patient to continue with their treatment.

Designing for equity in VR

The patient's comfort and engagement must always be the highest priority and for many users this platform will be their first experience trying virtual reality. Post stroke rehabilitation can be a long and frustrating journey, because of this it's important to make sure the technology never interferes with the recovery process. Even the smallest detail can have profound effects on the patient's experience, so every effort must be made to ensure the user is comfortable while in their VR treatment session.

The color pigment of the patient's hands are set before the beginning of any treatment sessions so there is no disconnect between their actual skin tone and their virtual representation.

Strokes also affect the way a person absorbs information. The platform allows for all instructions to be verbally communicated and written while a tutorial of the activities are being demonstrated.

Personas and user journey

Understanding the user needs of the platform is critical to the success of the treatment. The first priority for doctors is to discern the conditions and amount of disability of the people who will be using the system for rehabilitation. Once this is established the medical professionals can recommend an entry point to the platform. Strokes can cause a wide range of debilitations and thus every user will have different needs for their specific treatment. While the Neuro RecoVR platform is designed to rehabilitate several different post-stroke conditions, finding the best exercise module for each patient is incredibly important for long-term success. It's also imperative to remember that these patients are human beings and have their own goals and frustrations while going through the rehabilitation process. These details are essential to keep in mind when working with patients to complete their goals and raise their quality of life.

In addition to the three personas with post stroke disabilities we also created a fourth centered around spinal cord injury. While Neuro RecoVR is designed specifically for post stroke rehabilitation, the platform can also theoretically be used by patients with neurological trauma. We created the fourth persona "Logan" so that we could also determine how there might be additional beneficial uses for the platform.

Portrait of Cassandra

Cassandra

Age:
48
Work:
Marketing Director
Family:
Married, 2 children

Diagnosis:

Ischemic stroke on the right side of the brain. Cassandra has limited mobility in her left arm, but her lower body motor function is still relatively unaffected aside from a slight limp. In addition to limitation of her left arm motor function she also has trouble with vision on her left side and struggles with some memory loss.

Recommended treatment:

Once discharged from the hospital she should be sent home with the Neuro RecoVR headset and tasked with using it for at least 15-30 minutes a day. Both the Eye Spy and Barrel Breaker modules will assist in rehabilitating the motor function of her left arm and visuospatial awareness. She will also be scheduled for semi-regular sessions on the gait + awareness treadmill module to rehabilitate the weakness in her left leg while also focusing on her loss of left lateral vision.

Passions and hobbies:

Cooking, painting, gardening

Portrait of William

William

Age:
81
Work:
Retired Plumber
Family:
Married, 2 children, 4 grandchildren

Diagnosis:

Ischemic stroke on the left side of the brain. William has significant impairments to the right side of his body, currently leaving him wheelchair bound. In addition to the physical conditions William also has problems with his speech and organizational abilities.

Recommended treatment:

William will need a significant amount of sessions on the pre-gait module if he is to return to standing; even if he gathers enough strength to start walking again he will almost certainly require a walker. He will use both the pre-gait and gait modules to rehabilitate his lower body motor function while still receiving regular one-on-one physical therapy sessions. His upper body motor function might be too severe to use the VR headset, so traditional one-on-one physical therapy sessions should be utilized first until a stroke therapist can assess his ability to safely use the VR headset.

Passions and hobbies:

Solitaire, bingo, playing with his grandchildren

Portrait of Tamil

Tamil

Age:
76
Work:
Retired Seamstress
Family:
Widowed, 2 children, 2 grandchildren

Diagnosis:

Ischemic stroke on the right side of the brain. Tamil can walk but at a very slow pace and with a moderate limp on her left side. She also has limited use of her left arm and fine motor function. In addition to her physical impairments Tamil also has trouble with decision making and memory loss.

Recommended treatment:

Tamil will be scheduled for frequent treadmill sessions starting with the gait module to start to correct her step length and timing while walking; she will then be upgraded to the gait + awareness module to focus on distractions while moving. At the same time she can be using the VR headset modules for at least 15 minutes per day to exercise her left arm and fine motor function. With enough time using the headset and in-person physical therapy we are hopeful that she can return to doing her handmade embroidery.

Passions and hobbies:

Gem collecting, embroidery, meditation

Portrait of Logan

Logan

Age:
23
Work:
Bartender
Family:
Mother, Father, 2 sisters

Diagnosis:

Spinal cord trauma below the L1 vertebrae due to a snowboarding accident. Logan has significant impairments to his lower body motor function, though is not totally paralyzed from the waist down.

Recommended treatment:

With consistent physical therapy it is possible for Logan to regain the use of his lower body motor function. The pre-gait module should be the starting point for the treatment so that he is able to regain the ability to balance and stand again; from there the gait module should be used in conjunction with traditional lower body physical therapy so that he can build the strength to walk and hopefully one day return to snowboarding.

Passions and hobbies:

Snowboarding, skateboarding, surfing

Now that user personas have been defined, we can plot which Neuro RecoVR modules will be the most beneficial specifically for each patient's rehabilitation efforts.

Treadmill - In hospital care
WilliamLogan

Module 1 - Pre-Gait

  • Focus on stationary lower body motor-function
  • Focus on balance
  • Focus on weight distribution
  • Focus on increasing standing session lengths
Treadmill - In hospital care
WilliamTamilLogan

Module 2 - Gait

  • Focus on leg movement function
  • Focus on stride length
  • Focus on step pressure
  • Focus on increasing walk session time
Treadmill - In hospital care
TamilCassandra

Module 3 - Gait + Awareness

  • Lower body motor-function
  • Focus on attention to specific objects
  • Balance when adjusting area of focus
  • Focus on visual-spatial awareness
VR - At home care
TamilCassandra

Module 1 - Eye Spy

  • Focus on visual-spatial awareness
  • Focus on attention to specific objects
  • Focus on memory and thought retention
  • Focus on impaired limb upper body motor-function
VR - At home care
TamilCassandra

Module 2 - Barrel Breaker

  • Focus on upper body motor-function
  • Focus on visual-spatial awareness
  • Focus on object tracking
  • Focus on attention to specific objects

Ongoing patient testing

Both the in-hospital and out-patient Neuro RecoVR systems are currently in patient trials and early results look very promising. Additions, adjustments and features are constantly being added to the platform, all guided by the incoming data from user testing. Additional modules that focus on other areas of stroke rehabilitation such as speech and memory recovery are in development and will be integrated into the system when tested and approved by neurological therapists and doctors.

Photo of Mike Lohaus

I love collaborating with ambitious people and brands. Let's create something awesome together!

Best ways to reach me

Book an online meeting

Choose a date and time :)

Email

mike@mikelohaus.com

Social media