UX RESEARCH: FACIng dementia in dignity

Microsoft Design expo 2017 Challenge

Mixed-Reality application Design

Creative Field   Mixed-Reality, UX

Time                    20 February, 2017 - 18 April, 2017

Collaborator     Xingjian An, YiChi Ma, YiTing Liu, Yun Li


 "MEMO" is a Hololens Mixed Reality application that delivers customizable cognitive self-treatment for individuals with dementia. It helps people with early stage dementia to receive self-treatment any time in their homes. This application can be initially customized by doctors for each individual user and can track user progress so that doctors could adjust the content to each user's need. 


The inspiration of the design is from my family story. My grandma was diagnosed with dementia several years ago. She was a confident woman before. However, her dementia treatment depended on the doctor and our family, which made her unhappy and unconfident. She received cognitive training to delay her dementia progression. However, the trainings were tedious and she felt offended and frustrated when she had to train in front of other people. It made her feel depressed in her later years.

It’s my family’s story, but it’s also a common family story. According to recent medical reports, around 46.8 million people lived with dementia around the world in 2015. This may become your own story in the future. After the age of 65, the risk of dementia doubles every five years. This means that half of us may have dementia by the time we are 85. [1]






Part 1. Research about dementia

Dementia is not a specific disease. It's an overall term that describes a wide range of symptoms associated with a decline in memory or other thinking skills severe enough to reduce a person's ability to perform everyday activities. Alzheimer's is the most common type of dementia [1]

Dementia has three stages. For those in the early stages, it has been proven that professionally designed training can delay cognitive deterioration. Our target users are people with early-stage dementia. 


Part 2.  Researching existing medical treatment

People suffering from dementia receive both pharmaceutical and non-pharmaceutical treatment. Out of the non-pharmaceutical treatment, we found the following can be replicated and enhanced in MR technology: 

1. Reality orientation, which is a therapeutic approach to point out dates, locations, current surroundings frequently to patients. 

2. Cognitive and memory training, which uses a set of exercises or games to improve patients’ memories and cognitive functioning.

3. Reminiscence therapy, which help patients recall their memories by using photos and personal belongings.

4. Social engagement can delay patients’ further cognitive decline.




To get a better understanding of our users, we interviewed with several caregivers who have family members living with dementia and gained some valuable insights. 

  • People with dementia have been shown to become self-conscious and defensive as they start to lose their memories and are asked to do the same training over and over again.
  • People with dementia have to go to clinics to get training regularly. It consumes money and time to travel between home and clinics.
  • Some people with dementia are unwilling to do the training because training is tedious.
  • Some training can be done at home, but it would be hard for doctors to track the results.




According to the research and interview, we decided to focus on helping people with dementia receive cognitive training in dignity. Cognitive training is a set of exercises to improve cognitive functioning in the aspects of attention, perception, memory, language, and spatial cognition. Currently, the most common methods are paper-based games, such as crossword puzzles and matching games. With the increased availability of electronic devices, some applications are designed to help cognitive training. However, with both methods, users focus on what’s in their hands, so they are disconnected from the environment they live in. With MR, we can provide a spacial environment in which users can undergo self-treatment. 

Specificaliy, we designed a Hololens app solution. Here are four key features in our app:

  • The app can be used at home because with a familiar environment, users might feel calm and less stressed.
  • The app has a self-training therapy quality. Instead of asking users to perform treatment in front of people, the self-training method will ease users’ anxiety.
  • The users can train themselves instead of going to a clinic, which saves their families' money and time.
  • To provide more specific therapy, the app can also be customized by doctors and caregivers according to a user's personal interests, background, and their recent conditions. In addition, the users' families or caregivers can record instructions to make it easier for users to follow the therapy.



Based on the research and interview, we summarized some common characteristics of our users and then created a persona.


  • 75-year-old
  • Retired art history teacher
  • Fond of arts and new technology
  • Divorced, has 2 sons living in other countries

"I don’t want to be anyone’s burden."

John was an art-history teacher who is fond of art and new technology. He is 65 years old and recently discovered he has mild dementia. He hopes that he can maintain his ability to live individually. However, his families live far away from him and cannot take care of him all the time. Therefore, he hopes he can receive treatments at home by himself.




Based on these required features and persona, we built a Hololens working prototype, which provides cognitive training game for people with dementia. We also build an interactive web prototype for the doctor. This web prototype would allow doctors to customize the cognitive training game for people with dementia and track their performance.

Customizing and Tracking system prototype for doctors and caregivers

Hololens Working Prototype


USER TEST and interview


Initiately, we brought these prototypes to caregivers to try since they understand the target users and their treatment very well. 

We gained some positive feedbacks and advices from caregivers:

  • They really like the training contents are private to the users, so people with dementia don’t have to expose their weakness in front of other people.
  • They suggest more use of the spatial space. 
  • They suggest more intuitive gestures for people with dementia.


Following that, we sought feedback from an Alzheimer's Disease expert, professor Dr. Mary Mittelman at NYU Alzheimer's Disease Center, who provided valuable insights: 

  1. It is important to keep the person doing what they can for as long as possible. People with dementia will take pride in their accomplishment, which will positively impact other areas.

  2. It is nice to allow doctors to track these progress remotely.

  3. It is valuable to allow people to maintain their daily interactions and routines at home. 

  4. In order to help the users adjust to the new training experience, caregivers should be involved in the transition period.



According to the result of user testing, we want to further iterate our product:

1. We want to explore the social aspect in cognitive training and offer the option for caregivers to also use  the Hololens to interact with people with dementia. We believe it would better satisfy the emotional needs of people with dementia and allow caregivers to better empathize with people with dementia. 

2. We want to ensure that our product is intuitive and easy to use for people with dementia.

3. We want to enrich the training exercises that we offer in our system and fully utilize the spatial features of Mixed Reality.



[1] https://bit.ly/alzheimersday

[2] http://www.alz.org/what-is-dementia.asp

Thumbnail picture: http://www.alz.org/