Design Conversational Agent for Elderly with Heart Disease to Monitor Their Health Conditions
#HCI #Healthinformatics #VoiceAgent #VUI #AI
1.Project
We designed a proactive conversational agent with an AI Risk calculate model to evaluate a patient's health condition and give real-time medical suggestions. The conversational agent will initiate conversation with users on a daily basis and monitor their diet, sleep hours, mental health, bio data by asking users questions. One of the design challenges in our early stage is, what is the best time for agents to initiate the conversation for chronic disease monitoring. Two possible solutions came out:
(1) Monitoring users’ biodata and sending notification accordingly
(2) Initiate a conversation on a daily basis depending on users’ time availability. We went for the second solution, since the frequent alert might not encourage positive outcomes, and for chronic disease, a daily monitoring on a fixed time for patients is easier to do.
My Role
VUI Design, Researcher, AI Logic Design
Duration
5 Weeks
2. PROBLEM
The increase of the aging population presents challenges to social care and healthcare, in particular, the prevalence of heart diseases among the elderly and the need for long-term management is increasing healthcare costs. A good quality of self-monitoring is one of the most important steps to control the chronic disease before it becomes vital. However, it is a challenge that high quality self-monitoring involves a lot of factors, such as bio data, diet tracking, physical activity, sleep hours and mental health etc, which requires a lot of technical knowledge and equipment to measure. This pushes many older adults with chronic diseases away from high-quality monitoring processes. Our design goal is to design a voice interface combined with a medical assistant to offer real-time advice to older adults with heart diseases specifically. We will focus on improving speech input usability and voice interface accessibility for the elderly.
3.Design SOLUTION
3-1. Conceptual Design
Easy levels will require players to apply the least amount of grip strength. Harder levels require players to make full fists. The levels get more difficult as their rehabilitation progresses. The difficulty chosen for the grips drew inspiration from both the Sollerman test and the Bimanual Fine Motor Function (BFMF) scale. The most imperative gestures the game needs to emulate are the pulp pinch and lateral pinch since they are the most common gestures used in Activities of Daily Life(ADL).
.jpg)
Figure 01. The Conceptual Model of how the system works
The conceptual model of our system designed works as Figure 01. We learned from the real-world diagnostic process from clinicians and were inspired by natural communication to gather information. We designed our system by matching users’ input into pre-set categories and asking follow-up questions based on users’ responses. The AI will keep the conversation going until all risk contributions are collected by our AI system or the risk evaluation is completed.
3.2 Category User Inputs
We need two main functions to build any voice-based assistant. Firstly, the ability to listen to you and Secondly, the ability to respond to your commands. Besides these two core functions, our assistant will also need customized instructions. Installing and importing the necessary libraries is the first step. Before importing the libraries, use pip install to install them. Following are some of the key libraries used in this program: SpeechRecognition, Google’s text-to-speech package (gTTS), and Playsound package.
Diet
Diet
Diet
Diet
Diet
In addition, we use the Natural Language Toolkit (NLTK), which is a suite of libraries and programs for symbolic and statistical natural language processing for English written in Python. It contains text processing libraries for tokenization, parsing, classification, stemming, tagging and semantic reasoning. For our dataset there is no effective dataset that includes all the categories we need, what we did is to collect multiple Frequently Asked questions for each category and used that question to assess the user.
3.3 Prototype Sequence
We divide users into two categories: New users and existing users. For the new users, they will hear the goal of this conversational agent, a simple tutorial about how to use the conversational agent, a demographic information set-up including the medical history, and a preferred time for receiving notification. For existing users, the conversation will be initiated by an agent on a daily basis, and the existing users are asked questions about their health condition. Here are some examples of the dialogues based on different situations.
VoiceFlow Prototype 01: New User Set-up
4.TESTING PROCEDURE
User Interface Test
Since there is no standard of testing Voice User Interface, we follow the rule of usability heuristics for User Interface created by Jakob Nielson. We mainly focus on “User control and freedom”, “Error prevention”, “Flexibility and efficiency of use”, “Help users recognize, diagnose, and recover from errors” and “Help and documentation”. We did an in-group test first to test the functionality and usability for the first round, and based-on the testing results, we kept iterating the existing prototypes. Due to the lack of real users, another round of user testing will need to be done in the future to validate the design.
Couple of questions exist in our current design. Since we are using VoiceFlow to create the prototype, its voice recognition is not accurate when testers have an accent or are sick, which changes the voice tone. Another issue happened with tracking users’ medicine history. The VoiceFlow is hard to recognize the medicine’s name since it is not in its database. However, our self-developed AI can possibly solve this by feeding data in its data collection.
5.CONCLUSION
In this report, we present the process of building medical assistants that helps the elderly in monitoring their health. This pushes many older adults with chronic diseases away from high-quality monitoring processes. Even though people do an excellent job in constantly monitoring their health status, another challenge still exists. We will focus on improving speech input usability and voice interface accessibility for the elderly. Also, Older people may be more comfortable interacting via speech. We use a Voiceflow to design our prototype and Multinomial Naive Bayes has been used for classification and for the analysis of the categorical text data. For future work, we might use a better dataset and the state of art model Bidirectional Encoder Representations from Transformers BERT.