Remote Health Assessments with Multimodal Technologies
By Melike Belenli Gümüş
August 29, 2024
Introduction
The rise of telehealth in the post-pandemic era has significantly impacted healthcare delivery, allowing patients to receive assessments and monitoring remotely. The U.S. Department of Health and Human Services has taken steps to promote telehealth adoption, particularly automated remote health assessments, addressing challenges like limited clinic access and bias in traditional assessments. Artificial intelligence (AI), machine learning (ML), and natural language processing (NLP) play essential roles in modern healthcare, enabling computer systems to analyse data and adapt without explicit programming. Recent advancements in digital technologies have facilitated the measurement of various human-centric signal modalities for remote patient assessment, offering insights into a wide range of health conditions, particularly in mental health and neurological disorders.
AI and NLP in Mental Health
AI and NLP have significantly improved the assessment of mental health conditions. By analysing text data from social media, interviews, and electronic health records, these technologies can identify early indicators of mental illness. For instance, linguistic features such as part-of-speech, sentiment scores, and topic modelling are used to detect signs of depression, anxiety, and other mental health issues. Recent studies have shown that NLP techniques can classify conversations of individuals with mental health disorders with high accuracy. This capability supports early detection and intervention, potentially improving patient outcomes.
Neurological Disorder Assessments
NLP and AI also play a crucial role in assessing neurological disorders. Conditions such as Alzheimer’s disease (AD), traumatic brain injury (TBI), and autism can be evaluated through conversational analysis. Research has demonstrated that features like turn length, phonation time, and speech fluency are key indicators of these disorders. These findings demonstrate the potential of AI and NLP in providing objective and accurate assessments of neurological conditions.
Patient-Reported Outcome Measures (PROMs)
Traditional PROMs, such as the Parkinson’s Disease Questionnaire –39 and the ALS Functional Rating Scale – Revised for amyotrophic lateral sclerosis (ALS), often lack the granularity needed to capture disease severity accurately. AI and NLP can address this issue by analysing patient-reported data in their own words. This approach allows for a more detailed understanding of how diseases affect daily functioning.
Multimodal Analytics for Disease Progression
Combining multiple signal sources, such as audio, video, and text, offers a holistic view of patient health. Multimodal analytics can objectively monitor disease progression over time. For instance, studies have shown that frequent and remote speech analysis can detect early bulbar changes in ALS patients. This capability allows for timely interventions and better management of the disease. Similarly, video-based gait analysis can assess Parkinsonian gait in older adults, providing critical information for treatment planning.
Figure 1: Schematic illustrating the distinction between signal sources and modalities of health information relevant for remote health assessment and monitoring, alongside the benefits obtained by combining information from multiple modalities.
Case Study: The Modality Platform
The Modality platform exemplifies the benefits of multimodal technologies in remote health assessments. Utilising a cloud-based system and a virtual human guide, the platform conducts structured interactions with participants, capturing data from speech, text, orofacial movements, limb motor functions, and more. Furthermore, participants complete the Patient Report Of Problems (PROPs), a tool enabling them to detail their symptoms and severity in their own words. This comprehensive approach enhances clinical interpretability, performance, user experience, and technology robustness, making it a valuable tool for remote assessments across various health domains.
Figure 2: The Modality multimodal dialog platform for remote patient assessment uses a virtual human guide, Tina, to engage participants in an interactive conversation, record their audio and video signals, and extract multiple modalities of health information therefrom.
Conclusion
The integration of multimodal technologies in remote health assessments holds promise for early detection, accurate diagnosis, personalised precision health, digital clinical trials, and improved patient outcomes. While these technologies offer significant benefits, challenges such as signal diversity, atypical speech patterns, and privacy concerns must be addressed for widespread adoption. Collaborative efforts in overcoming these challenges will drive advancements in digital medicine and enhance healthcare practices. By leveraging multimodal analytics, healthcare providers can gain a comprehensive understanding of patient health. As AI and NLP continue to evolve, their impact on mental health and neurological assessments will offer new possibilities for patient care.
🌍 Will Yeztugo Approval and Access Strategy Deliver an Inflection Point in HIV/AIDS?
Yeztugo (lenacapavir), a twice-yearly injection, redefines long-acting HIV prevention with 96–100% efficacy. Gilead’s access strategy—insurance, assistance, and generics for 120 countries—targets global impact. Can it balance $28,218 U.S. costs with equity for 1.3M annual infections? My article explores health economics, public health, and market dynamics.
📖 Read how Yeztugo’s access strategy could transform long-acting HIV prevention
💊 Are you curious about how Canada plans to make essential prescription drugs accessible for everyone?
Canada’s Drug Agency has launched a national consultation to create a list of essential prescriptions that aim to ensure equitable access, improve health outcomes, and reduce costs. This initiative marks a key step towards universal pharmacare and highlights the importance of evidence-based practices in healthcare.
Dive into our latest article to learn more about the implications for patients, policymakers, and the broader healthcare system!
The recent overhaul of the US Advisory Committee on Immunization Practices (ACIP) raises critical concerns about the integrity and transparency in vaccine recommendations. The abrupt removal of longstanding members may disrupt the essential processes that have historically upheld public trust and immunization success.
Curious about the implications for health policy and public health? Dive into the full analysis to understand the balance needed between continuity and reform!
When you partner with Syenza, it’s like a Nuclear Fusion.
Our expertise are combined with yours, and we contribute clinical expertise and advanced degrees in
health policy, health economics, systems analysis, public finance, business, and project management.
You’ll also feel our high-impact global and local perspectives with cultural intelligence.
