Created and Validated by FDA: AI Drug Safety Surveillance Tool

By Staff Writer

October 3, 2024

Introduction

The accurate identification and timely updating of adverse reactions in drug labelling are critical for patient safety and effective drug use. Postmarketing surveillance plays a crucial role in identifying previously undetected adverse events (AEs) that emerge when a drug is used in broader and more diverse patient populations. Traditional methods of updating drug labelling with new AE information have been manual, time-consuming, and error-prone. The Food and Drug Administration (FDA) created and validated the LabelComp tool. This innovative artificial intelligence (AI) tool enhances the efficiency and accuracy of postmarketing drug safety surveillance. 

The Importance of Accurate Drug Labelling

Drug labelling provides healthcare professionals with a summary of essential scientific information needed for the safe and effective use of a drug. When a drug is approved, its known adverse reactions are included in the drug’s labelling. Real-world usage after a drug’s approval involves larger and more diverse patient populations compared to pre-approval clinical trials. Therefore, it is important to revise the Full Prescribing Information to provide accurate, up-to-date information on drug safety.

Prior research into safety-related labelling changes has explored the frequency and timing of safety issues and related labelling changes. These studies show that the majority of products have a new safety issue added after approval, and these updates continue throughout a drug’s lifecycle. However, previous efforts to characterise safety issues identified and added after approval have been tedious and time-consuming, involving manual comparisons of updated drug labelling with previous versions.

Introducing the LabelComp Tool

The LabelComp tool supports the comparison of FDA labelling documents within a web interface designed to identify product safety labelling changes. It has two main components. The first component uses text analytics to extract all texts from the two drug labelling documents being compared. The textual differences identified between the drug labelling documents are then highlighted to facilitate human expert review.

The second component leverages AI, specifically a Bidirectional Encoder Representations from Transformers (BERT) model, for natural language processing. The LabelComp tool uses the RxBERT model, a domain-specific BERT model developed by our group, which is based on BioBERT, with additional training on prescription drug labelling documents curated from FDALabel. RxBERT detects and highlights AE changes and their locations within specific sections of drug labelling documents. These sections include Boxed Warnings, Contraindications, Warnings and Precautions, Adverse Reactions, and Drug Interactions. 

Figure 1. LabelComp Tool Framework

Validation and Performance Evaluation

To determine the accuracy of the LabelComp tool in identifying safety-related labelling changes, researchers performed a validation study. Their goal was to create and validate a tool with high accuracy that could enable researchers and FDA reviewers to efficiently identify safety-related labelling changes.

The validation study of 87 drug labelling PDF pairs demonstrates the tool’s high accuracy. The F1 scores of overall performance ranged from 0.795 to 0.936 across different evaluation tiers, and the recall was at least 0.997, with only one missed AE out of 483 total AEs detected. This high recall indicates that LabelComp can identify nearly all newly added AEs, making it a significant advancement in drug safety surveillance.

Furthermore, one author conducted a post-hoc validation analysis by reviewing labels from 53 drugs approved by CDER in 2020. This analysis generated similar F1 scores across the three tiers and a recall of 1, confirming the tool’s high sensitivity.

Conclusion

The LabelComp tool is a significant contribution to pharmacovigilance, automating the identification of AE changes in drug labelling documents. Leveraging advanced AI techniques, including the RxBERT model for natural language processing, LabelComp effectively addresses the challenges of manual labelling comparison. Our validation study demonstrated LabelComp’s high accuracy in detecting new AEs in drug labelling, achieving a recall rate of 0.997.

As an open-source tool, LabelComp allows researchers to use, improve, and share enhancements with the public. You could adapt parts of the code to support other review or regulatory science activities. For example, the function that converts a PDF drug labelling document into machine-processable text is valuable. The ability to identify AEs in document sections also has potential for other uses.

LabelComp automates AE identification and provides detailed comparison outputs, marking a significant advancement despite its limitations. It doesn’t capture all AE changes and requires human validation for quality assurance. The FDA will continue developing and iterating AI tools for drug safety review and research. This aligns with its broader mission to use AI and emerging scientific approaches to advance regulatory science and public health. 

Reference url

Recent Posts

oral health Africa
    

Oral Health in Africa: Promoting Collaborative Solutions

🦷 Is oral health taking a back seat in public health discussions in Africa?

A new article reveals alarming statistics about the high prevalence of untreated dental diseases across the continent and a critical shortage of oral health professionals. It emphasizes the urgent need for collaborative action among healthcare providers and policy-makers to integrate oral health into broader public health frameworks.

Discover how strengthening partnerships can pave the way for improved health outcomes and resource allocation in oral health.

#SyenzaNews #globalhealth #HealthEconomics

tislelizumab NSCLC treatment
        

Early Benefit Assessment of Tislelizumab NSCLC Treatment: Insights and Implications

🧐 How is the evolving treatment landscape for NSCLC affecting patient access to tislelizumab?

The German Federal Joint Committee (G-BA) has launched an early benefit assessment for tislelizumab as a second-line treatment for adults with advanced NSCLC. This assessment notably focuses on PD-L1 negative patients and highlights the need for additional data to substantiate its value amidst a shifting emphasis on first-line immunotherapy.

Explore the nuances of this assessment and its implications for future research and market access in the full article.

#SyenzaNews #oncology #MarketAccess

colorectal cancer screening
    

Advances in Colorectal Cancer Screening: Access and Cost

🚀 Is blood-based screening redefining colorectal cancer detection?

The Shield blood test offers a non-invasive alternative to colonoscopy—boosting screening uptake, but raising questions around effectiveness and value.

🔍 Discover how this innovation could reshape patient care, payer strategy, and health system costs.

#SyenzaNews #HealthcareInnovation #CostEffectiveness #DigitalTransformation

When you partner with Syenza, it’s like a Nuclear Fusion.

Our expertise are combined with yours, and we contribute clinical expertise and advanced degrees in health policy, health economics, systems analysis, public finance, business, and project management. You’ll also feel our high-impact global and local perspectives with cultural intelligence.

SPEAK WITH US

CORRESPONDENCE ADDRESS

1950 W. Corporate Way, Suite 95478
Anaheim, CA 92801, USA

© 2025 Syenza™. All rights reserved.