Created and Validated by FDA: AI Drug Safety Surveillance Tool

By Staff Writer

October 3, 2024

Introduction

The accurate identification and timely updating of adverse reactions in drug labelling are critical for patient safety and effective drug use. Postmarketing surveillance plays a crucial role in identifying previously undetected adverse events (AEs) that emerge when a drug is used in broader and more diverse patient populations. Traditional methods of updating drug labelling with new AE information have been manual, time-consuming, and error-prone. The Food and Drug Administration (FDA) created and validated the LabelComp tool. This innovative artificial intelligence (AI) tool enhances the efficiency and accuracy of postmarketing drug safety surveillance. 

The Importance of Accurate Drug Labelling

Drug labelling provides healthcare professionals with a summary of essential scientific information needed for the safe and effective use of a drug. When a drug is approved, its known adverse reactions are included in the drug’s labelling. Real-world usage after a drug’s approval involves larger and more diverse patient populations compared to pre-approval clinical trials. Therefore, it is important to revise the Full Prescribing Information to provide accurate, up-to-date information on drug safety.

Prior research into safety-related labelling changes has explored the frequency and timing of safety issues and related labelling changes. These studies show that the majority of products have a new safety issue added after approval, and these updates continue throughout a drug’s lifecycle. However, previous efforts to characterise safety issues identified and added after approval have been tedious and time-consuming, involving manual comparisons of updated drug labelling with previous versions.

Introducing the LabelComp Tool

The LabelComp tool supports the comparison of FDA labelling documents within a web interface designed to identify product safety labelling changes. It has two main components. The first component uses text analytics to extract all texts from the two drug labelling documents being compared. The textual differences identified between the drug labelling documents are then highlighted to facilitate human expert review.

The second component leverages AI, specifically a Bidirectional Encoder Representations from Transformers (BERT) model, for natural language processing. The LabelComp tool uses the RxBERT model, a domain-specific BERT model developed by our group, which is based on BioBERT, with additional training on prescription drug labelling documents curated from FDALabel. RxBERT detects and highlights AE changes and their locations within specific sections of drug labelling documents. These sections include Boxed Warnings, Contraindications, Warnings and Precautions, Adverse Reactions, and Drug Interactions. 

Figure 1. LabelComp Tool Framework

Validation and Performance Evaluation

To determine the accuracy of the LabelComp tool in identifying safety-related labelling changes, researchers performed a validation study. Their goal was to create and validate a tool with high accuracy that could enable researchers and FDA reviewers to efficiently identify safety-related labelling changes.

The validation study of 87 drug labelling PDF pairs demonstrates the tool’s high accuracy. The F1 scores of overall performance ranged from 0.795 to 0.936 across different evaluation tiers, and the recall was at least 0.997, with only one missed AE out of 483 total AEs detected. This high recall indicates that LabelComp can identify nearly all newly added AEs, making it a significant advancement in drug safety surveillance.

Furthermore, one author conducted a post-hoc validation analysis by reviewing labels from 53 drugs approved by CDER in 2020. This analysis generated similar F1 scores across the three tiers and a recall of 1, confirming the tool’s high sensitivity.

Conclusion

The LabelComp tool is a significant contribution to pharmacovigilance, automating the identification of AE changes in drug labelling documents. Leveraging advanced AI techniques, including the RxBERT model for natural language processing, LabelComp effectively addresses the challenges of manual labelling comparison. Our validation study demonstrated LabelComp’s high accuracy in detecting new AEs in drug labelling, achieving a recall rate of 0.997.

As an open-source tool, LabelComp allows researchers to use, improve, and share enhancements with the public. You could adapt parts of the code to support other review or regulatory science activities. For example, the function that converts a PDF drug labelling document into machine-processable text is valuable. The ability to identify AEs in document sections also has potential for other uses.

LabelComp automates AE identification and provides detailed comparison outputs, marking a significant advancement despite its limitations. It doesn’t capture all AE changes and requires human validation for quality assurance. The FDA will continue developing and iterating AI tools for drug safety review and research. This aligns with its broader mission to use AI and emerging scientific approaches to advance regulatory science and public health. 

Reference url

Recent Posts

FDA warning Aspen Pharmacare
   

FDA Warning Aspen Pharmacare: Evaluating Compliance Violations and Their Impact

🚨 Is your pharmaceutical brand prepared for FDA scrutiny?

Aspen Pharmacare Holdings recently faced a serious FDA warning highlighting significant violations in its manufacturing practices. These issues not only jeopardize product quality but also threaten access to vital medicines for patients.

Explore the implications of regulatory compliance on brand reputation and global supply in the full article!

#SyenzaNews #pharmaceuticals #regulatoryaffairs #MarketAccess

pharmaceutical innovation 2025
      

Pharmaceutical Innovation and Market Access: Insights from Fast Company’s 2025 Top List

💊 Explore how top pharma companies are driving change in 2025! This article breaks down key innovations and smart access strategies from Fast Company’s most impactful list—giving you actionable insights into pharma innovation and market access 2025. 🚀

#SyenzaNews #pharmaceuticals #AIinHealthcare #innovation

global partnerships SCD
    

Global Partnerships SCD in Africa

🌍 How can global partnerships transform care for sickle cell disease in Africa?

In the latest article, we explore the significant strides made by the SickleInAfrica consortium, which highlights the power of collaboration in enhancing SCD management through comprehensive patient registries, clinical guidelines, and innovative research. These efforts not only improve health outcomes but also pave the way for sustainable solutions across the continent.

Dive deeper into this crucial conversation and discover how international forums are shaping the future of SCD care!

#SyenzaNews #globalhealth #HealthcareInnovation

When you partner with Syenza, it’s like a Nuclear Fusion.

Our expertise are combined with yours, and we contribute clinical expertise and advanced degrees in health policy, health economics, systems analysis, public finance, business, and project management. You’ll also feel our high-impact global and local perspectives with cultural intelligence.

SPEAK WITH US

CORRESPONDENCE ADDRESS

1950 W. Corporate Way, Suite 95478
Anaheim, CA 92801, USA

© 2025 Syenza™. All rights reserved.