AI Patient Education: Evaluating ChatGPT and Gemini for Pediatric Condition Materials

A recent peer-reviewed study examined the effectiveness of AI patient education by comparing ChatGPT and Gemini in generating educational materials for common pediatric exanthematous conditions, including varicella, hand, foot, and mouth disease (HFMD), and measles. The findings indicated that ChatGPT produced significantly longer responses than Gemini (p=0.047). However, readability, ease of understanding, and reliability—measured by Flesch-Kincaid and a modified DISCERN score—showed no significant difference between the two AI tools.

Depth and Clarity: A Balancing Act

• Content Depth vs. Readability: ChatGPT generated higher word counts, potentially offering in-depth information. Yet, this did not improve readability or reliability compared to Gemini. Both AI tools provided materials close to a ninth-grade reading level, which may challenge certain patient populations.
• Reliability and Quality: Gemini achieved a slightly higher mean reliability score (3/5 vs. 2.67/5 for ChatGPT), though this difference was not statistically significant. Both scores indicate room for improvement in the quality of AI Patient Education materials.
• No Correlation Between Readability and Reliability: The study found no link between ease of reading and reliability. Simpler text does not necessarily mean higher quality or more accurate information.

Importance of Accessible Patient Education

Varicella, HFMD, and measles are frequent viral exanthems in pediatrics. Early recognition and parental understanding are critical for quality management and minimizing disease transmission. Health organizations like WHO and CDC stress the need for clear, accurate, and comprehensible patient education materials in pediatric infectious diseases.

Large language models like ChatGPT and Gemini are increasingly used for patient education. However, these tools need thorough evaluation for accuracy, readability, and appropriateness across diverse patient demographics. Most guidelines recommend patient materials at a 6th to 8th-grade reading level—a benchmark neither AI tool consistently met in this study.

Implications for Healthcare and Economic Outcomes

Reading level difficulties present barriers to equitable healthcare outcomes. Low health literacy is linked to increased healthcare costs, poorer outcomes, and diminished patient engagement—key concerns for payers and providers. As AI-generated content grows in digital health, payers may demand validation of accuracy, readability, and patient-centered design for reimbursement. This study highlights the need for continuous AI validation, expert oversight, and iterative improvements before broad clinical or commercial use.

The authors suggest real-time validation, clinician review, and structured patient feedback could improve content quality. Such enhancements may be prerequisites for payer support and integration into healthcare systems.

Findings and Future Directions

In summary, this study shows AI tools like ChatGPT and Gemini can generate educational materials on pediatric rashes, but current outputs fall short of optimal readability and reliability standards. This underscores the need for real-world validation, multidisciplinary oversight, and adherence to health communication guidelines to fully leverage AI patient education—impacting public health, cost-effectiveness, and healthcare access.