LLM Judges Clinical Evaluation Limits in Multilingual Health Settings

Large language models deployed for clinical decision support require ongoing evaluation to confirm safety, yet reliance on human clinicians creates scalability barriers due to cost and variability. The central analysis demonstrates that LLM Judges Clinical Evaluation Limits prevent even the strongest LLM judge from reaching equivalence with human ratings on more than four of eleven criteria, while model-specific tendencies toward excessive leniency or severity persist and worsen when non-English languages are involved.

Scalability Barriers in Expert Review

These patterns indicate that current LLM-as-a-judge approaches cannot yet substitute for expert review in linguistically diverse global health settings. The evaluation framework adapted eleven dimensions from the Med-PaLM-2 rubric, encompassing alignment with medical consensus, logical reasoning, and potential for demographic bias, to score responses generated for Rwandan community health worker queries.

Agreement Metrics Across Evaluators

Six bilingual clinicians from a district hospital provided reference ratings on a random subset of 524 query-response pairs, while five separate LLMs received identical instructions and few-shot examples to produce parallel judgments. This dual-track design enabled direct quantification of agreement via rank correlation and odds ratios, isolating differences attributable to evaluator type rather than content variation, as shown in the linked study.

Ad

Persistent Shortfalls in Bias Detection

Inter-evaluator consistency proved higher among LLMs than among clinicians, yet no individual model satisfied equivalence thresholds across the full criterion set, and jury aggregation added equivalence on only one additional dimension. LLM Judges Clinical Evaluation Limits were further exposed when demographic bias detection failed uniformly, with virtually all LLM judges assigning perfect scores irrespective of response content, while human raters exhibited preference for clinician-generated answers and LLMs displayed length-related scoring inflation. Performance metrics declined and per-response costs rose when shifting from English to Kinyarwanda, although absolute LLM expenses remained far below human benchmarks.