•  
  •  
 

Document Type

Conference Proceedings

Abstract

Objective: We aimed to evaluate the ability of ChatGPT-4o to generate personalized, culturally appropriate, and readable educational materials for patients with diabetes using simulated realistic clinical notes. Methods: We prompted Gemini 2,5 to create four unique patient notes, incorporating characteristics such as age, ethnicity, health literacy level, glycemic control, comorbidities, and social determinants of health. We prompted ChatGPT-4o to create personalized education materials from the standpoint of a physician diagnosing a patient with diabetes on their visit. We created rubrics with 1-5 scores and used them to prompt Claude Sonnet 4 to evaluate the materials created by ChatGPT-40 based on accuracy (based on ADA guidelines), readability (Flesh-Kincaid), cultural relevance, and actionability (PEMAT). Results: Across all patient notes, Claude gave average scores of 3.75, 3.5, 5, and 4.75 for domains of readability, cultural relevance, accuracy, and actionability, respectively. Despite minor variations between readability and cultural relevance, the health education materials generated by ChatGPT still excelled in accuracy and actionability. Conclusion: Although our sample limits generalizability, ChatGPT demonstrated strong promise to be incorporated into clinical practice. ChatGPT has potential to both expedite and personalize diabetes education materials while accounting for many different factors. Future studies should explore ethical use of LLMs in generating patient health education materials at the point of diagnosis in real life settings.

Download

Share

COinS