Artificial Intelligence in Patient Communication: Performance of GPT-3.5 and GPT-4 in Coronary Bypass Surgery

Muhammet Fethi Sağlam; Emrah Uğuz; Kemal Erdoğan; Hüseyin Ünsal Erçelik; Murat Yücel; Cevat Ahmet Sert; Fatih Yamac; Erol Sener

Makale detayları

Artificial Intelligence in Patient Communication: Performance of GPT-3.5 and GPT-4 in Coronary Bypass Surgery

Yazarlar:

Muhammet Fethi Sağlam | Emrah Uğuz | Kemal Erdoğan | Hüseyin Ünsal Erçelik | Murat Yücel | Cevat Ahmet Sert | Fatih Yamac | Erol Sener

Yayıncı:

Duzce University

Ana Başlıklar:

Health Informatics and Information Systems

Dergi:

Konuralp Medical Journal Vol.17 Issue 1, pp.100-105

Belge Tipi:

Makale

Yayın Tarihi:

3/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001344315

pISSN:

1309-3878

Anahtar kelimeler:

Özet:

 Objective: This study aims to evaluate the ability of GPT-3.5 and GPT-4 to provide accurate, comprehensible, and clinically relevant responses to common patient questions about coronary bypass surgery. Method: A cross-sectional study was conducted at Ankara Yıldırım Beyazıt University Bilkent City Hospital with 80 cardiovascular surgery specialists. Participants rated the responses of GPT-3.5 and GPT-4 to 10 common patient questions about coronary bypass surgery based on four criteria: accuracy, understandability, clinical appropriateness, and overall evaluation. Statistical analysis included independent t-tests, Cronbachs Alpha reliability analysis, and Cohens d effect size calculation. Results: GPT-4 significantly outperformed GPT-3.5 across all metrics. The mean scores for GPT-4 were higher in accuracy 3.02 vs. 1.77, understandability 2.99 vs. 1.81, clinical appropriateness 2.96 vs. 1.78, and overall evaluation 2.98 vs. 1.77 p0.05 for all. Cronbachs Alpha values indicated good internal consistency 0.69 for all metrics, and Cohens d effect sizes demonstrated large differences 1.54 to 1.65. Conclusions: GPT-4 shows superior potential compared to GPT-3.5 in answering patient questions about coronary bypass surgery. Despite its strengths, occasional inaccuracies and incomplete responses highlight the need for further refinement. Future research should integrate patient feedback and evaluate the real-world clinical impact of these models to optimize their application in healthcare.

 Objective: This study aims to evaluate the ability of GPT-3.5 and GPT-4 to provide accurate, comprehensible, and clinically relevant responses to common patient questions about coronary bypass surgery. Method: A cross-sectional study was conducted at Ankara Yıldırım Beyazıt University Bilkent City Hospital with 80 cardiovascular surgery specialists. Participants rated the responses of GPT-3.5 and GPT-4 to 10 common patient questions about coronary bypass surgery based on four criteria: accuracy, understandability, clinical appropriateness, and overall evaluation. Statistical analysis included independent t-tests, Cronbachs Alpha reliability analysis, and Cohens d effect size calculation. Results: GPT-4 significantly outperformed GPT-3.5 across all metrics. The mean scores for GPT-4 were higher in accuracy 3.02 vs. 1.77, understandability 2.99 vs. 1.81, clinical appropriateness 2.96 vs. 1.78, and overall evaluation 2.98 vs. 1.77 p0.05 for all. Cronbachs Alpha values indicated good internal consistency 0.69 for all metrics, and Cohens d effect sizes demonstrated large differences 1.54 to 1.65. Conclusions: GPT-4 shows superior potential compared to GPT-3.5 in answering patient questions about coronary bypass surgery. Despite its strengths, occasional inaccuracies and incomplete responses highlight the need for further refinement. Future research should integrate patient feedback and evaluate the real-world clinical impact of these models to optimize their application in healthcare.

 Objective: This study aims to evaluate the ability of GPT-3.5 and GPT-4 to provide accurate, comprehensible, and clinically relevant responses to common patient questions about coronary bypass surgery. Method: A cross-sectional study was conducted at Ankara Yıldırım Beyazıt University Bilkent City Hospital with 80 cardiovascular surgery specialists. Participants rated the responses of GPT-3.5 and GPT-4 to 10 common patient questions about coronary bypass surgery based on four criteria: accuracy, understandability, clinical appropriateness, and overall evaluation. Statistical analysis included independent t-tests, Cronbachs Alpha reliability analysis, and Cohens d effect size calculation. Results: GPT-4 significantly outperformed GPT-3.5 across all metrics. The mean scores for GPT-4 were higher in accuracy 3.02 vs. 1.77, understandability 2.99 vs. 1.81, clinical appropriateness 2.96 vs. 1.78, and overall evaluation 2.98 vs. 1.77 p0.05 for all. Cronbachs Alpha values indicated good internal consistency 0.69 for all metrics, and Cohens d effect sizes demonstrated large differences 1.54 to 1.65. Conclusions: GPT-4 shows superior potential compared to GPT-3.5 in answering patient questions about coronary bypass surgery. Despite its strengths, occasional inaccuracies and incomplete responses highlight the need for further refinement. Future research should integrate patient feedback and evaluate the real-world clinical impact of these models to optimize their application in healthcare.

 Amaç: Bu çalışma, GPT-3.5 ve GPT-4ün koroner bypass cerrahisiyle ilgili yaygın hasta sorularına doğru, anlaşılır ve klinik olarak uygun yanıtlar verme yeteneğini değerlendirmeyi amaçlamaktadır. Yöntem: Ankara Yıldırım Beyazıt Üniversitesi Bilkent Şehir Hastanesinde 80 kalp ve damar cerrahisi uzmanı ile kesitsel bir çalışma yürütülmüştür. Katılımcılar GPT-3.5 ve GPT-4ün koroner bypass cerrahisi ile ilgili 10 yaygın hasta sorusuna verdiği yanıtları dört kritere göre değerlendirmiştir: doğruluk, anlaşılabilirlik, klinik uygunluk ve genel değerlendirme. İstatistiksel analiz bağımsız t-testlerini, Cronbach Alfa güvenilirlik analizini ve Cohens d etki büyüklüğü hesaplamasını içermektedir. Bulgular: GPT-4 tüm ölçütlerde GPT-3.5ten önemli ölçüde daha iyi performans göstermiştir. GPT-4 için ortalama puanlar doğruluk 3,02ye karşı 1,77, anlaşılabilirlik 2,99a karşı 1,81, klinik uygunluk 2,96ya karşı 1,78 ve genel değerlendirme 2,98e karşı 1,77 açısından daha yüksekti tümü için p0,05. Cronbachs Alpha değerleri iyi bir iç tutarlılık tüm ölçütler için 0,69 ve Cohens d etki büyüklükleri büyük farklılıklar 1,54 ila 1,65 göstermiştir. Sonuç: GPT-4, koroner bypass cerrahisi ile ilgili hasta sorularını yanıtlamada GPT-3.5e kıyasla üstün potansiyel göstermektedir. Güçlü yönlerine rağmen, zaman zaman ortaya çıkan yanlışlıklar ve eksik yanıtlar daha fazla iyileştirme ihtiyacının altını çizmektedir. Gelecekteki araştırmalar, hasta geri bildirimlerini entegre etmeli ve sağlık hizmetlerinde uygulamalarını optimize etmek için bu modellerin gerçek dünyadaki klinik etkilerini değerlendirmelidir.

Makale detayları

Artificial Intelligence in Patient Communication: Performance of GPT-3.5 and GPT-4 in Coronary Bypass Surgery

Yazarlar:

Muhammet Fethi Sağlam | Emrah Uğuz | Kemal Erdoğan | Hüseyin Ünsal Erçelik | Murat Yücel | Cevat Ahmet Sert | Fatih Yamac | Erol Sener

Yayıncı:

Duzce University

Ana Başlıklar:

Health Informatics and Information Systems

Dergi:

Konuralp Medical Journal Vol.17 Issue 1, pp.100-105

Belge Tipi:

Makale

Yayın Tarihi:

3/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001344315

pISSN:

1309-3878

Coronary bypass surgery

Artificial intelligence

GPT-3.5

GPT-4

Patient communicationKoroner bypass cerrahisi

Yapay zeka

GPT-3.5

GPT-4

Hasta iletişimi

Sayı

Makale detayları

Yazarlar: Muhammet Fethi Sağlam | Emrah Uğuz | Kemal Erdoğan | Hüseyin Ünsal Erçelik | Murat Yücel | Cevat Ahmet Sert | Fatih Yamac | Erol Sener

Yayıncı: Duzce University

Ana Başlıklar: Health Informatics and Information Systems

Dergi: Konuralp Medical Journal Vol.17 Issue 1, pp.100-105

Belge Tipi: Makale

Yayın Tarihi: 3/2025

Belge Dili: İngilizce

Document ID (Org No.): 2001344315

pISSN: 1309-3878

Sayı

Yazarlar:

Muhammet Fethi Sağlam | Emrah Uğuz | Kemal Erdoğan | Hüseyin Ünsal Erçelik | Murat Yücel | Cevat Ahmet Sert | Fatih Yamac | Erol Sener

Yayıncı:

Duzce University

Ana Başlıklar:

Health Informatics and Information Systems

Dergi:

Konuralp Medical Journal Vol.17 Issue 1, pp.100-105

Belge Tipi:

Makale

Yayın Tarihi:

3/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001344315

pISSN:

1309-3878