Evaluation of Artificial Intelligence Chatbots in the Management of Primary Tooth Traumas: A Comparative Analysis

Mihriban Gökcek Taraç

Makale detayları

Evaluation of Artificial Intelligence Chatbots in the Management of Primary Tooth Traumas: A Comparative Analysis

Yazarlar:

Mihriban Gökcek Taraç

Yayıncı:

Zonguldak Bulent Ecevit University

Ana Başlıklar:

Çocuk Diş Hekimliği

Dergi:

Journal of International Dental Sciences (Uluslararası Diş Hekimliği Bilimleri Dergisi) Vol.11 Issue 1, pp.22-31

Belge Tipi:

Makale

Yayın Tarihi:

3/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001344269

pISSN:

2149-8628

Anahtar kelimeler:

Özet:

 Aim: This study aimed to evaluate the reliability and consistency of four artificial intelligence AI chatbotsChatGPT 3.5, Google Gemini, Bing, and Claude AIas public sources of information on the management of primary tooth trauma. Materials and Methods: A total of 31 dichotomous questions were developed based on common issues and concerns related to dental trauma, particularly those frequently raised by parents. Each question, sequentially presented to the four AI chatbots, was repeated three times daily, with a one-hour interval between repetitions, over a five-day period, to assess the reliability and reproducibility of responses. Accuracy was determined by calculating the proportion of correct responses, with 95 confidence intervals estimated using the Wald binomial method. Reliability was assessed using Fleiss kappa coefficient. Results: All AI chatbots demonstrated high accuracy. Bing emerged as the most accurate model, achieving an accuracy rate of 96.34, while Claude had the lowest accuracy at 88.17. Consistency was classified as almost perfect for ChatGPT, Bing, and Gemini, whereas Claude exhibited a substantial level of agreement. These findings underscore the relative performance of AI models in tasks requiring high accuracy and reliability. Conclusion: These results emphasize the importance of critically evaluating AI-based systems for their potential use in clinical applications. Continuous improvements and updates are essential to enhance their reliability and ensure their effectiveness as public information tools.

 Aim: This study aimed to evaluate the reliability and consistency of four artificial intelligence AI chatbotsChatGPT 3.5, Google Gemini, Bing, and Claude AIas public sources of information on the management of primary tooth trauma. Materials and Methods: A total of 31 dichotomous questions were developed based on common issues and concerns related to dental trauma, particularly those frequently raised by parents. Each question, sequentially presented to the four AI chatbots, was repeated three times daily, with a one-hour interval between repetitions, over a five-day period, to assess the reliability and reproducibility of responses. Accuracy was determined by calculating the proportion of correct responses, with 95 confidence intervals estimated using the Wald binomial method. Reliability was assessed using Fleiss kappa coefficient. Results: All AI chatbots demonstrated high accuracy. Bing emerged as the most accurate model, achieving an accuracy rate of 96.34, while Claude had the lowest accuracy at 88.17. Consistency was classified as almost perfect for ChatGPT, Bing, and Gemini, whereas Claude exhibited a substantial level of agreement. These findings underscore the relative performance of AI models in tasks requiring high accuracy and reliability. Conclusion: These results emphasize the importance of critically evaluating AI-based systems for their potential use in clinical applications. Continuous improvements and updates are essential to enhance their reliability and ensure their effectiveness as public information tools.

 Aim: This study aimed to evaluate the reliability and consistency of four artificial intelligence AI chatbotsChatGPT 3.5, Google Gemini, Bing, and Claude AIas public sources of information on the management of primary tooth trauma. Materials and Methods: A total of 31 dichotomous questions were developed based on common issues and concerns related to dental trauma, particularly those frequently raised by parents. Each question, sequentially presented to the four AI chatbots, was repeated three times daily, with a one-hour interval between repetitions, over a five-day period, to assess the reliability and reproducibility of responses. Accuracy was determined by calculating the proportion of correct responses, with 95 confidence intervals estimated using the Wald binomial method. Reliability was assessed using Fleiss kappa coefficient. Results: All AI chatbots demonstrated high accuracy. Bing emerged as the most accurate model, achieving an accuracy rate of 96.34, while Claude had the lowest accuracy at 88.17. Consistency was classified as almost perfect for ChatGPT, Bing, and Gemini, whereas Claude exhibited a substantial level of agreement. These findings underscore the relative performance of AI models in tasks requiring high accuracy and reliability. Conclusion: These results emphasize the importance of critically evaluating AI-based systems for their potential use in clinical applications. Continuous improvements and updates are essential to enhance their reliability and ensure their effectiveness as public information tools.

 Amaç: Bu çalışma, dört yapay zeka sohbet botunun ChatGPT 3.5, Google Gemini, Bing ve Claude AI süt dişi travmasının yönetimiyle ilgili kamuya açık bilgi kaynakları olarak güvenilirliğini ve tutarlılığını değerlendirmeyi amaçlamıştır. Yöntem: Ebeveynlerin dental travmalar hakkında en sık sorduğu sorular temel alınarak, Evet veya Hayır şeklinde yanıtlanabilen 31 soru hazırlanmıştır. Her soru, dört yapay zeka sohbet botuna sırasıyla yöneltilmiş ve yanıtların güvenilirliğini ve tekrarlanabilirliğini değerlendirmek amacıyla beş gün boyunca, günde üç kez, birer saat arayla tekrarlanmıştır. Doğruluk, doğru yanıtların oranı hesaplanarak belirlenmiş ve 95 güven aralıkları Wald binom yöntemi kullanılarak tahmin edilmiştir. Güvenilirlik, Fleissin kappa katsayısı ile değerlendirilmiştir. Bulgular: Tüm yapay zeka sohbet botları yüksek doğruluk sergilemiştir. Bing, 96,34 doğruluk oranı ile en doğru model olarak öne çıkarken, Claude 88,17 doğruluk oranı ile en düşük performansı göstermiştir. Tutarlılık açısından ChatGPT, Bing ve Gemini neredeyse mükemmel düzeyde uyum gösterirken, Claude önemli düzeyde bir uyum sergilemiştir. Bu bulgular, yüksek doğruluk ve güvenilirlik gerektiren görevlerde Yapay Zeka modellerinin göreceli performansını vurgulamaktadır. Sonuç: Bu sonuçlar, klinik uygulamalarda potansiyel kullanımları açısından yapay zeka tabanlı sistemlerin eleştirel bir şekilde değerlendirilmesinin önemini ortaya koymaktadır. Güvenilirliklerini artırmak ve kamuya açık bilgi araçları olarak etkinliklerini sağlamak için sürekli iyileştirmeler ve güncellemeler gereklidir.

Makale detayları

Evaluation of Artificial Intelligence Chatbots in the Management of Primary Tooth Traumas: A Comparative Analysis

Yazarlar:

Mihriban Gökcek Taraç

Yayıncı:

Zonguldak Bulent Ecevit University

Ana Başlıklar:

Çocuk Diş Hekimliği

Dergi:

Journal of International Dental Sciences (Uluslararası Diş Hekimliği Bilimleri Dergisi) Vol.11 Issue 1, pp.22-31

Belge Tipi:

Makale

Yayın Tarihi:

3/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001344269

pISSN:

2149-8628

Artificial Intelligence

Chatbot

Dental Trauma.Yapay Zeka

Dental Travma

Chat GPT.

Sayı

Makale detayları

Evaluation of Artificial Intelligence Chatbots in the Management of Primary Tooth Traumas: A Comparative Analysis

Yazarlar: Mihriban Gökcek Taraç

Yayıncı: Zonguldak Bulent Ecevit University

Ana Başlıklar: Çocuk Diş Hekimliği

Dergi: Journal of International Dental Sciences (Uluslararası Diş Hekimliği Bilimleri Dergisi) Vol.11 Issue 1, pp.22-31

Belge Tipi: Makale

Yayın Tarihi: 3/2025

Belge Dili: İngilizce

Document ID (Org No.): 2001344269

pISSN: 2149-8628

Artificial Intelligence

Chatbot

Dental Trauma.Yapay Zeka

Dental Travma

Chat GPT.

Sayı

Yazarlar:

Mihriban Gökcek Taraç

Yayıncı:

Zonguldak Bulent Ecevit University

Ana Başlıklar:

Çocuk Diş Hekimliği

Dergi:

Journal of International Dental Sciences (Uluslararası Diş Hekimliği Bilimleri Dergisi) Vol.11 Issue 1, pp.22-31

Belge Tipi:

Makale

Yayın Tarihi:

3/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001344269

pISSN:

2149-8628