Comparative Performance Evaluation of Multimodal Large Language Models, Radiologist, and Anatomist in Visual Neuroanatomy Questions

Yasin Celal Güneş; Mehmet Ülkir

Makale detayları

Comparative Performance Evaluation of Multimodal Large Language Models, Radiologist, and Anatomist in Visual Neuroanatomy Questions

Yazarlar:

Yasin Celal Güneş | Mehmet Ülkir

Yayıncı:

Bursa Uludağ University

Ana Başlıklar:

Radyoloji ve Organ Görüntüleme | Anatomi

Dergi:

Uludağ Üniversitesi Tıp Fakültesi Dergisi Vol.50 Issue 3, pp.551-556

Belge Tipi:

Makale

Yayın Tarihi:

12/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001338904

pISSN:

1300-414X

Anahtar kelimeler:

Özet:

 This study examined the performance of four different multimodal Large Language Models LLMsGPT4-V, GPT-4o, LLaVA, and Gemini 1.5 Flashon multiple-choice visual neuroanatomy questions, comparing them to a radiologist and an anatomist. The study employed a cross-sectional design and evaluated responses to 100 visual questions sourced from the Radiopaedia website. The accuracy of the responses was analyzed using the McNemar test. According to the results, the radiologist demonstrated the highest performance with an accuracy rate of 90, while the anatomist achieved an accuracy rate of 67. Among the multimodal LLMs, GPT-4o performed the best, with an accuracy rate of 45, followed by Gemini 1.5 Flash at 35, ChatGPT4-V at 22, and LLaVA at 15. The radiologist significantly outperformed both the anatomist and all multimodal LLMs p0.001. GPT-4o significantly outperformed GPT4-V and LLaVA p0.001, but no significant difference was found between GPT-4o and Gemini 1.5 Flash p0.123. However, Gemini 1.5 Flash showed significant superiority over LLaVA p0.001 and also demonstrated a statistically significant difference compared to GPT4-V p0.004. This study highlights the significant performance gap between multimodal LLMs and medical professionals. While multimodal LLMs hold great potential in the medical field, they have not yet reached the level of accuracy of medical experts in correctly identifying neuroanatomical regions.

 This study examined the performance of four different multimodal Large Language Models LLMsGPT4-V, GPT-4o, LLaVA, and Gemini 1.5 Flashon multiple-choice visual neuroanatomy questions, comparing them to a radiologist and an anatomist. The study employed a cross-sectional design and evaluated responses to 100 visual questions sourced from the Radiopaedia website. The accuracy of the responses was analyzed using the McNemar test. According to the results, the radiologist demonstrated the highest performance with an accuracy rate of 90, while the anatomist achieved an accuracy rate of 67. Among the multimodal LLMs, GPT-4o performed the best, with an accuracy rate of 45, followed by Gemini 1.5 Flash at 35, ChatGPT4-V at 22, and LLaVA at 15. The radiologist significantly outperformed both the anatomist and all multimodal LLMs p0.001. GPT-4o significantly outperformed GPT4-V and LLaVA p0.001, but no significant difference was found between GPT-4o and Gemini 1.5 Flash p0.123. However, Gemini 1.5 Flash showed significant superiority over LLaVA p0.001 and also demonstrated a statistically significant difference compared to GPT4-V p0.004. This study highlights the significant performance gap between multimodal LLMs and medical professionals. While multimodal LLMs hold great potential in the medical field, they have not yet reached the level of accuracy of medical experts in correctly identifying neuroanatomical regions.

 This study examined the performance of four different multimodal Large Language Models LLMsGPT4-V, GPT-4o, LLaVA, and Gemini 1.5 Flashon multiple-choice visual neuroanatomy questions, comparing them to a radiologist and an anatomist. The study employed a cross-sectional design and evaluated responses to 100 visual questions sourced from the Radiopaedia website. The accuracy of the responses was analyzed using the McNemar test. According to the results, the radiologist demonstrated the highest performance with an accuracy rate of 90, while the anatomist achieved an accuracy rate of 67. Among the multimodal LLMs, GPT-4o performed the best, with an accuracy rate of 45, followed by Gemini 1.5 Flash at 35, ChatGPT4-V at 22, and LLaVA at 15. The radiologist significantly outperformed both the anatomist and all multimodal LLMs p0.001. GPT-4o significantly outperformed GPT4-V and LLaVA p0.001, but no significant difference was found between GPT-4o and Gemini 1.5 Flash p0.123. However, Gemini 1.5 Flash showed significant superiority over LLaVA p0.001 and also demonstrated a statistically significant difference compared to GPT4-V p0.004. This study highlights the significant performance gap between multimodal LLMs and medical professionals. While multimodal LLMs hold great potential in the medical field, they have not yet reached the level of accuracy of medical experts in correctly identifying neuroanatomical regions.

 Bu çalışma, dört farklı çok modlu Büyük Dil Modelinin GPT4-V, GPT-4o, LLaVA, Gemini 1.5 Flash görsel nöroanatomi çoktan seçmeli sorularındaki performansını, bir radyolog ve bir anatomistle karşılaştırarak incelemiştir. Kesitsel bir araştırma dizaynına dayanan çalışmada, Radiopaedia web sitesinden alınan 100 görsel soruya verilen yanıtlar değerlendirilmiştir. Yanıtların doğruluğu McNemar testi kullanılarak analiz edilmiştir. Sonuçlara göre, radyolog 90 doğruluk oranı ile en yüksek performansı sergilerken, anatomist 67 doğruluk oranı elde etmiştir. Çok modlu LLMler arasında en iyi performansı 45 doğruluk oranı ile GPT-4o göstermiştir; onu 35 ile Gemini 1.5 Flash, 22 ile ChatGPT4-V ve 15 ile LLaVA takip etmiştir. Radyolog, hem anatomiste hem de tüm çok modlu LLMlere kıyasla anlamlı derecede üstün bir performans sergilemiştir p0.001. GPT-4o, GPT4-V ve LLaVAya kıyasla anlamlı derecede daha iyi bir performans göstermiş p0.001, ancak Gemini 1.5 Flash ile arasında anlamlı bir fark gözlenmemiştir p0.123. Bununla birlikte, Gemini 1.5 Flash, LLaVAya karşı anlamlı bir üstünlük sağlamış p0.001 ve GPT4-V ile karşılaştırıldığında da istatistiksel olarak anlamlı bir fark ortaya çıkmıştır p0.004. Bu çalışma, çok modlu LLMler ile tıbbi uzmanlar arasındaki belirgin performans farkını ortaya koymaktadır. Çok modlu LLMler tıp alanında büyük bir potansiyel vaat etse de, nöroanatomik bölgeleri doğru bir şekilde tanımlama konusunda henüz tıbbi uzmanların doğruluk seviyesine ulaşamamaktadırlar.

Makale detayları

Comparative Performance Evaluation of Multimodal Large Language Models, Radiologist, and Anatomist in Visual Neuroanatomy Questions

Yazarlar:

Yasin Celal Güneş | Mehmet Ülkir

Yayıncı:

Bursa Uludağ University

Ana Başlıklar:

Radyoloji ve Organ Görüntüleme | Anatomi

Dergi:

Uludağ Üniversitesi Tıp Fakültesi Dergisi Vol.50 Issue 3, pp.551-556

Belge Tipi:

Makale

Yayın Tarihi:

12/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001338904

pISSN:

1300-414X

neuroanatomy

large language models

; GPT-4o

Gemini 1.5 Flashnöroanatomi

büyük dil modelleri

GPT-4o

Gemini 1.5 Flash

Sayı

Makale detayları

Yazarlar: Yasin Celal Güneş | Mehmet Ülkir

Yayıncı: Bursa Uludağ University

Ana Başlıklar: Radyoloji ve Organ Görüntüleme | Anatomi

Dergi: Uludağ Üniversitesi Tıp Fakültesi Dergisi Vol.50 Issue 3, pp.551-556

Belge Tipi: Makale

Yayın Tarihi: 12/2025

Belge Dili: İngilizce

Document ID (Org No.): 2001338904

pISSN: 1300-414X

Sayı

Yazarlar:

Yasin Celal Güneş | Mehmet Ülkir

Yayıncı:

Bursa Uludağ University

Ana Başlıklar:

Radyoloji ve Organ Görüntüleme | Anatomi

Dergi:

Uludağ Üniversitesi Tıp Fakültesi Dergisi Vol.50 Issue 3, pp.551-556

Belge Tipi:

Makale

Yayın Tarihi:

12/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001338904

pISSN:

1300-414X