ChatGPT: Is It Reliable as an Automated Writing Evaluation Tool?

Saliha Toscu

Makale detayları

ChatGPT: Is It Reliable as an Automated Writing Evaluation Tool?

Yazarlar:

Saliha Toscu

Yayıncı:

Anadolu University

Ana Başlıklar:

Alan Eğitimleri (Diğer)

Dergi:

Anadolu Journal of Educational Sciences International Vol.15 Issue 1, pp.329-349

Belge Tipi:

Makale

Yayın Tarihi:

3/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001341239

pISSN:

2146-4014

Anahtar kelimeler:

Özet:

 This study primarily aims to give an understanding of whether or not teachers could rely on AI technology, specifically ChatGPT, to score students writings. The study was conducted with the participation of EFL university students. The students were assigned different writing tasks for five weeks, and the tasks were scored by a teacher and ChatGPT separately. Then, their scores were compared to see the extent to which ChatGPT and teacher scores differed on the SPSS. The test results indicated no statistically significant differences in the scores the bot or the teacher gave. Additionally, the results were supported by the qualitative analysis of the teachers perception of ChatGPT use for automated writing evaluation. The teachers perceptions indicated their positive attitudes towards its use for the evaluation process and general use for enhancing instruction and learning, together with the concerns and suggestions to make the most of ChatGPT. The study gives insights into the integration of ChatGPT into the assessment process and its effectiveness for class practices.

 This study primarily aims to give an understanding of whether or not teachers could rely on AI technology, specifically ChatGPT, to score students writings. The study was conducted with the participation of EFL university students. The students were assigned different writing tasks for five weeks, and the tasks were scored by a teacher and ChatGPT separately. Then, their scores were compared to see the extent to which ChatGPT and teacher scores differed on the SPSS. The test results indicated no statistically significant differences in the scores the bot or the teacher gave. Additionally, the results were supported by the qualitative analysis of the teachers perception of ChatGPT use for automated writing evaluation. The teachers perceptions indicated their positive attitudes towards its use for the evaluation process and general use for enhancing instruction and learning, together with the concerns and suggestions to make the most of ChatGPT. The study gives insights into the integration of ChatGPT into the assessment process and its effectiveness for class practices.

 This study primarily aims to give an understanding of whether or not teachers could rely on AI technology, specifically ChatGPT, to score students writings. The study was conducted with the participation of EFL university students. The students were assigned different writing tasks for five weeks, and the tasks were scored by a teacher and ChatGPT separately. Then, their scores were compared to see the extent to which ChatGPT and teacher scores differed on the SPSS. The test results indicated no statistically significant differences in the scores the bot or the teacher gave. Additionally, the results were supported by the qualitative analysis of the teachers perception of ChatGPT use for automated writing evaluation. The teachers perceptions indicated their positive attitudes towards its use for the evaluation process and general use for enhancing instruction and learning, together with the concerns and suggestions to make the most of ChatGPT. The study gives insights into the integration of ChatGPT into the assessment process and its effectiveness for class practices.

 Bu çalışma öncelikle öğretmenlerin öğrencilerin yazılarını puanlamak için yapay zeka teknolojisine, özellikle chatgptye güvenip güvenemeyeceğine dair bir anlayış kazandırmayı amaçlamaktadır. Yarı deneysel bir araştırma tasarımına dayanan çalışma, EFL üniversitesi öğrencilerinin katılımıyla gerçekleştirilmiştir. Öğrencilere beş hafta boyunca farklı yazma ödevleri verildi ve ödevler bir öğretmen ve ChatGPT tarafından ayrı ayrı puanlandı. Daha sonra, ChatGPT ve bir öğretmenin puanlarının SPSSde ne ölçüde farklı olduğunu görmek için puanları birbirleriyle karşılaştırıldı. Testlerden elde edilen sonuçlar, ChatGPT veya öğretmen tarafından verilen puanlarda istatistiksel olarak anlamlı bir farklılık göstermedi. Ek olarak, sonuçlar öğretmenlerin otomatik yazma değerlendirmesi için ChatGPT kullanımı algısının nitel analizi ile desteklenmiştir. Öğretmenlerin algıları, değerlendirme sürecinde kullanımına ve öğretimi ve öğrenmeyi geliştirmek için genel kullanıma yönelik olumlu tutumlarını birlikte göstermiştir.

Makale detayları

ChatGPT: Is It Reliable as an Automated Writing Evaluation Tool?

Yazarlar:

Saliha Toscu

Yayıncı:

Anadolu University

Ana Başlıklar:

Alan Eğitimleri (Diğer)

Dergi:

Anadolu Journal of Educational Sciences International Vol.15 Issue 1, pp.329-349

Belge Tipi:

Makale

Yayın Tarihi:

3/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001341239

pISSN:

2146-4014

artificial intelligence

assessment

ChatGPT

writingyapay zeka

ChatGPT

değerlendirme

yazma

Sayı

Makale detayları

Yazarlar: Saliha Toscu

Yayıncı: Anadolu University

Ana Başlıklar: Alan Eğitimleri (Diğer)

Dergi: Anadolu Journal of Educational Sciences International Vol.15 Issue 1, pp.329-349

Belge Tipi: Makale

Yayın Tarihi: 3/2025

Belge Dili: İngilizce

Document ID (Org No.): 2001341239

pISSN: 2146-4014

Sayı

Yazarlar:

Saliha Toscu

Yayıncı:

Anadolu University

Ana Başlıklar:

Alan Eğitimleri (Diğer)

Dergi:

Anadolu Journal of Educational Sciences International Vol.15 Issue 1, pp.329-349

Belge Tipi:

Makale

Yayın Tarihi:

3/2025

Belge Dili:

İngilizce

Document ID (Org No.):

2001341239

pISSN:

2146-4014